Comment on Widespread Cloudflare outage blamed on mysterious traffic spike

<- View Parent
eRac@lemmings.world ⁨1⁩ ⁨week⁩ ago

It was actually the system Cloudflare uses to catch and block bots that went haywire.

They had a fake database you could query that would pull content from a bunch of different shard databases. They updated the config so that systems querying it could see the shards in addition to the main dummy DB. The tool that pulled data out of it assumed that it could only see the dummy, however, so it just asked for everything when it pulled a report to pass to the filtering system.

The filtering system assumed the report it received would be properly formed and crashed if it got one that was malformed.

source
Sort:hotnewtop