gaael
@gaael@lemm.ee
- Comment on Black Mirror AI 1 week ago:
From the Fabulous Systems (ScummVM’s sysadmin) blog post linked by Natanox:
About three weeks ago, I started receiving monitoring notifications indicating an increased load on the MariaDB server.
This went on for a couple of days without seriously impacting our server or accessibility–it was a tad slower than usual.
And then the website went down.
Now, it was time to find out what was going on. Hoping that it was just one single IP trying to annoy us, I opened the access log of the day
there were many IPs–around 35.000, to be precise–from residential networks all over the world. At this scale, it makes no sense to even consider blocking individual IPs, subnets, or entire networks. Due to the open nature of the project, geo-blocking isn’t an option either.
The main problem is time. The URLs accessed in the attack are the most expensive ones the wiki offers since they heavily depend on the database and are highly dynamic, requiring some processing time in PHP. This is the worst-case scenario since it throws the server into a death spiral.
First, the database starts to lag or even refuse new connections. This, combined with the steadily increasing server load, leads to slower PHP execution.
At this point, the website dies. Restarting the stack immediately solves the problem for a couple of minutes at best until the server starves again.
Anubis is a program that checks incoming connections, processes them, and only forwards “good” connections to the web application. To do so, Anubis sits between the server or proxy responsible for accepting HTTP/HTTPS and the server that provides the application.
Many bots disguise themselves as standard browsers to circumvent filtering based on the user agent. So, if something claims to be a browser, it should behave like one, right? To verify this, Anubis presents a proof-of-work challenge that the browser needs to solve. If the challenge passes, it forwards the incoming request to the web application protected by Anubis; otherwise, the request is denied.
As a regular user, all you’ll notice is a loading screen when accessing the website. As an attacker with stupid bots, you’ll never get through. As an attacker with clever bots, you’ll end up exhausting your own resources. As an AI company trying to scrape the website, you’ll quickly notice that CPU time can be expensive if used on a large scale.
I didn’t get a single notification afterward. The server load has never been lower. The attack itself is still ongoing at the time of writing this article. To me, Anubis is not only a blocker for AI scrapers. Anubis is a DDoS protection.
- Comment on Ahead of its 10 year Steam anniversary, Black Mesa has a new beta with improved Linux support and many fixes 1 month ago:
Black Mesaer?
- Comment on Beelink ME mini is a NAS with an Intel N200 processor and support for up to 6 SSDs 1 month ago:
This. I can’t afford reliable always-on storage now, but I plan to build for SSDs rather than HDDs because I don’t have a separate room to put it into.
- Comment on Kitten Space Agency - A community for/about the game by the same name that aims to be the spiritual successor to Kerbal Space Program 1 month ago:
You know you do. And even more, because it’s gonna be a cute kitty!
- Comment on After 50 million miles, Waymos crash a lot less than human drivers 2 months ago:
Much more so than having a car-centric infrastructure. If you start cherry-picking you’ll of course find cases where a car would have been more efficient but public transportation needs to be understood as a whole.
- Comment on A system to organise your life • Johnny.Decimal 3 months ago:
Adhd?