sudo
@sudo@programming.dev
- Comment on Bread mold 1 week ago:
Kraft American Singles is normal american cheese. All american cheese is “cheese product” because its solidified cheese sauce, not a cheese itself.
- Comment on I highly recommend journalctl-desktop-notification 1 week ago:
That seems like a flight of stairs up.
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
🤙
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
Anubis forces the site to reload when doing the normal PoW challenge! Meta Refresh is a sufficient mouse to block 99% of all bot traffic without being any more burdensome than PoW.
You’ve failed to demonstrate why meta-refresh is more burdensome than PoW and have pivoted to arguing the point I was making from the start as though it was your own. I’m not arguing with you any further. I’m satisfied that I’ve convinced any readers of our discussion.
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
You will have people complain about their anti-fingerprinting being blocked with every bot-managment solution. Your ability to navigate the internet anonymously is directly correlated with a bots ability to scrape. That has never been my complaint about Anubis.
My complaint is that the calculations Anubis forces you to do are absolutely negligible burden for a bot to solve. The hardest part is just having a JavaScript interpreter available. Making the author of the scraper write custom code to deal with your website is the most effective way to prevent bots.
Think about how much computing power AI data centers have. Do you think they give a shit about hashing some values for Anubis? No. They burn more compute power than a thousand Anubis challenges generating a single llm answer. PoW is a backwards solution.
Please Think. Captchas worked because they’re supposed to be hard for a computer to solve but are easy for a human. PoW is the opposite.
In the current shape Anubis has zero impact on usability for 99% of the site visitors, not so with meta refresh.
Again, I ask you: What extra burden does meta-refresh impose on users? How does setting a cookie and immediately refreshing the page burden the user more than making them wait longer while draining their battery before doing the exact same thing? Its strictly less intrusive.
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
And how do you actually check for working JS in a way that can’t be easily spoofed? Hint: PoW is a good way to do that.
Accessing the browsers API in any way is way harder to spoof than some hashing. I already suggested checking if the browser has graphics acceleration. That would filter out the vast majority of headless browsers too. PoW is just math and is easy to spoof without running any JavaScript. You can even do it faster than real JavaScript users something like Rust or C.
Meta refresh is a downgrade in usability for everyone but a tiny minority that has disabled JS.
What are you talking about? It just refreshes the page without doing any of the extra computation that PoW does. What extra burden does it put on users?
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
LOL
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
scrapers (currently) don’t want to spend extra on running headless chromium
WTF, That’s what I already? That was my entire point from the start!? You don’t need PoW to force headless usage. Any JavaScript challenge will suffice. I even said the Meta Refresh challenge Anubis provides is sufficient and explicitly recommended it.
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
Well in most cases it would by Python requests not curl. But yes, forcing them to use a browser is the real cost. Not just in CPU time but in programmer labor. PoW is overkill for that though.
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
Anubis is that it has a graded tier system of how sketchy a client is and changing the kind of challenge based on a a weighted priority system.
Last I checked that was just User-Agent regexes and IP lists. But that’s where Anubis should continue development, and hopefully they’ve improved since. Discerning real users from bots is how you do proper bot management. Not imposing a flat tax on all connections.
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
Then there was a paper arguing that PoW can still work, as long as you scale the difficulty in such a way that a legit user
Telling a legit user from a fake user is the entire game. If you can do that you just block the fake user. Professional bot blockers like Cloudflare or Akamai have machine learning systems to analyze trends in network traffic and serve JS challenges to suspicious clients. Last I checked, all Anubis uses is User-Agent filters, which is extremely behind the curve. Bots are able to get down to faking TLS fingerprints and matching them with User-Agents.
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
Its like you didn’t understand anything I said. Anubis does work. I said it works. But it works because most AI crawlers don’t have a headless browser to solve the PoW. To operate efficiently at the high volume required, they use raw http requests. The vast majority are probably using basic python
requestsmodule.You don’t need PoW to throttle general access to your site and that’s not the fundamental assumption of PoW. PoW assumes (incorrectly) that bots won’t pay the extra flops to scrape the website. But bots are paid to scape the website users aren’t. They’ll just scale horizontally and open more parallel connections. They have the money.
- Comment on Anubis is awesome and I want to talk aout it 2 weeks ago:
I’ve repeatedly stated this before: Proof of Work bot-management is only Proof of Javascript bot-management. It is nothing to a headless browser to by-pass. Proof of JavaScript does work and will stop the vast majority of bot traffic. That’s how Anubis actually works. You don’t need to punish actual users by abusing their CPU. POW is a far higher cost on your actual users than the bots.
Last I checked Anubis has an JavaScript-less strategy called “Meta Refresh”. It first serves you a blank HTML page with a
<meta>tag instructing the browser to refresh and load the real page. I highly advise using the Meta Refresh strategy. It should be the default.I’m glad someone is finally making an open source and self hostable bot management solution. And I don’t give a shit about the cat-girls, nor should you. But Techaro admitted they had little idea what they were doing when they started and went for the “nuclear option”. Fuck Proof of Work. It was a Dead On Arrival idea decades ago. Techaro should strip it from Anubis.
I haven’t caught up with what’s new with Anubis, but if they want to get stricter bot-management, they should check for actual graphics acceleration.
- Comment on Racism restaurant 3 weeks ago:
Made from white cheddar instead of yellow cheddar. Only difference is no annetto.
- Comment on Tabletop convection oven 1 month ago:
I’ve considtently cooked better rice on my stove than any rice cooker. It takes more practice to dial in but once you’ve got it, its better than a rice cooker.
ProTip you can just pick up the pot, hold the lid down and flip it to see if there’s too much or too little water left.
- Comment on [deleted] 2 months ago:
- Are you running as the systemd service?
- “Find” as in it doesn’t know where your folders are or “find” as in you told it where the media is but they’re not showing up? If former, configure your libraries in the UI. If latter, check logs, its probably a permissions issue.
- Can you log in to the webui from the host machine itself? If so but not other devices on the LAN, then there’s a probably a setting somewhere that’s set to
127.0.0.1that should be0.0.0.0.
- Comment on 2 months ago:
Liberals make up more of the consumer class than conservatives. That highly marketable strata of people that have disposable income tend to be affluent, college educated liberals. Its why they keep winning the culture war and it drives conservatives insane.
- Comment on [deleted] 2 months ago:
Turning and turning in the widening gyre The falcon cannot hear the falconer; Things fall apart; the centre cannot hold; Mere anarchy is loosed upon the world, The blood-dimmed tide is loosed, and everywhere The ceremony of innocence is drowned; The best lack all conviction, while the worst Are full of passionate intensity.
- Comment on [deleted] 2 months ago:
He’s clearly a centrist.
- Comment on [deleted] 2 months ago:
Is this Tyler Robinson?
- Comment on Jellyswarrm - reverse proxy all your Jellyfin servers from a single interface, presenting as a standard Jellyfin server, clients should work out of the box. 3 months ago:
Automatic Mapping
If a user already exists on one or more connected servers, they can log in directly with their existing Jellyfin credentials. Jellyswarrm will automatically create a local user and set up the necessary server mappings.
If the same username and password exist on multiple servers, Jellyswarrm will link those accounts together automatically. This provides a smooth experience, giving the user unified access to all linked servers.
Really should audit the implementation of that feature. So when you first log in it automatically sends you’re credentials to every connected server?
- Comment on Jellyswarrm - reverse proxy all your Jellyfin servers from a single interface, presenting as a standard Jellyfin server, clients should work out of the box. 3 months ago:
I always thought this would make more sense to implement client side in the media player. But its probably easier to implement this way.
- Comment on To install a new outlet with a dedicated circuit do they have to cut the drywall all the way from the electrical panel to the outlet? 3 months ago:
There’s many ways to avoid doing that and the electrician will prefer not to. Generally they prefer to route it laterally through the attic or basement and vertically in the walls. Sometimes, holes are cut in the wall to secure the wire or change its direction.
But if it must move laterally along some drywall and through all of the studs then yeah they’ll need to open up that drywall to drill holes in all the studs.
- Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges. 3 months ago:
Not much for open source solutions. A simple captcha however would cost scrapers more to crack than Anubis.
But when it comes to “real” bot management solutions: The least invasive solutions will try to match User-Agent and other headers against the TLS fingerprint and block if they don’t match. More invasive solutions will fingerprint your browser and even your GPU, then either block you or issue you a tracking cookie which is often pinned to your IP and user-agent. Both of those solutions require a large base of data to know what real and fake traffic actually looks like. Only large hosting providers like CloudFlare and Akamai have that data and can provide those sorts of solutions.
- Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges. 3 months ago:
Costs of solving PoW for Anubis is absolutely not a factor in any AI companies budget. Just the costs of answering one question is millions of times more expensive than running sha256sum for Anubis.
Just in case you’re being glib and mean the businesses will go under regardless of Anubis: most of these are coming from China. China is absolutely will keep running these companies at a loss for the sake of strategic development.
- Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges. 3 months ago:
Places like cloudflare and akamai are already using machine learning algorithms to detect bot traffic at a network level. You need to use similar machine learning to evade them. And since most of these scrapers are for AI companies I’d expect a lot of the scrapers to be LLM generated.
- Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges. 3 months ago:
Here’s one example of a proxy provider offering to pay developers to inject their proxies into their apps. (“100% ethical proxies” because they signed a ToS). Another is BrightData proxies traffic through users of their free HolaVPN.
IOT and smart TVs are also obvious suspects.
- Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges. 3 months ago:
Or your TV or IOT devices. Residential proxies are extremely shady businesses.
- Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges. 3 months ago:
The problem is primarily the resource drain on the server and tarpitting tactics usually increase thag resource burden by maintaining the open connections.
- Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges. 3 months ago:
This is what I’ve kept saying about POW being a shit bot management tactic. Its a flat tax across all users, real or fake. The fake users are getting making money to access your site and will just eat the added expense. You can raise the tax to cost more than what your data is worth to them, but that also affects your real users. Nothing about Anubis even attempts to differentiate between bots and real users.
If the bots take the time, they can set up a pipeline to solve Anubis tokens outside of the browser more efficiently than real users.