This is news to me! Thanks for enlightening me!
Comment on Anubis is awesome and I want to talk aout it
SmokeyDope@piefed.social 3 weeks agoTheres a compute option that doesnt require javascript. Its on site owners to configure IMO, though you can make the argument its not default I guess.
https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
From docs on Meta Refresh Method
Meta Refresh (No JavaScript)
The metarefresh challenge sends a browser a much simpler challenge that makes it refresh the page after a set period of time. This enables clients to pass challenges without executing JavaScript.
To use it in your Anubis configuration:
# Generic catchall rule
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: CHALLENGE
challenge:
difficulty: 1 # Number of seconds to wait before refreshing the page
algorithm: metarefresh # Specify a non-JS challenge method
This is not enabled by default while this method is tested and its false positive rate is ascertained. Many modern scrapers use headless Google Chrome, so this will have a much higher false positive rate.
Dojan@pawb.social 2 weeks ago
z3rOR0ne@lemmy.ml 2 weeks ago
Yeah I actually use the noscript extension and i refuse to just whitelist certain sites unless I’m very certain I trust them.
I run into Anubis checks all the time and while I appreciate the software, having to consistently temporarily whitelist these sites does get cumbersome at times. I hope they make this noJS implementation the default soon.
Prathas@lemmy.zip 2 weeks ago
Wait, you keep temporarily allowing then over and over again? Why temporary?
z3rOR0ne@lemmy.ml 2 weeks ago
Most of the Anubis encounters I have are to redlib instances that are shuffled around, go down all the time, and generally are more ephemeral than other sites. Because I use another extension called Libredirect to shuffle which redlib instance I visit when clicking on a reddit link, I don’t bother whitelisting them permanently.
I already have solved this on my desktop by self hosting my own redlib instance via localhost and using libredirect to just point there, but on my phone I still do the whole nojs temp unblock random redlib instance. Eventually I plan on using wireguard to host a private redlib instance on a vps so I can just not deal with this.
This is a weird case I know, but its honestly not that bad.