Comment on Are the new RSS feeds misbehaving?
auster@thebrainbin.org@thebrainbin.org 3 weeks ago (4/4) Remembered I could ask some AI to make things easier.
For kbin.earth feed, Grok says it's Cloudflare bot protection, likely because neither Newsboat or Feeder pass a valid browser agent.
For TBB, Grok for some reason had a hiccup, so asked GPT-5.4 on DuckDuckGo's AI bridge, and it said:
The problem is that the URL is not returning a valid RSS/Atom feed. It’s returning an HTML login page instead, so feed readers see a normal webpage with <form>, <input>, etc., not feed XML. That’s why one reader says “no feeds on the page,” another says “unsupported feed format,” and another says “malformed xml string.”
From the pasted page, the site is protected by login and the feed endpoint appears to require authentication or is redirecting to the login page. The presence of a CSRF token and sign-in form confirms the response is HTML, not feed data.
oh okay, well yeah I enabled the "force users to login" option, because the scrapers are just causing tremendous load otherwise. On gehirneimer it is currently disabled so maybe that is the sole reason?
So it is just the login requirement or did I miss something else? Allowing anonymous access to the endpoint would solve it then, but makes it also easy for scrapers to fetch content.
Appears to have been the solution. Tested a link yesterday night, after BentiGorlich's comment on the spike graph, and the direct links are back to being login-protected, though now feeds seem to be working.
...or maybe the login requirement was reactivated for everything and what I saw were the ripples of the small window it stayed active, given my test just now.
Seems to be the case? Left an instance of Newsboat running as I went to do my chores, and just checking now, it already fetched a few communities' posts.
Though it seems concerning to me that scrappers are getting past the current protections. (Hopefully I'm not one of them, considering how much I still follow...)
Well when I remove the requirement to login this happens. I guess you know what happened in the middle :D