Comment on AI companies are violating a basic social contract of the web and and ignoring robots.txt
KingThrillgore@lemmy.ml 9 months ago
I explicitly have my robots.txt set to block out AI crawlers, but I don’t know if anyone else will observe the protocol. They should have tools I can submit a sitemap.xml against to know if i’ve been parsed. Until they bother to address this, I can only assume their intent is hostile and if anyone is serious about building a honeypot and exposing the tooling for us to deploy at large, my options are limited.
phx@lemmy.ca 9 months ago
The funny (in an “wtf” not “haha” sense) thing is, individuals such as security researchers have been charged under digital trespassing laws for stuff like accessing publicly available ststems and changing a number in the URL in order to get access to data that normally wouldn’t, even after doing responsible disclosure.
Meanwhile, companies completely ignore the standard mentions to say “you are not allowed to scape this data” and then use OUR content/data to build up THEIR datasets, including AI etc.
Shit is fucked
FartsWithAnAccent@lemmy.world 9 months ago
Remember Aaron Swartz
jkrtn@lemmy.ml 9 months ago
Corporations are people except when it comes to liability. Compare the consequences of stealing several thousand dollars from someone by fraud vs. stealing several thousand dollars from someone by fraud as an LLC.
Pretzilla@lemmy.world 9 months ago
Just thought of a nasty hack the browser makers (or hackers) could use to scrape unlisted sites - by surreptitiously logging user browser history for a crawl list
Spotlight7573@lemmy.world 9 months ago
While there are some extensions that do this, last I saw Google didn’t use Chrome for populating Search:
…perficient.com/…/does-google-use-chrome-to-disco…
brbposting@sh.itjust.works 9 months ago
Perhaps some web extensions already do this and phone home about it.