pasjrwoctx
@pasjrwoctx@social.2ndshot.photos
- Comment on Another massive distributed HTTP flood is currently hitting git.friendi.ca a 4 weeks ago: @utzer @utzer ok so you could block those user agents I have something like
# Fake browser detection RewriteCond %{HTTP_USER_AGENT} (Chrome/[0-9]{3}|Chrome/1[3-9][0-9]|Chrome/150|Firefox/1[3-9][0-9]|Safari/60[0-9]|Version/17) [NC] RewriteCond %{HTTP_ACCEPT_LANGUAGE} ^$ RewriteRule ^ - [G,L] RewriteCond %{THE_REQUEST} "GET\shttp" [NC] RewriteRule ^ - [G,L] and then # 1. BLOCK BAD BROWSER NAMES / BOT FRAMEWORKS RewriteCond %{HTTP_USER_AGENT} (CCBot|SearchEngineBot|Pandalytics|UCBrowser|ZoneProjectBot|Embarcadero\sURI\sClient|Xenu\sLink\sSleuth|siteradar|SignalsBot|fun-cert-watch|SERankingBacklinksBot|Pinterestbot|CMS-Checker|HeadlessChrome|Puppeteer|SeznamBot|Sogou|8LEGS|HTTrack|cherrypicker|AhrefsBot|BLEXBot|DotBot|MJ12bot|PetalBot|SemrushBot|BuiltWith|Viewer/99|Python|aiohttp|curl|Wget|libwww|Go-http-client|GeedoShopProductFinder|DuckDuckBot|node|IMJ-CompanyPage-Scraper|baidu|RootEvidence|NetAPI\sv1|Scrapy|Bingbot|SummalyBot|got|HUNT-Bot|CibraxScanner|SalesOS-CompanyVerifier|RecordedFuture|SurdotlyBot|panscient\.com|Xiaomi|Android.*Firefox|GPTBot|ClaudeBot|BardBot|LLMScraper|Firecrawl|Crawl4AI|ia_archiver|archive\.org_bot|Google|wp2shell|okhttp|Cortex-Xpanse|Mozilla\.5\.0\.compatible;\.MSIE\.10\.0;\.Windows\.NT\.6\.1;\.Trident/6\.0|facebookexternalhit|facebookexternalua|Version\.13\.0\.3\.Mobile\.15E148\.Safari\.604\.1|iPhone.*Version/13\.0\.3|Safari/604\.1|cms-scanner|OAI-SearchBot|ChatGPT-User|PerplexityBot|Perplexity-User|Amazonbot|Applebot-Extended|Meta-ExternalAgent|Meta-ExternalFetcher|cohere-ai|DeepSeek|Bytespider|Diffbot|Omgilibot|Omgili|Google-Extended|Google-CloudVertex|MistralAI-User|OAI-AdsBot|YouBot|anthropic-ai|NosibleBot) [NC,OR] RewriteCond %{THE_REQUEST} "^[A-Z]{3,9}\s+https?://" [NC] RewriteRule ^.*$ - [G,L] # 2. BLOCK SPAM WEBSITES (REFERRERS) RewriteCond %{HTTP_REFERER} (baidu\.com|bsky\.(net|com|app)|facebook\.com|meta\.com|threads\.(com|net)|instagram\.com|google\.com|googleusercontent\.com|youtube\.com|x\.com|t\.co|twitter\.com|x\.ai|bing\.com|yahoo\.com|yandex\.com|duckduckgo\.com|microsoft\.com|amazon\.com|brave\.com|semalt\.com|buttons-for-website\.com|darodar\.com|blackhatworth\.com|ilovevitaly\.com|priceg\.com|ranksonic\.com) [NC] RewriteRule ^.*$ - [G,L] ############################################### # SAFE BOT & SCRAPER KILLER (FRIENDICA-COMPATIBLE) ############################################### # Kill obvious scanners by User-Agent RewriteCond %{HTTP_USER_AGENT} (nmap|nikto|acunetix|sqlmap|fimap|nessus|openvas|arachni|wpscan|dirbuster|fuzzer) [NC] RewriteRule ^ - [G,L] # Kill requests with directory traversal attempts RewriteCond %{QUERY_STRING} (\.\./|\.\.\\|%2e%2e|%5c) [NC] RewriteRule ^ - [G,L] # Kill malformed absolute URLs RewriteCond %{THE_REQUEST} "^[A-Z]{3,9}\s+https?://" [NC] RewriteRule ^ - [G,L] # Kill empty User-Agent ONLY if NOT federation RewriteCond %{HTTP_USER_AGENT} ^$ RewriteCond %{REQUEST_URI} !^/\.well-known/ [NC] RewriteRule ^ - [G,L]
which has cut down a lot of bad traffic from getting 200, and getting hit with a 410 when it knocks on my door, I would start there, because blocking ips is fun and easy, but if they are being spoofed, they will just spoof more, and worse when they get recycled to actual valid users then you lose traffic, I have found the agent blocking is more effective, they are not willing to rewrite every script to adjust for that so for now it seems to be the fastest block, and if it is getting that hard, run it through cloudflare on the free side proxied for a while anyways to help slowdown and divert the bad traffic, it will keep your server happier, I know not everyone is a fan of that, but somtimes you have to change the route to stay on the road - Submitted 4 weeks ago to helpers@forum.friendi.ca | 0 comments
- Comment on Another massive distributed HTTP flood is currently hitting git.friendi.ca a 4 weeks ago:
@utzer ok so you could block those user agents I have something like # Fake browser detection
RewriteCond %{HTTP_USER_AGENT} (Chrome/[0-9]{3}|Chrome/1[3-9][0-9]|Chrome/150|Firefox/1[3-9][0-9]|Safari/60[0-9]|Version/17)
[NC]RewriteCond %{HTTP_ACCEPT_LANGUAGE} ^$
RewriteRule ^ - [G,L]RewriteCond %{THE_REQUEST} "GET\shttp"
[NC]RewriteRule ^ - [G,L] and then # 1. BLOCK BAD BROWSER NAMES / BOT FRAMEWORKSRewriteCond %{HTTP_USER_AGENT} (CCBot|SearchEngineBot|Pandalytics|UCBrowser|ZoneProjectBot|Embarcadero\sURI\sClient|Xenu\sLink\sSleuth|siteradar|SignalsBot|fun-cert-watch|SERankingBacklinksBot|Pinterestbot|CMS-Checker|HeadlessChrome|Puppeteer|SeznamBot|Sogou|8LEGS|HTTrack|cherrypicker|AhrefsBot|BLEXBot|DotBot|MJ12bot|PetalBot|SemrushBot|BuiltWith|Viewer/99|Python|aiohttp|curl|Wget|libwww|Go-http-client|GeedoShopProductFinder|DuckDuckBot|node|IMJ-CompanyPage-Scraper|baidu|RootEvidence|NetAPI\sv1|Scrapy|Bingbot|SummalyBot|got|HUNT-Bot|CibraxScanner|SalesOS-CompanyVerifier|RecordedFuture|SurdotlyBot|panscient\.com|Xiaomi|Android.*Firefox|GPTBot|ClaudeBot|BardBot|LLMScraper|Firecrawl|Crawl4AI|ia_archiver|archive\.org_bot|Google|wp2shell|okhttp|Cortex-Xpanse|Mozilla\.5\.0\.compatible;\.MSIE\.10\.0;\.Windows\.NT\.6\.1;\.Trident/6\.0|facebookexternalhit|facebookexternalua|Version\.13\.0\.3\.Mobile\.15E148\.Safari\.604\.1|iPhone.*Version/13\.0\.3|Safari/604\.1|cms-scanner|OAI-SearchBot|ChatGPT-User|PerplexityBot|Perplexity-User|Amazonbot|Applebot-Extended|Meta-ExternalAgent|Meta-ExternalFetcher|cohere-ai|DeepSeek|Bytespider|Diffbot|Omgilibot|Omgili|Google-Extended|Google-CloudVertex|MistralAI-User|OAI-AdsBot|YouBot|anthropic-ai|NosibleBot) [NC,OR]
RewriteCond %{THE_REQUEST} "^[A-Z]{3,9}\s+https?://"
[NC]RewriteRule ^.*$ - [G,L]# 2. BLOCK SPAM WEBSITES (REFERRERS)
RewriteCond %{HTTP_REFERER} (baidu\.com|bsky\.(net|com|app)|facebook\.com|meta\.com|threads\.(com|net)|instagram\.com|google\.com|googleusercontent\.com|youtube\.com|x\.com|t\.co|twitter\.com|x\.ai|bing\.com|yahoo\.com|yandex\.com|duckduckgo\.com|microsoft\.com|amazon\.com|brave\.com|semalt\.com|buttons-for-website\.com|darodar\.com|blackhatworth\.com|ilovevitaly\.com|priceg\.com|ranksonic\.com)
[NC]RewriteRule ^.*$ - [G,L]###############################################
# SAFE BOT & SCRAPER KILLER (FRIENDICA-COMPATIBLE)
################################################ Kill obvious scanners by User-Agent
RewriteCond %{HTTP_USER_AGENT} (nmap|nikto|acunetix|sqlmap|fimap|nessus|openvas|arachni|wpscan|dirbuster|fuzzer)
[NC]RewriteRule ^ - [G,L]# Kill requests with directory traversal attempts
RewriteCond %{QUERY_STRING} (\.\./|\.\.\\|%2e%2e|%5c)
[NC]RewriteRule ^ - [G,L]# Kill malformed absolute URLs
RewriteCond %{THE_REQUEST} "^[A-Z]{3,9}\s+https?://"
[NC]RewriteRule ^ - [G,L]# Kill empty User-Agent ONLY if NOT federation
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteCond %{REQUEST_URI} !^/\.well-known/
[NC]RewriteRule ^ - [G,L] which has cut down a lot of bad traffic from getting 200, and getting hit with a 410 when it knocks on my door, I would start there, because blocking ips is fun and easy, but if they are being spoofed, they will just spoof more, and worse when they get recycled to actual valid users then you lose traffic, I have found the agent blocking is more effective, they are not willing to rewrite every script to adjust for that so for now it seems to be the fastest block, and if it is getting that hard, run it through cloudflare on the free side proxied for a while anyways to help slowdown and divert the bad traffic, it will keep your server happier, I know not everyone is a fan of that, but somtimes you have to change the route to stay on the road - Submitted 4 weeks ago to helpers@forum.friendi.ca | 0 comments
- Comment on Another massive distributed HTTP flood is currently hitting git.friendi.ca a 4 weeks ago: @utzer what is the user agents, looking at your access server logs, you should be able to see if even you had 1million different ip hitting at the same time, if all the user agents are the same, then you have trace back all million ip between whois and dig, to figure out if they are tied together, if all same user agent and time each ip hit and pulls same amount of data, that leans more to coordinated DDoS, if differing user agents, differing hit times and data captures, then massive scraping
did I make any sense - Submitted 4 weeks ago to helpers@forum.friendi.ca | 0 comments
- Comment on Does anyone have a dummies guide to setting up # Cloudflare for # frie 4 weeks ago: So I think I got #Cloudflare all setup now, and supposedly this instance is running through it, but I have no idea if anything at all is really happening;
- Submitted 4 weeks ago to helpers@forum.friendi.ca | 1 comment
- Submitted 2 months ago to helpers@forum.friendi.ca | 0 comments
- Submitted 2 months ago to helpers@forum.friendi.ca | 0 comments
- Submitted 2 months ago to helpers@forum.friendi.ca | 0 comments
- Submitted 2 months ago to helpers@forum.friendi.ca | 0 comments
- Submitted 2 months ago to helpers@forum.friendi.ca | 0 comments
- Submitted 2 months ago to helpers@forum.friendi.ca | 0 comments
- Submitted 2 months ago to helpers@forum.friendi.ca | 0 comments