Comment - FBXL Lotide

Comment on Another massive distributed HTTP flood is currently hitting git.friendi.ca a

utzer@f.utzer.de@f.utzer.de ⁨17⁩ ⁨hours⁩ ago

@pasjrwoctx Thanks, I checked the user agents. The pattern looks highly coordinated.

Out of 164,435 Anubis challenge requests, there were only 160 distinct user-agent strings. The top ten account for 96.79% of all requests and were used almost perfectly evenly:

Firefox 120 / Windows — 16,066
Chrome 118 / Windows — 16,052
Firefox 121 / macOS — 15,949
Edge 119 / Windows — 15,917
Chrome 119 / Windows — 15,874
Chrome 119 / macOS — 15,872
Edge 120 / Windows — 15,863
Chrome 120 / macOS — 15,853
Firefox 121 / Windows — 15,812
Chrome 120 / Windows — 15,761

These are old browser versions from around 2023. Combined with almost one IP per request, this looks like a single coordinated tool rotating through a fixed list of spoofed browser identities and a very large residential/proxy network. It does not prove whether the objective is scraping or disruption, but it is clearly not ordinary independent crawling.

original

Sort:hotnew top

pasjrwoctx@social.2ndshot.photos ⁨17⁩ ⁨hours⁩ ago
@utzer ok so you could block those user agents I have something like # Fake browser detection
RewriteCond %{HTTP_USER_AGENT} (Chrome/[0-9]{3}|Chrome/1[3-9][0-9]|Chrome/150|Firefox/1[3-9][0-9]|Safari/60[0-9]|Version/17)
[NC]RewriteCond %{HTTP_ACCEPT_LANGUAGE} ^$
RewriteRule ^ - [G,L]
RewriteCond %{THE_REQUEST} "GET\shttp"
[NC]RewriteRule ^ - [G,L] and then # 1. BLOCK BAD BROWSER NAMES / BOT FRAMEWORKS
RewriteCond %{HTTP_USER_AGENT} (CCBot|SearchEngineBot|Pandalytics|UCBrowser|ZoneProjectBot|Embarcadero\sURI\sClient|Xenu\sLink\sSleuth|siteradar|SignalsBot|fun-cert-watch|SERankingBacklinksBot|Pinterestbot|CMS-Checker|HeadlessChrome|Puppeteer|SeznamBot|Sogou|8LEGS|HTTrack|cherrypicker|AhrefsBot|BLEXBot|DotBot|MJ12bot|PetalBot|SemrushBot|BuiltWith|Viewer/99|Python|aiohttp|curl|Wget|libwww|Go-http-client|GeedoShopProductFinder|DuckDuckBot|node|IMJ-CompanyPage-Scraper|baidu|RootEvidence|NetAPI\sv1|Scrapy|Bingbot|SummalyBot|got|HUNT-Bot|CibraxScanner|SalesOS-CompanyVerifier|RecordedFuture|SurdotlyBot|panscient\.com|Xiaomi|Android.*Firefox|GPTBot|ClaudeBot|BardBot|LLMScraper|Firecrawl|Crawl4AI|ia_archiver|archive\.org_bot|Google|wp2shell|okhttp|Cortex-Xpanse|Mozilla\.5\.0\.compatible;\.MSIE\.10\.0;\.Windows\.NT\.6\.1;\.Trident/6\.0|facebookexternalhit|facebookexternalua|Version\.13\.0\.3\.Mobile\.15E148\.Safari\.604\.1|iPhone.*Version/13\.0\.3|Safari/604\.1|cms-scanner|OAI-SearchBot|ChatGPT-User|PerplexityBot|Perplexity-User|Amazonbot|Applebot-Extended|Meta-ExternalAgent|Meta-ExternalFetcher|cohere-ai|DeepSeek|Bytespider|Diffbot|Omgilibot|Omgili|Google-Extended|Google-CloudVertex|MistralAI-User|OAI-AdsBot|YouBot|anthropic-ai|NosibleBot) [NC,OR]
RewriteCond %{THE_REQUEST} "^[A-Z]{3,9}\s+https?://"
[NC]RewriteRule ^.*$ - [G,L]
# 2. BLOCK SPAM WEBSITES (REFERRERS)
RewriteCond %{HTTP_REFERER} (baidu\.com|bsky\.(net|com|app)|facebook\.com|meta\.com|threads\.(com|net)|instagram\.com|google\.com|googleusercontent\.com|youtube\.com|x\.com|t\.co|twitter\.com|x\.ai|bing\.com|yahoo\.com|yandex\.com|duckduckgo\.com|microsoft\.com|amazon\.com|brave\.com|semalt\.com|buttons-for-website\.com|darodar\.com|blackhatworth\.com|ilovevitaly\.com|priceg\.com|ranksonic\.com)
[NC]RewriteRule ^.*$ - [G,L]
###############################################
# SAFE BOT & SCRAPER KILLER (FRIENDICA-COMPATIBLE)
###############################################
# Kill obvious scanners by User-Agent
RewriteCond %{HTTP_USER_AGENT} (nmap|nikto|acunetix|sqlmap|fimap|nessus|openvas|arachni|wpscan|dirbuster|fuzzer)
[NC]RewriteRule ^ - [G,L]
# Kill requests with directory traversal attempts
RewriteCond %{QUERY_STRING} (\.\./|\.\.\\|%2e%2e|%5c)
[NC]RewriteRule ^ - [G,L]
# Kill malformed absolute URLs
RewriteCond %{THE_REQUEST} "^[A-Z]{3,9}\s+https?://"
[NC]RewriteRule ^ - [G,L]
# Kill empty User-Agent ONLY if NOT federation
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteCond %{REQUEST_URI} !^/\.well-known/
[NC]RewriteRule ^ - [G,L] which has cut down a lot of bad traffic from getting 200, and getting hit with a 410 when it knocks on my door, I would start there, because blocking ips is fun and easy, but if they are being spoofed, they will just spoof more, and worse when they get recycled to actual valid users then you lose traffic, I have found the agent blocking is more effective, they are not willing to rewrite every script to adjust for that so for now it seems to be the fastest block, and if it is getting that hard, run it through cloudflare on the free side proxied for a while anyways to help slowdown and divert the bad traffic, it will keep your server happier, I know not everyone is a fan of that, but somtimes you have to change the route to stay on the road
original
- utzer@f.utzer.de@f.utzer.de ⁨17⁩ ⁨hours⁩ ago
  @pasjrwoctx thanks, that is not what I will try, but just realized I can block the top10 or top11 user-agents and this will block 95% of this bullshit access.
  original
  - utzer@f.utzer.de@f.utzer.de ⁨17⁩ ⁨hours⁩ ago
    @pasjrwoctx and it works!
    original
    tom_s@friendica.ambag.es@friendica.ambag.es ⁨16⁩ ⁨hours⁩ ago
    @utzer @pasjrwoctx TLS-/JA3-Fingerprint: Taucht aus allen 50.000 IPs nur ein oder zwei Fingerprints auf?
    tcpdump oder tshark, Anubis kann JA3 selbst berechnen, wenn man es dort als Log-Feld ergänzt
    original
    -> View More Comments
    tom_s@friendica.ambag.es@friendica.ambag.es ⁨17⁩ ⁨hours⁩ ago
    @utzer @pasjrwoctx
    Das hat einige Begleiterscheinungen, die man nicht möchte.
    Prüf aus dem Log das tatsächliche Header-Profil des Floods (Accept-Language leer/nicht leer, Anzahl eindeutiger Sec-CH-UA/Accept-Kombinationen). Wenn da ein konstanter Unterschied zu echten Browsern ist, kann ich dir eine gezielte Regel bauen, die genau diesen Header-Set blockt statt der UA-Version. Das ist der einzige UA-nahe Filter, der hier sauber zwischen Flood und echten Nutzern trennt.
    original
    -> View More Comments

utzer@f.utzer.de@f.utzer.de ⁨17⁩ ⁨hours⁩ ago

@pasjrwoctx

More details regarding user-agent distribution

Retained log window: 2026-08-14 17:50–22:42 CEST
Challenge requests: 164,435
Distinct user-agent strings: 160
Share represented by the top ten: 96.79%

The almost perfectly even distribution among the first ten identities is particularly striking.

 COUNT  USER AGENT
------  ------------------------------------------------------------

 16106  Mozilla/5.0 (Windows NT 10.0; Win64; x64)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/118.0.0.0 Safari/537.36

 16097  Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0)
        Gecko/20100101 Firefox/120.0

 15993  Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0)
        Gecko/20100101 Firefox/121.0

 15959  Mozilla/5.0 (Windows NT 10.0; Win64; x64)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0

 15924  Mozilla/5.0 (Windows NT 10.0; Win64; x64)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/119.0.0.0 Safari/537.36

 15914  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/119.0.0.0 Safari/537.36

 15899  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/120.0.0.0 Safari/537.36

 15887  Mozilla/5.0 (Windows NT 10.0; Win64; x64)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0

 15855  Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0)
        Gecko/20100101 Firefox/121.0

 15800  Mozilla/5.0 (Windows NT 10.0; Win64; x64)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/120.0.0.0 Safari/537.36

  2871  Mozilla/5.0 (compatible; crawler)

   709  Mozilla/5.0 (X11; Linux x86_64)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/148.0.7778.0 Safari/537.36

   413  Mozilla/5.0 (Windows NT 10.0; Win64; x64)
        AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/144.0.0.0 Safari/537.36

    86  Chrome/135.0.0.0 — Windows
    40  Chrome/116.0.0.0 — Windows
    40  Chrome/107.0.0.0 — Windows
    39  Chrome/148.0.0.0 — macOS
    34  Chrome/150.0.0.0 — Windows
    33  Chrome/131.0.0.0 — Windows
    32  Chrome/133.0.0.0 — Windows
    25  Edge/135.0.0.0 — Windows
    24  Chrome/146.0.0.0 — macOS
    23  Chrome/145.0.0.0 — Windows
    23  Chrome/110.0.0.0 — Windows
    22  Chrome/126.0.0.0 — Linux
    22  Chrome/123.0.0.0 — macOS
    21  Chrome/146.0.0.0 — Windows
    21  Chrome/109.0.0.0 — Windows
    21  Chrome/136.0.0.0 — macOS
    20  Chrome/99.0.4844.51 — Windows
    20  Chrome/149.0.0.0 — Windows
    20  Chrome/145.0.0.0 — macOS
    19  Chrome/133.0.0.0 — macOS
    18  Chrome/105.0.0.0 — Windows
    17  Chrome/134.0.0.0 — Windows
    17  Chrome/111.0.0.0 — Windows
    17  Safari/15.3 — macOS
    17  Chrome/131.0.0.0 — macOS
    16  Edge/99.0.1150.30 — Windows
    16  Chrome/104.0.5112.81 — Windows
    16  Firefox/135.0 — macOS
    16  Safari/18.0 — macOS
    16  Safari/15.5 — macOS
    15  Chrome/100.0.4896.75 — Windows
    15  Chrome/149.0.0.0 — macOS
    14  Firefox/137.0 — Windows
    14  Chrome/147.0.0.0 — Windows
    14  Chrome/124.0.0.0 — Windows
    14  Chrome/117.0.0.0 — Windows
    14  Chrome/112.0.0.0 — Windows
    14  Chrome/108.0.0.0 — Windows
    14  Chrome/101.0.4951.67 — Windows
    14  Safari/26.0 — macOS
    14  Safari/17.0 — macOS
    14  Chrome/124.0.0.0 — macOS
    13  Chrome/104.0.0.0 — Windows
    13  Safari/18.4 — macOS
    13  Chrome/150.0.0.0 — macOS
    12  Firefox/133.0 — macOS
    12  Chrome/135.0.0.0 — macOS
    11  Chrome/148.0.0.0 — Windows
    11  Chrome/103.0.0.0 — Windows
    10  Edge/101.0.1210.47 — Windows
    10  Safari/18.3.1 — macOS
    10  Chrome/134.0.0.0 — macOS
     9  Chrome/106.0.5249.119 — Windows
     9  Chrome/147.0.0.0 — macOS
     8  Chrome/142.0.0.0 — Windows
     7  Firefox/137.0 — Ubuntu Linux
     7  Firefox/125.0 — Linux
     7  Chrome/132.0.0.0 — Linux
     7  Chrome/151.0.0.0 — Windows
     7  Chrome/130.0.0.0 with CCleaner — Windows
     7  Chrome/139.0.0.0 with Safari WebKit string — macOS
     6  Chrome/137.0.0.0 — Linux
     6  Chrome/114.0.0.0 — Linux
     6  Firefox/153.0 — Windows
     6  Edge/151.0.0.0 — Windows
     5  Chrome/131.0.0.0 — Linux
     5  Chrome/134.0.0.0 — ChromeOS
     5  Chrome/106.0.0.0 — Windows
     5  Chrome/128.0.0.0 — macOS
     5  Chrome/151.0.0.0 — Android
     4  Firefox/136.0 — Ubuntu Linux
     4  Chrome/129.0.0.0 — Linux
     4  Firefox/140.0 — Windows
     4  Firefox/135.0 — Windows
     4  Chrome/138.0.0.0 — Windows
     4  Edge/136.0.0.0 — Windows
     4  Chrome/129.0.0.0 — Windows
     4  Safari/17.6 — macOS
     4  Safari/17.5 — macOS
     4  Chrome/147.0.0.0 — Android
     3  Chrome/136.0.0.0 — Linux
     3  Chrome/130.0.0.0 — Linux
     3  Chrome/124.0.0.0 — Linux
     3  Chrome/133.0.0.0 — ChromeOS
     3  Obsidian/1.8.10, Electron/34.2.0 — Windows
     3  Chrome/58.0.3029.110 — Windows
     3  Edge/138.0.0.0 — Windows

original

tom_s@friendica.ambag.es@friendica.ambag.es ⁨17⁩ ⁨hours⁩ ago
@utzer @pasjrwoctx Perfekte UA-Rotation, ein IP pro Request, ist ein professionelles, dauerhaft laufendes Scraping-Setup, das die PR-/Issue-Daten von friendica-addons harvestet. DDoS-Angreifer geben sich keine Mühe, UAs realistisch und gleichverteilt zu rotieren.
original