admiralpatrick

@admiralpatrick@lemmy.world

This is a remote user, information on this page may be incomplete. View at Source ↗

Ask me anything!

This is my mod account on LemmyWorld. My main user is @ptz@dubvee.org

⁨Comment⁩ on ⁨Admins: Instnace randomly running extremely slowly? Check for this⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
Very nice!
⁨Comment⁩ on ⁨Admins: Instnace randomly running extremely slowly? Check for this⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
Unfortunately, there’s many many reasons that could be the case. I’m just putting this out there since it’s easy to check for and mitigate against.
⁨Comment⁩ on ⁨Admins: Instnace randomly running extremely slowly? Check for this⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
No, that’s just /api/v3/user which returns both posts and comments.
⁨Comment⁩ on ⁨Admins: Instnace randomly running extremely slowly? Check for this⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
Good idea with the f2b integration.

I thought about that before just blocking unscoped requests to that endpoint in Nginx.
⁨Comment⁩ on ⁨Admins: Instnace randomly running extremely slowly? Check for this⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
That was my thought, but also wasn’t sure since there might be a use-case I’m unfamiliar with. I vaguely recall seeing a feature request for Photon a while back to be able to just browse comments, so I assume that would be how it worked.

But yeah as it is now, it can be abused.
⁨Comment⁩ on ⁨Admins: Instnace randomly running extremely slowly? Check for this⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
That’s my normal go-to, but more than once I’ve accidentally blocked locations that Let’s Encrypt uses for secondary validation, so I’ve had to be more precise with my firewall blocks
⁨Comment⁩ on ⁨Admins: Instnace randomly running extremely slowly? Check for this⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
Lemmy. I added a comment above LW wouldn’t let me edit the post.

Mine’s only extended with some WAF rules and I’ve got a massive laundry list of bot user agents that it blocks, but otherwise it’s pretty bog standard.

If instances have Anubis setup correctly (i.e. not in front of /api/…) then that might not help them since this is calling the API endpoint.
⁨Comment⁩ on ⁨Admins: Instnace randomly running extremely slowly? Check for this⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
Can’t edit the post (Thanks Cloudflare! /s) but additional info:
- I truncated the log excerpts. The user agent string in the requests is blank
- This is for Lemmy admins only. It might apply to others, but I can’t say since I only have this test Lemmy instance to reference
- My Nginx solution may have room for improvement; I was just trying to block that behavior without breaking comments in posts and move on with my day. Suggestions for improvement are welcome.
Admins: Instnace randomly running extremely slowly? Check for this
Submitted ⁨⁨1⁩ ⁨week⁩ ago⁩ to ⁨fediverse@lemmy.world⁩ | ⁨34⁩ ⁨comments⁩
⁨Comment⁩ on ⁨What are the activity_id formats for various platforms?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:

but I send you a PM

Oh, sorry. One of the new features in this dev branch is the ability to disable PMs and mentions. I’ve been running with those turned off. Seems like that feature is working lol.

I turned DMs back on and found the message - will try to join here when I’m back on desktop. Dunno how active I can be right now, but I am eventually going to start on Piefed so would be nice to have a sounding board.

Some of the devs are already working on shared logic/libraries between apps.

Nice!
⁨Comment⁩ on ⁨What are the activity_id formats for various platforms?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:
Oh, I meant just if the instance isn’t know, I thought resolving would make it “aware” of that instance. I could be wrong. But yeah, the instance would have to federate with the other one for it to be able to resolve, though. e.g. it won’t resolve an object from an instance that is on the current instance’s “block” list.
⁨Comment⁩ on ⁨What are the activity_id formats for various platforms?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:
I believe you can, yeah, and I also think that “bootstraps” that instance to yours if it doesn’t already know about it. But in that case, the way I have the search written, it’ll “fall back” to regular search which also does resolveObject. That just takes longer.

The ap_id check is just to short-circuit that behavior to avoid the lengthy, often unnecessary, search and quickly redirect you to your instance’s local copy.

Have had that working for about a week now, and it’s pretty nice. Please do steal this feature lol.
⁨Comment⁩ on ⁨What are the activity_id formats for various platforms?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:
At startup, it calls /api/v3/federated_instances and stores the result to a lookup variable. Then I’ve got a couple of helper functions that accept either an instance ID or a domain name which looks them up from the lookup variable.
⁨Comment⁩ on ⁨Email on your own domain is easy⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:
Email on your own domain: Yep, super easy.

Email from home IP or from the IPv4 you get assigned with a VPS: Super difficult
⁨Comment⁩ on ⁨What are the activity_id formats for various platforms?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:

I think you would be better served by checking for the Link header

Can’t really do that, client-side. CORS is a perpetual cockblock (though I understand why it is), and I’d rather not make an internal API endpoint to do the lookup.

The application polls Lemmy’s getFederatedInstances API endpoint at startup, so it has a list of every activity pub server your instance knows about. That’s the first and primary check for the URL that’s being searched.

The second check is just to rule out non activity pub URLs that point to a federated instance (e…g. lemmy.world/modlog, lemm.world/pictrs/image/blah.webp, etc).

Goal isn’t to “catch 'em all” but to catch the most used ones. If there’s one I don’t account for, either by omission or because the federated platform didn’t exist when I made the patterns, then it will just fall back to a regular search which also includes trying to resolve it as a federated URL (which is the current behavior in all prior versions).

The goal is just to simply short-circuit the search behavior if the query is a known ap_id URL in order to avoid a lengthy search process and quickly redirect you to your instance’s local copy.
⁨Comment⁩ on ⁨What are the activity_id formats for various platforms?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:
I’m making an “omnisearch” box.

Paste in an AP_ID into the search field, and it auto-resolves it and redirects you to your instance’s local copy (which is very fast) instead of going through the whole search process (which is slow). To prevent false positives, I’m matching the various ap_id formats and only doing the resolution on those; anything else gets passed to search.

Anything else that falls through the cracks just gets passed to search as usual (which also does a resolveObject lookup).

It’s to make life easier.
⁨Comment⁩ on ⁨What are the activity_id formats for various platforms?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:
We’ve had this discussion :)

This application is written against the Lemmy API. It only speaks API. Eventually it’ll speak Piefed API as well, but right now, only Lemmy API.

Lemmy and Piefed only do server-to-server Activity Pub and not client-to-server AP. Clients have to use the API to interact with them. This is a Lemmy (and eventually Piefed) client.
⁨Comment⁩ on ⁨What are the activity_id formats for various platforms?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:
Cool, thanks. I was close with /user guessing from memory.

I think the /users/…/post_id will be sufficient. It just needs to know that the given URL is an AP_ID before passing it off to the API call to resolveObject. Since it already knows instance.domain.tld is a federated instance, it just needs to see if the path is an AP_ID or the HTML (or something else). Thus, I don’t have to parse the whole thing, just check that enough of it matches.

Thanks!
What are the activity_id formats for various platforms?
Submitted ⁨⁨2⁩ ⁨weeks⁩ ago⁩ to ⁨fediverse@lemmy.world⁩ | ⁨19⁩ ⁨comments⁩
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
If you have DB access, the values are in the local_site_rate_limit table. You’ll probably have to restart Lemmy’s API container to pick up any changes if you edit the values in the DB.
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
One of these days your mom’s gonna stop paying for your Mullvad subscription. Whaddya gonna do then?
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
Awesome! Win-win.
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
“Message” bucket is kind of a general purpose bucket that covers a lot of different endpoints. I had to ask the lemmy devs what they were back when I was adding a config section in Tesseract for the rate limits.

These may be a little out of date, but I believe they’re still largely correct:

Image
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
That’s a consideration, yeah, but they’d have to all be hitting lemmy.zip (your instance) and all from the same /32 IPv4 address.

(AFAIK) CG-NAT still uses port address translation so there’s an upper limit to the number of users behind one IP address. They also are distributed geographically. So everyone would need to be in the same area on the same instance to really have that be an issue.

The more likely scenario would be multiple people in the same household using the same instance. But 20 comments per minute, divided by two people in the house would still be 10 comments per minute. That’s still probably more than they could reasonably do.
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
nginx.org/en/docs/…/ngx_http_proxy_module.html

$proxy_add_x_forwarded_for is a built-in variable that either adds to the existing X-Forwarded-For header, if present, or adds the XFF header with the value of the built-in $remote_ip variable.

The former case would be when Nginx is behind another reverse proxy, and the latter case when Nginx is exposed directly to the client.

Assuming this Nginx is exposed directly to the clients, maybe try changing the bottom section like this to use the $remote_addr value for the XFF header. The commented one is just to make rolling back easier. Nginx will need to be reloaded after making the change, naturally.
```
     # Add IP forwarding headers
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header Host $host;
      # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-For $remote_addr;
```
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
Yeah, you are setting it, but that’s assuming the variable $proxy_add_x_forwarded_for has the correct IP. But the config itself is correct.

Do you have a separate location block for /api by chance, and is the proxy_set_header directive set there, too? Unless I’m mistaken, location blocks don’t inherit that from the / location.
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
I replied to your other comment, but most likely cause is the API server not getting the correct client IP. If that’s not setup correctly, then it will think every request is from the reverse proxy’s IP and trigger the limit.

Unless they’re broken again. Rate limiting seems to break every few releases, but my instance was on 0.19.12 before I shut it down, and those values worked.
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
Not sure. I had mine set to 20 per 60 for a long time without issue.

Most likely cause would be the Lemmy API service not getting the correct client IP and seeing all API requests come from the reverse proxy’s IP.

Are you sending the client IP in the X-Forwarded-For header? Depending on how your inbound requests are routed, you may have to do that for every reverse proxy in the path.
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:

So, a ‘Comments’ Rate limit: 10, Per second: 60, means a maximum of 10 comments per minute, correct?

Correct, per client IP.

Maybe the reason you see 99999999 is due to troubleshooting

Could be. I try not to speculate on “why” when I don’t have access to the answer lol.

I don’t recall any of them being from mander (unless they were dealt with before I started testing?), but thanks for taking preventative measures :)
⁨Comment⁩ on ⁨[PSA] Watch for the antiyanks troll and consider adjusting your rate limits⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:
You’ll have to talk to the lemmy devs about that. I’m a retired admin, but last I was aware, they’re based on client IP.