Submitted 14 hours ago by paequ2@lemmy.today to selfhosted@lemmy.world
https://arvind.io/posts/using-fail2ban-to-protect-exposed-services/
I’m wondering if I’m starting to outgrow Tailscale… my wife keeps having networking issues on Android due to Tailscale, the Nvidia Shield kills the Tailscale app randomly, and my parents’ TV doesn’t have a Tailscale app…
I feel like the time is approaching to publicly expose some of my services to the internet…
Any other tips?
Exposing services to the internet is a whole other game. Try wireguard first, i never had issues and use it mostly from my tablet.
I am kinda interested in WireGuard, but how does it work with multiple non-PC devices on different home networks? Tailscale runs seamlessly on my Arch laptop, but Android, TVs, and streaming sticks have hiccups from time to time.
I have services that I want to share with my non-techie family. If a service stops working, they suck at debugging and fixing the issue on their own.
Tailscale is WireGuard under the hood, if you didn’t know. It’s an overlay network that uses WireGuard to make the actual connections, and has some very clever “stuff” to get the clients actually to connect, even if behind firewalls without needing port forwarding.
Using WireGuard directly basically just changes the app you use, which may or may not help with your issues. But the connecting technology is the exact same.
I’ve never used tailscale but use wireguard extensively.
There’s not much of a learning curve for you as the administrator. You have to discard some misconceptions you might bring from other VPNs but really after 30 minutes of looking at configs you’ll get it.
I use wireguard for my small team of 5 people to access self hosted services. You install wireguard, load the config, and then it just works.
The trick, if it can be called that, is using public dns for private services.
On your server, suppose you have service-a service-b and service-c in containers with ip addresses in the 10.0.2.0/24 range. Then you’d have a reverse proxy like traefik at 10.0.2.1. You’d also create a wireguard container with an IP in that same 10.0.2.0/24 range, and configure it’s wireguard adapter to be 10.0.12.1 or soomething so you have “2” for the containers and “12” for the wireguard clients.
Then in wireguard configurations you direct all traffic for 10.0.2.0/24 through the tunnel but everything else just uses their devices normal internet connection.
Finally create a public dns record pointing to the reverse proxy like *.mydomain.com > 10.0.12.1
now whatever.mydomain.com will resolve to your reverse proxy but is still only available to devices connected to the wireguard container on your server.
I use tailscale for my non-tech family.
I run a rPi with tailscale, pihole and nginx on it in their house. They connect to the their WiFi, get adblocking for free. They go to “homarr.sever” pihole captures this sends it to nginx and gets reverse proxies to a homarr LXC on my server. From their they can click links to the services which are at on “service/.######.xyz”. Again pihole captures the request, sends it to nginx which reverse proxies it over tailscale to the appropriate LXC.
My primary scare is wg being another potential single point of software failure, preventing remote access
You can run it on two servers at once. I have WireGuard running on my router (which can’t give full speed) and my server (which can run at full speed). Also running on another machine for fun.
I know a lot of people shit on it from a decentralization perspective, but I use Cloudflare to expose all my services. Then anyone who hits my sites has to go through Cloudflare’s detections first. I have all my services behind a reverse proxy (nginx proxy manager) running locally, and that’s the only though exposed to the Internet through my router, also that ONLY allows connections at all from Cloudflare IPs or my local network. My home IP is obfuscated, my services can only be accessed using the ports I define, and things are happy. I also block as much as possible on my router, and have automatic updates on all my server VMs/LXCs.
You could also set up a Cloudflare tunnel to go to the reverse proxy and avoid needing to expose anything to the direct Internet.
Just turn off caching for any media servers domains/subdomains if you go with Cloudflare, or else it will try to cache any media on their servers and it’s technically a ToS violation so people get their accounts banned. It’s a simple setup to disable cache though.
Crowdsec is absolutetly the way to go. Jhst be aware you need the engine and what the call a “bouncer” both running, but they have easy instructions about how to install both.
Yes, I run many services and website on the public web at home. Harden your server first. Like disabling root ssh login.
Also enable auto updates on your server.
Use your router/server to block many some counties. You could also use block lists, there any many bad ip lists out there.
Configure rate limits in Nginx.
You also mentioned fail2ban. You can define many rules and actions. Like blocking ips that might go over your previously defined rate limits. Or 4xx action for ips that request a lot of non existing pages (404 errors) .
Also captcha won't cut it anymore today. Try https://github.com/TecharoHQ/anubis
Of course expose only what you want to expose, so only open ports in your firewall you really want to open. Ideally put everything behind a reverse proxy like Nginx.
Let's start with all of the things mentioned above. Ping me later if want to know more or have questions.
Harden your server first
Do you have any tutorials or guides on this handy?
Use your router/server to block some counties using geoip
Yeah, definitely all my users are in the same town/region/country as me. So this could be doable.
Configure rate limits in Nginx
Hm, currently using Caddy as my reverse proxy. I guess there’s some module for this.
only open ports in your firewall you really want to open
The only port I need open is 443 for accessing Jellyfin and Immich. I can definitely block 22 from the public internet.
Caddy is also fine.
I wrote a blog about server hardining and you might find it useful: https://blog.melroy.org/2023/server-hardening/
Do you have any tutorials or guides on this handy?
Now that’s a deeeeep rabbit hole. I tend to go overboard on hardening and security, however, one good place to start is installing Lynis and run a scan. Lynis will spit out a rather extensive list of areas you need to harden or adjust and a score for your server. It will also give links where you can go and read up on the specific item in question. Now, not every one of the bullets in the list will apply, but you should give each careful consideration. Lynis is Free and Open Source Software (FOSS).
lynis audit systemI ran a scan just for demonstration purposes so you can see what the end results are. This is just a snippet:
* Configure minimum password age in /etc/login.defs [AUTH-9286] https://cisofy.com/lynis/controls/AUTH-9286/ * Configure maximum password age in /etc/login.defs [AUTH-9286] https://cisofy.com/lynis/controls/AUTH-9286/ * Default umask in /etc/login.defs could be more strict like 027 [AUTH-9328] https://cisofy.com/lynis/controls/AUTH-9328/ * To decrease the impact of a full /home file system, place /home on a separate partition [FILE-6310] https://cisofy.com/lynis/controls/FILE-6310/ * To decrease the impact of a full /tmp file system, place /tmp on a separate partition [FILE-6310] https://cisofy.com/lynis/controls/FILE-6310/ * To decrease the impact of a full /var file system, place /var on a separate partition [FILE-6310] https://cisofy.com/lynis/controls/FILE-6310/
Be mindful of where you get your hardening tutorials. There are hundreds of thousands out there. I would stick with authoritative sources.
What kinds of things are you planning to expose? What I expose I hide behind a reverse proxy with IP whitelists. Whatever I don’t need access to on the go I don’t expose.
What kinds of things are you planning to expose?
Primarily Jellyfin and Immich.
What I expose I hide behind a reverse proxy with IP whitelists.
Do all your clients have fixed IPs? I have some clients that are phones or laptops, but I would imagine those change as people drive around to different cities or connect to different coffee shop WiFi.
It depends on what service - some, like Jellyfin, are accessed only from home IPs which are static (for music through Jellyfin I use offline mode to prevent too much mobile traffic), so I can add those specific IPs in the whitelist. Otger services I need to access from elsewhere, and I can add entire subnets (i.e. for my phone carrier network or VPN servers). Those change once in a while and that is annoying. Other services I want publically available.
Jellyfin especially still has some unsecured endpoints where it would be wise to take some.extra precautions. I think the risk some people seem to think this poses is a little overblown (i.e. rights holders finding your instance and reverse mapping your entire library and suing you to oblivion), but better not risk it.
I wouldn’t
Is mTLS an option?
You also could setup a ingress node on the network to route certain IP addresses over Tailscale.
I personally prefer Netbird
I’ve had to stop using it. In the last few months I have more and more suddenly lost all connectivity outside of my tailscale network. I tried excluding apps but I still will randomly fail to receive SMS or calls, suddenly getting them delivered in a rush when a disconnect from tailscale.
GAAH! OK! I’M NOT CRAZY!
The exact same thing is happening to my wife’s phone. We’re both on Pixel 8s, have the same VPN settings, but for some magic reason Tailscale breaks only her phone. She has to turn off Tailscale and reboot her phone to regain connectivity.
These shenanigans is why I’m considering just exposing things to the public internet. I’m using Tailscale on several device types and Tailscale adds friction to all of my devices (except Arch where everything always works).
I understand the friction is there for a good reason, but my family doesn’t. They just see that Jellyfin doesn’t work and that all of this is buggy and maybe they just should sign up for Netflix instead of dealing with all of these bugs.
I’ve had a pretty good experience with it aside from this recent problem with my phone. It’s a big deal right now - I have a number of self hosted services I use on my phone. When I left it enabled, it would happen multiple times a day, though it would get fixed if I just quickly disable-enable it… at least until it randomly happens again in an hour or two.
Unfortunately, my mother has been having a number of health issues so there is no fucking way I’m going to risk missing calls and texts…so I just deal with being disconnected for now. I really wish there was a solution or something I could do to figure out what’s going wrong.
fail to receive SMS or calls, suddenly getting them delivered in a rush when I disconnect from tailscale.
I think that’s a known issue, but the last time I looked there didn’t seem to be a known cause or a fix.
I have all my services behind a reverse proxy and use Crowdsec to monitor and block automated attacks. I also have pocket-id for auth, I use SSO for apps that support it and others just require authentication to access them at all. The docs are pretty solid, it was easy to set up.
Make sure you know the services running on your server, the most likely way you could get attacked is by just leaving some vulnerable or misconfigured software running and accessible.
I recommend looking into setting up site-to-site VPN configurations between routers at each location. I have this set up between my home network and my parents’ network.
Once you have it properly configured you can simply have the router itself handle routing of specific traffic over the VPN connection, instead of needing each device to connect to the VPN individually.
it’s a bit more complicated to set up and maintain but not anything previously complex, and absolutely worth it for your use case IMO
site-to-site VPN configurations between routers at each location … the router itself handle routing of specific traffic over the VPN connection, instead of needing each device to connect to the VPN individually.
Interesting. This could help with my home network and my parent’s, but I also need to handle cases where I don’t control the network. I have clients that are phones. My family would expect that the services keep working even if they’re in a different city on cellular or at a friend’s house.
Site-to-site and individual client setups are not mutually exclusive. They can co-exist.
I have Wireguard on both my phone and laptop - and tailscale should work the same way - which I only activate when I’m away from my home network.
You could even set it up where each “roaming” device is always connected to their “home” network by VPN, which uses site-to-site to further route the traffic where it needs to go.
observantTrapezium@lemmy.ca 10 hours ago
Exposing stuff to the internet shouldn’t be that scary… I haven’t had any incident so far in 8 years. Yes, you see plenty of illegitimate access attempts in the logs, but if everything is properly patched, it should be OK.