Submitted 9 months ago by MigratingtoLemmy@lemmy.world to selfhosted@lemmy.world
Logcheck. It took ages to make sure innocent logs are ignored, but now I get an email as soon as anything non-routine happens on my servers. I get emails with logs from every update, every time I log in, etc. This has given me the most confidence that nothing unexpected is happening on my servers. Of course, one needs to make sure that the firewall is configured well, and that you use ssh keys etc., but logcheck is how I know I'm doing enough.
Very nice idea, and it’s quite simple too. I’ll consider improving logging on my side too. Thanks
Never used it “in anger” but:
I have my firewall plugged into a metered outlet (plugged into a UPS). I have it set up to send me alerts if power draw increases beyond a certain threshold. I’ve tested it and wireguard is measurable (yay) but so are DDOS attacks. If I get that alert, I can choose to turn off that plug and take my whole network offline until I get home and can sort that out.
Care to share what outlet you had success with? I’m comfortable with Home Assistant and ZigBee/Z-Wave. Something this critical probably shouldn’t be wireless, but I digress. I’m also interested in what software you’re using for monitoring and alerts (if you’re willing to share). Cheers!
I just use a pretty generic z-wave plug and home assistant. In the past I did more complex setups that actually determine what process is spiking and so forth. But eventually realized that “this is doing a lot of compute…” is a catch all for a LOT of potential issues.
And I guess I don’t understand what you mean by “shouldn’t be wireless”. It is inherently going to be wireless because you will be on your phone on the other side of the planet. If you genuinely suspect you will be vulnerable to attacks of this scale then you… probably have other things to worry about.
But as a safety blanket?
Aren’t you just DoSing yourself at that point?
Well damn that’s clever. I will have to try that.
Or you could trigger automation that turns it off for hours, then turns it back on. That way you could get around the need to physically turn it on, in case everyone is away.
With my firewall disabled a lot of my internal network (including home assistant) will fall over sooner than later.
But that is also a recipe for mass stress. Because I know “something happened”. And now I know “in six hours, I need to check in and make sure that ‘something’ is still not happening”. Which is extra shitty if I got the notification late evening local time.
I have friends/neighbors that I trust to swing by and push a button in the event I need to bring it back up before I get home. But if I have reached the point of “it is possible my wireguard credentials were compromised?” then I really don’t need to be able to download the next episode of ATLA NOW.
My most paranoid config is disabling Ipv4
That’s it. If someone wants to attack me, they will need to adopt IPv6!
they will need to adopt IPv6!
And find your IP in a /56 or /64 range (depending on what your ISP gives you). Good luck.
I’ve replaced reconnaissance commands (a handful of them found here: www.cybrary.it/…/linux-commands-used-attackers)
With shell scripts which send me a notification via pushover. I’m running several internet-facing services, and the moment those get run because someone is doing some sleuthing inside the machine, I get notified.
I doesn’t stop people getting in, I’ve set up other things for that – but on the off chance that there is some zero-day that I don’t know about yet, or they’ve traversed the network laterally somehow, the moment they run one of those commands, I know to kill-switch the entire thing.
That’s a very good idea. Something to think about, especially if you have open ports and are paranoid enough (aren’t we all? Hehe). Thanks
My security is fairly simplistic but I’m happy with it
Could you please elaborate how you do the honeypotting?
I just used fail2ban on the commonly used default ports such as 22, 21 Etc, any requests on those ports get sent into purgatory, so the ip gets blacklisted any connections from it hangs until it times out. It’s a super basic setup iptables logs whenever a request is not in the current firewall (last rule in the chain) and then fail2ban reads the log and handles the block
What honeypot are you using?
Genuine question. What security concerns does luksCrypt have?
Really all I do is setup fail2ban on my very few external services, and then put all other access behind wireguard.
Logs are clean, I'm happy.
Standard and well-tested setup. Thanks for your reply!
I understand some of these words.
all buzz :P
Only remote access by wireguard and ssh on non standard port with key based access.
Fail2ban bans after 1 attempt for a year Logs are encrypted and mailed off site daily
System updates over tor connecting to onion repos.
Nginx only has one exposed port 443 that is accessible by wireguard or land. Certs are signed by letsencrypt. Paths are ip white listed to various lan or wireguard ips.
I dont allow devices I dont admin on the network so they go on their own subnet. This is guests phones and their windows laptops.
Linux only on the main network.
I also make sure to backup often.
Can you explain why you use onion repos? I’ve never heard of that, and I’ve heard of kind of a lot of things.
Onion repositories are package repositories hosted on tor hidden services. The connection goes through six hops and is end to end encrypted. In addition to further legitimizing the tor network with normal everyday usage it has the benefit of hiding what packages have been installed on a system.
Whonix has some notes about it here if you want to read more.
With Debian it’s just the apt-tor package, and the project maintains an official list at… onion.Debian.org iirc?
I don’t know if serving onion traffic is more expensive for Debian/mirror maintainers so idk if this is something everybody should use
System updates over tor connecting to onion repos.
How does this help, assuming your DNS isn’t being spoofed?
Please see my reply below with links.
Linux only on the main network.
Is that a security benefit?
If big corporations hoovering your data should be on everyone’s threat list, then yea, i’d say its a huge benefit.
Well I dont trust closed source software and so what I can to avoid it when I can. At least foss can be audited. Also all the linux devices on the main network are devices I admin.
I guess it cuts the attack surface profile down a bit?
Fail2ban bans after 1 attempt for a year.
Fail2ban yes; one year, however, is IMO a bit excessive.
Most ISP IP assignments do tend to linger - even with DHCP the same IP will be re-assigned to the same gateway router for quite a number of sequential times - but most IPs do eventually change within a few months. I personally use 3 months as a happy medium for any blacklist I run. Most dynamic IPs don’t last this long, almost all attackers will rotate through IPs pretty quickly anyhow, and if you run a public service (website, etc.), blocking for an entire year may inadvertently catch legitimate visitors.
Plus, you also have to consider the load such a large blocklist will have on your system, if most entries no longer represent legitimate threat actors, you’ll only bog down your system by keeping them in there.
Fail2ban can be configured to allow initial issues to cycle back out quicker, while blocking known repeat offenders for a much longer time period. This is useful in keeping block lists shorter and less resource-intensive to parse.
My block list is very small actually due to the non standard ssh port. Everything else goes through wireguard.
If it was open to the public then yes I’d have to reconsider the ban length.
How do you all that have your services on your LAN accessing it over wireguard when external pass the wife/kids/family test? If I had to have my wife activate a VPN before she could access our nextcloud or bitwarden, she’d just never use it
Is always on not an option?
Always on wireguard kills battery life on mobile for me so I guess that’s a no.
always on they wouldnt know about it and if the connection failed or the wg service crashed on their phone then the services wouldn’t work. It adds a complexity that you don’t want when you’re trying to pass the wife test. Plus yes battery.
Using SPA firewall knocking (fwknop) to open ports to ssh in. I suppose if I was really paranoid, the most secure would be an air gap, but there’s only so much convenience I’ll give up for security.
I’m going to save your comment because it has opened up a new technique for network security that I had never thought of before. Thanks a bunch
I’m an enterprise guy, so that’s the explanation for non home use things.
I bet this can still get exploited, just would take effort hopefully none does for a home network.
I’m still one shitty windows zero day click away from getting my workstation or browser tokens owned though, I can feel it.
Also laughing because that’s how some companies get owned, IP stolen, etc.
There has to be balance, if your life using their system sucks so hard you can’t do your job or meet production marks, you get creative.
My industry has to prioritize security over productivity. It’s almost impossible to get work done.
Ha yeah.
Id say the same for trellix.
You should try doing things with installs or updating apps when the edr product blocks write access to all temp locations. You have to do an exclusion for every installer, signing cert, or turn it off to install programs.
I’m still one shitty windows zero day click away from getting my workstation or browser tokens owned though, I can feel it.
As somebody taking like 0% of all that measures and not having any problem, luck was involved for sure, unless they have a good reason to attack you in particular… I feel like you will be fine…
Ha, probably. It’s fun to learn stuff though.
Working in this field, almost every company has been beached, IP stolen, etc.
Sometimes your home IP gets hit in an automated scan for a vulnerability and then auto exploited by automation. I’m hoping not to get random chance added to a botnet.
You seem to have a great setup. Since this comment touches on slightly advanced topics, I’ll ask this here:
Your measures seem to be focussed more on security than privacy - which is great! It’s my fault for not specifying it in the post, but I’d definitely like to know if you have done anything specific to keep your network private as well as secure.
Thanks for your wonderful comment - saved!
I guess the firmware is as good as possible. All network devices are just computers and can be exploited. I use a Cisco router as my actual gateway. Sophos is inline after that.
Privacy. 🤔
Not much. I have certain traffic go through a VPN to the Internet, but that’s split tunneled.
I use incognito? That doesn’t really do anything, ha.
I’m slowly killing web browser tracking and cookie stuff that group policy allows.
Since you’re running x86 for your router, do you actively prevent ME from trying to connect to the Internet?
I am aware of the ME, but I can’t really do anything about it. Current ARM SBCs are not suitable for a router/firewall (at least in my experience). I’m not that concerned about it though.
Wouldn’t that last bullet mean you’re not updating the windows machines whatsoever? Would this not cause more security issues in the long run, considering “connected tot he internet” isn’t a requirement to spread an infection.
It might sound ridiculous, but I currently also run a WSUS server to get Windows updates. But I will probably replace my entire Windows setup with a better solution. Since I don’t run Windows bare-metal anymore, I’m looking forward to using offline Windows VMs on my Proxmox host and just accessing the internet directly from my Linux machine.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
| Fewer Letters | More Letters |
|---|---|
| DNS | Domain Name Service/System |
| NAT | Network Address Translation |
| VPN | Virtual Private Network |
[Thread #493 for this sub, first seen 6th Feb 2024, 16:55] [FAQ] [Full list] [Contact] [Source code]
After reading this thread I’m apparently not paranoid enough.
Internet facing services are on their own firewalled vlan (dmz), behind a rev proxy, and I have crowdsec running on the proxy and router.
Anything that can get away with putting up on a vps I have (e.g. this Lemmy server). But some things have storage/compute requirements I’m not willing to shell out for.
I’ve got systems that can detect suspicious activities in the net, which result in a shutdown of the router. And not like “could you please shut down” but a hard power off type of shutdown.
Now that’s the kind of paranoid I was hoping to see in here. High five, pal.
Oh, you have a setup that signals to your power source to shut off internet when it detects an anomaly on the internet? That’s quite specific, and I’m having trouble trying to understand the use-case here, but it’s definitely included in the paranoid-list. Thanks!
Notifications on system file access
Notifications on root login/sudo
Declarative OS, tmpfs root, disabled sudo
Bastion server, but right now I don’t have a proper router to do it at home
Yubikey, or a separate phone on Graphene OS for otp, keys, etc
Authelia + fascist fail2ban (or some CSF)
Most of these are pretty normal, but usually you don’t do them all at once 😄 also, I don’t really like hiding my services from the open internet, authelia is fine tuned to let people only access what they are supposed to. And regular users of my server usually don’t notice that I even have it
I’d love to hear more about your Authelia setup. I’m using Authentik, but placing to do the same thing. I haven’t opened my server up to the Internet yet (just built it on Friday), but what I’d like to do is have a webserver that supplies a login page, and you can’t access anything else until you’ve logged in
I’m a Nixos user, I wouldn’t be much help unless you do Nixos. But it’s a whole new rabbit hole which would take you months/years to learn and setup 😅
What I can say, you can do “access from home network”, “access from VPN network”, “1fa/2fa from the internet” OR “access for / and /api, but 1fa/2fa for stuff like /admin, /admin-settings, or just /login or /logged-in”
Fail2ban is fun, also maybe have a look at crowdsec
Declarative OS, tmpfs root, disabled sudo
How do you change anything about the OS/do updates? iirc nixos requires elevated privileges for that?
There are many ways to do many things in nixos
For updates you can do automatic updates
Also, there are many deployment tools, like deploy-rs, morph, colmena, bento. They all have different approaches. Some you use ssh to deploy a remote system. Some just fetch the configuration and autodeploy it.
There are many ways how you can play with this. So you can disable sudo, and deploy with ssh only from some or a specified ip. Or you can keep ssh for root disabled and just deploy home-manager. It’s really a lot, you imagination is the only limit
NAT 🥴
You might be interested in setting up network bound encryption via Clevis and Tang. I use a hidden pi zero in my house acting as a Tang server. It’s great being able to reboot any of my encrypted servers without having to manually unlock disks.
Do you recommend any resources about this? I’d be interested in learning how to implement this.
I know about it, but it kinda defeats the purpose (the purpose being police raid protection)
Neat post and great comments. Saved. Thanks. :)
My personal setup includes:
I‘m aware that stuff might go horribly wrong but so far it hasnt.
I’m somewhat paranoid therefore running several isolated servers. And it’s still not bulletproof and will never be!
Air gapping? I keep a offline backup just in case.
Following for my own edification!
I used to have all VMs in my QEMU/KVM server on their own /30 routed network to prevent spoofing. It essentially guaranteed that a compromised VM couldn’t give itself the IP of say, my web server and start collecting login creds. Managing the IP space got painful quick.
I have Nginx Proxy Manager set up to let me access services running on other ports on the machine with a local network only access list just so my traffic even in my own network will use TLS. The likelihood that anyone is sniffing traffic on my own network is extremely small, but I’m paranoid. (Can’t let anyone see that I’m running Ubuntu Server. How embarrassing.)
For about a year I was running a full out of band IPS on my network. My core switch was set up with port mirroring to spit out a copy of all traffic on one port so that my Suricata server could analyze it. Then, this was fed into ElasticSearch and a bunch of big data crap looked for anomalies.
It was cool. Basically useless because all it did was complain about the same IP crawler bots as my nginx logs. But fun to setup and ultimately good for my career lol.
Mine’s pretty simple, I have a “don’t open ports until ABSOLUTELY NECESSARY”, wireguard works well enough for everything else I need to access remotely. I also keep SSH disabled on any machine that has direct access to the internet.
I am clearly not paranoid enough. For a while I was running an open source router inline between the network AP and the fiber to Ethernet box and running nids but the goddamn thing kept crapping out every few days so i took it back out until I can find a more stable solution.
I have plans if I can ever get around to it. I want the smart TV, printer and other shitty things on a separate network from the more trusted devices. I don’t know how yet but I would like to set up 802.1X for the trusted stuff.
npftables blocks all incoming except a particular set of ips. any connections from those ips hit pubkey authentication.
I’ve never had a problem
One day I will setup my security onion, but I’m procrastinating
betterdeadthanreddit@lemmy.world 9 months ago
Nice try, attacker trying to get me to do their reconnaissance work for them. I’m on to you.
MigratingtoLemmy@lemmy.world 9 months ago
It would be funny if that were the case. I was just hoping to be a little more paranoid from you lot and maybe improve on the things I’ve thought about
betterdeadthanreddit@lemmy.world 9 months ago
Yeah, just having a little fun in the role of a paranoid admin. My setup isn’t worth mentioning since it fits my threat model (i.e. nobody gives a shit about my network, just don’t be the low hanging fruit) but I’m interested in other replies. Hope you get some useful responses here.
hakunawazo@lemmy.world 9 months ago
No, honestly I’m not an attacker, but your local bank. We just need your help to update our systems. Please provide us the following credentials to continue using our phish- *ugh* services.
Credit card number: _____________
CVV: ___ Expiration date: ______
Steamymoomilk@sh.itjust.works 9 months ago
Spfff me, never Anyway please tell me more about your IP adress and your private keys.
Auli@lemmy.ca 9 months ago
Ok my ip address is 192.168.3.200.
Feathercrown@lemmy.world 9 months ago
My private key has a 3 in it
ripcord@lemmy.world 9 months ago
Hey its me your friend Aaron how are you. Hey i was just wandering, what is your credit card details again? just wandering