Submitted 8 months ago by Ek-Hou-Van-Braai@piefed.social to selfhosted@lemmy.world
My Homelab currently consists of 3 Mini PC's
They are all just plugged into the router my ISP provided, I'd like to get a new router that runs open-source software and create a new network from it. I have no idea where to begin.
What hardware would you recommend?
If possible I'd like to in the future attach a sim card to my network as a backup for the occasion that the ISP connection drops. (just a nice to have)
I use a dual NIC mini PC running OpnSense. Ot would support USB sims. I actually have two of the routers connected woth a network cable. If one goes down, the other takes over.
I started with pfsense on Virtualbox, then quickly moved to HyperV on Windows, where I had 3 locations running this as their routers for almost 2 years, even through COVID when I couldn’t get to some locations. I never had a single issue, just got annoyed at the constant Windows updates rebooting the systems and internet going down when it did.
I then moved to Untangle, still on Hyper V, then moved off VM onto baremetal on an HP Elitedesk 800 with 10GbE card that cost about £100 total, which ran wonderfully until Untangle got sold out.
So, then onto OPNSense on the same Elitedesk (after reading about PFSenses silly games they played), and this ran perfectly for about 18 months, and with solid 1Gbps on Wireguard, then after all these years of messing with routers, I finally switched to a Unifi UDM Pro SE last year and I couldn’t be happier. It does all I need, plus also CCTV recording (away from Blue Iris). I no longer have to worry that my DIY routers are going to fail on me. So, I would recommend Unifi hardware, despite it not being open source, mainly because, well, internet is a crucial service, especially in my household. And the UDM does WAN fail over well with my backup 4G modem.
I did once however, move from PFSense to Untangle on a remote machine. Because Untangle had a GUI, I fired up a VM on the same Windows machine as PFSense, set it all up with the same NIC settings, then adjusted Hyper V so that the Untangle VM booted and the PfSense one didn’t, then rebooted the machine and waited nervously for a few minutes, then boom, up popped the Untangle router! It felt good getting that done. It was only at my parents house, but still, it required a 90min journey if it failed.
With all the drama of Windows 11 in recent years, I’m glad I switched away from HyperV when I did.
I’m a fan of pfSense, myself. But other suggestions here for OSes have been reasonable. I have a netgate router feeding an eero wap with a second wap creating a bridged wifi network. Future-proofing with 10GB on a wired switch if a good idea. I got a Unifi 2.5GB switch with 10GB uplink for that. The difference in performance moving large data around is massive. I have 10GB between my primary machine, the one that I run as my always-on server, and my NAS. It’s awesome. Everything else is 2.5GB.
Assuming you want to replace it all, not just home lab use…
Do you want the router to also be 10" rack-mountable? That seems like it’d be a big input into the hardware you get.
The short answer: For a router, either find an off-the-shelf Wifi router that is supported by OpenWRT (very nice and very easy), or (and this is my personal preference) build your own firewall mini PC which will be much more complex and powerful to the point of complete overkill but also fully controllable right down to the network stack (and what’s the point of a homelab if not fiddling around with such things?).
You can run OpenWRT directly on full AMD64 PC if you want, or even just a Raspberry Pi (some people appear to have had good luck with the 4B and 5, though I don’t know the specifics of that approach) The famous PfSense would be another option, based on BSD. I used to use that, but I really wanted something directly Linux-based.
Which brings us to the fact that you can also even use a standard Linux distro like Debian and install all the tools you want on top of that and set up all the firewall yourself from scratch. That is actually what I do, using Linux kernel’s nftables for NAT Masquerading/IP forwarding and managing it currently with foomuuri which is essentially just a very lightweight nftables configuration manager. It doesn’t do anything you can’t do directly with nftables, but even though it’s perfect for me but I’m not sure I would recommend it in general. They have some very simple examples, but the documentation is pretty sparse, you need to either understand nftables under the hood or infer what you can by reading between the lines of the few examples you can find. A more mature and traditional Linux firewall like firewalld might be preferable if you want. Either way, this is definitely a much more complex route though, and fighting with firewall rules to get things to work is not everybody’s idea of “fun”. It is powerful though, and infinitely flexible. If you want it to “just work” without hassle, stick to the single-purpose devices and use OpenWRT as the OS designed to do this. It’s way simpler.
If you do decide do go the DIY firewall route though, all you really need for a firewall PC is at least a second NIC (some motherboards have two wired NIC onboard already, you can use one for WAN and the other + WiFi for LAN) or you can a PCIe network card that has multiple ports. I wouldn’t really recommend using one of your existing Mini PCs for this, as it’s really not a good idea to share the firewall/network appliance functionality shared with other services, both for security and for configuration complexity reasons. The firewall really works best and is easiest to configure when it is truly just a gateway for the network, putting traffic from one side out the other side, plus whatever fundamental network/firewall services you need to accomplish that. When you start also trying to selectively route some of that traffic to actual services on the firewall itself, it gets really complex and ugly really fast, and even if you can get it working which is often very nontrivial, it’s also very fragile and it’s easy to blow open holes in your security this way.
I’ve actually now got a pair of mini-PC firewalls, both set up using foomuuri, uCARP and Kea to do failover with each other so if one goes offline the other takes over its IP and starts routing traffic until it comes back. It’s not perfect or completely bulletproof but it’s pretty good for an amateur! In a pinch (when my previous, non-redundant firewall died) I’ve also used an GL.iNet travel router as my network’s primary router temporarily and their routers support an expansion board with 5G/SIM support so that could be an option too. I have to say it worked perfectly and was actually pretty nice, my only hesitation is that the travel router (at least the one I have, Beryl AX) seems to run a bit hot and I’m not sure it’s really intended for 24/7/365 operation (plus I need it for when I travel). They do make home routers too though, so maybe worth looking into, they’re really nice hardware running their own fork of OpenWRT out of the box.
Opnsense on basically anything. That’s what I’d recommend as a platform, so see if they have recommended hardware for cell network support.
Or if you’re okay with commercial products, cradlepoint makes good cell network hardware. But you should still have a separate firewall/router and just use the cradlepoint as a modem.
Yeah. Would be my recommendation, too. For the size of the lab a Zimba seems a good choice if something new is what OP wants, otherwise a MiniPC.
I’m running it virtualized in proxmox, mostly for the challenge. And boy was it a challenge. Runs great now though!
I don’t really recommend virtualizing network infrastructure. If you break proxmox, you will probably lose Internet access entirely.
They are not open source but I just started dipping my toes into the Mikrotik ecosystem and the hardware has been pretty nice from what I have seen. I am not a network guy, just a homegamer coming from normal asus routers though. They have a couple of options for adding cell service via sim cards but I have not looked too far into it.
I’ve had the opposite experience with Mikrotik.
I really wanted to like it, but (I say this as a former Cisco instructor) their approach to UI and documentation is terrible (the docs don’t tell you what’s what, just tell you how to setup a specific config, without explaining what they’re doing or why, even worse, they start numbering their eth interfaces from 1 - it took me a while to figure this out).
Worse, it was unstable as hell. I setup one just as a test, with one laptop connected via ethernet. Every couple days I wouldn’t be able to even ping the laptop - I’d have to reboot the router, manually, since it had become unresponsive.
This with a simple config (just eth2 is LAN, eth1 is external), and no rules.
It may have been a faulty unit, but as a consumer I can’t risk assuming this, especially given the very poor docs and clumsy UI/config approach - it all indicates this is a very immature product, definitely not something I’d recommend to a newbie.
I hope they can really improve - the form factor is excellent, the price point is unbeatable, the capabilites of the hardware are extensive.
I somewhat agree on your comment about documentation and UI (altough once you get used to it, it’s manageable) but just to add with my experience on these things: for me they’ve been rock solid. I’ve used them both at home and professionally (mostly on small-ish networks) for at least 10 years and they just seem to run just fine.
Currently my home router is RB4011iGS+ and there’s been absolutely no problems with it in the 4-5 years it’s been on my network. I’m not saying all their models are as reliable and there’s not that many models I’ve had my hands on, but my experience with them has so far been pretty good.
Yeah the docs are not good, I have been lucky to have a friend with lots of experience in their ecosystem who has been schooling me up on it. Once I got the basic configuration setup its been fine.
I may regret saying that in a bit when I go to add my other components, like my adguard/pi-hole, vpns, ip cameras, and other networked devices but the basic test setup I have now seems to be stable enough to deploy.
I have not seen the connection loss issues but I will keep an eye out for it.
OpenWRT on a 5009? Why? You’ll lose the switch integration and a whole lot of speed, not to mention features…
The op mentioned wanting open source. Just providing information and it got me curious.
First things first, make sure your ISP’s device can be put into a bridge mode, or that you can get a modem instead, otherwise you’re going to be dealing with double NAT issues.
Impromptu2599@lemmy.world 8 months ago
Check out GNS3, it will let you build it an entire lab virtually.