Comment

Comment on Feedback on Network Design and Proxmox VM Isolation

DeltaTangoLima@reddrefuge.com ⁨8⁩ ⁨months⁩ ago

Yeah, it’s definitely overkill putting each VM into its own broadcast domain. What you’re looking to achieve is known zero trust architecture (ZTA). The primary concept is that you never implicitly trust a particular piece of traffic, and always verify it instead.

The most common way I’ve seen this achieved is exactly what you’re talking about - more micro-segmentation of your network.

The design principles are usually centred around what the crown jewels are in your network. For most companies applying ZTA, that’s usually their data, especially customer data.

Ideally you create a segment that holds that data, but no processing/computer/applications. You can also create additional segments for more specific use cases if you like, but I’ve rarely seen this get beyond three primary segments: server; database; data storage.

In your case, you can either create three separate VLANs on your Proxmox cluster, with your your OPNsense firewall having an interface defined in each, or use the Proxmox firewall. I’d go the former - OPNsense is a lot more capable than the Proxmox firewall, especially if you turn on intrusion detection.

I’m not using any further segmentation beyond my VMs sitting in their own VLAN from my phsyical, but here’s a screenshot of my networking setup on Proxmox. I wrote this reply to another post here on Selfhosted, talking about how my interfaces are setup. As I said in there, it’s a bit of a headfuck getting it done, but very easy to manage once setup.

BTW, this isn’t overkill if it’s what you want to do. You’re teaching yourself some very valuable skills and, and you clearly have a natural talent for thinking both vertically and horizontally about your security. This shit is gold when I interview young techs. One of my favourite interview moments is when I ask about their home setups, and then get to see their passion ignite when they talk about it.

source

Sort:hotnew top

filister@lemmy.world ⁨8⁩ ⁨months⁩ ago
I want to do exactly that on my fw router. I have installed two identical SSDs and the plan is to install Proxmox and run Opnsense on top of it. I also want to segment my WLAN and allocate the 5GHz to phones and laptops and tablets. 2.4GHz to IoT devices and the Guest WLAN for occasional guests. Each one of them should be in a separate VLAN.

My ISP router would have the WLAN disabled and I will run it through a Netgear RX7800 running OpenWRT. The idea is to run Opnsense with intrusion detection and serve as the primary gateway for every device in my network.

Any guide or hint how I can achieve that would be highly appreciated.

source
- DeltaTangoLima@reddrefuge.com ⁨8⁩ ⁨months⁩ ago
  VLANs are absolutely the key here. I run 4 SSIDs, each with its own VLAN. You haven’t mentioned what switch hardware you’re using, but I’m assuming it’s VLAN-capable.
  
  The (high-level) way I’d approach this would be to first assign a VLAN for each purpose. In your case, sounds like three VLANs for the different WLAN classes (people; IoT; guest) and at least another for infrastructure (maybe two - I have my Proxmox VMs in their own VLAN, separate to physical infra).
  
  VLANS
  
  Sounds like 5 VLANs. For the purposes of this, I’ll assign them thusly:
  
  vlan10: people, 192.168.10.0/24
  
  vlan20: physical infrastructure, 192.168.20.0/24
  
  vlan30: Proxmox/virtual infra, 192.168.30.0/24
  
  vlan40: IoT, 192.168.40.0/24
  
  vlan50: guest, 192.168.50.0/24
  
  That’ll give you 254 usable IP addresses in each VLAN. I’m assuming that’ll be enough. ;)
  
  SWITCH
  
  On your switch, define a couple of trunk ports tagging appropriate VLANs for their purpose:
  
  One for your Nighthawk, tagging VLANs 10, 20, 40 and 50 (don’t need 30 - Proxmox/VMs don’t use wireless)
  
  One for your Proxmox LAN interface, tagging all VLANs (you ultimately want to route all traffic through OPNsense)
  
  If you had additional wired access points for your wireless network, you’d create additional trunk ports for those per item 1. If you have additional Proxmox servers in your cluster, ditto for item 2 above.
  
  WIRELESS
  
  I’m not that familiar with OpenWRT, but I assume you can create some sort of rules that lands clients into VLANs of your choice, and tags the traffic that way. That how it is on my Aruba APs.
  
  For example, anything connecting to the IoT SSID would be tagged with vlan40. Guest with vlan50, and so on.
  
  PROXMOX
  
  Create a Linux Bridge interface for the LAN interface, bridging the physical interface connected to SWITCH item 2, above
  
  Create Linux VLAN interfaces on the bridge interface, for each VLAN (per my screenshot example)
  
  You haven’t mentioned internet/WAN but, if you’re going to use OPNsense as your primary firewall/router in/out of your home network, you’d also create a Linux Bridge interface to the physical interface connecting your internet
  
  ** OPNSENSE**
  
  This is the headfuck stage (at least, it was for me at first). Simply put, you need to attach the Proxmox interfaces to your OPNsense VM.
  
  I’m not going to attempt to explain it in reduced, comment form - no way I could do it justice. This guide helped me immensely in getting mine working.
  
  If you have any issues after attempting this, just sing out mate, and I’ll try and help out. Only ask is that we try and deal with it in comment form here where practical, when Googlers in the future land here in the Fediverse.
  
  source
  - filister@lemmy.world ⁨8⁩ ⁨months⁩ ago
    Thanks a lot for the detailed reply, and the time you took to answer an Internet stranger. Much appreciated.
    
    source
Pete90@feddit.de ⁨8⁩ ⁨months⁩ ago
Thank you so much for your kind words, very encouraging. I like to do some research along my tinkering, and I like to challenge myself. I don’t even work in the field, but I find it fascinating.

The ZTA is/was basically what I was aiming for. With all those replies, I’m not so sure if it is really needed. I have a NAS with my private files, a nextcloud with the same. The only really critical thing will be my Vaultwarden instance, to which I want to migrate from my current KeePass setup. And this got me thinking, on how to secure things properly.

I mostly found it easy to learn things when it comes to networking, if I disable all trafic and then watch the OPNsense logs. Oh, my PC uses this and this port to print on this interface. Cool, I’ll add that. My server needs access to the SMB port on my NAS, added. I followed this logic through, which in total got me around 25-30 firewall rules making heavy use of aliases and a handfull of floating rules.

My goal is to have the control for my networking on my OPNsense box. There, I can easily log in, watch the live log and figure out, what to allow and what not. And it’s damn satisfying to see things being blocked. No more unknown probes on my nextcloud instance (or much reduced).

The question I still haven’t answered to my satisfaction is, if I build a strict ZTA or fall back to a more relaxed approach like you outlined with your VMs. You seem knowledgable. What would you do, for a basic homelab setup (Nextcloud, Jellyfin, Vaultwarden and such)?

source
- DeltaTangoLima@reddrefuge.com ⁨8⁩ ⁨months⁩ ago
  
  What would you do, for a basic homelab setup (Nextcloud, Jellyfin, Vaultwarden and such)?
  
  I guess my first question is are you intending to open up any of these to be externally available? Once you understand the surface area of a potential attack, you can be a lot more specific about how you protect yourself.
  
  I have just about everything blocked off for external access, and use an always-on Wireguard VPN to access them when I’m not home. That makes my surface area a lot smaller, and easier to protect.
  
  source
  - Pete90@feddit.de ⁨8⁩ ⁨months⁩ ago
    Only Nextcloud if externally available so far, maybe I’ll add Vaultwarden in the future.
    
    I would like to use a VPN, but my family is not tech literate enough for this to work reliably.
    
    I want to protect these public facing services by using an isolated Traefik instance in conjunction with Cloudflare and Crowdsec.
    
    source