Comment on XCP-NG vs PROXMOX security hardening?
moonpiedumplings@programming.dev 1 week ago
Firstly, Xen is considered by secure by Qubes — but that’s mainly the security of the hypervisor and virtualization system itself. They make a very compelling argument that escaping a Xen based virtual machine is going to be more difficult than a KVM virtual machine.
But threat model matters a lot. Qubes aims to be the most secure OS ever, for use cases like high profile journalists or other people who absolutely need security, because they will literally get killed without it.
Amazon moved to KVM because, despite the security trade off’s, it’s “good enough” for their usecase, and KVM is easier to manage because it’s in the Linux kernel itself, meaning you get it if you install Linux on a machine.
In addition to that, security is about more than just the hypervisor. You noted that Promox is Debian, and XCP-NG is Centos or a RHEL rebuild similar to Rocky/Alma, I think. I’ll get to this later.
Xen (and by extension XCP-NG) was better known for security whilst KVM (and thus Proxmox)
I did some research on this, and was planning to make a blogpost and never got around to this.
Name | Summary | Full Article | Notes |
---|---|---|---|
Performance Evaluation and Comparison of Hypervisors in a Multi-Cloud Environment | Compares WSL (kind of Hyper-V), VirtualBox, and VMWare-Workstation. | springer.com, html | Not honest comparison, since WSL is likely using inferior drivers for filesystem access, to promote integration with host. |
Performance Overhead Among Three Hypervisors: An Experimental Study using Hadoop Benchmarks | Compares Xen, KVM, and an unnamed commercial hypervisor, simply referred to as CVM. | ||
Hypervisors Comparison and Their Performance Testing (2018) | Compares Hyper-V, XenServer, and vSphere | springer.com, html | |
Performance comparison between hypervisor- and container-based virtualizations for cloud users (2017) | Compares xen, native, and docker. Docker and native have neglible performance differences. | ieee, html | |
Hypervisors vs. Lightweight Virtualization: A Performance Comparison (2015) | Docker vs LXC vs Native vs KVM. Containers have near identical performance, KVM is only slightly slower. | ieee, html | |
A component-based performance comparison of four hypervisors (2015) | Hyper-V vs KVM vs vSphere vs XEN. | ieee, html | |
Virtualization Costs: Benchmarking Containers and Virtual Machines Against Bare-Metal (2021) | VMWare workstation vs KVM vs XEn | springer, html | Most rigorous and in depth on the list. Workstation, not esxi is tested. |
The short version is: it depends, and they can fluctuate slightly on certain tasks, but they are mostly the same in performance.
default PROXMOX and XCP-NG installations.
What do you mean by hardening? If you are talking about hardening the management operating system (Proxmox’s Debian or XCP’s RHEL-like), or the hypervisor itself?
I agree with the other poster about CIS hardening and generally hardening the base operating system used. But I will note that XCP-NG is more designed to be an “appliance” and you’re not really supposed to touch it. I wouldn’t be suprised if it’s immutable nowadays.
For the hypervisor itself, it depends on how secure you want things, but I’ve heard that at Microsoft Azure datacenters, they disable hyperthreading because it becomes a security risk. In fact, Spectre/Meltdown can be mitigated by disabling hyper threading. Of course, their are other ways to mitigate those two vulnerabilities, but by disabling hyper threading, you can eliminate that entire class of vulnerabilities — at the cost of performance.
marauding_gibberish142@lemmy.dbzer0.com 1 week ago
Thank you for the wonderful comment. I am talking about the operating system (Debian vs CentOS if I remember correctly) when I mention Hardening.
I haven’t seen a concrete example of anyone applying CIS policies on the XCP-NG base, neither have I seen any mentions of securing the XCP-NG base by companies using them in production. I understand that having a walled-off
dom0
is great and I like that about Xen, but not seeing dialogue on base OS level security is making me a bit uncomfortable about XCP-NG. Not sure if it is immutable, if it is then that would relieve some of my worries.Personally, I think Proxmox is somewhat unsecure too. I believe something like following relevant STIG recommendations, kernel self-protection, hardened malloc and other things (there’s a huge list but I’ll be brief) should be essential. Ideally I would’ve preferred that the Proxmox project took some of the measures that the Kicksecure project does in hardening debian but I don’t see any mention of something like that. If I end up wanting to run Proxmox, I’ll install Debian, distro-morph it to Kicksecure and then follow the instructions for Proxmox (not sure how I’ll keep from using the Proxmox custom kernel but we’ll see).
moonpiedumplings@programming.dev 1 week ago
Proxmox is unique from other projects, in it’s much more hacky, and much of the stack is custom rather than standards. Like for example: For networking, they maintain a fork of the Linux’s older networking stack, called
ifupdown2
, whereas similar projects, like openstack, or Incus, use either the standard Linux kernel networking, or a project calledopenvswitch
.I think Proxmox is definitely secure enough, but I don’t know if I would really trust it for higher value usecases due to some of their stack being custom, rather than standard and mantained by the wider community.
If you’re interested in deploying a hypervisor on top of an existing operating system, I recommend looking into Incus or Openstack. They have packages/deployments than can be done on Debian or Red Hat distros, and I would argue that they are designed in a more secure manner (since they include multi tenancy) than Proxmox. In addition to that, they also use standard tooling for networking, like both can use Linux Bridge (in-kernel networking) for networking operations.
I would trust Openstack the most when it comes to security, because it is designed to be used as a public cloud, like having your own AWS, and it is deployed with components publicly accessible in the real world.
marauding_gibberish142@lemmy.dbzer0.com 1 week ago
I had looked into openstack a while back but left it thinking it was too complex. I was looking at Apache’s Cloudstack then.
I see now that a contributor has got Debian in the official list of supported distributions. Which means my distro-morphing idea should work in theory with OpenStack. This is a great idea, thanks. I will look at OpenStack more seriously now. Does look like it will need some effort though
moonpiedumplings@programming.dev 1 week ago
I don’t recommend doing a manual install though, as it’s extremely complex compared to automated deployment solutions like kolla-ansible, openstack-ansible, or openstack-helm/genestack. They make the install much more simpler and less time consuming, while still being intensely configurable.