Running it as a VM also introduces many other potential sources of inefficiency. I always recommend running a firewall on dedicated bare metal hardware, it is a very specialized task with very particular requirements on behalf of both the hardware and the software. That doesn’t mean you need to use a pre-built appliance, but it does explain why it’s so common, and running it on a VM on a server that is doing other stuff is likely contributing to your issues significantly.

Personally, I run my firewall/router on a very stripped-down Debian with almost no non-essential services and a custom built kernel. I hand-picked a multi-port PCIe x4 Intel NIC with good Linux compatibility and drivers, and I’m using foomuuri to handle the routing and kea to handle DHCP/DNS for my internal network. This is a very minimal, bare-bones configuration and I wouldn’t really recommend it unless you really know what you’re doing, and it’s absolutely not “idiot mode networking” and if that’s what you want you’re going to have a real bad time. But it works for me, so it’s proof that it is possible.