Proxmox thick provisions virtual machines. If you give the VM 4GB of ram the host will use at least 4GB of ram.

If you are running ZFS you also probably have an ARC cache