The nice thing about vm with nginx proxy manager or just nginx running on the same host as the rest(or majority) of vms is that internal traffic doesn’t traverse other devices. It looks more secure setup this way. That being said chances of some packet sniffer running on your network between proxy and destination VM is low.

I’m in similar situation as you. I run overpowered router that barely sees any CPU usage.

I haven’t tried nginx but I’m running HA proxy on opnsense router. I saw in community forums most people use that. After going through tutorial for one service it’s pretty easy to grasp configuration concept and replicate for other services. I think only one confusing option is that backends pools can have backends configured and you can have only one in use. Test syntax button ensures you don’t make mistakes. HA proxy has powerful options for backend more than you probably need. I moved router management port to higher number and setup proxy to run on 443. Then wildcard DNS entry points to router and that allows to keep adding services as needed.