I don’t know if SoftEther has an option so you don’t tunnel everything, and just use the virtual LAN IPs for games, file transfers, etc.

And I don’t know your actual technical level or the people you play with, but, for people that can go as far as opening ports and installing a server, and getting others to connect to it, I would suggest Headscale (the free self-hosted version of Tailscale) as a next step, or if inclined to learn something a bit more hands on Wireguard.

With those you can configure it so, only the desired traffic goes through (like games or files sharing using the virtual LAN IP), and the rest goes out normally, or configure exit nodes, so if/when desired, all traffic is tunneled like what you have now.