On your first part, clarifying your intent, I think that you are overcomplicating yourself by expecting traffic to come to the server via domain name (pass through proxy) from Router A network and by IP:Port from Router B network, you can access all, from anywhere through domains and subdomains, and avoid using numbers.

If you can’t set up a DNS directly on Router A, you can set it per device you would want to access the server through port forwarding of Router B, meaning setting the laptop to use itself as primary DNS and as secondary use external, and any other device you would want in that LAN do the same, It is a bit tedious to do per device instead but still possible.

Wouldn’t this link to the 192.168.0.y address of router B pass through router A, and loop back to router B, routing through the slower cable? Or is the router smart enough to realize he’s just talking to itself and just cut out `router A from the traffic?

No, the request would stop on Router B, and maintain all traffic, on the 10.0.0.* network it would not change subnets, or anything.

Remember that all my advice so far is so you don’t use any IP or Port anywhere, and your experience is seamless on any device using domains, and subdomains, the only place where you would need to put IP or ports, is on the reverse proxy itself, to tell anything reaching it, where the specific app/service is, as those would need to be running on different ports but be reached through the reverse proxy on defaults 80 or 443, so that you don’t have to put numbers anywhere.