That’s called split DNS. You can probably use that term to figure out if a particular router supports it. Basically, you would tell the router “if a DNS request for a specific URL is coming from a local IP, use a different (usually local) DNS table”. So like you can tell it “if a device asks for this URL, route it to this local IP instead.” So the DNS request never actually leaves your network.

It can be handy for cases where you don’t always want to be reliant on an external DNS server. For instance, if your internet is spotty. You don’t want your Jellyfin to stop working just because your internet went out; Everything is local, so it should be able to connect. But if you’re only using an external DNS provider, it won’t be able to connect without internet. So split DNS will allow you to connect to local services even when your internet is out.

The big downside to split DNS is that you often run into DNSSEC (DNS over https) warnings. Since the URL was intercepted before it actually reached an external DNS server, the traffic isn’t taking the path that the service “expected” it to take. So it may throw some warnings, or refuse to connect because it thinks your traffic is being intercepted, (because… Well… It is being intercepted… By you.)