To mitigate the risks you could put the local server into its own network where it cannot reach anything else in your home.