Modern Android Do Not Disturb is configurable enough for you to do this. Allow your family contacts through, block the rest.