On my Fritzbox, I needed to add all my Subdomains to a list under:

DNS Rebind Protection Your FRITZ!Box suppresses DNS responses that refer to IP addresses in its own home network (DNS rebind protection). Here you can specify exceptions for which DNS rebind protection should not apply. Do this by entering the complete name of the host (domain name including the subdomain) in the list.

That’s under network settings, advanced. In case you have that.