I assume I need to have a domain name through a DNS service like cloudflare in order to make use of it

Yes, you’re correct here.

Once I have my DNS setup, how do I associate it with my server or point it through the nginx reverse proxy?

You begin by forwarding ports 80 and 443 to your Nginx proxy server. These are the standard ports for http and https requests, respectively. So your Nginx will immediately be able to tell if a request is http or https based on which port it is coming in on.

Next, you would set an A name record on your domain manager. This A name will point a subdomain to a specific IPv4 address. So for instance, maybe the name is “abs” and the IP is your home WAN IP. So whenever an http or https request comes in on “abs.{your domain}” it will get redirected to your WAN IP. If you wanted to use IPv6, that would be an AAAA name record instead… But if this is your first foray into self-hosting, you probably don’t want to use IPv6.

On Nginx’s side, it receives all of those incoming http and https requests because the ports are forwarded to it. You configure it to take requests for those subdomains, and route them to your various devices accordingly. You’ll also need to do some config for SSL certificates, which will allow https requests to resolve successfully. You can either use a single certificate for the entire site, or an individual certificate for each subdomain. Neither is “more” correct for your needs, (though I’m sure people will argue about that in responses to this).

So for instance, you send a request to https://abs/.{your domain}. The domain manager forwards this to your WAN IP on port 443. Nginx receives this request, resolves the SSL certificate, and forwards the request to the device running abs. So your ABS instance isn’t directly accessible from the net, and needs to bounce off of Nginx with a valid https request in order to be accessible.

You’ll want to run something like Fail2Ban or Crowdsec to try and prevent intrusion. Fail2Ban listens to your various services’ log files, and IP-bans repeated login failures. This is to help avoid bots that find common services (like ABS) and try to brute-force them by spamming common passwords. You can configure it to do timeouts with increasing periods. So maybe the first ban is only 5 minutes, then 10, then 20, etc…

Lastly, you would probably want to run something like Cloudflare-DDNS to keep that WAN IP updated. I’m assuming you don’t have a static IP, and you don’t want your connections to break every time your IP address changes. DDNS is a system that routinely checks your WAN IP every few minutes, and pushes an update to your provider if it has changed. So if your IP address changes, you’ll only be down for (at most) 5 minutes. This will require some extra config on your provider’s part, to get an API key and to configure the DDNS service to point at your various A name records.

If you need any help setting the individual services up, let me know. I personally suggest docker-compose for setting up the entire thing (Nginx, DDNS, and Fail2Ban) as a single stack, but that’s purely because it’s what I know and it makes updates easy. But this comment is already long enough, and each individual module could be just as long.