To explain how HAproxy and competing tools solve this:

Two servers are prepared to be the single reverse proxy, but one is active. They constantly communicate with a “heartbeat”. When the active one fails to send a heartbeat, the secondary executes the steps to become the active primary. When the primary’s heart starts beating again, it becomes active again.

So there can be a few seconds of downtime, but the failover is automatic.