If going the route of a backup solution, is it feasible to install OpenWRT on all of my devices, with the expectation that I can do some sort of automated backups of all settings and configurations, and restore in case of a router dying?

My two cents: use a “full” computer as your router (with either something like OPNsense or any “regular” linux distro if you don’t need the GUI) and OpenWRT on your access points.

Unless you use the GUI and backup/restore the configuration (as you would with proprietary firmwares), OpenWRT is frankly a pain to configure and deploy. At the moment I’m building custom images for all my devices, but (next time™) I’m gonna ditch all that, get an x86 router and just manually manage OpenWRT on my wifi APs (I only have two and they both have the same relatively straightforward config).

It’s a pain that I know can be solved with buying dedicated access points (…right?)

Routers and access points are just computers with network interfaces (there may be level-2-only APs, but honestly I’ve never heard of any)… most probably your issue is that the firmware of your “routers as access points” doesn’t want to be configured as a dumb AP.