Update: I took some suggestions and ideas from the comments here and built a system for this:

• Create the mountpoint directories
• Add the mounts to /etc/fstab with the user,noauto option
• I’m using quadlets, so add a new [service] section to my container file and add ExecStartPre and ExecStopPost entries with mount and umount commands

This allows me to have the mounts only active when the container is running. And I can still have rootless podman containers. Seems to be working well in the limited testing I did yesterday.