You’re not wrong, but I think you might be leaving some future capabilities on the table, that’s it.
There is nothing wrong with running everything through Portainer at all. It’s how I started myself. The downside is that it’s limited if you ever wish to do e.g. HA OS or a sandboxed OS for testing/playing around. Automatic backups, re-sizing LXC’s or giving more memory is also easier to do with a GUI than in CLI. At least for me hehe.

That’s the great thing about self hosting though: if you’re happy with it, then it’s perfect!
Don’t change anything because someone tells you to if it works for you, friend!