Yes. You can simply not expose SMTP at all and just use the IMAP/JMAP part. Unless you need also JMAP, I am not sure it brings you a lot to the table you wouldn’t also get from a good old dovecot. IMO the big advantage of Stalwart is the all-in-one package it delivers plus the good defaults. It also shines when you want a multi node deployment. For a single node IMAP only it might not be the best choice, in my opinion. But it would work, if you want to.