This is a complete digression but do you know if there is a consumer hardware that can be reliably erased?

behave as an affordable HSM

Like, to create a hardware keystore? No, I don’t, sorry. If I wanted one myself, I’d probably just buy an existing one and hope that they did things correctly. :-)