The comics kind of address this:

Killing the holder of the Infinity Gauntlet usually results in someone equally bad picking it up.

So just killing Thanos often isn’t a complete solution, after the stones have been gathered.