I love the Vita, and you’re right, you can hack it to accept SD Cards, use native PSP/PS1 emulation in any game and a lot of homebrew ports.

PS: If you’re willing to get third party PC Handhelds, the Ayaneo Air 1S is the closest thing the Vita form factor I know. 5.5" OLED screen, but the bezel is thicker.