I’ve run koboldcpp on a steam deck. You have to stick to small model files, of course, like maybe 4Gb, but you can get decent speed if you do.