Actually I agree. I guess I was just still annoyed after reading just previously about how llms are somehow not neural networks, and in fact not machine learning at all…

Btw, you can absolutely finetune llms on classical regression problems if you have the required data (and care more about prediction quality than statistical guarantees.) The resulting regressors are often quite good.