Not OP but I use these on the regular.

I’d still agree with the OP that there are hard limits to what these can do. I’ve gotten Claude stuck in loops before on removing unrelated code, then adding it back, then removing it again hoping it’ll fix something.

And OP is still correct. At the heart of all of this it’s “given input x guess the probability of response Y”. Even frontier models don’t think. They can output tokens to call tools to try and get more input x but it’s still a best guess.

You can also give them too much context and get “context rot” which makes their output absolutely horrible too. I think cursor had a problem with that where too many Claude skills caused cursor to hallucinate and go nuts.