Comment on Sam Altman Says If Jobs Gets Wiped Out, Maybe They Weren’t Even “Real Work” to Start With
galaxy_nova@lemmy.world 19 hours agoI mean if you use the code base you’re working in as context it’ll probably learn the code base faster than you will, although I’m not saying that’s a good strategy, I’d never personally do that
kescusay@lemmy.world 18 hours ago
The thing is, it really won’t. The context window isn’t large enough, especially for a decently-sized application, and that seems to be a fundamental limitation. Make the context window too large, and the LLM gets massively offtrack very easily, because there’s too much in it to distract it.
And LLMs don’t remember anything. The next time you interact with it and put the whole codebase into its context window again, it won’t know what it did before, even if the last session was ten minutes ago. That’s why they so frequently create bloat.