Comment on ChatGPT's Atlas: The Browser That's Anti-Web
paraphrand@lemmy.world 1 week agoDuring setup, Atlas pushes very aggressively for you to turn on “memories” (where it tracks and stores everything you do and uses it to train an AI model about you)
I wonder, do memories really train a model about the user? Or are they just shoved in the context window strategically? Possibly selected by a small performant model in the background based on relevance to the current context window?
Training millions of mini models on people would be really interesting, and I don’t think I’ve noticed anything saying that is happening, yet. Even tho it seems like a logical idea.
ilinamorato@lemmy.world 1 week ago
I’m sure it’s RAG at best. There’s no way I can conceive of that they’re actually training individual models for each user in a performant or economical fashion.
More likely, as you said, they’re just zero-shotting the relevant personal data into the context window. And honestly, I’d be a little surprised if they had a smaller model trying to evaluate relevance; a simple heuristic or basic frequency analysis algorithm would probably perform about as well and be a lot cheaper. The big final model can probably toss away the noise well enough.