Comment on OpenAI's GPT Trademark Request Has Been Denied
kogasa@programming.dev 4 months agoYou can’t really say any GPT model has nothing to do with OpenAI. They invented the architecture. But the name GPT predates their commercial products using the technology.
NevermindNoMind@lemmy.world 4 months ago
I don’t know enough to know whether or not that’s true. My understanding was that Google’s Deep mind invented the transformer architecture with their paper “all you need is attention.” A lot, if not most, LLMs use a transformer architecture, though your probably right a lot of them base it on the open source models OpenAI made available. The “generative” part is just descriptive of the model generating outputs (as opposed to classification and the like), and pre trained just refers to the training process.
But again I’m a dummy so you very well may be right.
kogasa@programming.dev 4 months ago
The attention paper from Google introduced transformers, OpenAI introduced generative pretraining as a technique that allows transformers to achieve very good performance on downstream tasks with very little additional fine tuning. This paper and the subsequent release of the pretrained GPT models directly lead to the LLM boom.
…amazonaws.com/…/language_understanding_paper.pdf