Comment on How AI works is often a mystery — that's a problem
msage@programming.dev 10 months ago“Rumors claim that GPT-4 has 1.76 trillion parameters”
I’m not sure even unlimited time would help understand what’s really going on.
You could build another model to try to decipher te first, but how much could you trust it?
wikibot@lemmy.world [bot] 10 months ago
Here’s the summary for the wikipedia article you mentioned in your comment:
Generative Pre-trained Transformer 4 (GPT-4) is a multimodal large language model created by OpenAI, and the fourth in its series of GPT foundation models. It was initially released on March 14, 2023, and has been made publicly available via the paid chatbot product ChatGPT Plus, and via OpenAI’s API. As a transformer-based model, GPT-4 uses a paradigm where pre-training using both public data and “data licensed from third-party providers” is used to predict the next token. After this step, the model was then fine-tuned with reinforcement learning feedback from humans and AI for human alignment and policy compliance.: 2 Observers reported that the iteration of ChatGPT using GPT-4 was an improvement on the previous iteration based on GPT-3.5, with the caveat that GPT-4 retains some of the problems with earlier revisions. GPT-4 is also capable of taking images as input on ChatGPT. OpenAI has declined to reveal various technical details and statistics about GPT-4, such as the precise size of the model.^article^ ^|^ ^about^