Comment on Microsoft CEO warns that we must 'do something useful' with AI or they'll lose 'social permission' to burn electricity on it

<- View Parent
sobchak@programming.dev ⁨2⁩ ⁨weeks⁩ ago

Dunno if that’s true or not. Generally, much more compute is used in inference than training, since you only train once, then use that model for millions of queries or whatever. However, some of these AI companies may be training many models constantly to one-up each-other and pump their stock; dunno. The “thinking” model paradigm is also transferring a lot more compute to inference. IIRC OpenAI spent $300k of compute just for inference to complete a single benchmark a few months ago (and found that, like training, exponentially increasing amounts of compute are needed for small gains in performance).

source
Sort:hotnewtop