Lol there is no way in hell this is sustainable
Bing Chat so hungry for GPUs, Microsoft will rent them from Oracle
Submitted 1 year ago by throws_lemy@lemmy.nz to technology@lemmy.world
https://www.theregister.com/2023/11/07/bing_gpu_oracle/
Comments
db0@lemmy.dbzer0.com 1 year ago
Deebster@programming.dev 1 year ago
Environmentally or economically? Actually, which one you mean doesn’t matter, cos it’s not either way:
dojan@lemmy.world 1 year ago
That second one is hilarious, because base CoPilot is absolute garbage. Less helpful than Intellisense.
I’ve had some very niche use cases for CoPilot Chat, but even that is just hidden away gathering dust most of the time.
RanchOnPancakes@lemmy.world 1 year ago
Yes. This seems VERY profitable and sustainable. Yes indeed.
partial_accumen@lemmy.world 1 year ago
It probably is very profitable and it doesn’t need to be sustainable. This is likely a short term (3 years is short) deal.
This is the tip off for me:
In this case, Microsoft is using the system alongside its Azure Kubernetes Service to orchestrate Oracle’s GPU nodes to keep up with what’s said to be demand for Bing’s AI features.This doesn’t really look like this is about Bing AI features but rather GPUs are in demand so much that MS can “rent in bulk” from Oracle, and then rent out Azure’s own GPUs to Azure customers willing to pay per second for GPU usage at retail prices.
RanchOnPancakes@lemmy.world 1 year ago
I mean all the processing needed for AI in general. its the new… “big thing” they’ll find some use. Give it to everyone. realize it won’t make money if its free. Try to find a way to charge. It won’t be good enough for that. People will move on. It’ll end up in its niches where it makes senses and is considered a cost savings, and the world will move on. A ton of firms will merge and go bankrupt and then all the big tech companies will move onto the next “big thing”.
SARGEx117@lemmy.world 1 year ago
Hey they can have my old GPU if they give me a new blank laptop.
I’ve always wanted to try linux
Damage@slrpnk.net 1 year ago
So when will CPUs integrate the hardware necessary to compete with GPUs on these tasks? This situation is ridiculous, the device designed for this isn’t able to keep up with the device designed for something else entirely
jmcs@discuss.tchncs.de 1 year ago
You are looking at it wrong by taking the names too literally. GPUs are simply processing units optimized for parallel computation and CPUs processing units optimized for general purpose sequential computation. And these optimizations require architectural trade-offs, so to be efficient at both types you’ll need to have both a CPU and GPU.
So think of it this way, a CPU is actually a General-purpose Sequential Processing Unit and a GPU as a Parallel Processing Unit, but renaming them would only add to the confusion.
Deebster@programming.dev 1 year ago
GPUs are a lot closer to AI processors (tensor cores and similar) than CPUs. Graphics processing is about doing lots of simple computations simultaneously, which is what AI does - lots and lots of matrix maths. CPUs are more general purpose but can’t compete on raw speed because of this (and some of the hacks to try to get more speed are causing security problems).
autotldr@lemmings.world [bot] 1 year ago
This is the best summary I could come up with:
Demand for Microsoft’s AI services is apparently so great – or Redmond’s resources so tight – that the software giant plans to offload some of the machine-learning models used by Bing Search to Oracle’s GPU supercluster as part of a multi-year agreement announced Tuesday.
The partnership essentially boils down to: Microsoft needs more compute resources to keep up with the alleged “explosive growth” of its AI services, and Oracle just happens to have tens of thousands of Nvidia A100s and H100 GPUs available for rent.
Microsoft was among the first to integrate a generative AI chatbot into its search engine with the launch of Bing Chat back in February.
You all know the drill by now: you can feed prompts, requests, or queries into Bing Chat, and it will try to look up information, write bad poetry, generate pictures and other content, and so on.
In this case, Microsoft is using the system alongside its Azure Kubernetes Service to orchestrate Oracle’s GPU nodes to keep up with what’s said to be demand for Bing’s AI features.
Oracle claims its cloud super-clusters, which presumably Bing will use, can each scale to 32,768 Nvidia A100s or 16,384 H100 GPUs using a ultra-low latency Remote Direct Memory Access (RDMA) network.
The original article contains 580 words, the summary contains 207 words. Saved 64%. I’m a bot and I’m open source!
Fedizen@lemmy.world 1 year ago
I heard rumors that azure ran on oracle, this is probably why. Microsoft’s pursuit of advanced chatbot technology is surely to be a loser in the long run.
deleted@lemmy.world 1 year ago
Of course, they must understand every click you make in Windows.
Migrate to Linux.
Deebster@programming.dev 1 year ago
Er, Bing Chat is AI-assisted search, not analytics.