I’ll toss in my two cents.

It’s mainly about handling and processing vast amounts of data. Many times more than you or I may deal with on a day to day basis. First, you have to have somewhere to put it all. Then, you’ve got to load whatever you’re working with into memory. So you need terabytes of RAM. When you’re dealing with that much data, you need beefy CPUs with crazy fast connections with a ton of bandwidth to move it all around at any kind of reasonable pace.

Imagine opening a million chrome tabs, having all of them actively playing a video, and needing to make sense of the cacophony of sound. Only instead of sound, it’s all text, and you have to read all of it all at once to do anything meaningful with it.

If you make a change to any of that data, how does it affect the output? What about a million changes? All that’s gotta be processed by those beefy CPUs or GPUs.

Part of the reason AI data enters need so much memory is because they’ve got to load increasingly large amounts of training data all at once, and then somehow have it be able to be accessed by thousands of people all at once.

But if you want to understand every permutation of whatever data you’re working with, it’s gonna take a ton of time to essentially sift through it all.

And all that’s gotta be hardware? You have to make doubly sure that the results you get are accurate, so redundancies are built in. Extremely precise engineering of parts, how they’re assembled, and how they’re ultimately used is a lot of what makes supercomputers what they are. Special CPUs, RAM with error correction, redundant connections, backups… it all takes a lot of time, space, and money to operate.