Comment on Thanks a lot, AI: Hard drives are already sold out for the entire year, says Western Digital

• tal@lemmy.today ⁨3⁩ ⁨weeks⁩ ago

  I don’t know if you’re saying this, so my apologies if I’m misunderstanding what you’re saying, but this isn’t principally ECC memory that’s being produced.

  I suppose that a small portion of AI-related sales might go to ECC DDR5 sticks, because some of that hardware will probably use it, but what they’re really going to be using in bulk is high-bandwidth-memory (HBM), which is going to be non-modular, connected directly to the parallel compute hardware.

  HBM achieves higher bandwidth than DDR4 or GDDR5 while using less power, and in a substantially smaller form factor.[13] This is achieved by stacking up to eight DRAM dies and an optional base die which can include buffer circuitry and test logic.[14] The stack is often connected to the memory controller on a GPU or CPU through a substrate, such as a silicon interposer.[15][16] Alternatively, the memory die could be stacked directly on the CPU or GPU chip. Within the stack the dies are vertically interconnected by through-silicon vias (TSVs) and microbumps. The HBM technology is similar in principle but incompatible with the Hybrid Memory Cube (HMC) interface developed by Micron Technology.[17]

  The HBM memory bus is very wide in comparison to other DRAM memories such as DDR4 or GDDR5. An HBM stack of four DRAM dies (4‑Hi) has two 128‑bit channels per die for a total of 8 channels and a width of 1024 bits in total. A graphics card/GPU with four 4‑Hi HBM stacks would therefore have a memory bus with a width of 4096 bits. In comparison, the bus width of GDDR memories is 32 bits, with 16 channels for a graphics card with a 512‑bit memory interface.[18] HBM supports up to 4 GB per package.

  source

  • Toes@ani.social ⁨3⁩ ⁨weeks⁩ ago

    Ah, thanks for the information. I was already aware most of it was going to GPU type hardware. I just naturally assumed all those gpus need servers with lots of ram.

    source
• AllHailTheSheep@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

  AI doesn’t necessarily use ddr tho, they stick to HBM which is a different thing entirely.

  source

  • Toes@ani.social ⁨3⁩ ⁨weeks⁩ ago

    Yes, I understand that.

    source