Open Menu
AllLocalCommunitiesAbout
lotide
AllLocalCommunitiesAbout
Login

'Consider a system with no DRAM' replaced by a 'recycling fiber loop': John Carmack envisages bold future to avoid AI-driven RAM crisis

⁨194⁩ ⁨likes⁩

Submitted ⁨⁨1⁩ ⁨month⁩ ago⁩ by ⁨tal@lemmy.today⁩ to ⁨technology@lemmy.world⁩

https://www.techradar.com/computing/memory/a-cure-for-the-memory-crisis-john-carmack-envisions-fiber-cables-replacing-ram-for-ai-usage-which-would-mean-a-better-future-for-us-all

source

Comments

Sort:hotnewtop
  • CubitOom@infosec.pub ⁨1⁩ ⁨month⁩ ago

    Or we could just like…not do the terrible thing that is bad in everyday.

    source
  • edgemaster72@lemmy.world ⁨1⁩ ⁨month⁩ ago

    and then the datacenters adopt that tech and hoard it all too

    source
    • JasonDJ@lemmy.zip ⁨1⁩ ⁨month⁩ ago

      That’s the idea. It’s pretty worthless for home use, but for AI workloads, it might make sense, the problem is that it’s not quite scalable yet.

      Essentially, if you’ve got 256Tb/s going over 200km of fiber, that means that there’s quite literally 32,000,000,000 bytes (32GB) “in flight”, living on the fiber at any period of time.

      So it’s essentially it’s a revolving sushi belt of bytes, roughly as large as London (inside M25), moving at nearly the speed of light.

      Of course, it doesn’t have to be the size of London. You could wind it into something about the size of a softball. Theoretically.

      It’s a cool idea and Carmack is no doubt a brilliant man. It seems far fetched but it’s kind of been done before… en.wikipedia.org/wiki/Core_rope_memory

      source
      • Morphit@feddit.uk ⁨1⁩ ⁨month⁩ ago

        It’s an optical delay-line memory. Early computer memories were acoustic in some manner.

        I can’t imagine that the latency of ‘delay line RAM’ would be acceptable to anyone today. Maybe there’s some clever multiplexing that could improve that but it would surely add more complexity that just making more RAM ICs.

        source
        • -> View More Comments
      • Schmoo@slrpnk.net ⁨1⁩ ⁨month⁩ ago

        moving at nearly the speed of light.

        Couldn’t resist being a bit of a stickler but 🤓 erm… technically it is moving at the speed of light through a medium, which is slightly less than c, the speed of light in a vacuum. Fun fact, when things move faster than the speed of light through a medium - such as water - it produces Cherenkov radiation, the glowing blue light associated with some nuclear reactors, which is sorta like a sonic boom but with light instead of sound.

        source
        • -> View More Comments
      • meme_historian@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

        Also would it really be Random Access Memory? Seems like we would have to optimize a lot of things for sequential data access

        source
      • vacuumflower@lemmy.sdf.org ⁨1⁩ ⁨month⁩ ago

        Also optical fiber is used a lot on battlefields now. It just remains there. There’s a lot to be assembled.

        source
    • mrnobody@reddthat.com ⁨1⁩ ⁨month⁩ ago

      Then all the necessary mineral prices will shoot up 3,648%.

      source
      • Goodlucksil@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

        Is that a decimal comma or a digit separator comma

        source
        • -> View More Comments
  • brucethemoose@lemmy.world ⁨1⁩ ⁨month⁩ ago

    The issue with AI is “now”

    Can they power with solar? Nuclear? Hell, even a natural gas plant? Nope, the data centers need the power right this second, so they get gas turbines on site. Same with cooling; evaporative is just the quickest and cheapest to set up.

    Same with its architecture. There’s no time to fix temperature/sampling issues, no time to try bitnet or any of a bazillion interesting papers that came out. A shippable product (model) is needed yesterday; just scale up what we have. “Fail” a single experiment? Your team is fired, which is exactly what happened at Meta.

    Everything has to happen right now because of corporate FOMO. So, while this is an interesting musing and maybe Intel or someone will play with it, the actual AI labs could not care less because they can’t get it immediately.

    source
  • RobotToaster@mander.xyz ⁨1⁩ ⁨month⁩ ago

    I’m pretty sure 200km of fibre isn’t going to be cheap either

    source
    • adespoton@lemmy.ca ⁨1⁩ ⁨month⁩ ago

      Fibre is just strands of extruded glass; one of the most common substances on earth.

      Sure beats the blood minerals needed for memory, and to scale up, you just extrude longer strands.

      source
    • e8CArkcAuLE@piefed.social ⁨1⁩ ⁨month⁩ ago

      there is a bit of surplus of fibre wire in Ukraine, i hear… /s

      source
      • UniversalBasicJustice@quokk.au ⁨1⁩ ⁨month⁩ ago

        I’m not sure which job sounds less appealing; collecting it or splicing it

        source
    • acosmichippo@lemmy.world ⁨1⁩ ⁨month⁩ ago

      could be cheaper than enterprise grade DIMMs.

      source
    • CosmoNova@lemmy.world ⁨1⁩ ⁨month⁩ ago

      Probably cheaper than tens of thousands of satellites.

      source
  • solrize@lemmy.ml ⁨1⁩ ⁨month⁩ ago

    Delay line memory in gigabytes? Bold indeed.

    source
  • just_another_person@lemmy.world ⁨1⁩ ⁨month⁩ ago

    This is… incredibly stupid. This man has done so many drugs he no longer realizes how computers or electricity works.

    source
    • Shadow@lemmy.ca ⁨1⁩ ⁨month⁩ ago

      en.wikipedia.org/wiki/Delay-line_memory

      source
      • nova_ad_vitum@lemmy.ca ⁨1⁩ ⁨month⁩ ago

        The lack of investment in more production capacity for RAM is based on a roughly 3-year horizon for this insane extra AI demand.

        Creating workable consumer-grade alternatives with delay line memory of all things would take longer than that, and the market would collapse the moment AI demand for RAM dried up. This is one of those things that is theoretically possible but due to both technology and market conditions will absolutely not be a thing.

        source
        • -> View More Comments
      • just_another_person@lemmy.world ⁨1⁩ ⁨month⁩ ago

        It’s not that we don’t know what it is, it, again, is just INCREDIBLY FUCKING STUPID.

        source
    • doorknob88@lemmy.world ⁨1⁩ ⁨month⁩ ago

      John Karmack is on drugs?

      source
      • greybeard@feddit.online ⁨1⁩ ⁨month⁩ ago

        Maybe he confused him with John McAfee.

        source
  • whaleross@lemmy.world ⁨1⁩ ⁨month⁩ ago

    Throw in some AI and a Blockchain and you’ll get the cryptobros hooked. Then use it to store NFTs.

    source
    • boonhet@sopuli.xyz ⁨1⁩ ⁨month⁩ ago

      It’s being proposed for AI literally. As in AI doesn’t TECHNICALLY need RAM, it could also use SAM and this stuff could provide excellent sequential access performance.

      source
  • ms_lane@lemmy.world ⁨1⁩ ⁨month⁩ ago

    It’s an interesting idea, but what’s the floor size for a pair of 200TB/s fibre transceivers vs. 32GB of HBM?

    It’s it’s not significantly less, this doesn’t seem like it’d be particularly helpful outside the 200TB/s of streaming data.

    source
    • tal@lemmy.today ⁨1⁩ ⁨month⁩ ago

      I’m assuming that the point is the bandwidth.

      goes looking for HBM bandwidth

      en.wikipedia.org/wiki/High_Bandwidth_Memory

      It says that HBM 4, which is coming out this month, can do 2 TiB/s.

      source
  • eleitl@lemmy.zip ⁨1⁩ ⁨month⁩ ago

    They never mention the word latency even once. It’s a delay line SAM and speed of light in glass is some 200000 km/s. This is hard drive latency.

    source
  • tal@lemmy.today ⁨1⁩ ⁨month⁩ ago

    Note that this is from last month, though I haven’t seen it submitted.

    source
  • NotMyOldRedditName@lemmy.world ⁨1⁩ ⁨month⁩ ago

    So we’ll soon have houses built with a place to hold a spool of 200km multi fiber cable (which shouldn’t be too big, Ukraine drones carry 40km worth of single strand) and we can plug out computers into it.

    source
    • KairuByte@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

      You can carry multiple wavelengths over a single strand.

      source
      • NotMyOldRedditName@lemmy.world ⁨1⁩ ⁨month⁩ ago

        The article was saying the spool would give them 32gb, hence the multi strand thought. Were going to want hundreds of gb to run a decent model.

        source
  • architect@thelemmy.club ⁨1⁩ ⁨month⁩ ago

    No. I will always have computers. Fuck you.

    source
  • sturmblast@lemmy.world ⁨1⁩ ⁨month⁩ ago

    Save us John

    source
    • ooterness@lemmy.world ⁨1⁩ ⁨month⁩ ago

      Were you taking to John Carmack or John Connor?

      source
      • KillerWhale@orcas.enjoying.yachts ⁨1⁩ ⁨month⁩ ago

        Yes

        source
      • Sxan@piefed.zip ⁨1⁩ ⁨month⁩ ago

        I prefer þe real John… John Carter.

        source
  • humanspiral@lemmy.ca ⁨1⁩ ⁨month⁩ ago

    while bandwidth is high, storage is low. Even dropping speed to 10Tb/sec, it would mean 1.25GB of effective ram.

    source
  • sepi@piefed.social ⁨1⁩ ⁨month⁩ ago

    Random Access how?

    source
    • boonhet@sopuli.xyz ⁨1⁩ ⁨month⁩ ago

      If you read the article, it’s sequential access but that’s fine for AI use.

      source
      • sepi@piefed.social ⁨1⁩ ⁨month⁩ ago

        I read the article title and it said RAM. Now you’re trying to pull a sam altman bamboozle - “it’s not random, it’s sequential” - then it ain’t RAM.

        Fuck the law and fuck the article yeehaw

        source
        • -> View More Comments
  • geekwithsoul@piefed.social ⁨1⁩ ⁨month⁩ ago

    I don’t pretend to understand how this would actually work, but wouldn’t this essentially be like token ring networking but used as memory?

    source
    • cmnybo@discuss.tchncs.de ⁨1⁩ ⁨month⁩ ago

      It’s delay line memory. It was common back in the days of vacuum tube computers.

      source
    • tal@lemmy.today ⁨1⁩ ⁨month⁩ ago

      A little bit, but normally Token Ring didn’t just keep data running around in a circle on and on — Token Ring works more like a roundabout, where you enter at a given computer on the ring and then exit at another device. Without looking, I suspect that, like Internet Protocol packets, Token Ring probably had a TTL (time-to-live) field in its packets.

      Also, I’m assuming that an implementation of Carmack’s idea would have only one…I don’t know the right term, might be “repeater”. You need to have some device to receive the data and then retransmit them to keep the signal strong and from spreading out. You wouldn’t want to have a ton of those, because otherwise it’d add cost. On Token Ring, you’d have a bunch.

      source