Comment on China solves 'century-old problem' with new analog chip that is 1,000 times faster than high-end Nvidia GPUs

Quazatron@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

• nymnympseudonym@piefed.social ⁨3⁩ ⁨weeks⁩ ago

  You might benefit from watching Hinton’s lecture; much of it details technical reasons why digital is much much better than analog for intelligent systems

  BTW that is the opposite of what he set out to prove
  He says the facts forced him to change his mind

  https://m.youtube.com/watch?v=IkdziSLYzHw

  source

  • CeeBee_Eh@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

    much of it details technical reasons why digital is much much better than analog for intelligent systems

    For current LLMs there would be a massive gain in energy efficiency if analogue computing was used. Much of the current energy costs come from stimulating what effectively analogue processing on digital hardware. There’s a lot lost in the conversation, or “emulation” of analogue.

    source
  • yeahiknow3@lemmy.dbzer0.com ⁨3⁩ ⁨weeks⁩ ago

    I wish researchers like Hinton would stick to discussing the tech. Anytime he says anything about linguistics or human intelligence he sounds like a CS major smugly raising his hand in Phil 101 while the rest of the class groans.

    Again, I respect Hinton as a computer scientist. But the guy is philosophically illiterate. I am not exaggerating.

    source
  • Quazatron@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

    Thank you for the link, it was very interesting.

    Even though analogue neural networks have the drawback that you can’t copy the neuron weights (currently, but tech may evolve to do it), they can still have use cases in lower powered edge devices.

    I think we’ll probably end up with hybrid designs, using digital for most parts except the calculations.

    source

    • nymnympseudonym@piefed.social ⁨3⁩ ⁨weeks⁩ ago

      For low power neural nets look up “spiking neural networks”

      source
• bulwark@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  That’s a good point. The model weights could be voltage levels instead of digital representations. Lots of audio tech uses analog for better fidelity.I also read that there’s a startup using particle beams for lithography. Exciting times.

  source

  • nymnympseudonym@piefed.social ⁨3⁩ ⁨weeks⁩ ago

    At least one Nobel Laureate had exactly the opposite opinion (see the Hinton lecture above)

    source
  • vrighter@discuss.tchncs.de ⁨3⁩ ⁨weeks⁩ ago

    what audio tech uses analog for better fidelity?

    source

    • bulwark@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

      Vinyl record, analog tube amplifiers, a good part of speakers 🤌

      Honestly though digital compression now is so good it probably sounds the same.

      source

      • vrighter@discuss.tchncs.de ⁨3⁩ ⁨weeks⁩ ago

        speakers are analog devices by nature.

        The other two are used for the distortions they introduce, so quite literally lower fidelity. Whether some people like those distortions is irrelevant.

        You want high fidelity: lossless digital audio formats.

        source

        • -> View More Comments
• Trainguyrom@reddthat.com ⁨3⁩ ⁨weeks⁩ ago

  That and the way companies have been building AI they have been doing so little to optimize compute to instead try to get the research out faster because that’s what is expected in this bubble. I’m absolutely fully expecting to see future research finding plenty of ways to optimize these major models.

  But also R&D has been entirely focused on digital chips I would not be at all surprised if there were performance and/or efficiency gains to be had in certain workloads by shifting to analog circuits

  source