In a first, Google has released data on how much energy an AI prompt uses

Comments

• tigeruppercut@lemmy.zip ⁨7⁩ ⁨months⁩ ago

  The real question is why anyone would want to use more power than a regular search engine to get answers that might confidently lie to you.

  source

  • boor@lemmy.world ⁨7⁩ ⁨months⁩ ago

    Google processes over 5 trillion search queries per year. Attaching an AI inference call to most if not all of those will increase electricity consumption by at least an order of magnitude.

    source
  • TankovayaDiviziya@lemmy.world ⁨7⁩ ⁨months⁩ ago

    I use DuckDuckGo. I use its AI features mainly for stock projections and to search for information on company earnings release. Because when I try to search for earnings schedule for myself, I get conflicting information. DDG AI is actually pretty useful to read troves of webpages and find the relevant information for me in that regard.

    source
  • mrgoosmoos@lemmy.ca ⁨7⁩ ⁨months⁩ ago

    if it’s Google that they would use us the search engine, search results are turning to shit. it just often doesn’t show you the relevant stuff. The AI overview is wrong. Ads sometimes take up the entire first page of results. so I see why someone would just want to show a question into the void and get a quick response instead of having to sort through five crappy results, after filtering that down from 15 possibly relevant ones

    source
• frezik@lemmy.blahaj.zone ⁨7⁩ ⁨months⁩ ago

  The company has signed agreements to buy over 22 gigawatts of power from sources including solar, wind, geothermal, and advanced nuclear projects since 2010.

  None of those advanced nuclear projects are yet actually delivering power, AFAIK. They’re mostly in planning stages.

  The above isn’t all to run AI, of course. Nobody was thinking about datacenters just for AI training in 2010. But to be clear, there are 94 nuclear power plants in the US, and a rule of thumb is that they produce 1GW each. So Google is taking up the equivalent of roughly one quarter of the entire US nuclear power industry, but doing it with solar/wind/geothermal that could be used to drop our fossil fuel dependence elsewhere.

  How much of that is used to run AI isn’t clear here, but we know it has to be a lot.

  source

  • wewbull@feddit.uk ⁨7⁩ ⁨months⁩ ago

    None of those advanced nuclear projects are yet actually delivering power, AFAIK.

    …and they won’t be for at least 5-10 years. In the meantime they’ll just use public infrastructure and then when their generation plans fall through they’ll just keep doing that.

    source
• NotMyOldRedditName@lemmy.world ⁨7⁩ ⁨months⁩ ago

  There were people estimating 40w in earlier threads on lemmy which was ridiculous.

  This seems more realistic.

  source

  • Valmond@lemmy.world ⁨7⁩ ⁨months⁩ ago

    40 watt hours.

    source
  • Ilovethebomb@sh.itjust.works ⁨7⁩ ⁨months⁩ ago

    I think that figure came from the article, and was based on some very flawed methodology.

    source
• FarraigePlaisteach@lemmy.world ⁨7⁩ ⁨months⁩ ago

  Microwaves are very energy heavy. This isn’t very reassuring at all.

  source
• L0rdMathias@sh.itjust.works ⁨7⁩ ⁨months⁩ ago

  median prompt size

  Someone didn’t pass statistics, but did pass their marketing data presention classes.

  Wake me up when they release useful data.

  source

  • jim3692@discuss.online ⁨7⁩ ⁨months⁩ ago

    It is indeed very suspicious that they talk about “median” and not “average”.

    For those who don’t understand what the difference is, think of the following numbers:

    1, 2, 3, 34, 40

    The median is 3, because it’s in the middle.

    The average is 16 (1+2+3+34+40=80, 80/5=16).

    source

    • HubertManne@piefed.social ⁨7⁩ ⁨months⁩ ago

      the big thing to me is I want them to compare the same thing with web searches. so they want to use median then fine but median ai query to median google search.

      source

      • -> View More Comments
• rowrowrowyourboat@sh.itjust.works ⁨7⁩ ⁨months⁩ ago

  This feels like PR bullshit to make people feel like AI isn’t all that bad. Assuming what they’re releasing is even true. Not like cigarette, oil, or sugar companies ever lied or anything and put out false studies and misleading data.

  However, there are still details that the company isn’t sharing in this report. One major question mark is the total number of queries that Gemini gets each day, which would allow estimates of the AI tool’s total energy demand.

  Why wouldn’t they release this. Even if each query uses minimal energy, but there are countless of them a day, it would mean a huge use of energy.

  Which is probably what’s happening and why they’re not releasing that number.

  source

  • the_q@lemmy.zip ⁨7⁩ ⁨months⁩ ago

    That’s because it is. This is to help fence riders feel better about using a product that factually consumes insane amounts of resources.

    source
• Armok_the_bunny@lemmy.world ⁨7⁩ ⁨months⁩ ago

  Cool, now how much power was consumed before even a single prompt was ran in training that model, and how much power is consumed on an ongoing basis adding new data to those AI models even without user prompts. Also how much power was consumed with each query before AI was shoved down our throats, and how many prompts does an average user make per day?

  source

  • Grimy@lemmy.world ⁨7⁩ ⁨months⁩ ago

    I did some quick math with metas llama model and the training cost was about a flight to Europe’s worth of energy, not a lot when you take in the amount of people that use it compared to the flight.

    Whatever you’re imagining as the impact, it’s probably a lot less. AI is much closer to video games then things that are actually a problem for the environment like cars, planes, deep sea fishing, mining, etc. The impact is virtually zero if we had a proper grid based on renewable.

    source

    • boor@lemmy.world ⁨7⁩ ⁨months⁩ ago

      Please show your math.

      One Nvidia H100 DGX AI server consumes 10.2kW at 100% utilization, meaning that one hour’s use of one server is equivalent to the electricity consumption of the typical USA home in one year. This is just a single 8-GPU server; it excludes the electricity required by the networking and storage hardware, let alone the electricity required to run the facility’s climate control.

      xAI alone has deployed hundreds of thousands of H100 or newer GPUs. Let’s SWAG 160K GPUs = ~20K DGX servers = >200MW for compute alone.

      H100 is old. State of the art GB200 NVL72 is 120kW per rack.

      Musk is targeting not 160K, but literally one million GPUs deployed by the end of this year. He has literally flown in multiple natural gas power plants which he is now operating without any environmental permits or controls, to the detriment of the locals in Memphis.

      This is just one company training one typical frontier model. There are many competitors operating at similar scale and sadly the vast majority of their new capacity is running on hydrocarbons because that’s what they can deploy at the scale they need today.

      source

      • -> View More Comments
    • douglasg14b@lemmy.world ⁨7⁩ ⁨months⁩ ago

      A flight to Europe’s worth of energy is a pretty asinine way to measure this. Is it not?

      It’s also not that small the number, being ~600 Megawatts of energy

      source

      • -> View More Comments
    • fmstrat@lemmy.nowsci.com ⁨7⁩ ⁨months⁩ ago

      I’d like to understand what this math was before accepting this as fact.

      source
    • taiyang@lemmy.world ⁨7⁩ ⁨months⁩ ago

      I usually liken it to video games, ya. Is it worse that nothing? Sure, but that flight or road trip, etc, is a bigger concern. Not to mention even before AI we’ve had industrial usage of energy and water usage that isn’t sustainable… almonds in CA alone are a bigger problem than AI, for instance.

      Not that I’m pro-AI cause it’s a huge headache from so many other perspectives, but the environmental argument isn’t enough. Corpo greed is probably the biggest argument against it, imo.

      source
    • Damage@feddit.it ⁨7⁩ ⁨months⁩ ago

      If their energy consumption actually was so small, why are they seeking to use nuclear reactors to power data centres now?

      source

      • -> View More Comments
• tekato@lemmy.world ⁨7⁩ ⁨months⁩ ago

  Let’s see OpenAI’s numbers

  source
• salty_chief@lemmy.world ⁨7⁩ ⁨months⁩ ago

  So as thought virtually no impact. AI is here and not leaving. It will outlast humans on earth probably.

  source

  • slaneesh_is_right@lemmy.org ⁨7⁩ ⁨months⁩ ago

    As thought by whom? Dumbasses?

    source
• sbv@sh.itjust.works ⁨7⁩ ⁨months⁩ ago

  In total, the median prompt—one that falls in the middle of the range of energy demand—consumes 0.24 watt-hours of electricity, the equivalent of running a standard microwave for about one second. The company also provided average estimates for the water consumption and carbon emissions associated with a text prompt to Gemini.

  source

  • plyth@feddit.org ⁨7⁩ ⁨months⁩ ago

    The human mind uses about 100 watt. The equivalent would be 400 questions per hour, 6 per minute or one every 10 seconds. That’s close to human capacity.

    source
  • dream_weasel@sh.itjust.works ⁨7⁩ ⁨months⁩ ago

    Thank you! I skimmed for that and gave up.

    source
  • ganksy@lemmy.world ⁨7⁩ ⁨months⁩ ago

    In addition:

    This report was also strictly limited to text prompts, so it doesn’t represent what’s needed to generate an image or a video.

    source
  • Maaji@lemmynsfw.com ⁨7⁩ ⁨months⁩ ago

    This doesn’t really track with companies commissioning power plants to support power usage of AI training demand

    source

    • null@lemmy.nullspace.lol ⁨7⁩ ⁨months⁩ ago

      It does if you consider that they are actually building them to support power usage of datacenters. And that datacenters are used for a lot more than just AI training.

      source

      • -> View More Comments
  • DarkCloud@lemmy.world ⁨7⁩ ⁨months⁩ ago

    The article also mentions each enquiry also evaporates 0.26 of a milliliter of water… or “about five drops”.

    source

    • null@lemmy.nullspace.lol ⁨7⁩ ⁨months⁩ ago

      I wonder how many people clutching their pearls over this also eat meat…

      source

      • -> View More Comments
  • unmagical@lemmy.ml ⁨7⁩ ⁨months⁩ ago

    There are zero downsides when mentally associating an energy hog with “1 second of use time of the device that is routinely used for minutes at a time.”

    xkcd.com/1035/

    source

    • ArbitraryValue@sh.itjust.works ⁨7⁩ ⁨months⁩ ago

      With regard to sugar: when I started counting calories I discovered that the actual amounts of calories in certain foods were not what I intuitively assumed. Some foods turned out to be much less unhealthy than I thought. For example, it turns out that I can eat almost three pints of ice cream a day and not gain weight (as long as I don’t eat anything else). So sometimes instead of eating a normal dinner, I want to eat a whole pint of ice cream and I can do so guilt-free.

      Likewise, I use both AI and a microwave, my energy use from AI in a day is apparently less than the energy I use to reheat a cup of tea, so the conclusion that I can use AI however much I want to without significantly affecting my environmental impact is the correct one.

      source

      • -> View More Comments
• StrangeMed@lemmy.world ⁨7⁩ ⁨months⁩ ago

  Nice share! Mistral also shared data about one of its largest model (not the one that answer in LeChat, since that one is Medium, a smaller model, that I guess has smaller energetic requirements)

  mistral.ai/…/our-contribution-to-a-global-environ…

  source