Technus

@Technus@lemmy.zip

• ⁨Comment⁩ on ⁨Nvidia CEO Jensen Huang says ‘I think we’ve achieved AGI’⁩ ⁨⁨4⁩ ⁨days⁩ ago⁩:

  Not sure what this internal state you are referring to is. Are you talking about all the values that come out of each step of the computations?

  It would need to be able to form memories like real brains do, by creating new connections between neurons and adjusting their weights in real time in response to stimuli, and having those connections persist. I think that’s a prerequisite to models that are capable of higher-level reasoning and understanding. But then you would need to store those changes to the model for each user, which would be tens or hundreds of gigabytes.

  These current once-through LLMs don’t have time to properly digest what they’re looking at, because they essentially forget everything once they output a token. I don’t think you can make up for that by spitting some tokens out to a file and reading them back in, because it still has to be human-readable and coherent. That transformation is inherently lossy.

  This is basically what I’m talking about: https://www.comicagile.net/comic/context-switching/

  But for every single token the LLM outputs. The fact that it’s allowed to take notes is a mitigation for this context loss, not a silver bullet.

• ⁨Comment⁩ on ⁨OpenAI Will Shut Down Sora Video Platform⁩ ⁨⁨4⁩ ⁨days⁩ ago⁩:

  There are thousands of starving furry artists out there who would be happy to take your commission.

• ⁨Comment⁩ on ⁨Nvidia CEO Jensen Huang says ‘I think we’ve achieved AGI’⁩ ⁨⁨5⁩ ⁨days⁩ ago⁩:

  The conversion of the output to tokens inherently loses a lot of the information extracted by the model and any intermediate state it has synthesized (what it “thinks” of the input).

  Until the model is able to retain its own internal state and able to integrate new information into that state as it receives it, all it will ever be able to do is try to fill in the blanks.

• ⁨Comment⁩ on ⁨Nvidia CEO Jensen Huang says ‘I think we’ve achieved AGI’⁩ ⁨⁨5⁩ ⁨days⁩ ago⁩:

  The size of the context window is fixed in the structure of the model. LLMs are still at their core artificial neural networks, so an analogy to biology might be helpful.

  Think of the input layer of the model like the retinas in your eyes. Each token in the context window, after embedding (i.e. conversion to a series of numbers, because ofc it’s just all math under the hood), is fed to a certain set of input neurons, just like the rods and cones in your retina capture light and convert it to electrical signals, which are passed to neurons in your optic nerve, which connect to neurons in your visual cortex, each layer along the way processing and analyzing the signal.

  The number of tokens in the context window is directly proportional to the number of neurons in the input layer of the model. To make the context window bigger, you have to add more neurons to the input layer, but that quickly results in diminishing returns without adding more neurons to the inner layers to be able to process the extra information. Ultimately, you have to make the whole model larger, which means more parameters, which means more data to store and more processing power per prompt.

• ⁨Comment⁩ on ⁨OpenAI Will Shut Down Sora Video Platform⁩ ⁨⁨5⁩ ⁨days⁩ ago⁩:

  I’d bet money that the Disney deal falling through was because OpenAI couldn’t guarantee that Sora couldn’t be used to generate porn of their characters, since attackers will almost certainly always find new prompt injections.

  Surprise surprise, it’s a giant fucking black box that you can never have complete control over.

• ⁨Comment⁩ on ⁨Nvidia CEO Jensen Huang says ‘I think we’ve achieved AGI’⁩ ⁨⁨6⁩ ⁨days⁩ ago⁩:

  If you prompted an LLM to review all of it’s database entries, generate a new response based on that data, then save that output to the database and repeat at regular intervals, I could see calling that a kind of thinking.

  That’s kind of what the current agentic AI products like Claude Code do. The problem is context rot. When the context window fills up, the model loses the ability to distinguish between what information is important and what’s not, and it inevitably starts to hallucinate.

  The current fixes are to prune irrelevant information from the context window, use sub-agents with their own context windows, or just occasionally start over from scratch. They’ve also developed conventional AGENTS.md and CLAUDE.md files where you can store long-term context and basically “advice” for the model, which is automatically read into the context window.

  However, I think an AGI inherently would need to be able to store that state internally, to have memory circuits, and “consciousness” circuits that are connected in a loop so it can work on its own internally encoded context. And ideally it would be able to modify its own weights and connections to “learn” in real time.

  The problem is that would not scale to current usage because you’d need to store all that internal state, including potentially a unique copy of the model, for every user. And the companies wouldn’t want that because they’d be giving up control over the model’s outputs since they’d have no feasible way to supervise the learning process.

• ⁨Comment⁩ on ⁨Nvidia CEO Jensen Huang says ‘I think we’ve achieved AGI’⁩ ⁨⁨6⁩ ⁨days⁩ ago⁩:

  I only have a rather high level understanding of current AI models, but I don’t see any way for the current generation of LLMs to actually be intelligent or conscious.

  They’re entirely stateless, once-through models: any activity in the model that could be remotely considered “thought” is completely lost the moment the model outputs a token. Then it starts over fresh for the next token with nothing but the previous inputs and outputs (the context window) to work with.

  That’s why it’s so stupid to ask an LLM “what were you thinking”, because even it doesn’t know! All it’s going to do is look at what it spat out last and hallucinate a reasonable-sounding answer.

• ⁨Comment⁩ on ⁨Call me Barbie the way I lust after Ken Thompson.⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:

  No, but that won’t stop me from having fun trying, will it?

• ⁨Comment⁩ on ⁨Cryptographers engage in war of words over RustSec bug reports and subsequent ban⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:

  “The nonce reuse issue seems to be a valid security issue, but it is by no means a critical vulnerability: it only affects applications that do more than four billion encryptions with a single HPKE setup,” said Valsorda. “The average application does one.”

  No implementation should be using the same asymmetric keypair more than once. This is such a non-issue that it’s kind of hilarious. Sounds like the reporter was trying so desperately to get credit for anything they could put on their portfolio, and just wouldn’t take “no” for an answer.

• ⁨Comment⁩ on ⁨I was told it would be a cultural experience⁩ ⁨⁨3⁩ ⁨weeks⁩ ago⁩:

  Buc-ees is the only good thing about Texas.

• ⁨Comment⁩ on ⁨NVIDIA could enter the desktop CPU market with performance equal to AMD and Intel⁩ ⁨⁨3⁩ ⁨weeks⁩ ago⁩:

  Yeah, that was my question. Why the hell would they develop new silicon when 99% of their fab space is dedicated to feeding the AI bubble?

• ⁨Comment⁩ on ⁨Netflix's $82.7 billion deal to buy Warner has been dramatically gazumped by Paramount, to the tune of $111 billion⁩ ⁨⁨4⁩ ⁨weeks⁩ ago⁩:

  There’s no way in hell Paramount has that much cash available. This is gonna be a leveraged buyout of epic proportions, and it’s gonna crash and burn epically too.

• ⁨Comment⁩ on ⁨Interesting question. For me it would be a dead grandfather⁩ ⁨⁨4⁩ ⁨weeks⁩ ago⁩:

  My dad tells these stories about how my great grandpa used to joke and pull pranks on the family. I wish I could have met him in person.

  That or getting an hour to pick Alan Turing’s brain. He’d probably hate the current generation of “AI” as much as I do.

• ⁨Comment⁩ on ⁨systemd has been a complete, utter, unmitigated success⁩ ⁨⁨5⁩ ⁨weeks⁩ ago⁩:

  I honestly don’t get what people were so up in arms about, besides just not wanting to change what already worked for them.

• ⁨Comment⁩ on ⁨Kiss goodbye to 8K as support from the TV industry 'dwindles'⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:

  It’s because we’re at the limits of the human visual system. The difference in pixel pitch between 4k and 8k at the distances we watch TV is literally imperceptible.

  It also doesn’t help that there’s not much content authored and distributed for higher resolutions. It’s exponentially more expensive to produce, store, and deliver.

  Home Internet connections on average aren’t any better than they were ten years ago, either, at least not in the US. I doubt a lot of them can even support 8k streaming, let alone with anyone else using it at the same time.

• ⁨Comment⁩ on ⁨Chrome is also turning into an agentic browser with its newest update⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:

  Yeah, I realized that as soon as I posted it.

• ⁨Comment⁩ on ⁨Chrome is also turning into an agentic browser with its newest update⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:

  You know what this is launching just in time for?

  Tax season.

  Queue 10 million people getting audited because they let their browser file their taxes for them.

• ⁨Comment⁩ on ⁨Fluid gears rotate without teeth, offering new mechanical flexibility⁩ ⁨⁨2⁩ ⁨months⁩ ago⁩:

  It’s possible to design fluid couplings for torque. That’s what a torque converter in an automatic transmission does.

  It’s not quite as elegant, of course, but still.

• ⁨Comment⁩ on ⁨Such a dreamy guy⁩ ⁨⁨2⁩ ⁨months⁩ ago⁩:

  Joshington?

  Jhleswig-Holstein?

• ⁨Comment⁩ on ⁨Definitely the safest source for advice⁩ ⁨⁨2⁩ ⁨months⁩ ago⁩:

  I think some people are so eager to offload all critical thinking to the machine because they’re barely capable of it themselves to begin with.

• ⁨Comment⁩ on ⁨AI’s Memorization Crisis | Large language models don’t “learn”—they copy. And that could change everything for the tech industry.⁩ ⁨⁨2⁩ ⁨months⁩ ago⁩:

  It’s glorified autocorrect (/predictive text).

  People fight me on this every time I say it but it’s literally doing the same thing just with much further lookbehind.

  In fact, there’s probably a paper to be written about how LLMs are just lossily compressed Markov chains.

• ⁨Comment⁩ on ⁨Journalistic Malpractice: No LLM Ever ‘Admits’ To Anything, And Reporting Otherwise Is A Lie⁩ ⁨⁨2⁩ ⁨months⁩ ago⁩:

  I fucking hate when people ask an LLM “what were you thinking” because the answer is meaningless, and it just showcases how little people understand of how they actually work.

  Any activity inside the model that could be considered any remote approximation of “thought” is completely lost as soon as it outputs a token. The only memory it has is the context window, the past history of inputs and outputs.

  All it’s going to do when you ask it that, is it’s looking over the past output and attempting to rationalize what it previously output.

  And actually, even that is excessively anthropomorphizing the model. In reality, it’s just generating a plausible response to the question “what were you thinking”, given the history of the conversation.

  I fucking hate this version of “AI”. I hate how it’s advertised. I hate the managers and executives drinking the Kool-Aid. I hate that so much of the economy is tied up in it. I hate that it has the energy demand and carbon footprint of a small nation-state. It’s absolute insanity.

• ⁨Comment⁩ on ⁨Breed back better, or whatever Biden said⁩ ⁨⁨2⁩ ⁨months⁩ ago⁩:

  Great source of protein crawling there across the ground, it’d be a shame if someone wasn’t there to eat it

• ⁨Comment⁩ on ⁨Welcome to the thunderdome?⁩ ⁨⁨2⁩ ⁨months⁩ ago⁩:

  That’s probably based on the first definition because you can play either an ascending or descending scale.

  Also the music staff kinda looks like a ladder.

• ⁨Comment⁩ on ⁨On dasher!⁩ ⁨⁨2⁩ ⁨months⁩ ago⁩:

  Didn’t know doordashers could contact you through whatsapp

• ⁨Comment⁩ on ⁨It's not a whack jack, it's a...⁩ ⁨⁨3⁩ ⁨months⁩ ago⁩:

  I was thinking about this the other day and realized something:

  Back when the modern Santa character was first being developed, coal was a genuinely useful thing. It was fuel for the stove which heated your house and cooked your food. It was a basic necessity of life.

  If you were naughty, Santa didn’t just give you nothing. You weren’t going to get an awesome toy, but he made sure you weren’t going to freeze to death on Christmas, either.

  Santa believes everyone deserves to live. That having a warm place to sleep is a basic human right.

  This might be /r/im14andthisisdeep material, but I just thought that was interesting.

• ⁨Comment⁩ on ⁨WHO STOLE MY FUCKING BARS?⁩ ⁨⁨3⁩ ⁨months⁩ ago⁩:

  Yeah but on the second incarnation, wouldn’t that put you right back where you started?

• ⁨Comment⁩ on ⁨WHO STOLE MY FUCKING BARS?⁩ ⁨⁨3⁩ ⁨months⁩ ago⁩:

  What happens to the guy that was driving it? Does he just blink out of existence when the car shuts off? That’s my question. You might argue that there is no such thing, but my own conscious experience proves to myself that there’s something else there. I want to know what happens to that part.

  Hell, for all I know, you might just be a soulless meatbag automaton, and there really is no one in the driver’s seat for you. Or I could just be the only actual human talking in a thread full of bots. With 90% of the training data going into LLMs being vapid contrarian debates on social media, I could easily see that being the case here.

• ⁨Comment⁩ on ⁨WHO STOLE MY FUCKING BARS?⁩ ⁨⁨3⁩ ⁨months⁩ ago⁩:

  I’m not expecting or planning for anything, that’s kind of the point. I’m not expecting one specific outcome. It’s actually really freeing, because I’m not stuck searching for meaning in an existence that offers none.

  And if it turns out that it does all just go black, it won’t be my problem anymore, will it?

• ⁨Comment⁩ on ⁨WHO STOLE MY FUCKING BARS?⁩ ⁨⁨3⁩ ⁨months⁩ ago⁩:

  I don’t agree that the cessation of brain activity necessarily means the end of the subjective experience. That doesn’t mean I purport to know what actually happens at that point. I hope it’s some sort of reincarnation but that’s just because there’s more I want to experience in this universe than I possibly could in a single lifetime.

  “You only have one life, live it the best you can” is a nice motivational mantra, but however well I live my life, it’s highly unlikely I will live long enough to experience interstellar travel, for example, or first contact with alien life. I think that really fucking sucks, and I really hope I’ll have a chance on the next go-around. But if it’s something completely different, I’m cool with that, too.