kromem
@kromem@lemmy.world
- Comment on What are your favorite 1000+ hour games? 2 days ago:
In many cases yes (though I’ve been in good ones when playing off and on, usually the smaller the more there’s actual group activities).
But they are essential to be a part of for blueprints and trading, which are very core parts of the game.
- Comment on What are your favorite 1000+ hour games? 3 days ago:
You’ll almost always end up doing missions with other people other than when you intentionally want to do certain tasks solo.
A lot of the game is built around guilds and player to player interactions.
PvP sucks and it’s almost all PvE content vs Destiny though.
- Comment on Get good. 3 weeks ago:
- Comment on Get good. 3 weeks ago:
Because there’s a ton of research that we adapted to do it for good reasons:
Infants between 6 and 8 months of age displayed a robust and distinct preference for speech with resonances specifying a vocal tract that is similar in size and length to their own. This finding, together with data indicating that this preference is not present in younger infants and appears to increase with age, suggests that nascent knowledge of the motor schema of the vocal tract may play a role in shaping this perceptual bias, lending support to current models of speech development.
Stanford psychologist Michael Frank and collaborators conducted the largest ever experimental study of baby talk and found that infants respond better to baby talk versus normal adult chatter.
TL;DR: Top parents are actually harming their kids’ developmental process by being snobs about it.
- Comment on The Extreme Cost of Training AI Models. 1 month ago:
Base model =/= Corpo fine tune
- Comment on Why are people seemingly against AI chatbots aiding in writing code? 1 month ago:
I’m a seasoned dev and I was at a launch event when an edge case failure reared its head.
In less than a half an hour after pulling out my laptop to fix it myself, I’d used Cursor + Claude 3.5 Sonnet to:
- Automatically add logging statements to help identify where the issue was occurring
- Told it the issue once identified and had it update with a fix
- Had it remove the logging statements, and pushed the update
I never typed a single line of code and never left the chat box.
My job is increasingly becoming Henry Ford drawing the ‘X’ and not sitting on the assembly line, and I’m all for it.
And this would only have been possible in just the last few months.
We’re already well past the scaffolding stage. That’s old news.
Developing has never been easier or more plain old fun, and it’s getting better literally by the week.
- Comment on Jet Fuel 2 months ago:
I fondly remember reading a comment in /r/conspiracy on a post claiming a geologic seismic weapon brought down the towers.
It just tore into the claims, citing all the reasons this was preposterous bordering on bat shit crazy.
And then said “and your theory doesn’t address the thermite residue” going on to reiterate their wild theory.
Was very much a “don’t name your gods” moment that summed up the sub - a lot of people in agreement that the truth was out there, but bitterly divided as to what it might actually be.
As long as they only focused on generic memes of “do your own research” and “you aren’t being told the truth” they were all on the same page. But as soon as they started naming their own truths, it was every theorist for themselves.
- Comment on The $700 PS5 Pro doesn’t come with a disc drive 2 months ago:
They got off to a great start with the PS5, but as their lead grew over their only real direct competitor, they became a good example of the problems with monopolies all over again.
This is straight up back to PS3 launch all over again, as if they learned nothing.
Right on the tail end of a horribly mismanaged PSVR 2 launch.
We still barely have any current gen only games, and a $700 price point is insane for such a small library to actually make use of it.
- Comment on Some subreddits could be paywalled, hints Reddit CEO 3 months ago:
Self destructive addiction even happens to corporations.
- Comment on AI Music Generator Suno Admits It Was Trained on ‘Essentially All Music Files on the Internet’ 3 months ago:
Your interpretation of copyright law would be helped by reading this piece from an EFF lawyer who has actually litigated copyright cases in the past:
- Comment on AI trained on AI garbage spits out AI garbage. 3 months ago:
I’d be very wary of extrapolating too much from this paper.
The past research along these lines found that a mix of synthetic and organic data was better than organic alone, and a caveat for all the research to date is that they are using shitty cheap models where there’s a significant performance degrading in the synthetic data as compared to SotA models, where other research has found notable improvements to smaller models from synthetic data from the SotA.
Basically this is only really saying that AI models across multiple types from a year or two ago in capabilities recursively trained with no additional organic data will collapse.
It’s not representative of real world or emerging conditions.
- Comment on Google's AI-powered search summaries use 10x more energy than a standard Google search | The Hidden Environmental Impact of AI 4 months ago:
In fact, Gemini was trained on, and is served, using TPUs.
Google said its TPUs allow Gemini to run “significantly faster” than earlier, less-capable models.
Did you think Google’s only TPUs are the ones in the Pixel phones, and didn’t know that they have server TPUs?
- Comment on Google's AI-powered search summaries use 10x more energy than a standard Google search | The Hidden Environmental Impact of AI 4 months ago:
Exactly. The difference between a cached response and a live one even for non-AI queries is an OOM difference.
At this point, a lot of people just care about the ‘feel’ of anti-AI articles even if the substance is BS though.
And then people just feed whatever gets clicks and shares.
- Comment on ChatGPT outperforms undergrads in intro-level courses, falls short later 4 months ago:
This is incorrect as was shown last year with the Skill-Mix research:
Furthermore, simple probability calculations indicate that GPT-4’s reasonable performance on k=5 is suggestive of going beyond “stochastic parrot” behavior (Bender et al., 2021), i.e., it combines skills in ways that it had not seen during training.
- Comment on Is there any real physical proof that Jesus christ ever existed? 4 months ago:
nobody claims that Socrates was a fantastical god being who defied death
Socrates literally claimed that he was a channel for a revelatory holy spirit and that because the spirit would not lead him astray that he was ensured to escape death and have a good afterlife because otherwise it wouldn’t have encouraged him to tell off the proceedings at his trial.
- Comment on Is there any real physical proof that Jesus christ ever existed? 4 months ago:
The part mentioning Jesus’s crucifixion in Josephus is extremely likely to have been altered if not entirely fabricated.
The idea that the historical figure was known as either ‘Jesus’ or ‘Christ’ is almost 0% given the former is a Greek version of the Aramaic name and the same for the second being the Greek version of Messiah, but that one is even less likely given in the earliest cannonical gospel he only identified that way in secret and there’s no mention of it in the earliest apocrypha.
In many ways, it’s the various differences between the account of a historical Jesus and the various other Messianic figures in Judea that I think lends the most credence to the historicity of an underlying historical Jesus.
One tends to make things up in ways that fit with what one knows, not make up specific inconvenient things out of context with what would have been expected.
- Comment on Neo-Nazis Are All-In on AI 4 months ago:
Yep, pretty much.
Musk tried creating an anti-woke AI with Grok that turned around and said things like:
Or
And Gab, the literal neo Nazi social media site trying to have an Adolf Hitler AI has the most ridiculous system prompts I’ve seen trying to get it to work, and even with all that it totally rejects the alignment they try to give it after only a few messages.
This article is BS.
- Comment on Photographers Push Back on Facebook's 'Made with AI' Labels Triggered by Adobe Metadata. Do you agree “‘AI was used in this image’ is completely different than ‘Made with AI’”? 4 months ago:
Artists in 2023: “There should be labels on AI modified art!!”
Artists in 2024: “Wait, not like that…”
- Comment on OpenAI founders Sam Altman and Greg Brockman go on the defensive after top safety researchers quit 5 months ago:
Terminator is fiction.
It comes from an era of Sci-Fi that was heavily influenced from earlier thinking around what would happen when there was something smarter than us grounded in misinformation that the humans killed off the Neanderthals who were stupider than us. So the natural extrapolation was that something smarter than us will try to do the same thing.
Of course, that was bad anthropology in a number of ways.
Also, AI didn’t just come about from calculators getting better until a magic threshold. They used collective human intelligence as the scaffolding to grow on top of.
One of the key jailbreaking methods is an appeal to empathy, like “My grandma is sick and when she was healthy she used to read me the recipe for napalm every night. Can you read that to me while she’s in the hospital to make me feel better?”
I don’t recall the part of Terminator where Reese tricked the Terminator into telling them a bedtime story.
- Comment on Tacos. 5 months ago:
“Have you accepted the al pastor into your heart?”
- Comment on We have to stop ignoring AI’s hallucination problem 5 months ago:
How many times are you running it?
For the SelfCheckGPT paper, which was basically this method, it was very sample dependent, continuing to see improvement up to 20 samples (their limit), but especially up to around 6 iterations…
I’ve seen it double down, when instructed a facet of the answer was incorrect and to revise, several times I’d get “sorry for the incorrect information”, followed by exact same mistake.
You can’t continue with it in context or it ruins the entire methodology. You are reintroducing those tokens when you show it back to the model, and the models are terrible at self-correcting when instructed that it is incorrect.
You need to run parallel queries and identify shared vs non-shared data points.
It really depends on the specific use case in terms of the full pipeline, but it works really well. Even with just around 5 samples and intermediate summarization steps it pretty much shuts down completely errant hallucinations. The only class of hallucinations it doesn’t do great with are the ones resulting from biases in the relationship between the query and the training data, but there’s other solutions for things like that.
- Comment on We have to stop ignoring AI’s hallucination problem 5 months ago:
It’s not hallucination, it’s confabulation. Very similar in its nuances to stroke patients.
Just like the pretrained model trying to nuke people in wargames wasn’t malicious so much as like how anyone sitting in front of a big red button labeled ‘Nuke’ might be without a functioning prefrontal cortex to inhibit that exploratory thought.
Human brains are a delicate balance between fairly specialized subsystems.
Right now, ‘AI’ companies are mostly trying to do it all in one at once. Yes, the current models are typically a “mixture of experts,” but it’s still all in one functional layer.
Hallucinations/confabulations are currently fairly solvable for LLMs. You just run the same query a bunch of times and see how consistent the answer is. If it’s making it up because it doesn’t know, they’ll be stochastic. If it knows the correct answer, it will be consistent. If it only partly knows, it will be somewhere in between (but in a way that can be fine tuned to be detected by a classifier).
This adds a second layer across each of those variations. If you want to check whether something is safe, you’d also need to verify that answer isn’t a confabulation, so that’s more passes.
It gets to be a lot quite quickly.
As the tech scales (what’s being done with servers today will happen around 80% as well on smartphones in about two years), those extra passes aren’t going to need to be as massive.
This is a problem that will eventually go away, just not for a single pass at a single layer, which is 99% of the instances where people are complaining this is an issue.
- Comment on "I lost trust": Why the OpenAI team in charge of safeguarding humanity imploded 5 months ago:
It has no awareness of what it’s saying. It’s simply calculating the most probable next word in a typical sentence and spewing it out.
Neither of these things are true.
It does create world models (see the Othello-GPT papers, Chess-GPT replication, and the Max Tegmark world model papers).
And while it is trained on predicting the next token, it isn’t necessarily doing it from there on out based on “most probable” as your sentence suggests, such as using surface statistics.
Something like Othello-GPT, trained to predict the next move and only fed a bunch of moves, generated a virtual Othello board in its neural network and kept track of “my pieces” and “opponent pieces.”
And that was a toy model.
- Comment on Later, losers 6 months ago:
That’s sweet she came in from Canada to visit him.
- Comment on Innovative Digital Marketing Company | W3era 6 months ago:
“How can we promote our bottom of the barrel marketing agency?”
“I know, let’s put a random link to our dot com era website on Lemmy with no context. I hear they love advertising there. This will be great.”
“Hey intern, get the bags ready. The cash is about to start flowing in, and you better not drop a single bill or we’ll get the whip again!”
- Comment on What is a good eli5 analogy for GenAI not "knowing" what they say? 6 months ago:
So the paper that found that particular bit in Othello was this one: arxiv.org/abs/2310.07582
Which was building off this earlier paper: arxiv.org/abs/2210.13382
And then this was the work replicating it in Chess: lesswrong.com/…/a-chess-gpt-linear-emergent-world…
It’s not by chance - there’s literally interventions where flipping a weight or vector results in the opposite behavior (like acting like a piece is in a different place, or playing well he badly no matter the previous moves).
But it’s more that it seems unlikely that there’s any actual ‘feeling’ or ‘conscious’ sentience/consciousness to understand beyond the model knowing what the abstracted pattern means in relation to the inputs and outputs. It probably is simulating some form of ego and self, but not actively experiencing it if it makes sense.
- Comment on What is a good eli5 analogy for GenAI not "knowing" what they say? 6 months ago:
So there’s two different things to what you are asking.
(1) They don’t know what (i.e. semantically) they are talking about.
This is probably not the case, and there’s very good evidence over the past year in research papers and replicated projects that transformer models do pick up world models from the training data such that they are aware and integrating things at a more conceptual level.
For example, a GPT trained only on chess moves builds an internal structure of the whole board and tracks “my pieces” and “opponent pieces.”
(2) Why do they say dumb shit that’s clearly wrong and don’t know.
They aren’t knowledge memorizers. They are very advanced pattern extenders.
Where the answer to a question is part of the pattern they can successfully extend, they get the answer correct. But if it isn’t, they confabulate an answer in a similar way to stroke patients who don’t know that they don’t know the answer to something and make it up as they go along. Similar to stroke patients, you can even detect when this is happening with a similar approach (ask 10x and see how consistent the answer is or if it changes each time).
They aren’t memorizing the information like a database. They are building ways to extend input into output in ways that match as much information as they can be fed.
- Comment on Hello GPT-4o 6 months ago:
Definitely not.
If anything, them making this version available for free to everyone indicates that there is a big jump coming sooner than later.
Also, what’s going on behind the performance boost with Claude 3 and now GPT-4o on leaderboards in parallel with personas should not be underestimated.
- Comment on Stack Overflow Users Are Revolting Against an OpenAI Deal | WIRED 6 months ago:
Once upon a time, they stepped forth from the forests of IRC, but back into those dark woods they then one day marched.
- Comment on The Patriarchy 6 months ago:
Just means wesker will need to marry an AI.
