Mniot

@Mniot@programming.dev

• ⁨Comment⁩ on ⁨ELI5. Limit of current gen AI/LLMs⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:

  The “agents” and “agentic” stuff works by wrapping the core innovation (the LLM) in layers of simple code and other LLMs. Let’s try to imaging building a system that can handle a request like “find where I can buy a video card today. Make a table of the sites, the available cards, their prices, and how they compare on a benchmark.” We could solve this if we had some code like

  search_prompt = llm(f"make a list of google web search terms that will help answer this user's question. present the result in a json list with one item per search. <request>{user_prompt}</request>")
  results_index = []
  for s in json.parse(search_prompt):
    results_index.extend(google_search(s))
  results = [fetch_url(url) for url in results_index]
  summarized_results = [llm(f"summarize this webpage, fetching info on card prices and benchmark comparisons <page>{r}</page>") for r in results]
  
  return llm(f"answer the user's original prompt using the following context: <context>{summarized_results}</context> <request>{user_prompt}</request>")
  

  It’s pretty simple code, and LLMs can write that, so we can even have our LLM write the code that will tell the system what to do! (I’ve omitted all the work to try to make things sane in terms of sandboxing and dealing with output from the various internal LLMs).

  The important thing we’ve done here is instead of one LLM that gets too much context and stops working well, we’re making a bunch of discrete LLM calls where each one has a limited context. That’s the innovation of all the “agent” stuff. There’s an old Computer Science truism that any problem can be solved by adding another layer of indirection and this is yet another instance of that.

  Trying to define a “limit” for this is not something I have a good grasp on. I guess I’d say that the limit here is the same: max tokens in the context. It’s just that we can use sub-tasks to help manage context, because everything that happens inside a sub-task doesn’t impact the calling context. To trivialize things: imagine that the max context is 1 paragraph. We could try to summarize my post by summarizing each paragraph into one sentence and then summarizing the paragraph made out of those sentences. It won’t be as good as if we could stick everything into the context, but it will be much better than if we tried to stick the whole post into a window that was too small and truncated it.

  Some tasks will work impressively well with this framework: web pages tend to be a TON of tokens but maybe we’re looking for very limited info in that stack, so spawning a sub-LLM to find the needle and bring it back is extremely effective. OTOH tasks that actually need a ton of context (maybe writing a book/movie/play) will perform poorly because the sub-agent for chapter 1 may describe a loaded gun but not include it in its output summary for the next agent. (But maybe there are more ways of slicing up the task that would allow this to work.)

• ⁨Comment⁩ on ⁨The developers of PEAK, explaining how they decided on pricing for their game. ⁩ ⁨⁨5⁩ ⁨weeks⁩ ago⁩:

  It works against the general population, if this particular one doesn’t, don’t get too busy strutting, there is almost certainly something else that does work on you.

  That is very well put! I feel like I’ve talked to so many people who see one ad that doesn’t land and say, “ads don’t work on me.”

• ⁨Comment⁩ on ⁨Yunohost, the *arr stack and VPN⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:

  Set your expectations: networking is complex and the configuration you’re hoping for is particularly complex. It sounds to me like you’re looking for a split-horizon configuration where local traffic stays local but internet traffic is routed over VPN. But also you want that configuration only for specific apps.

  It’s not the *arr programs that are tricky, it’s that any service you try to configure this way will be some of the hardest sysadmin work.

• ⁨Comment⁩ on ⁨[deleted]⁩ ⁨⁨1⁩ ⁨month⁩ ago⁩:

  The turbo-hell part is that the spam comments aren’t even being written for humans to see. The intention is that ChatGPT picks up the spam and incorporates it into its training.

  I worked at a company that sold to doctors and the marketing team was spending most of their effort on this kind of thing. They said that nowadays when doctors want to know “what should I buy to solve X?” or “which is better A or B?” they ask ChatGPT and take its answer as factual. They said that they were very successful in generating blog articles for OpenAI to train on so that our product would be the preferred answer.

• ⁨Comment⁩ on ⁨Windows 11 could actually become the same kind of mistake Sony made with the PS3⁩ ⁨⁨3⁩ ⁨months⁩ ago⁩:

  I’d be curious to hear how you end up liking it.

  As someone who spends a lot of time on the command-line, I’ve generally preferred MacOS over Windows as my not-Linux OS. But my impression is that for people who like the Windows or Linux GUI, MacOS is a bigger (and less pleasant) change.

  And even on the command-line, MacOS is a different *nix distro and makes seme pretty weird choices (launchd, plists, /etc is actually /private/etc, …) whereas you could have vanilla Ubuntu inside WSL2.

• ⁨Comment⁩ on ⁨Beware, another "wonderful" conservative instance to "free us" has appeared ⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  In the novel Snow Crash, the residents of the white-supremacist micro-state franchises visit other places because they have no non-whites to attack at home.

• ⁨Comment⁩ on ⁨YSK Lemvotes.org will show you votes on any post, comment, or by user, or anything on the fediverse⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  If by any chance, one’s identity is reviled, their entire behaviour history would be out in the wild.

  So close to a sweet meter. What do you think of

  “If, by some chance, one’s handle’s reviled / their foul history would be out in the wild.”

  ? It’s not perfect. Probably just a little more work-shopping.

• ⁨Comment⁩ on ⁨Exactly Six Months Ago, the CEO of Anthropic Said That in Six Months AI Would Be Writing 90 Percent of Code⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  The theory is that the new hire gets better over time

  It always amazes me how few people get this. Have they only ever made terrible hires?

  The way that a company makes big profits is by hiring fresh graduates and giving them a cushy life while they grow into good SWEs. By the time you’re paying $200k for a senior software engineer, they’re generating far more than that in value. And you only had to invest a couple years and some chump change.

  But now businesses only think in the short-term and so paying $10k for a month of giving Anthropic access to our code base sounds like a bargain.

• ⁨Comment⁩ on ⁨Exactly Six Months Ago, the CEO of Anthropic Said That in Six Months AI Would Be Writing 90 Percent of Code⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  Executives are mostly irrelevant as long as they’re not forcing the whole company into the bullshit.

  I’m seeing a lot of this, though. Like, I’m not technically required to use AI, but the VP will send me a message noting that I’ve only used 2k tokens this month and maybe I could get more done if I was using more…?

• ⁨Comment⁩ on ⁨Exactly Six Months Ago, the CEO of Anthropic Said That in Six Months AI Would Be Writing 90 Percent of Code⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  More as an alternative to a search engine.

  In my ideal world, StackOverflow would be a public good with a lot of funding and no ads/sponsorship.

  Since that’s not the case, and everything is hopelessly polluted with ads and SEO, LLMs are momentarily a useful tool for getting results. Their info might be only 3/4 correct, but my search results are also trash. Who knows what people will do in a year when the LLMs have been eating each others slop and are also being stuffed with ads by their owners.

• ⁨Comment⁩ on ⁨Exactly Six Months Ago, the CEO of Anthropic Said That in Six Months AI Would Be Writing 90 Percent of Code⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  Coding junkie is where you sneak away from your friends and code a few lines in the bathroom

• ⁨Comment⁩ on ⁨Exactly Six Months Ago, the CEO of Anthropic Said That in Six Months AI Would Be Writing 90 Percent of Code⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  We’re talking here about garbage code that we don’t want. If the choice is “let me commit bad code that causes problems or else I will quit using computers”… is this a dilemma for you?

• ⁨Comment⁩ on ⁨Exactly Six Months Ago, the CEO of Anthropic Said That in Six Months AI Would Be Writing 90 Percent of Code⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  It does seem silly, but it’s perfectly aligned with the marketing hype that the AI companies are producing.

• ⁨Comment⁩ on ⁨Reality Is Ruining the Humanoid Robot Hype: The obstacles to scaling up humanoids that nobody is talking about⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  The disconnect is coming here:

  I just don’t see the line where any of that is impossible. It seems inevitable

  I agree with you that robots building robots is not impossible. I disagree with you that it’s therefore inevitable. I strongly disagree with you that it’s therefore inevitable in the immediate future.

• ⁨Comment⁩ on ⁨⁩ ⁨⁨6⁩ ⁨months⁩ ago⁩:

  The people cheering are just honoring Charlie’s legacy.

• ⁨Comment⁩ on ⁨Is it realistic to hope that lemmy grows to the size of the bigger social media platforms?⁩ ⁨⁨6⁩ ⁨months⁩ ago⁩:

  This, but in a hopeful voice instead of sarcastic 🙂

  (Being surrounded by people who think more progressively will tend to shift people’s views)

• ⁨Comment⁩ on ⁨Intel CPU Temperature Monitoring Driver For Linux Now Unmaintained After Layoffs⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  OK, cool. That definitely helps things.

  I think where we’re disagreeing is that I think in a capitalist society the promise of money will inevitably corrupt the government (because it’s made of people). Maybe it can be avoided if the government performs additional regulatory action to stop anyone from getting too wealthy, but that sounds like beyond the limits that you want to set for government.

• ⁨Comment⁩ on ⁨Intel CPU Temperature Monitoring Driver For Linux Now Unmaintained After Layoffs⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  Without force

  Whoah, whoah. Why’d you rule that out?

  Your business plan: quality goods at reasonable prices. My business plan: hire some goons to kill you and take your stuff.

  Historically, this has a lot of precedence.

• ⁨Comment⁩ on ⁨Intel CPU Temperature Monitoring Driver For Linux Now Unmaintained After Layoffs⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  In an unfettered marketplace, what stops a dominant player from introducing fetters?

• ⁨Comment⁩ on ⁨Intel CPU Temperature Monitoring Driver For Linux Now Unmaintained After Layoffs⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  When I read your message, I get the impression that you think of “The Government” as this independent actor. I see it as a system that is primarily controlled by wealthy people. Either directly or through their funding advertisements (including astroturfing/bot-farms) to promote what they want.

  So the larger companies do get government assistance… because they are the government. And this isn’t some kind of weird coincidence. It’s fundamental to capitalism’s operation. You can’t have a system that’s based on capital and then have it be unbiased towards entities who have vastly more capital!

• ⁨Comment⁩ on ⁨Intel CPU Temperature Monitoring Driver For Linux Now Unmaintained After Layoffs⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  socialized capitalism

  I think I understand your complaint, but I’d say “free market” rather than “capitalism”. But regardless of what we call it, it doesn’t actually exist unless you have a more powerful external system regulating it.

  Start with a truly free-market capitalist system. One company manages to temporarily pull ahead (through luck and skill). The rational thing for the company to do isn’t “make better products” (that’s hard) but “destroy competing companies” (much easier). And the end-product would be that the company becomes a government so it can force consumers to pay.

  So I’d argue that socialized capitalism (which I’m picturing as a socialist system that permits certain specific free markets and handles the fallout of business failures) is what you actually want.

• ⁨Comment⁩ on ⁨Too bad we can't have good public transportation⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  Capitalism is when there’s an owner-class controlling production via capital. It doesn’t really matter what they’re producing or at what cost or who’s consuming.

• ⁨Comment⁩ on ⁨Too bad we can't have good public transportation⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  No, because cross-country trains and heavy use of them to move goods and people predates cars by quite a bit. Trains were a key component of the North winning the Civil War, for example.

  Lots of existing train infrastructure needed to be torn out to make room for car infrastructure.

• ⁨Comment⁩ on ⁨Too bad we can't have good public transportation⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  Are you suggesting that’s why the US hasn’t improved trains? Is there something about train improvements specifically that you think is harmful?

• ⁨Comment⁩ on ⁨hubris go brrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  Lots of things seem reasonable if you skip the context and critical reasoning. It’s good to keep some past examples of this that personally bother you in your back pocket. Then you have it as an antidote for examples that don’t bother you.

• ⁨Comment⁩ on ⁨Jack Dorsey says his 'secure' new Bitchat app has not been tested for security⁩ ⁨⁨8⁩ ⁨months⁩ ago⁩:

  Looking at the code, it reads like it was written by LLM: chatty commit messages, lack of spelling/capitalization errors, bullet points galore, shit-ton of “Fix X” commits that don’t read like they’re increasingly-frustrated, worthless comments randomly scattered like “i + 1 // add 1 to i” without any other comments on the page.

  No security review because none of the code has been reviewed and he doesn’t know what’s in it.

• ⁨Comment⁩ on ⁨Holy sh*t: Jack Dorsey just Announced Bitchat(A secure, decentralized, peer-to-peer messaging app for iOS and macOS that works over Bluetooth mesh networks) Licensed Under Public Domain.⁩ ⁨⁨8⁩ ⁨months⁩ ago⁩:

  I once did some programming on the Cybiko, a device from 2000 that could form a wireless mesh network with peers. The idea was that you could have a shopping mall full of teens and they’d be able to chat with each other from one end to the other by routing through the mesh. It was a neat device!

• ⁨Comment⁩ on ⁨ChatGPT Mostly Source Wikipedia; Google AI Overviews Mostly Source Reddit⁩ ⁨⁨9⁩ ⁨months⁩ ago⁩:

  This is good advice for all tertiary sources such as encyclopedias, which are designed to introduce readers to a topic, not to be the final point of reference. Wikipedia, like other encyclopedias, provides overviews of a topic and indicates sources of more extensive information.

  The whole paragraph is kinda FUD except for this. Normal research practice is to (get ready for a shock) do research and not just copy a high-level summary of what other people have done. If your professors were saying, “don’t cite encyclopedias, which includes Wikipedia” then that’s fine. But my experience was that Wikipedia was specifically called out as being especially unreliable and that’s just nonsense.

  I personally use ChatGPT like I would Wikipedia

  Eesh. The value of a tertiary source is that it cites the secondary sources (which cite the primary). If you strip that out, how’s it different from “some guy told me…”? I think your professors did a bad job of teaching you about how to read sources. Maybe because they didn’t know themselves. :-(

• ⁨Comment⁩ on ⁨ChatGPT Mostly Source Wikipedia; Google AI Overviews Mostly Source Reddit⁩ ⁨⁨9⁩ ⁨months⁩ ago⁩:

  I think it was. When I think of Wikipedia, I’m thinking about how it was in ~2005 (20 years ago) and it was a pretty solid encyclopedia then.

  There were (and still are) some articles that are very thin. And some that have errors. Both of these things are true of non-wiki encyclopedias. When I’ve seen a poorly-written article, it’s usually on a subject that a standard encyclopedia wouldn’t even cover. So I feel like that was still a giant win for Wikipedia.

• ⁨Comment⁩ on ⁨ChatGPT Mostly Source Wikipedia; Google AI Overviews Mostly Source Reddit⁩ ⁨⁨9⁩ ⁨months⁩ ago⁩:

  I think the academic advice about Wikipedia was sadly mistaken. It’s true that Wikipedia contains errors, but so do other sources. The problem was that it was a new thing and the idea that someone could vandalize a page startled people. It turns out, though, that Wikipedia has pretty good controls for this over a reasonable time-window. And there’s a history of edits. And most pages are accurate and free from vandalism.

  Just as you should not uncritically read any of your other sources, you shouldn’t uncritically read Wikipedia as a source. But if you are going to uncritically read, Wikipedia’s far from the worst thing to blindly trust.