Comment on Jack Dorsey's New Company Falling Apart as It Forces Employees to Use AI
MagicShel@lemmy.zip 3 weeks ago
At work today we had a little presentation about Claude Cowork. And I learned someone used it to write a C (maybe C++?) compiler on Rust in two weeks at a cost of $20k and it passed 99% of whatever hell test suite they use for evaluating compilers. And I had a few thoughts.
I think this is a cool thing in the abstract. But in reality, they cherry picked the best possible use case in the world and anyone expecting their custom project is going go anything like this will be lighting huge piles of money on fire.
A C compiler in two weeks is a difficult, but doable, grad school class project (especially if you use lex and yacc instead of hand-coding the parser). And I guarantee 80 hours of grad student time costs less than $20k.
Frankly, I’m not impressed with the presentation in your anecdote at all.
Here is the original cite that my company pulled that from if you want more details.
I’ve never written a compiler, nor in Rust, so I have no idea the effort involved. I’m just boggling over the price tag. I’ll bet that’s the cost of an entire offshore team.
Yeah, the thing also has limited scope and requires some meddling to point to necessary includes as evidenced by the first issue, afair. And the code produced is subpar I heard
It’s even simpler than that: using an LLM to write a C compiler is the same as downloading an existing open source implementation of a C compiler from the Internet, but with extra steps as the LLM was actually fed with that code and is just re-assembling it back together but with extra bugs - plagiarism hidden behind an automated text parrot interface.
A human can beat the LLM at that by simply finding and downloading an implementation of that more than solved problem from the Internet, which at worse will take maybe 1h.
The LLM can “solve” simple and well defined problems because its basically plagiarizing pieces of existing code that solve those problems.
Hey, so I started this comment to disagree with you and correct some common misunderstandings that I’ve been fighting against for years. Instead, as I was formulating my response, I realized you’re substantially right and I’ve been wrong — or at least my thinking was incomplete. I figured I’d mention because the common perception is arguing with strangers on the internet never accomplishes anything.
LLMs are not fundamentally the plagiarism machines everyone claims they are. If a model reproduces any substantial text verbatim, it’s because the LLM is overtrained on too small of a data set and the solution is, somewhat paradoxically, to feed it more relevant text. That has been the crux of my argument for years.
That being said, Anthropic and OpenAI aren’t just LLM models. They are backed by RAG pipelines which are verbatim text that gets inserted into the context when it is relevant to the task at hand. And that fact had been escaping my consideration until now. Thank you.
Even the LLM part might be considered Plagiarism.
Basically, unlike humans it cannot assemble an output based on logical principles (i.e. assembled a logical model of the flows in a piece of code and then translate it to code), it can only produce text based on an N-space of probabilities derived from the works of others it has “read” (i.e. fed to it during training).
That text assembling could be the machine equivalent of Inspiration (such as how most programmers will include elements they’ve seen from others in their code) but it could also be Plagiarism.
Ultimately it boils down to were the boundary between Inspiration and Plagiarism stands.
As I see it, if for specific tasks there is overwhelming dominance of trained weights from a handful of works (which, one would expect, would probably be the case for a C-compiler coded in Rust), then that’s a lot more towards the Plagiarism side than the Inspiration side.
Granted, it’s not the verbatim copying of an entire codebase that would legally been deemed Plagiarism, but if it’s almost entirely a montage made up of pieces of a handful of codebases, could it not be considered a variant of Plagiarism that is incredibly hard for humans to pull off but not so for an automated system.
Note that obviously the LLM has no “intention to copy”, since it has no intent at all, what I’m saying is that the people who made it have intentionally made an automated system that copies elements of existing works, which normally assembles the results from very small textual elements (same as a person who has learned how letters and words work can create a unique work from letters and words) but with the awareness that in some situations that automated system they create is producing output based on an amount of sources which is very low to the point that even though it’s assembling the output token by token, it’s pretty much just copying whole blocks from those sources same as a human manually copying a text from a source to their own work.
In summary, IMHO LLMs don’t always plagiarize, but can sometimes do it when the number of sources that’s what ultimately created the volume of the N-dimensional probabilistic space they’re following is very low.
I agree with you on a technical level. I still think LLMs are transformative of the original text and if
when the number of sources that’s what ultimately created the volume of the N-dimensional probabilistic space they’re following is very low.
then the solution is to feed it even more relevant data. But I appreciate your perspective. I still disagree, but I respect your point of view.
I’ll give what you’ve written some more thought and maybe respond in greater depth later but I’m getting pulled away. Just wanted to say thanks for the detailed and thorough response.
https://harshanu.space/en/tech/ccc-vs-gcc/ has a good overview how bad it really is
Thank you. Great addition. That was a very interesting read, though I need to be more awake for reading technical writing like that 🥱.
That was interesting to read if not a bit jargon heavy. Thanks for sharing
Also, software development is already the best possible use case for LLMs: you need to build something abiding by a set of rules (as in a literal language, lmao), and you can immediately test if it works.
In e.g. a legal use case instead, you can jerk off to the confident sounding text you generated, then you get chewed out by the judge for having hallucinated references. Even if you have a set of rules (laws) as a guardrails, you cannot immediately test the what the AI generated - and if an expert needs to read and check everything in detail, then why not just do it themselves in the same amount of time.
We can go on to business, where the rules the AI can work inside are much looser, or healthcare, where the cost of failure is extremely high. And we are not even talking about responsibilities, official accountability for decisions.
I just don’t think what is claimed for AI is there. Maybe it will be, but I don’t see it as an organic continuation of the path we’re in. We might have another dot com boom when investors realize this - LLMs will be here to stay (same as the internet did), but they will not become AGI.
I also often get assigned projects where all the tests are written out beforehand and I can look at an existing implementation while I work…
Don’t forget that there are tons of C compilers in the dataset already
99% pass rate? Maybe that’s super impressive because it’s a stress test, but if 1% of my code fails to compile I think I’d be in deep shit.
Also - one of the main arguments of vibe coding advocators is that you just need to check the result several times and tell the AI assistant what needs fixing. Isn’t a compiler test suite ideal for such workflow? Why couldn’t they just feed the test failures back to the model and tell it to fix them, iterating again and again until they get it to work 100%?
Maybe they did, that’s how they got to 99%. The remaining issues are so intricate/complex the LLM just can’t solve them no matter how many test cases you give it.
I would be interested in knowing what language it was for sure, as there is a huge difference between a C and a C++ compiler in terms of complexity
I think this is the reported github.com/anthropics/claudes-c-compiler.
And here’s a pretty good article about it arstechnica.com/…/sixteen-claude-ai-agents-workin…
I just posted where I found the source in another comment. It would have probably the information you’re interested in.
I wanna make sure I got this right. They used $20,000 in fees in 2 weeks to make a compiler? Also, to what end? Like what’s the expected ROI on that?
Well it’s Anthropic, creators of Claude. It’s a way to show off and convince people AI can do it. $20k is what it would cost you or me, but it’s just free for them.
I don’t even hate AI but it’s kinda sickening the way they overstate the capabilities. But let me tell you how excited the top leadership at my company is about this…
$20k is what it would cost you or me, but it’s just free for them.
No it isn’t. This is not regular software where the bulk of the price is the licensing. With slope-as-a-service, the bulk of the price is the data center operation cost - which Anthropic is certainly not getting for free.
I mean there is a cost associated with it, just like there is a cost associated with having free soda in the break room, but it was free for the person doing the project. It’s absorbed into operational costs.
pulsewidth@lemmy.world 3 weeks ago
Agree with all points. Additionally, compilers are also incredibly well specified via ISO standards etc, and have multiple open source codebases available, eg GCC which is available in multiple builds and implementations for different versions of C and C++, and DQNEO/cc.go.
So there are many fully-functional and complete sources that Claude Cowork would have pulled routines and code from.
xep@discuss.online 3 weeks ago
The vibe codes compiler is likely unmaintainable, so it can’t br updated when the spec changes even assuming it did work and was real. So you’d have to redo the entire thing. It’s silly.
exu@feditown.com 3 weeks ago
Updates? You just vibecode a new compiler that follows the new spec
MagicShel@lemmy.zip 3 weeks ago
“I want to add a command line option that auto generates helloworld.exe”
“That’ll be $21,000.”
killabeezio@lemmy.world 2 weeks ago
Nah bro. Just tell the agent to be a super duper distinguished software developer and write no bugs and keep the code maintainable /s