I use it to generate unit tests, it’ll get the bulk of the code writing done and does a pretty good job at coverage, usually hitting 100%. All I have to do for the most part is review the tests to make sure they’re doing the right thing, and mock out some stuff that it missed.
Comment on '3d-printing a screw' is a way to describe how AI integration is stupid most of the time
kescusay@lemmy.world 3 days agoSo there are a few very specific tasks that LLMs are good at from the perspective of a software developer:
- Certain kinds of analysis tasks can be done very quickly and efficiently with Copilot in agent mode. For instance, having it assess your existing code for adherence to stylistic standards where a violation isn’t going to trigger a linting error.
- Quick script writing is something it excels at. There are all kinds of circumstances where you might need an independent script, such as a database seed file. It’s not part of the application itself, but it’s a useful utility to have, and Copilot is good at writing them.
- Scaffolding a new application. If you’re creating something brand new and know which tools you want to use for it, but don’t want to go through the hassle of setting everything up yourself, having Copilot do it can be a real time saver.
And that’s… pretty much it. I’ve experimented with building applications with “prompt engineering,” and to be blunt, I think the concept is fundamentally flawed. The problem is that once the application exceeds the LLM’s context window size, which is necessarily small, you’re going to see it make a lot more mistakes than it already does, because - just as an example - by the time you’re having it write the frontend for a new API endpoint, it’s already forgotten how that endpoint works.
As the application approaches production size in features and functions, the number of lines of code becomes an insurmountable bottleneck for Copilot. It simply can’t maintain a comprehensive understanding of what’s already there.
Omgpwnies@lemmy.world 2 days ago
lectricleopard@lemmy.world 1 day ago
Legit. Do you need to feed it your code base at all? How does it know what needs to be tested otherwise?
Omgpwnies@lemmy.world 13 hours ago
yeah, it’s the copilot plugin for intellij, basically right click and choose generate tests, it’ll read the file and … well…
Downside to that approach is that it doesn’t know what some function calls do if they’re not part of that file, so it tends to miss places that need to be mocked out.
Occasionally it writes a test that’s “wrong”, and I have to fix the test… very rarely, the “wrong” test is actually “right” based on say a method signature or decision tree, and the method itself needs changing.
kescusay@lemmy.world 2 days ago
You’re right, unit tests are another area where they can be helpful, as long as you’re very careful to check them over.
WorldsDumbestMan@lemmy.today 2 days ago
They are getting faster, having larger context windows, and becoming more accurate. It is only a matter of time until AI simply copy-cats 99.9% of the things humans do.
kescusay@lemmy.world 2 days ago
Actually, there’s growing evidence that beyond a certain point, more context drastically reduces their performance and accuracy.
I’m of the opinion that LLMs will need a drastic rethink before they can reach the point you describe.
WorldsDumbestMan@lemmy.today 2 days ago
We have 100M context AI, we just need better attention mechamisms.
lectricleopard@lemmy.world 1 day ago
This sounds to me like saying you have enough feathers in the grocery bag you’re holding. All you need now is a beak, and you’ll make yourself a duck.
X doubt
JustTesting@lemmy.hogru.ch 3 days ago
one other use case where they’re helpful is ‘translation’. Like i have a docker compose file and want a helm chart/kubernetes yaml files for the same thing. It can get you like 80% there, and save you a lot of yaml typing.
Wont work well if it’s mo than like 5 services or if you wanted to translate a whole code base from one language to another. But converting one kind of file to another one with a different language or technology can work ok. Anything to write less yaml…