I’m not really sure why it was such a small sample size. It definitely casts doubt on some of their conclusions. I also have issues with some methodology used. I think a better study that came out a week or two ago was the one that showed visible neurological decline from AI use.
Comment on Study finds AI tools made open source software developers 19 percent slower
Sagan_Wept@lemmynsfw.com 3 months ago
Their sample size was 16 people…
bulwark@lemmy.world 3 months ago
kromem@lemmy.world 3 months ago
Where the most experienced minority only had a few weeks of using AI inside an IDE like Cursor.
mspencer712@programming.dev 3 months ago
I got flamed pretty hard for pointing out that this sample size really needs to be in the title, but it needs to be said. Thank you. Sixteen people is basically a forum thread, and not a very popular one.
It’s still useful information and a good read, but a lot of people don’t click through to the article, they just remember the title and move on.
tankfox@midwest.social 3 months ago
Who are in the process of learning to do something new, versus the workflow that they’ve been trained in and have a lot of experience in.
Where was the sample of non-coders tasked with doing the same thing, using AI to help or learning without assistance?
Where was the sample of coders prohibited from looking anything up and having to rely solely on their prior knowledge to do the job?
It might help define what’s actually being tested.