AI Code Assistants - advantages and limitations

[practicalli-johnny]

Article title
AI Code Assistants - advantages and limitations

Article description or content
Artificial intelligence has become a significantly overloaded term over the last 7 decades since it was coined.

NOTE: The term "artificial intelligence" was coined by John McCarthy. He introduced the term in 1956 at the Dartmouth Workshop, which is widely considered the starting point for the field of AI research.

While the idea of machines exhibiting intelligent behavior had been explored before, McCarthy's use of the term "artificial intelligence" and his organization of the Dartmouth workshop are credited with formally establishing the field as a distinct area of study.

• https://en.wikipedia.org/wiki/John_McCarthy_(computer_scientist)
• https://en.wikipedia.org/wiki/History_of_artificial_intelligence

Research

https://developers.slashdot.org/story/25/07/12/016247/ai-slows-down-some-experienced-software-developers-study-finds

AI research nonprofit METR conducted the in-depth study on a group of seasoned developers earlier this year while they used Cursor, a popular AI coding assistant, to help them complete tasks in open-source projects they were familiar with. Before the study, the open-source developers believed using AI would speed them up, estimating it would decrease task completion time by 24%. Even after completing the tasks with AI, the developers believed that they had decreased task times by 20%. But the study found that using AI did the opposite: it increased task completion time by 19%. The study's lead authors, Joel Becker and Nate Rush, said they were shocked by the results: prior to the study, Rush had written down that he expected "a 2x speed up, somewhat obviously.

The slowdown stemmed from developers needing to spend time going over and correcting what the AI models suggested. "When we watched the videos, we found that the AIs made some suggestions about their work, and the suggestions were often directionally correct, but not exactly what's needed," Becker said. The authors cautioned that they do not expect the slowdown to apply in other scenarios, such as for junior engineers or engineers working in codebases they aren't familiar with. Still, the majority of the study's participants, as well as the study's authors, continue to use Cursor today. The authors believe it is because AI makes the development experience easier, and in turn, more pleasant, akin to editing an essay instead of staring at a blank page. "Developers have goals other than completing the task as soon as possible," Becker said. "So they're going with this less effortful route."

Creative Commons license
All ideas and content submitted should be copyright free. Submitted content will be used in any Practicalli works and distributed under a Creative Commons license to ensure that content remains freely available.