Software development is one of the early use cases for generative AI. Thousands of companies big and small are already using tools such as GitHub Copilot to speed up how they build new applications and services.
But while AI may boost production, it could also be detrimental to overall code quality, according to a new research project from GitClear, a developer analytics tool built in Seattle.
The study analyzed 153 million changed lines of code, comparing changes done in 2023 versus prior years, when AI was not as relevant for code generation. Some of the findings include:
- “Code churn,” or the percentage of lines thrown out less than two weeks after being authored, is on the rise and expected to double in 2024. The study notes that more churn means higher risk of mistakes being deployed into production.
- The percentage of “copy/pasted code” is increasing faster than “updated,” “deleted,” or “moved” code. “In this regard, the composition of AI-generated code is similar to a short-term developer that doesn’t thoughtfully integrate their work into the broader project,” said GitClear founder Bill Harding.
The bottom line, per Harding: AI code assistants are very good at adding code, but they can cause “AI-induced tech debt.”
“Fast code-adding is desirable if you’re working in isolation, or on a greenfield problem,” he told GeekWire. “But hastily added code is caustic to the teams expected to maintain it afterward.”
In other words, more quantity doesn’t always lead to better quality.
AI is like a “brand new credit card here that is going to allow us to accumulate technical debt in ways we were never able to do before,” Armando Solar-Lezama, a professor at MIT, told The Wall Street Journal in a story last year about AI coding tools.
The rise of AI coding could also impact how engineers are compensated.
“If engineering leaders are making salary decisions based on lines of code changed, the combination of that plus AI creates incentives ripe for regrettable code being submitted,” Harding said.
Harding said it’s tough to say whether AI tools will be a net positive for software development. He pointed to the benefits of using AI to get custom-tailored code answers, from places such as Phind. But he also said reading bad code “is the most willpower-graining component of the job” for developers.
A study by McKinsey last year found that a “massive surge in productivity” from AI coding is possible, but it depends on task complexity and developer experience. “Ultimately, to maintain code quality, developers need to understand the attributes that make up quality code and prompt the tool for the right outputs,” the study said.
Harding previously led an eBay competitor called Bonanza, which was acquired last year to help fund the growth of Alloy.dev, a 15-person company that operates GitClear and note-taking app Amplenote.
