The role of artificial intelligence (AI) in software development has evolved significantly over time. Not everyone may be comfortable with letting AI compose their code, but its utility in testing and scrutinizing code is undeniable.

AI-powered software creation is no longer confined to the realms of science fiction. GitHub Copilot is a leading example, with over 1.2 million developers depending on AI to generate code, thereby saving time and reducing costs for businesses. However, the conversation often centers on Copilot's still-emerging code-generation abilities, and overlooks the potentially greater and more immediate benefits AI and large language models (LLMs) offer to software development.

Jaana Dogan, a prominent software engineer at GitHub, advocates for a shift in perspective. Dogan posits that there is an undue emphasis on code generation, while the value of LLMs in code analysis is largely disregarded. In essence, astute developers might want to leverage AI-based software development more for scrutinizing their code rather than composing it.

Acknowledging the Robot's Contribution

It's common knowledge among developers that code testing is crucial. However, software testing and test-driven development are often more discussed than practiced. Developers might struggle to impartially test their own code or find the process tedious and time-consuming. There are various testing methods, from unit testing to integration testing to regression testing, all of which carry a price: they seem to hinder development speed. Initially, development productivity may seem to decelerate, but thorough testing ultimately leads to more efficient long-term outcomes.

As I've mentioned before, prioritizing development speed over impact is a detrimental strategy. The ultimate aim of a software developer should be to write minimal code for maximum effect, and testing is an indispensable tool for achieving this. This is where LLMs, powering tools like Copilot, can make a significant difference.

Drawing from an 80s song by Styx, "Thank you very much, Mr. Roboto / For doing the jobs nobody wants to." This sentiment resonates with many developers who would rather avoid the necessary but unattractive task of software testing. What if AI could shoulder this responsibility?

Detecting Deficiencies

LLMs can assist in coding by generating routine code, enabling developers to concentrate on higher-value coding tasks. However, depending entirely on the 'black-box' AI could be somewhat unsettling. As pointed out by Ben Kehoe, former cloud robotics research scientist for iRobot, some predictions about AI suggest that it will eventually take over all responsibilities from humans. This assumption implicitly means that the accountability of the individual for the task would simply dissipate. Experienced developers might be comfortable allowing LLMs to generate code, confident in their ability to identify any inaccuracies. For novice developers, on the other hand, reliance on LLMs might lead them into scenarios where they might not fully comprehend their continuing responsibility for the work.

The use of LLMs for code review carries lesser risks. Dogan shares her personal experience of how useful LLMs were in spotting missing test cases, unreleased leaking resources, or even pointing out the faults in her IAM policy. However, she maintains that they are not highly beneficial for suggesting optimizations, ordering results by priority, or for code deletion or cleanups.

Nevertheless, they can aid a developer in identifying issues in unfamiliar languages; for instance, a Java developer might employ LLM to review Go code. According to Dogan, LLMs can also be useful in "navigating small blocks of code [or] specific algorithms I'm not familiar with." 

AI is not here to replace software developers, at least not in the foreseeable future. If used correctly, AI can actually aid developers in improving at their job.

Conclusion

In conclusion, LLMs, and AI in general, can supplement developers' efforts. While they can't replace human creativity, they can take on some of the more monotonous tasks (like testing), enabling developers to focus on tasks that require a more human touch. The integration of AI into software development isn't about replacing developers, but about empowering them to work more effectively and creatively. The future of software development with AI isn't a battlefield but a partnership, where AI assists in mundane tasks, leaving developers to focus on the big picture, strategy, and innovation.