Secure code development goes beyond tools and software – it is a complex activity grounded in risk management and involves an understanding of a developer’s strengths and weaknesses.
Recognizing your developers’ level of expertise goes a long way, and helps determine where security issues are most likely to occur, and which developer is best suited for which task.
As Generative AI large language models (LLMs) write more code, managing them requires the same approach.
Each model has its own preferred coding style, including having strong points in areas like code creation, and weak ones regarding security blind spots. Ultimately, AI models each have their own unique personalities, too.
Although AI-generated code is “machine-made”, each model and the code it produces requires a customized approach to the way it is secured and reviewed, so as to not fall short of mitigating potential risk and vulnerabilities introduced by LLMs.
Organizations need to establish precise security reviews, with human developers anchoring the process to implement effective security controls while also managing the specific coding temperament of each LLM used.
AI-generated code should not be treated differently than code produced by human developers, and should undergo the same personalized risk assessments.
Key Takeaways
- LLMs exhibit distinct “coding personalities” that influence code quality, complexity, documentation, and security outcomes.
- A Sonar analysis found that leading AI coding models share common weaknesses, including security blind spots, technical debt, and the potential to introduce high-severity vulnerabilities.
- No single model excels at every task, making it important to match LLM strengths and weaknesses to specific development use cases.
- Developers need strong security skills to effectively review, validate, and secure AI-generated code throughout the software development lifecycle.
- Organizations should treat AI-generated code like human-written code by applying the same risk assessments, secure coding practices, and review processes.
With Code Creation, LLMs Are People Too
In a recent report from Sonar, the company undertook an in-depth analysis of five leading LLMs: Anthropic’s Claude Sonnet 4 and 3.7, OpenAI’s GPT-4o, Meta’s Llama 3.2 90B and the open source OpenCoder-8B.
The report compares the models’ strengths to different degrees, such as producing syntactically valid code, demonstrating technical competence and working across different programming languages.
It also highlighted common weaknesses, including a glaring lack of security awareness, particularly in leaving software vulnerable to attacks of the highest severity levels, a lack of engineering discipline and a penchant for producing messy code.
Another key finding that underscores the importance of risk management is the fact that no LLMs are quite alike; they each have their own quirks, preferences and security blind spots.
Much like individual human developer personalities, each model has its own style, referred to in the report as a “coding personality”, that can impact how human developers review and analyze AI-generated output.
The report identified three primary, measurable traits of the LLMs tested: complexity and communication, verbosity and documentation.
A more verbose model, for instance, generates many more lines of code to perform a task than a comparatively taciturn model, which can make code review more involved.
Likewise, verbose models tend to produce more complex solutions. They may be necessary for certain tasks, but they require more cognitive effort from code reviewers than a model with a more straightforward approach.
The models tested also exhibited a range of communication styles, as evidenced by the amount of documentation they provide to explain their work.
Together, these metrics constitute a personality type, comparable to human developer types, according to the report:
- The senior architect (Claude Sonnet 4). Good at building complex, enterprise systems, but all that impressive code could be hiding serious bugs.
- The rapid prototyper (OpenCoder 8B). Fast and furious, good at getting projects underway quickly, but with a font of technical debt, which can create challenges later.
- The unfulfilled promise (Llama 3.2 90B). Lots of potential, but middle-of-the pack performance, with critical security blind spots.
- The efficient generalist (GPT-4o). Neither exceedingly verbose nor concise, it can be used for a variety of jobs and tends to avoid the most serious security issues, but it can be careless, opening the door to mistakes that compromise quality and reliability.
- The balanced predecessor (Claude 3.7 Sonnet). Produces highly functional code with excellent documentation but can also produce high-severity vulnerabilities.
This is an excellent representation of how different models have their own strengths and weaknesses, which are incredibly valuable to know when deciding which LLM should be applied to certain projects, just as you might trust a senior architect with a critical application before giving the job to a junior programmer.
But it also underscores the importance of equipping developers with the skills they need to review AI-generated code effectively and efficiently.
Developers Need Security Proficiency to Manage AI Personalities
Ultimately, while LLMs have their benefits and can be efficient at improving code, a critical factor is how carefully they are managed within the software development lifecycle (SDLC).
Developers must be upskilled in security proficiency to ensure that risk can be successfully managed.
It’s not enough for humans to be in the loop, they need the education and skills to ensure that AI-generated code is trustworthy and that technical debt doesn’t pile up in a development environment accelerated by AI assistants.
Equally important, developers must also be able to prompt AI models in a way that helps generate secure code and perform competent security reviews of the AI’s output.
Understanding an AI model’s coding personality can help, but upskilling developers is essential.
The optimal way to reduce risk and ensure that software is secure is to prevent vulnerabilities from the very start of the SDLC.
No matter the source of the created code, developer education is a nonnegotiable way to assure secure coding, code review and testing.
CISOs should also prioritize models that are less risky for the tasks at hand and empower developers through education and upskilling.
By considering LLM “personalities” and enforcing developer security proficiency, organizations can close gaps in security knowledge, target technical debt and help mitigate the growing risk within their own codebases.
This guest article was written by Matias Madou, Ph.D., CTO & Co-Founder at Secure Code Warrior.