I wanted to write this post to bring together a few thoughts on the use of generative AI in software development and the enthusiasm it has attracted in our industry, where the ability to produce things faster seems to justify giving up on understanding what we are producing. I understand that enthusiasm when it comes from colleagues trying to spare themselves some of their daily workload; I find it much less convincing when it serves to claim expertise they have chosen to outsource, or to dress up as innovation the automation of practices that were never particularly innovative in the first place.
We had already addressed this subject when we introduced our AI policy for software development, which asks everyone who works with us or wants to contribute to our projects to read and understand the code they write and, above all, take responsibility for it, even when they have used a generative assistant. In this article, I also want to look at the context in which these tools are adopted: the companies we work for, the value they place on our expertise, and how readily faster production is celebrated as innovation, with little concern for what is being produced or who will have to deal with it when it stops working.
The first thing I want to point out is that, when we ask it to generate something for us, generative AI is not a tool that helps us create it using our own abilities. It is a statistical system trained on other people's work, often without their consent. We are delegating work that we might not even be able to do ourselves.
That distinction holds regardless of how useful it is for producing a mountain of often useless bullshit better and faster, which accounts for a substantial share of the IT services produced by capitalist companies in Italy and around the world. The intellectual property behind this bullshit is not worth much either, since it largely amounts to endlessly copying other people's systems, solutions, and ideas: a widespread practice long before generative AI arrived.
It is hardly surprising, then, that industries where much of the work involves recycling what already exists, including my own, I should stress, would enthusiastically welcome a tool that automates the process and helps them get through their work and earn their paycheck faster. When the value of your work is in the hands of a CEO who is as arrogant as they are incompetent, passing off yet another rehash as innovation, perhaps on LinkedIn, genAI aptly represents the industrial refinement of an already firmly established mediocrity.
I understand, and in many cases share, the choice many colleagues make to use these tools to spare themselves at least some of the crap they are forced to produce every day at work, perhaps under some half-assed remote working arrangement “granted” instead of a pay raise. The condition, however, is having the intellectual honesty to acknowledge how little of what we have had generated is actually ours, without trying to pass off the ability to write prompts as creative talent or claim credit for work done for us. If we can understand, verify, and correct that work, our contribution has a recognizable value; if all we have done is request changes until the result looked acceptable, we should at least avoid claiming expertise we do not have.
With all that said, I still have vastly more respect for those who use AI to assist with analysis, verification, and technical support than for those who blindly have entire chunks of code generated without really understanding how they work.
There are plenty of examples: assistance with penetration testing and vulnerability assessment, threat modeling, security auditing, code review, and identifying bugs and potential regressions. Then there is generating unit and integration tests, preparing inputs for fuzz testing, and analyzing results and gaps in test coverage. The same applies to assistance with static analysis and dependency checks, performance analysis and identifying bottlenecks, assisted debugging, and log analysis. Even technical documentation can benefit, provided, perhaps, that you read it before handing it over and putting your name on it.
These uses still require the right tools: asking a model to examine code is not the same as running a scan, just as a hypothesis about performance needs to be checked against actual measurements. AI can help us set up checks and interpret their results; we still need to establish that they were performed correctly and that the conclusions are supported by evidence.
Then there is the whole of DevSecOps and the CI/CD pipeline: from configuring and verifying pipelines to automating checks, perhaps provided that we have first specified what they are supposed to check; from checking policies to monitoring deployments, through to assistance with incident analysis and remediation.
References such as the NIST Secure Software Development Framework and the OWASP Application Security Verification Standard can help structure these checks. The OWASP SAMM guidance on AI-assisted development also addresses how to integrate these tools into development and security processes. These are activities where the value we seek from AI lies in helping us analyze, verify, and improve the security of what we are building. This use does not resolve the disputes over the work used to train the models, but it serves a purpose I consider far more worthwhile than mass-producing content to sell as our own.
Even an inexperienced developer, an enthusiast with no particular expertise, and even the vibe coder, just above the CEO who does not know shit and the morons who like their posts on LinkedIn, can gain something concrete from this: help recognizing problems they would otherwise be unable to identify.
An overlooked vulnerability can have far more serious consequences than a few badly written lines of code. If an assistant points out that an endpoint allows access to another customer's data, or that a credential has ended up in the repository, it is flagging a problem we need to address. We then need to verify that the finding is valid and work out how to fix it, seeking help from someone with the necessary expertise if needed.
A reassuring response from the model, however, does not guarantee that the software is secure. Even the GitHub Copilot documentation calls for generated code to be verified and presents assisted code review as a supplement to human review. Having the same model write both the code and the tests can also preserve the same misunderstanding in both.
Addressing at least some of these gaps, provided we verify the results and do not mistake statistical generators for guarantees of security, means using these technologies to prevent disasters, rather than churning out bullshit without the slightest idea of what we are doing.
References
- NIST SP 800-218: Secure Software Development Framework. Practices for reducing vulnerabilities throughout the software development life cycle.
- OWASP Application Security Verification Standard: requirements for designing and verifying web application security.
- OWASP SAMM: AI-Assisted Development. Guidance on integrating generative assistants into development and security processes.
- GitHub: responsible use of Copilot agents, covering capabilities, limitations, and guidance for reviewing generated code.
- U.S. Copyright Office: Copyright and Artificial Intelligence. Reports on model training and copyright protection for generated outputs under U.S. law.
