According to Harness, AI code generation is exposing severe delivery pipeline limitations within traditional software architectures.
GitHub Copilot recently passed its five-year mark, prompting industry-wide evaluation of large language models (LLMs) in software engineering environments. Martin Reynolds, Field CTO at Harness, provided an objective assessment of the current deployment architecture.
“GitHub Copilot’s fifth anniversary should’ve been a milestone,” says Reynolds. “Instead, it exposed something the industry has been avoiding: you can’t bolt AI onto legacy delivery infrastructure and call it transformation.”
Organisations integrate AI assistants to increase developer output, yet they process that increased output through static, manual continuous integration and continuous deployment (CI/CD) environments. The resulting system congestion negates the initial speed gains.
Pipeline congestion and testing overload
“The ongoing criticism of GitHub’s usage-based pricing and reliability underscores a broader infrastructure problem that hasn’t kept pace with the speed of AI coding,” Reynolds explains.
Production data from enterprise implementations shows a direct correlation between high Copilot adoption and failing build pipelines. Engineering teams deploy generative models to write boilerplate functions, generating massive volumes of pull requests. Legacy CI/CD systems respond by queuing these requests, exhausting cloud compute budgets, and failing to execute security scans within acceptable timeframes.
“Copilot got developers writing code faster, then left them stranded with pipelines, pricing models, and governance frameworks that were never designed for this volume or velocity,” says Reynolds.
Implementing generative coding tools requires an entirely rebuilt approach to continuous testing. Traditional software development life cycles (SDLC) rely on human-paced code commits. Developers typically submit two or three substantial pull requests daily. AI assistance increases that output exponentially, flooding the pipeline with code requiring automated static application security testing (SAST), dynamic analysis, and peer review.
“Faster code generation only helps if the rest of the SDLC can keep pace with testing, security reviews, governance, and cost controls,” states Reynolds. When platform teams fail to upgrade their testing automation alongside their coding tools, the delivery pipeline fractures under the load.
Platform engineering leads report severe degradation in build times after rolling out AI assistants across large developer fleets. Compute costs escalate as automated test suites run repeatedly against AI-generated code containing subtle logic errors or security vulnerabilities.
“What we’re seeing across enterprise teams is that the ‘use as much AI as possible’ approach is hitting a wall, because the foundational elements of the SDLC weren’t updated to support the new, more automated era of software engineering,” Reynolds explains.
Companies effectively pay twice: first to license the AI and generate the code, and second to cover the inflated cloud infrastructure bills required to process, test, and inevitably reject a high percentage of that generated output.
To absorb the output of AI coding assistants, platform teams deploy heavily-modified Kubernetes clusters dedicated entirely to ephemeral build runners.
Traditional static server allocations collapse under the burst traffic generated by AI-assisted development teams. Engineers configure these dynamic environments to spin up hundreds of isolated containers in response to an influx of pull requests, execute unit tests simultaneously, and terminate immediately to conserve cloud budget. This adjustment represents the baseline requirement for enterprise AI adoption.
Static and dynamic application security testing pipelines require identical modernisation. Legacy SAST tools scan entire monolithic codebases overnight. AI code generation demands instantaneous, incremental scanning at the developer endpoint before the code ever reaches the central repository.
Deploying localised and highly-tuned scanning engines directly into the IDE prevents AI-generated security flaws from entering the continuous integration queue, preserving compute resources and maintaining deployment velocity.
Evaluating the ROI of using AI for code generation
Determining the exact return on investment (ROI) regarding AI coding assistants remains a contested exercise inside enterprises. Financial officers demand precise metrics linking subscription costs to deployed features, but current deployment structures obscure this data entirely.
“Costs are under scrutiny, ROI stays murky, and companies can’t tell the difference between AI spend that ships valuable code and AI spend that generates expensive noise,” states Reynolds.
The macro outlook on these costs is growing increasingly stark. According to data from Gartner, by 2028, AI coding costs are projected to overtake the average developer’s salary due to rising LLM token consumption and the shift to consumption-based licensing models.
The shift from predictable seat-based licensing to consumption-based pricing among AI vendors introduces variable cost structures for software engineering workloads. Many vendors currently lack transparency into how token consumption is calculated and billed, severely limiting an enterprise’s ability to forecast and control budgets.
“Organisations are rapidly moving from experimentation to scaled deployment of AI coding agents, but many are underestimating the financial impact of rising token consumption,” says Nitish Tyagi, Senior Principal Analyst at Gartner.
“Token discipline will not emerge through developer choice alone, as developers tend to optimise for speed and convenience over cost efficiency. Without a governed engineering operating model, costs can escalate faster than the productivity gains these tools are designed to deliver.”
Currently, token overspending is heavily driven by governance gaps, including ungoverned autonomy in agent-driven workflows, bloated context windows, and a total absence of structured feedback mechanisms to optimise usage. As light users become mainstream users, reliance increases, driving further growth in token consumption and overall spend.
Platform engineering directors often now track the exact percentage of AI-generated code that passes automated testing and reaches production. If an engineering squad demonstrates a high rate of rejected pull requests, the system automatically throttles API access to the generative model. This hard constraint forces developers to review AI output locally rather than relying on cloud-based CI/CD pipelines to catch basic syntax errors.
Implementing a disciplined operating model
To curb these snowballing expenses and avoid massive budget blowouts, Gartner advises tech leaders to act fast and establish a disciplined strategy for daily AI usage:
- Create a scenario-based decision framework: Map out exactly when to hand tasks over to AI coding assistants and how much freedom they should have. Every engineering task ought to fall into one of three clear categories: entirely human-driven, a human working alongside an agent, or fully-autonomous.
- Align model selection with task complexity: Implement intelligent model routing strategies. Engineering and platform teams should direct simpler, high-frequency tasks to smaller and more cost-effective models, reserving expensive frontier models exclusively for complex, high-value development work.
- Mandate context engineering practices: Train developers to rigorously optimise the input context provided to AI systems. This means stripping away fluff, summarising data where possible, and cutting out non-essential info to aggressively lower token usage while keeping output quality sharp.
- Establish guardrails and limits: Weave automated safety nets – like token caps, escalation triggers, and real-time tracking – straight into the daily development workflow to stop runaway costs before they start.
- Make token audits routine: Require teams to analyse heavy token usage during normal sprint retrospectives. It’s an easy way to pinpoint waste, tweak coding habits, and help different squads swap money-saving tips.
Software engineers face pressing demands to deploy next-gen agentic AI models. These advanced systems operate with higher autonomy, attempting to manage entire repositories, resolve complex bugs, and execute infrastructure changes without human intervention. However, releasing these autonomous agents into legacy delivery pipelines introduces unacceptable risk.
“Organisations need to ensure they’re not just racing deeper into agentic AI. They need to get the basics right first, so AI-generated work can move through the SDLC without creating new risk at every stage,” Reynolds concludes.
Establishing those basics requires stripping down and rebuilding the entire deployment mechanism. Financial controls must integrate directly into the development environment, evaluating the infrastructure cost of proposed code changes before the pull request initiates a build sequence.
Looking back at five years of GitHub Copilot, it’s obvious that cranking out code is just a tiny part of the puzzle. The teams that actually win are putting their energy into overhauling the delivery pipeline, ensuring the underlying architecture can effortlessly compile, validate, and ship whatever code the AI throws at it.
See also: Google Cloud details full-stack AI architecture for developers
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including the Cyber Security & Cloud Expo. Click here for more information.
Developer is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.