By Simon Rizkalla, VP, Customer Advocacy, New Relic
Development velocity has always been tied to business success, which is one reason why there’s been an increase in reliance on AI to ship products faster. New Relic’s AI Coding Report, found that 67% of tech leaders reported that AI now generates or significantly refactors between 51% and 75% of an organisation’s weekly code output. However, apps also need to work seamlessly to solve pressing customer pain points.
While 94% of tech leaders rate AI code as being higher quality than code written by humans, 82% have had at least one major production failure tied to AI-generated code in the past six months. It’s clear that businesses cannot rely on AI coding if it fails at the customer touchpoint. Part of the reason is that 62% of teams ship AI-generated code without reviewing it: code reaches production and lives there, without the right application monitoring in place. But the tricky part is that conventional APMs aren’t the best option either, as they weren’t made for the AI coding era.
Finding the “right” application monitoring for AI code
Monitoring AI-coded applications can’t be done in the same way as monitoring traditional apps. Conventional apps work with predictable logic, but AI introduces complexities. AI monitoring needs to capitalise on the speed of technology without compromising on business outcomes. Otherwise, customer confidence can deteriorate.
AI coding also introduces the issue of rising token consumption, which can eat into business margins and drive up infrastructure and API costs. For instance, an AI agent may blindly edit the same code without verifying the output, or repeatedly call the same tool and get stuck in a loop, without making any progress. This can spiral costs out of control without a businesses’ knowledge – until the bill arrives. These kinds of AI-specific complexities make observability indispensable, with research showing that in India, 73% have deployed AI monitoring, but most are running it across multiple, disconnected tools. This delivers fragmented coverage and leaves blind spots. Some 86% report an increase in the time senior staff now spend fixing AI-generated code, and those teams can’t fully trust their tools.
Monitoring AI-coded applications using multiple tools invites more incidents than reliability. Consolidated observability that spans the entire lifecycle is necessary and covers the prompt, response, user feedback, and shipped code, as well as visibility into what AI coding assistants are spending.
Maximum speed, minimum firefighting
The promise of AI coding disappears as soon as an incident occurs. It’s vital that code holds up under real-world conditions, not just in sandboxes or during review. If it breaks the moment traffic spikes, or when a user takes an untested path, the code looks good but adds little value to the business.
Intelligent observability is built for this era of AI-written code. It covers the entire AI stack through 360-degree visibility and monitors the entire lifecycle. This granular-level visibility is what helps catch AI drift or resource overconsumption early, with real-time visibility able to spot issues before AI starts showing minor errors that eventually become incidents.
Consolidated visibility also avoids the need for piecing together insights from disconnected tools, removing guesswork from the equation. Businesses can connect the dots faster, get to the root of the problem, and quickly take action, compressing the mean time to resolve (MTTR). They can also configure sessions, daily or weekly budget limits, and receive alerts in real-time, while also providing insights into periodic token spend. These features allow businesses to stay on top of AI expenses and help prevent bill shock.
Observability also enables teams to improve the performance of applications based on end-user feedback. This helps businesses analyse the performance of AI-generated code and make improvements to enhance the overall experience of customers. Intelligent observability also helps compare costs incurred by different models, which is extremely useful when companies use more than one AI model to run their business. It comprehensively compares token consumption across toolsets, identifying the tools breaking the bank and doubling down on those that deliver the most value at the lowest cost.
Business outcomes over vanity metrics
Speed should never be the ultimate goal for AI coding – it’s more about creating a path to achieve business outcomes. To leverage the benefits of AI’s development velocity, AI-generated code should be treated as unverified until being observed in real-world scenarios. Code that breaks at a customer touchpoint is worthless, no matter how long it took to generate. The investment only pays off if the code stays observable and performs the way it should across every touchpoint. Speed without visibility isn’t an advantage – it’s a looming risk moving faster.