A critical category of production incidents is currently slipping through the cracks for many engineering teams, primarily because existing postmortem frameworks are ill-equipped to categorize them.
The scenario unfolds when an AI agent, operating with incomplete contextual awareness, initiates a technically correct action that triggers a cascade failure across infrastructure. By the time the incident review convenes, teams are often left debating whether the root cause lies with the agent’s decision-making or a flaw in the underlying infrastructure, as the methodologies for analyzing these distinct domains have historically remained disconnected.
The potential for such exposures is no longer theoretical. A significant 79% of organizations now deploy some form of AI agent in production, with an overwhelming 96% planning to expand their usage. While Gartner forecasts that 33% of enterprise software will incorporate agentic AI by 2028, the firm simultaneously warns that 40% of these projects will falter due to inadequate risk management controls.
What these statistics fail to capture is the failure mode occurring precisely between these two figures: agents that are operational, not canceled, yet quietly generating infrastructure events that remain unclassified as risks.
My experience spans six years dedicated to building enterprise-scale infrastructure automation systems. Initially, at Cisco, I led AI-driven lifecycle platforms deployed across more than 20 global enterprise clients. Subsequently, at Splunk, I focused on designing AI-assisted root cause analysis and observability workflows utilized in thousands of enterprise environments.
During this period, I also contributed to a patent for an intent-based chaos engineering methodology. Across all these roles, I consistently observed organizations making a fundamental structural error: treating autonomous agents and chaos engineering as separate disciplines. In reality, they are intrinsically linked aspects of the same discipline, and the gap between them is now quietly giving rise to the next wave of significant production incidents.
The Crucial Judgment Call That Agents Often Omit
To grasp the significance of this issue, it’s essential to first understand the inherent limitations in how enterprises currently manage chaos, even before introducing agents.
Most mature engineering organizations have established chaos engineering programs, incorporating practices like game days, blast radius controls, and SLO-gated experiments. When a human engineer orchestrates a chaos experiment, a critical element is present: a human makes a judgment call regarding the system’s current capacity to withstand the proposed perturbation. This involves reviewing dashboards, assessing error budget burn rates, and evaluating the stability of dependencies. While imperfect and often reliant on intuition, this human-in-the-loop process ensures a crucial assessment is made before introducing stress.
The introduction of an autonomous remediation agent—one capable of restarting services, rerouting traffic, scaling resources, or modifying configurations in response to detected anomalies—eliminates this vital human judgment step. The agent identifies an anomaly and executes an action, which is essentially a chaos event. This occurs without an explicit check of the SLO burn rate, a calculation of the blast radius, or any human assessment of whether the current moment is appropriate for introducing additional stress into a system that might already be contending with pressure from multiple other sources.
I have witnessed a specific failure mode play out repeatedly: A remediation agent detects elevated latency in a microservice and responds by restarting the service cluster—a seemingly logical action based on its training data and narrow operational view. However, the agent is unaware that three other services are simultaneously handling peak traffic, the shared connection pool is already at 87% utilization, and a dependent database is executing a background index rebuild. The agent’s restart action then triggers a “thundering herd” problem against the recovering service.
What began as a latency spike, the very issue the agent was designed to resolve, escalates into a cascading failure that the agent was never equipped to model. The true blast radius of the agent’s action was not merely the service restart itself, but everything downstream, impacted by a system state the agent lacked complete visibility into.
Crucially, no existing chaos engineering program had tested for this specific confluence of events, and no blast radius calculation had accounted for the agent as an active participant. This is because we have been hesitant to classify agents as chaos injectors, when, in fact, we should be doing so.
According to the AI Incidents Database, reported AI-related incidents saw a 21% increase between 2024 and 2025. This figure likely underestimates the actual exposure, as most organizations lack incident classification schemas that identify an autonomous agent’s action as the primary trigger for a cascade. Consequently, incidents are often logged as mere service restarts, connection pool saturations, or latency events, rendering the agent’s role invisible in postmortem analyses.
Absorb Capacity: An Under-Managed Resource in Most Systems
The fundamental issue lies in the absence of a shared understanding and consistent measurement of “absorb capacity” across enterprise systems. Absorb capacity represents a real-time estimation of how much additional stress a system can endure before breaching its Service Level Objective (SLO) commitments. Current chaos engineering programs manage this implicitly through human judgment and static thresholds that only activate after a limit has already been transgressed. Agents, conversely, do not manage this capacity at all.
Through structured primary research involving Site Reliability Engineering (SRE) and platform engineering practitioners from organizations such as Intuit and GPTZero, I have been developing a resilience budget model. The core principle of this model is to treat absorb capacity as a continuously recalculated, consumable resource, rather than a static threshold that one aims to avoid breaching.
A resilience budget is informed by four critical live signal classes:
- SLO Burn Rate: This serves as the primary input, directly quantifying the deviation between the system’s current performance and its most critical commitment. If a system is consuming its monthly error budget at five times the expected rate, its resilience budget is effectively depleted, irrespective of other resource utilization metrics like CPU.
- P99 Latency Trend: The trajectory of P99 latency is more informative than its absolute value. A service exhibiting an upward trend over 40 minutes signals a different state of stress than one that has remained stable at the same absolute latency level.
- Dependency Saturation State: This is the most frequently overlooked signal. A chaos experiment or agent action that assumes a shared connection pool is readily available, when it is actually operating at 87% capacity, will invariably lead to failure modes that were never anticipated during the design phase.
- Application Behavioral Signals: Metrics such as session completion rates, shifts in API call patterns, and degradation in conversion rates can reveal system stress earlier than infrastructure metrics. This is because end-users typically experience performance degradation before monitoring systems like Prometheus report it.
What differentiates this model as a “budget” rather than a mere “threshold” is its consumable nature. Every chaos experiment depletes the available capacity, and every agent action draws from it. In complex, multi-team organizations where numerous experiments and agents might operate concurrently, this budget becomes a shared resource. Without a transparent ledger tracking consumption, two teams running experiments on overlapping dependencies could collectively exhaust the resilience budget, resulting in a combined blast radius neither team had planned for. Introducing autonomous agents operating entirely outside this ledger system causes the entire accounting process to collapse.
The Role and Limitations of Large Language Models in Chaos Engineering
Several engineering organizations are currently leveraging Large Language Models (LLMs) to generate chaos hypotheses. These models analyze dependency graphs and incident postmortem corpora, yielding results that are directionally valuable. LLMs can surface plausible failure modes that experienced SREs recognize as worthy of investigation, and they can generate hypotheses more rapidly than manual methods, particularly when working with extensive historical postmortem data.
However, a significant limitation emerges from the staleness of dependency graphs. This poses a hard constraint. A hypothesis generated from a dependency graph that does not reflect services extracted last month or a new shared library dependency introduced two sprints ago will inherently propose experiments with inaccurate blast radius assumptions. The issue isn’t that the model makes an error; it’s that the model is unaware it’s making one. It will confidently present incorrect information regarding a system boundary that is no longer operative. In the realm of chaos engineering, such confident incorrectness in a production environment can directly lead to an unplanned outage.
Research from Stanford’s Trustworthy AI Research Lab has demonstrated that model-level guardrails alone are insufficient; fine-tuning attacks successfully bypassed leading models in the majority of tested scenarios. For chaos hypothesis generation, this implies that a model incapable of reliably maintaining its own safety boundaries cannot be trusted to accurately model the blast radius of an action it has never encountered within a dependency graph it has not verified.
When hypothesis generation draws from postmortem corpora, the staleness problem is significantly mitigated. Postmortems document failures that demonstrably occurred within the system at specific points in time, providing a signal validated by production realities. This represents a tractable near-term application for AI in this domain, proving genuinely beneficial for organizations with mature incident documentation practices.
What AI cannot, and should not, be tasked with is making execution decisions when signals are ambiguous. Such judgment requires an awareness of factors that exist entirely outside any monitoring system: pending deployments that altered the dependency landscape an hour prior, on-call staffing levels during a holiday weekend, or a client commitment that renders any additional risk unacceptable until Monday. A model lacking access to such context should not be making these critical calls. This is not a temporary limitation awaiting a more advanced model; it is a structural constraint inherent to machine observability. Building an agent architecture that disregards this constraint is fundamentally constructing a system that will inevitably make a consequential decision based on incomplete information, without a human in the loop to intervene.
Implications for Enterprise Production Agent Governance
The governance implications are straightforward to articulate but challenging to implement. Every autonomous agent action that interacts with infrastructure must be registered against the same live signal layer that governs chaos experiments. The same SLO burn rates, latency trends, and dependency saturation states that a human engineer would scrutinize before initiating an experiment should serve as gating conditions for agent actions, dictating both permission and timing. If the resilience budget falls below a predefined threshold, the agent must either wait or escalate; it should not proceed with execution.
Furthermore, agent actions need to be treated and modeled as experiments, not merely logged as events. When an agent restarts a service, the critical question extends beyond whether the restart was technically successful. It must also assess whether the blast radius of that action was proportionate to the available absorb capacity and what cascading effects it induced across dependent systems. This data constitutes chaos engineering intelligence and must be incorporated into the resilience budget model, informing subsequent decisions made by both the agent and the human team.
In genuinely ambiguous situations—when the resilience budget score is unclear, when a recent deployment has altered the system topology in ways not captured by the agent’s context window, or when dependency states are in flux—the execution decision must be deferred to a human operator. This is not a concession regarding agent autonomy but a fundamental engineering requirement dictated by the current technological landscape. A circuit breaker mechanism that escalates ambiguous cases to human oversight is not a deficiency in agent architecture; it is the critical component that renders the architecture trustworthy enough for production deployment. Intent-based verification formalizes this by defining expected agent behavior prior to deployment and continuously validating adherence to those boundaries under live system conditions.
Organizations that reliably operate autonomous agents at scale are not those with the most sophisticated AI models. Instead, they are the ones that recognized, before critical failures occurred, that every agent action constitutes a chaos event and proactively built their governance frameworks accordingly.
The initial practical step, while perhaps unglamorous, is crucial: audit every autonomous agent currently interacting with infrastructure. Map its action surface against your live SLO burn rate signals and establish explicit floor conditions below which the agent is mandated to pause or escalate. This audit is likely to reveal agents operating entirely outside your existing resilience accounting mechanisms.
The vast majority of organizations deploying agents at scale today have several such agents. Identifying them proactively is far preferable to discovering them through a production incident.
Sayali Patil possesses over six years of experience at Cisco Systems and Splunk, where she was instrumental in developing the reliability and automation systems essential for scaling enterprise AI infrastructure.
Business Style Takeaway: The increasing deployment of autonomous AI agents necessitates a paradigm shift in risk management, integrating their actions into existing chaos engineering frameworks. Organizations must proactively audit these agents and implement governance layers that account for their potential impact on system resilience to prevent novel and complex production incidents.
Details can be found on the website : venturebeat.com