Erosion Patterns

The dominant narrative around trust failure in AI systems focuses on catastrophic events: the agent that makes a disastrous financial decision, the autonomous vehicle that causes an accident, the chatbot that produces harmful content. These events are dramatic, newsworthy, and relatively rare.

The far more common - and far more dangerous - mode of trust failure is erosion. Trust erodes through accumulated micro-failures that individually seem insignificant: the agent that is slightly late in reporting, the recommendation that is slightly off-target, the constraint that is slightly exceeded. No single event triggers alarm. But over weeks and months, the human's willingness to delegate quietly diminishes until the relationship is abandoned.

Erosion is dangerous precisely because it is invisible. There is no moment of crisis, no error message, no dramatic failure. The human simply stops delegating - and often cannot articulate why. The AXD designer's task is to make erosion visible, measurable, and preventable through designed intervention patterns.

Silent degradation occurs when the agent's performance declines without reporting it. The agent continues to operate, continues to produce results, but the quality of those results gradually deteriorates. The human, who has delegated precisely because they do not want to monitor every action, has no visibility into the decline.

In agentic commerce, silent degradation manifests as the purchasing agent that gradually accepts worse prices, the portfolio manager that slowly drifts from the investment strategy, or the negotiation agent that incrementally concedes more than necessary. Each individual action is within tolerance. The cumulative effect is significant.

Design intervention: Proactive performance reporting. The agent must be designed to monitor its own performance against baseline metrics and report deviations before they accumulate. This requires the agent to maintain a model of its own expected performance - a form of computational self-awareness that is a core requirement of trust architecture.

Expectation drift occurs when the agent's behaviour diverges from the human's mental model of what the agent should be doing. The agent may be performing well by objective measures, but the human's expectations have evolved - or were never precisely aligned with the agent's actual behaviour in the first place.

This pattern is particularly insidious because neither party is "wrong." The human's expectations are legitimate. The agent's behaviour is competent. But the gap between them widens over time, creating a growing sense of unease that the human may not be able to articulate.

In agentic commerce, expectation drift manifests when a shopping agent optimises for price while the human has gradually shifted to prioritising quality, or when a financial agent maintains a risk profile that no longer matches the human's evolving life circumstances.

Design intervention: Periodic alignment rituals. The system must be designed to periodically surface the agent's current operating parameters and invite the human to confirm, adjust, or recalibrate. These are not interruptions - they are trust maintenance ceremonies that prevent the slow divergence of expectation and behaviour.

Opacity accumulation occurs when the agent's decision-making becomes progressively less legible to the human. Early in the relationship, the human may understand the agent's reasoning. But as the agent handles more complex tasks, encounters more edge cases, and develops more sophisticated strategies, its reasoning becomes opaque.

The human does not lose trust in a single moment of confusion. Instead, each slightly-opaque decision adds a thin layer of uncertainty. Over time, these layers accumulate into a wall of incomprehension that makes the human feel they have lost control - even if the agent is performing well.

Design intervention: Adaptive explainability. The agent's communication must scale with complexity. As decisions become more complex, the explanation mechanisms must become more sophisticated - not more verbose, but more structurally clear. This requires designing explanation systems that can operate at multiple levels of abstraction, allowing the human to drill down only when they need to.

Recovery stall occurs when the system lacks mechanisms to rebuild trust after a failure. The agent makes a mistake - perhaps a significant one - and the system has no designed pathway for acknowledgment, explanation, correction, and gradual trust restoration.

Without recovery mechanisms, the failure becomes permanent. The human's trust drops to the level established by the failure and never recovers. The agent may continue to perform well, but the human's willingness to delegate remains frozen at the post-failure level. Over time, the human either abandons the agent or constrains it to such a narrow scope that it becomes useless.

Design intervention: Recovery architecture. Every agentic system must include designed recovery pathways that address three phases: acknowledgment (the agent recognises and reports the failure), explanation (the agent provides an honest account of what went wrong and why), and demonstration (the agent proves through subsequent behaviour that it has learned from the failure). Recovery is not automatic - it must be designed as carefully as the primary trust formation pathway.

Consent decay occurs when the original delegation agreement becomes stale. The human granted authority under specific circumstances that have since changed - but the agent continues to operate under the original mandate. The consent was valid when given but has decayed through changed circumstances. Design intervention: consent refresh mechanisms that periodically revalidate the delegation agreement against current conditions.

Competence plateau occurs when the agent's capabilities fail to grow with the human's expectations. Early in the relationship, the agent's competence exceeds the human's expectations, building trust rapidly. But as the human's expectations rise - as they should in a healthy relationship - the agent's competence plateaus. The gap between expectation and capability widens, eroding trust even though the agent's absolute performance has not declined. Design intervention: the Autonomy Gradient, which makes the agent's growing (or stable) competence visible and manages expectations about the pace of capability expansion.

Relationship fatigue occurs when the maintenance burden of the human-agent relationship exceeds the value it provides. Every trust maintenance mechanism - alignment rituals, performance reviews, consent refreshes - requires human attention. If these mechanisms are too frequent, too demanding, or poorly designed, the human experiences fatigue and disengages. Design intervention: adaptive maintenance scheduling that calibrates the frequency and depth of trust maintenance interactions to the relationship's maturity and the human's engagement capacity.

The seven erosion patterns are not independent - they interact and compound. Silent degradation accelerates expectation drift. Opacity accumulation makes recovery stall more likely. Consent decay feeds relationship fatigue. The AXD designer must address erosion as a systemic phenomenon, not a collection of isolated risks.

This requires three design commitments. First, erosion monitoring: instrumentation that detects the early signals of each erosion pattern before they become visible to the human. Second, erosion intervention: designed mechanisms that arrest erosion when detected, through proactive communication, alignment rituals, and adaptive explanation. Third, erosion prevention: architectural decisions that make certain erosion patterns structurally impossible - for example, mandatory performance reporting that prevents silent degradation by design.

Trust erosion is the quiet killer of agentic relationships. The organisations that master erosion design will build the most durable human-agent relationships in the agentic economy. Those that ignore it will watch their users silently walk away.