Making Architectural Guarantees Observable, Enforceable, and Reviewable

Drift Detection

Making Architectural Guarantees Observable, Enforceable, and Reviewable

Traditional observability answers “What happened?” but fails to answer the most critical question: “Did the system behave according to what we said it would do?” This gap leads to Silent Architectural Drift: scenarios where a service returns a "Success" signal (HTTP 200) to a user while internally failing to meet its persistence or atomicity guarantees.

This guide outlines the implementation of Intent Drift Detection for Node.js/TypeScript services.

1. The Core Concept: The Three Truths

We define the request lifecycle through three distinct layers:

* Intent: What we believed would happen (The Declaration).

* Observation: What actually happened (The Fact).

* Outcome Signal: What we told the caller (The Promise).

Drift occurs when the Outcome Signal contradicts the Intent based on the Observation.

2. Responsibility Split (The Governance Model)

To maintain integrity, we separate the reporting of facts from the enforcement of truth.

* Service Layer (Feature Teams): Perform the work (DB, Kafka, APIs). They report facts to internal probes but never decide if the system is "drifting."

* Middleware Layer (Platform/Arch): Intercepts the final response, compares probes against the HTTP status, detects drift, and assigns severity.

3. Reference Implementation

A. Context Propagation

Using AsyncLocalStorage, we ensure the "Belief" (Intent) survives across asynchronous boundaries.

// src/context/intentContext.ts

import { AsyncLocalStorage } from 'async_hooks';

export interface IntentContext {

declarationId: string;

belief: string;

}

export const intentContextStore = new AsyncLocalStorage<IntentContext>();

B. The Control Loop (Middleware)

This middleware is the "Truth Enforcer." It intercepts the 'finish' event of the response to verify if the outcome matches the internal probes.

// src/middleware/intentDrift.middleware.ts

export const intentDriftMiddleware =

(intentDeclaration: { declarationId: string; belief: string }) =>

(req: Request, res: Response, next: NextFunction) => {

const startTime = Date.now();

res.locals.intentProbes = { mongoWrite: 'PENDING', kafkaAck: 'PENDING' };

intentContextStore.run(intentDeclaration, () => {

res.on('finish', async () => {

const probes = res.locals.intentProbes;

const success = res.statusCode >= 200 && res.statusCode < 300;

const persistenceFailed = probes.mongoWrite !== 'SUCCESS';

if (success && persistenceFailed) {

await IntentSignalModel.create({

signalType: 'IntentDrift',

metadata: { service: 'adapter-v1', timestamp: new Date(), traceId: req.headers['x-trace-id'] },

intent: intentContextStore.getStore(),

observation: { probes, statusCode: res.statusCode },

drift: { gapDetected: true, severity: 'HIGH', description: 'Success returned but DB write failed' }

});

}

});

next();

});

};

4. Operationalizing via Analytics (The Governance Layer)

We don't just log these signals; we query them to drive architectural decisions. These MongoDB Aggregation Queries run inside your Grafana dashboards or MongoDB Compass for audit reviews.

A. The "Liar" Metric (Success Status vs. Internal Failure)

Run in: Grafana (Real-time)

Identify services that are misleading clients by returning 2xx while internal persistence fails.

// MongoDB Pipeline for Grafana

[

{ "$match": {

"observation.statusCode": { "$gte": 200, "$lt": 300 },

"observation.probes.mongoWrite": { "$ne": "SUCCESS" }

}},

{ "$group": { "_id": "$metadata.service", "count": { "$sum": 1 } } }

]

B. High-Level Drift Health (Severity Distribution)

Run in: MongoDB Compass (Sprint Review)

Visualize the volume of architectural violations this week.

db.intent_signals.aggregate([

{ $match: { "drift.gapDetected": true } },

{ $group: { _id: "$drift.severity", count: { $sum: 1 } } }

])

5. How to Run & Consume This Data

| Layer | Where Queries Run | Frequency | Purpose |

|---|---|---|---|

| Real-Time | Grafana | Every 1m | Alerts on "HIGH" severity drift. |

| Audit | Mongo Compass | On-Demand | Root cause analysis of "Ghost" bugs. |

| Governance | Cron Job / Script | Weekly | Reporting Drift Ratios to Leadership. |

Final Architectural Statement

We are moving from Observability (Watching things happen) to Verification (Verifying promises). By implementing Intent Drift Detection, we ensure that our system's behavior matches its documentation.

This is the source of truth for our architectural guarantees.

Next Step: Would you like me to draft the README.md for the internal @org/intent-runtime npm package so your developers can begin integrating the probe helpers immediately?