AI post-deployment monitoring and incident response
Primary statement
AI post-deployment monitoring operates per NIST AI RMF MANAGE-4.1 (capturing user/AI actor input, evaluating system performance, drift detection) + MANAGE-2.3 (procedures for previously-unknown risks) + MEASURE-2.7 (adversarial ML security evaluation) + EU AI Act post-market monitoring + DPDPA incident notification flow.
Audit-fatigue payoff
A single AI post-deployment monitoring programme — drift detection + adversarial ML monitoring + incident escalation to AI governance — satisfies post-deployment requirements across all 7 contributing frameworks.
Strictness matrix
Scope
Scope: post-deployment monitoring capturing user input + AI actor input + ongoing evaluation + drift detection across all deployed AI systems.
Ceiling source: nist_ai_rmf:MANAGE-4.1
Rationale: NIST AI RMF MANAGE-4.1 specifies the comprehensive monitoring scope.
Threshold
Threshold: procedures triggered by previously-unknown risks — emergent capabilities, harmful outputs, misuse patterns.
Ceiling source: nist_ai_rmf:MANAGE-2.3
Rationale: MANAGE-2.3 sets the emergent-risk threshold uniquely.
Method
Method: drift detection + adversarial ML security evaluation (evasion, poisoning, model extraction) + emergent capability monitoring + user feedback capture + incident escalation to AI governance + integration with broader incident response.
Ceiling source: nist_ai_rmf:MEASURE-2.7
Rationale: NIST AI RMF MEASURE-2.7 adversarial ML evaluation is uniquely AI-specific.
Frequency
Monitoring: continuous. Periodic review: at planned intervals. Adversarial ML evaluation: annual + on material model change.
Ceiling source: nist_ai_rmf:GOVERN-1.5
Rationale: GOVERN-1.5 planned periodic review is the audit anchor.
Evidence
Evidence: monitoring plan + drift detection logs + adversarial ML evaluation reports + user feedback capture records + emergent-risk incident records + post-deployment review minutes.
Ceiling source: nist_ai_rmf:MANAGE-4.1
Rationale: MANAGE-4.1 evidence is comprehensive for AI post-deployment.
Auditor test pattern
Step 1: Inspect AI monitoring plan. Step 2: Sample one deployed AI system; verify drift detection. Step 3: Verify adversarial ML evaluation. Step 4: Verify user feedback capture mechanism. Step 5: Sample emergent-risk incident.
Common findings
Common findings: (1) Monitoring covers model accuracy but not adversarial ML; (2) Drift detection absent; (3) User feedback not fed back to AI governance; (4) Emergent capability monitoring theoretical.