Post-incident debriefings

1) Why post-incident parsing is needed

Post-incident debriefing (post-mortem/AAR) is a structured process for training an organization after a failure. The goal is not to find blame, but to identify root and contributing causes and consolidate measurable actions (CAPAs) that reduce the risk of recurrence and the cost of incidents, improving SLO, MTTR and customer/regulatory confidence.

2) Principles (Just Culture)

Without accusations: we analyze systems, decisions and context, not personalities.
Facts are more important than opinions: timeline, logs, metrics, trails, artifacts of changes.
E2E view: from symptoms on the client to internal dependencies and external providers.
Verifiability: each hypothesis is supported by experiment/data.
Loop closure: parsing CAPA → → checkpoints → retests.

3) When to run parsing and what formats are

Required: SEV-0/1; violation of SLA/regulatory requirements; data leakage; significant PR risk.
Accelerated (light): SEV-2 with noticeable influence or recurring symptoms.
Communication AAR: if the failure affects the status page/support, we check the SLAs of updates and the quality of messages.

Terms: draft for 48-72 hours, the final version - up to 5 working days (unless otherwise agreed).

4) Roles and responsibilities

RCA Lead: organizes the process, leads the meeting, is responsible for the quality of the report and CAPA.
Incident Commander (IC): Provides incident facts and solutions.
Tech Leads (by Systems): Cause analysis that confirms artifacts.
Comms/Support/Legal: assessment of communications and compliance requirements.
Scribe: protocol, evidence gathering, compliance with structure.

Product/Business Stakeholders - Customer Impact/Turnover, CAPA Prioritization

5) Preparation: what to collect before the meeting

Time line (UTC): T0 detection → Tn recovery; releases/feature flags/configs, status of providers.
Observability data: SLI/SLO graphs, error-rate, percentiles, logs, traces, screenshots.
Context of changes: links to PR/deploy, DB migrations, feature flags, work plans.
Impact: affected cohorts/regions/providers, downtime minutes, SLA credits.
Communications: drafts/posts on the status page, support answers, internal announcements.
Politicians/playbooks: what should have happened in the process where there were deviations.

6) Analytical procedures (select combination)

5 Why: rapid autopsy of the causal chain (risk - oversimplification).
Fishbone Chart: People/Process/Platform/Policy/Partner/Product.
Fault Tree Analysis (FTA) - deduction from event to multiple causes (AND/OR).
Change Analysis: What changed during the incident vs stable condition.
Causal Graph: Causal graph for complex microservices and external dependencies.
Human Factors Review: Fatigue, information noise, irrelevant runbooks' and.

7) Report structure (template)

1. Executive Summary-What, when, who was affected, the final status.
2. Impact: SLI/SLO, users, regions/providers, min downtime, financial/regulatory effects.
3. Timeline (UTC): key events, releases, IC solutions, communications.
4. Observations and data: graphs, logs, traces, diffuses of configs/schemes.
5. Hypotheses and tests: accepted/rejected, references to experiments/simulations.
6. Root causes: system/process/technical (clear wording).
7. Contributing factors: why not noticed/stopped earlier.
8. What worked/what didn't work: processes, tools, people.
9. CAPA: corrective and preventive actions with owners/deadlines/success metrics.
10. Verification plan: D + 14/D + 30 control points, closing criteria.
11. External versions: client/regulatory (no sensitive data).
12. Applications: artifacts, links to tickets/PR, screenshots of dashboards.

8) CAPAs: how to make actions work

Each action has an owner, a deadline, and an effect KPI (for example, a change-failure-rate reduction of X%, a zero repeat of 90 days, a burn-rate reduction in spikes).
Separate Corrective and Preventive measures.
Link to policy-as-code: alerts, SLO-gates, autoscale/limits, GitOps.
CAPA enters the public backlog with reviews at weekly operational meetings.

9) Effect check and closure

Checkpoints: D + 7 (intermediate), D + 14/D + 30 (main), D + 90 (total).
Verification: tests/simulations (game day), shadow traffic, observability (stable SLIs in the green zone), no relapses.
Closing is only possible with completed CAPAs and validated metrics.

10) Communications and Compliance

Internal: clear status for product/support/management, SLA updates are met.
External: status page, mailings to clients/partners; language without blame, a clear prevention plan.
Regulatory: notification deadlines, depersonalization of examples, unchangeable storage of reports and artifacts.

11) Process Maturity Metrics

Report publication time: actual vs SLA (e.g. ≤5 working days).
CAPA completion rate:% of activities closed on due date.
Reopen rate: the proportion of repeat incidents in 90 days.

Proportion of systemic causes vs "human error."

Alert hygiene: a decrease in false pages, the growth of alerts covered with runbooks.
DORA metrics change: MTTR, change-failure-rate before/after.

12) Checklists

Before parsing

• RCA owner and membership defined.
• Collected timeline and artifacts (logs/graphs/releases/flags).
• Impact assessed by cohort/region/provider.
• Drafts of Impact and Timeline sections have been prepared.
• Relevant policies/playbooks are mapped to actual actions.

During

• Accepted/rejected hypotheses and grounds were recorded.
• Root and contributing causes identified.
• A CAPA plan with KPIs and deadlines has been created.
• Report versions for external parties are agreed (if necessary).

After

• Report published on time, access by role.
• CAPAs are logged, owners are confirmed.
• Test points and mini-simulation are assigned for verification.
• Updated runbook/SOP/alerts/documentation.

13) Anti-patterns

"Guilty man X" - repeat → without systemic reasons.
Report without CAPA or without owners/deadlines - paper for paper.
No facts/artifacts - conclusions on sensations.
Too common language ("database overload") without specific changes.
Ignoring communications and compliance are reputational risks.
Closure without effect testing - relapses after weeks.

14) Mini templates

Report header

Incident: INC-2025-10-31 (SEV-1)
Window: 2025-10-31 18: 05-18: 47 UTC
Owner of the analysis: @ rca-lead
Affected: EU region, payments (success -28% peak)
Status: corrected; 48 hours monitoring

Root cause formulation (example)

CAPA (fragment)

Enable canary routing to PSP-A (1%→5%→25%), owner: @ payments-tl, until: 2025-11-07, KPI: zero P1 incidents when providers release 30 days.
Reconfigure timeouts/retrays with a total ≤ SLA time of 800 ms, owner: @ platform-sre, up to: 2025-11-05, KPI: p99 <600 ms under load N.
Add Business SLI by BIN Cohort, Owner: @ data-lead, to: 2025-11-10, KPI: Degradation Detection <5 min.

15) Embedding in daily practice

Weekly RCA reviews: CAPA status, new lessons, process updates.
Directory of post-mortems in wiki with tags (service, SEV, reasons) and search.
Simulations based on incident in 2-4 weeks to verify measures.
Including lessons in on-call onboarding and updating training scenarios.

16) The bottom line

Post-incident parsing is a mechanism for systemic improvement. When facts are collected, causality is proven, actions are measurable and verified, the organization accumulates reliability operating capital: MTTR and repeat incidents fall, release predictability and customer confidence increase.