Operations and → Management Operating Responsibility Culture

Culture of operational responsibility

1) Why do you need it

Technology provides tools, but people and their behavior create reliability. An operational responsibility culture makes the platform predictable, accelerates disaster recovery, reduces "noise" and turns incidents into fuel for improvement.

• A unified understanding of "what reliability is" and who is responsible for it.
• Transparent roles, responsibilities and authorizations in operations.
• A safe environment for discussing errors and quick adjustments.
• Rhythmic improvement of SLO, reaction time and cost of operations.

2) Principles (core of culture)

1. You build it — you run it. The team owns the quality of their domain from code to on-call.
2. SLO-first. Decisions are evaluated through impact on SLO and error budget.
3. Blameless & factual. Post-mortems without accusations, only facts, data and actions.
4. Small & reversible. Small changes, phicheflags, canaries, quick rollback.
5. Safety to speak up. Everyone can raise a "red flag" without fear.
6. Evidence over opinions. Data and artifacts are more important than opinions and status.
7. Continuously learn. Incidents → hypotheses → experiments → standards.

3) Roles and ownership

Domain owner (Payments/Bets/Games/KYC): SLO, on-call, improvement roadmap, error budget.
Incident manager (rotation): coordination of reaction, timeline, quality of communications.
SRE/Platform: reliability tools (observability, alerts, phicheflags, canaries).
Team Lead/EM: expectations, development of competencies, observance of rituals.
Business stakeholder: aligns SLO/priorities, accepts risks/compromises.

4) SLO as liability contract

The only source of truth: the definition of metrics, windows, exceptions.
Error budget: explicit risk boundaries → gate to releases/experiments.
SLO discussion: "will this release burn 20% of the budget? ».
Revision once a quarter: together with the product and business.

5) On-call and incident preparedness

Clear expectations: reaction time, channels, authority (stop valve right).
Training: emulation of incidents, shadow duty, DR exercises.
Artifacts: live runbook 'and, escalation matrix, update templates.
Caring for people: shift load, compensation, rotation, "no heroics" policy.

• Access and VPN verified.
• Notification channels and backup contacts are healthy.
• Runbook updated ≤ 30 days ago.
• DR participation in the last 90 days.

6) Communications in operations

Uniform templates: short updates on incidents, "handover" packages between shifts.
Publicity of decisions: key compromises are recorded in writing.
Annotations on graphs: releases, feature flags, provider windows.
SLO panels for all: transparency of statuses and error budget.

[HH: MM] P2 Games latency ↑ p99 to 420 ms (base + 28%). Canary rolled back.
ETA of the next update: 20 min. Owner: squad-games. Next steps: tuning the breaker, checking provider Y.

7) Postmortem without charges

Facts and timeline: who, when, what did, on the basis of what data.

Systemic causes: processes, tools, interfaces, not "culprits."

Actions with deadlines: corrective and preventive.
Lessons learned: standards, checklists, runbook updates.

Impact: <metrics/revenue/users>
Timeline: <UTC+TZ>
Root cause: <system cause, not personalities>
Fix now: <3 actions + owners + ETA>
Prevent: <3 process/tool improvements>
Signals to watch: <SLO/metrics>

8) Rituals and cadence

Weekly Ops Review (30 min): SLO, incidents, alerts, action progress.
Monthly reliability retrospective: lessons, trends, updating standards.
Quarterly Reliability Review: SLO/budget revisions, roadmap integration.
Game-days/Chaos: planned scenarios of failures and development of the feilover.

9) Motivation, growth and trajectories

Competencies: he-call, incident-management, observability, SLO-engineering, FinOps.
Career levels: reliability contribution expectations (initiatives, post-mortems, mentoring).
Intangible motivation: recognition, authorship of improvements, "Reliability Champion" of the quarter.
Material: on-call compensation, bonuses for achieving SLO-KPI.

10) Policies and norms of conduct (fragments)

• Any he-call can pause the release/feature when SLO is threatened.
• The decision is fixed and revised after the risk is eliminated.

• Major changes only with phicheflags and canaries.
• Autogates by SLO metrics; deviation → pause/rollback.

• All critical information is in common channels, without private solutions.
• ETS (Estimated Time to Status) is mandatory for P1/P2 incidents.

11) Culture Metrics (Maturity KPIs)

SLO Coverage: proportion of critical paths with formally described SLO/alerts.
Pre-Incident Detect Rate: percentage of incidents intercepted at the degradation stage.
MTTR/MTTD: Dynamics by Quarter.
Change Failure Rate: Post-release pullbacks/regressions.
Postmortem Action SLA: Proportion of actions closed on time.
Alert Fatigue Index: On-call/shift alerts.
Handoff Quality Score: Quality of passing between shifts.
Psychological Safety Pulse: short regular survey (anonymous).

12) Implementation checklist

• Domains, owners, on-call and SLO are defined.
• Stop-valve, post-mortem and communications policies have been adopted.
• Raised SLO panel and release annotations.
• Rituals launched: weekly Ops review and template handovers.
• Post-mortem templates and an action tracker have been developed.
• The first game-day/DR exercise was held.
• Set up culture KPIs and review monthly.

13) Anti-patterns

Cult of heroes: save at the last minute instead of system fixes.
People's fault: finding the "culprit," not the cause.

Hidden solutions: private chats, "verbal agreements."

Big nightly releases: no flags or canaries.
Metrics without action: reports are there, solutions are not.
Chaotic handovers: there is no template and confirmation of acceptance.

14) Tools and artifacts (minimum)

SLO/alert directory with owners.
Runbook repository (by domain, update ≥ monthly).
Templates: postmortem, handover, incident update, degradation plan.
Панели: SLO Overview, Incidents, Change Safety, Providers.
Activity tracker: a single backlog with SLAs and owners.

15) Embedding in HR circuits

Onboarding: trainings in SLO, on-call, postmortems.
Performance evaluation: Contributions to reliability and culture are part of the performance review.
Mentoring: shadow duty, pair incident management.
Pulse surveys: quarterly safety/burnout assessment.

16) 30/60/90 - start-up plan

• Assign domain owners and on-call, fix at least by SLO (p95, success rate).
• Adopt stop valve and post-mortem policies, approve templates.
• Launch a weekly Ops review and handover ritual.

• Conduct 2 game-day/DR exercises, raise the SLO and Change Safety panel.
• Build a canary and auto gates via SLO on 1-2 critical services.
• Run culture metrics (MTTR, Action SLA, Pulse survey).

• Trend analysis, updating SLO/budgets, integrating improvements into the roadmap.
• Introduce a "Reliability Champion" and a mentoring program.
• Adjust rituals and policies based on retrospective results.

17) Templates (fragments)

scope: P1/P2 and repeated P3 timeline: ≤72 hours format: impact, timeline, root cause, actions (now/prevent), owners/due dates review: monthly on Reliability Review blameless: specifying personalities only as a fact of time stamp

SLO summary     Incidents and ETAs    Providers and quotas    Releases/Canaries    Risks/observations     Action items

- Ficheflags/canary set up
- SLO alerts and annotations included
- Rollback plan and "safe mode" defined
- Provider windows considered
- Responsible on-call confirmed

18) FAQ

Q: How do you measure "culture," not just technique?
A: Enter Pulse-poll (psychological safety), Action SLA postmortems, share of public decisions, HQS handovers.

Q: What to do about "heroism"?

A: Thank you, but record systemic changes so that heroism is not required. In performance, consider prevention and improvement, not just "feats."

Q: How do you convince a business of the value of culture?
A: Show connection: lower MTTR/Change Failure Rate → higher conversion/revenue, fewer fines and late-night pages, predictable releases.