Operations and Management → Operator Feedback System

Operator feedback system

1) Why do you need it

Operators see reality before anyone else: alert noise, "blind spots" of dashboards, inconvenient SOPs, pain points of providers and releases. If this experience does not turn into change, the company pays with the growth of MTTR, Change Failure Rate and on-call burnout.

• Consistently collect and digitize shift experience.
• Quickly convert feedback into SOP/alert/dashboard/process fixes.
• Support psychological safety and recognition of the contribution of operators.
• Give transparency: processing status, benefit metrics and economic impact.

2) Principles

1. One Inbox, Many Views: one input feedback stream, different showcases for platform/domains.
2. Actionable> Opinion: capture observation + fact + desired outcome.
3. Traceable: each feedback has an ID, processing owner, status and term.
4. Safe & Fair: anonymity is permissible; personal accusations are prohibited.
5. Close the Loop: mandatory response and demonstration of the result (modified SOP, new alert, etc.).
6. Docs-as-Code: changes in knowledge - through PR with reference to feedback.

3) Collection channels and formats

Structured form (recommended): in the portal/bot (5-7 fields, autocomplete shift).
Shortcat from the incident: "Add feedback" directly from the INC/ticket card.
Handover package: Observations and suggestions section.

Retro/clinics: weekly 30-min analysis of "TOP feedback of the week."

Anonymous form: for sensitive topics (about processes/culture).
Auto candidates: collecting "noisy" alerts and broken links as a potential feedback.

Category: [Alerts/Dashboards/SOP/Tools/Processes/Providers/Comms]
Domain: [Payments/Bets/Games/KYC/Platform]
Description: <what was observed and where>
Data: <links to panels/logs/tickets>
Desired outcome: <how to understand what has become better>
Impact: [P1..P4] (see scale)
Option: Anonymous []

4) Taxonomy and tags

• Alerts (noise/threshold/hysteresis/duplicates)
• Dashboards (metrics/broken links/incomprehensible graphs)
• SOP/Runbook (obsolete/incomplete/no Rollback)
• Processes (handover/incidents/releases/escalations)
• Tools (bots/orchestrator/observability UX)
• Providers (quotas/SLA/feilover)
• Communications (Tone/ETA/Templates)

Теги: `#p99`, `#quota`, `#burn-rate`, `#grafana-link-broken`, `#sop-dod-missing`, `#alert-fatigue`, `#handover`, `#psp-switch`, `#feature-flags`, `#postmortem`.

5) Impact scales and prioritization

• P1 - affects SLO/revenue/security (immediate processing).
• P2 - impairs MTTR/on-call/operability (SLA 5 op. days).
• P3 - Useful Improvement/UX (SLA 15 op. days).
• P4 - nice-to-have/discussion (if available).

Scoring (ideas): 'Score = Impact (P) × Reach × Confidence/Effort', compatible with RICE/WSJF roadmap.

6) SLAs and processing statuses

Статусы: `New → Triaged → In Progress → Waiting Info → Shipped → Verified → Closed`

• Acknowledgement: ≤ 2 employees day (comment + owner).
• Triaged: ≤ 5 slave days (priority, plan).
• First Fix: ≤ 15 slave days to P2/P3 (or transfer to Roadmap with date).
• Close the Loop: mandatory update to the author/channel and the entry "what has changed."

7) RACI (who is responsible for what)

8) Integrations and automation

Incidents/Tickets: Create Feedback button with autocomplete links and context.
Docs-as-Code: PR template, where the field'closes _ feedback _ id' is required.
Observation: collections of "broken links," "outdated panels," "alerts without an owner" → auto-feedback.
AI summaries: once a week - clustering feedback, themes and duplicates; draft responses.
Handover: automatic squeeze "feedback per shift" in # ops-handover.

yaml id: FBK-2025-1147 author: oncall@payments (anon: false)
domain: payments category: alerts impact: P2 title: "Noisy alert ProviderQuota90 for PSP-X"
evidence:
- grafana: /d/providers/psp-x? from=...
- incident: INC-457 problem: "Fires when usage> 0. 85 at brief peaks, no effect on SLO"
desired_outcome: "Add hysteresis/time window, reduce false pages"
owner: squad-observability links: []
status: triaged due: 2025-11-15

9) Procedures (SOP) for feedback

SOP: Admission and triage

1. Check completeness of form (category/domain/impact/evidence).
2. Assign owner and priority.
3. Check duplicates/cluster (AI hint).
4. Reply to author (ETA/plan).
5. Create tasks (alerts/dashboards/SOP/tools).

SOP: Close the Loop

1. Link to PR/ticket/deploy.
2. Short "what changed" entry + effect metric (before/after).
3. Update status'Verified'after confirmation by operator/shift.

4. In # ops-changelog - a card "that was improved by feedback."

10) Dashboards and quality metrics

Feedback Overview: incoming/processed, SLA, distribution by category/domain.
Alert Hygiene: noisy rules before/after, pages/shift, false-positive rate.
Docs Health: expired SOPs, Docs-as-Code coverage, broken links.
Operator Experience (OX): Pulse survey: "how much do tools help?" (0–10).
Impact: estimate of savings (decrease in FTE-hours, MTTR, decrease in incidents).

• Acknowledgement SLA ≥ 95%.
• Close-Rate 30 days ≥ 70% (P2/P3).
• Alert Fatigue − 30% for the quarter in top categories.
• Overdue SOPs (review-SLAs) = 0.
• Operator NPS/OX ≥ +30.
• The share of feedback with measurable Outcome ≥ 60%.

11) Psychological safety and anonymity

Anonymous feed is allowed (only the coordinator is visible by default).
Ban on personal accusations and "witch hunts." Focus on facts/data.
Quarterly "Voice of Operator" meetup: open stage for proposals.
"Red security button": channel for sensitive signals (ethics/compliance).

- Delete personal attacks/secrets/PII.
- We return to the author with a request to reformulate according to the template.
- Disclaimer: feedback is not a promise of implementation, but a response with status is required.

12) Relationship to Roadmap and Prioritization

Weekly - selection of TOP-f/topics → the Roadmap initiative (RICE/WSJF).
Each P1/P2 class feedback affecting SLO is required to have an initiative or change in the nearest sprint.
In the Roadmap card - the field'source: feedback_ids' for traceability.

13) Remuneration and recognition

Reliability Champion (quarterly): best feedback with measurable effect.
Badges for contribution (Docs/SOP/Alert Hygiene).
Public # ops-changelog mentioning authors (if not anonymously).

14) Anti-patterns

"Proposal box" without statuses and deadlines.
Nobody fills out giant forms of →.

Feedback without data: "make it convenient."

Lack of anonymity and security "only in words."

There is no closing of the cycle: "thank you, we will take into account" instead of changes or unfolded failure.
Landfill in chat without a single registry and metrics.

15) Checklists

• Category/domain/impact specified.
• There is evidence (panels/logs/tickets).
• Owner and ETA assigned.
• Duplicates verified.
• Reply sent to author.

• Changes applied (alerts/dashboards/SOPs/tools).
• Effect measured (before/after).
• Author notified, 'Verified' status.
• Added to # ops-changelog.

16) Templates

Feedback: <short title>
ID: FBK-YYYY-NNNN
Author: <Nickname or Anonymous>
Domain/Category: <.../...>
Impact: P1/P2/P3/P4
Description:
Data/References:
Desired outcome:
Risks/Dependencies:
Processing Owner:
ETA/Term:
Статус: New/Triaged/In Progress/Waiting Info/Shipped/Verified/Closed
Outcome (after closing):

Closes: FBK-YYYY-NNNN
Changes: <what is updated in SOP/Runbook/policies>
Before/After: <screen/metric>
Communication Plan: <links to # ops-changelog/instructions>

17) 30/60/90 - launch plan

• Launch a single form/bot, feedback storage and basic Overview dashboard.
• Approve taxonomy, influence scale and SLA.
• Assign RACI, train operators and owners of triage.
• Include the "Add Feedback" button in the incident cards and handover template.

• Enable AI clustering/deduplication and auto-candidates (broken links/noisy alerts).
• Embed Docs-as-Code PR bundle and Roadmap source.
• Conduct 2 "SOP clinics" and 1 "Voice of Operator."
• Reduce Alert Fatigue by 2 categories by ≥15%.

• Close ≥70% P2/P3, achieve Acknowledgement SLA ≥95%.
• Reach Operator OX ≥ + 30, enter rewards/badges.
• Weekly # ops-changelog, regular retro feedback.
• Record standards and metrics in OKR (next quarter).

18) FAQ

Q: How do you avoid drowning in a flood of sentences?
A: Single sign-on, rigid taxonomy, SLAs and scoring. Weekly sorting and Roadmap link.

Q: And if feedback "hurts," but without data?
A: Politely return with a pattern of data/examples. The AI bot helps: it tells you what links to attach.

Q: How to protect yourself from "personal showdowns"?
A: Moderation, anonymous option, "facts/data/outcome" policy, ban on personalities.

Q: What if there is no resource?
A: Publicly record "Not Doing Now" with reason and revision date. Bind to Roadmap.