Operational cost optimization
1) Goals and principles
The goal is to reduce costs per unit of business value while maintaining SLO and product quality.
Principles: measure → optimize → automate; ROI prioritization "SLO-first" (savings are not detrimental to the user experience); cost transparency (showback/chargeback).
2) Taxonomy of expenses (what exactly we optimize)
Infrastructure: compute (CPU/GPU), storage (SSD/obj), network/egress, CDN/WAF, backups, logging/observability.
Data & Streaming: Brokers (Kafka), DWH/OLAP (ClickHouse/BigQuery), Caches (Redis/Mem), ETL/Orchestrations.
Payment chain: processing, KYC/AML, scoring, anti-fraud, chargeback fund.
Product/marketing: bonuses, free backs, affiliates (CPA/RevShare), traffic purchases, promotional events.
Operations/people: support, risk analytics, compliance, manual workflows.
Licenses/partners: game engines, live casino providers, SaaS services.
3) Metrics and unit-economics
Baseline:- $/RPS, $/transaction (deposit/bet/withdrawal), $/active player/month, $/GB-ingest logs, $/TB-storage/month, $/ML-inference.
- Summary KPI: Cost to Serve per segment (geo/device/channel).
- $/RPS = (OPEX_infra + OPEX_data + OPEX_3rd + OPEX_ops) / avg_RPS
- $/transaction = (OPEX_platform + fees_payment + antifraud + support )/ N_tx
- LTV: CAC: CTS - Lifetime Value: Customer Acquisition Cost: Cost to Serve.
4) FinOps circuit and "SLO-aware" savings
Showback/chargeback: distribution of costs by products/teams/services.
Budgets and alerts: monthly limits and deviation warnings.
SLO-first: any optimizations are tested: is SLI normal? p95/p99, error-rate, availability.
Experiments: A/B savings (included compression, reduced the retention of logs - did not worsen SLI?).
5) Compute: right-sizing and autoscaling
Right-sizing: profiling CPU/mem, reducing pod/VM sizes, removing "headroom for headroom."
Autoscaling: HPA/KEDA by load metrics/queues; night/regional recessions - aggressive scale-in.
Price models: Reserved/Savings plans, Spot/Preemptible for batch/ETL, hybrid regions.
Runtime updates: modern versions of JVM/Go/Node can give − 10-30% CPU.
6) Storage and data
Storage classes: hot SSD for OLTP, cold/archive for history and logs.
TTL/retention: rules for indexes/logs/traces (for example, 7-14 days of p99-parts, aggregates - longer).
Compression and format: Parquet/ORC for lake, ZSTD for logs, deduplication.
DWH/OLAP: materialized view/aggregates, unloading "expensive" requests; ad-hoc constraint.
Streaming: compact topics, batch-size/acks are optimal for $/msg, fan-out control.
7) Network, egress and CDN
Minimize egress: edge caching, pinning traffic within a region/feast.
CDN economics: cache-hit growth by versioning, reasonable TTL, image-resize at the edge.
Compression and protocols: HTTP/2/3, gzip/br, WebP/AVIF for media.
API chattiness: aggregation/butching, gRPC protocols for chat/streaming.
8) DB and cache: $/request
Profiling: top N slow and frequent queries; indexes covering queries.
CQRS/Reader Cues: Offloading OLTP through read-replica and denormalization.
Cache strategies: key directory cache, session/token, hot rank lists; follow hit-ratio and eviction.
Transaction limit: short transactions, pagination limits and N + 1 requests.
Architecture: async/queues instead of synchronous chains, idempotency and retry jitter.
9) Observability and logs
Sampling traces: dynamic, in case of incidents - increase.
Logs by profile: structured, without excessive DEBUG level on sale.
Ingest filtering: cut off noise (no health checks), aggregation of metrics instead of raw logs.
SLO dashboards: fewer scattered panels → fewer metrics → less ingest.
10) Payments and anti-fraud (external fees)
Mix of providers: routing at the lowest commission, taking into account conversion/risk scoring.
Reduced failures - Correct 3-D Secure/retries → less workload and fees.
Anti-fraud rules: Targeting risk, not "everything," so as not to overpay for checks.
Chargeback control: preventive triggers for anomalies in rates and conclusions.
11) Bonuses, free backs and marketing expenses
Limits and funnel: personalization of bonuses for LTV/risk scoring → less "overfeeding."
Anti-abuse: account dedup, velocity rules, output caps for promo.
Traffic: SmartLink and post-click filters, low-quality source cull, post-view attribution with windows.
Tournament economics: prize pool ↔ expected uplift ARPPU/retention; track ROI.
12) Operations and people
Automation of routine: playbooks, ranbooks, auto-typing incidents.
Support: macros, first-level bots, VIP prioritization; self-service in your personal account.
QA and environments: ephemeral-env by PR, test data as a service, turning off standing stands.
13) Governance and Processes
Policy-as-Code: resource limits, prohibition of expensive instances without justification.
Change management: canary releases - fewer rollbacks and alterations.
Cost catalog: single tags/labels for all resources; a "who pays for what" report.
Weekly Reviews: Top 10 Budget Eaters, Action Plan and Owners.
14) Implementation Roadmap (12 weeks)
Weeks 1-2: cost inventory, tags/tags, summary dashboard $/RPS, savings goals.
3-4: right-sizing, HPA/KEDA, log cleaning/TTL, trace sampling.
5-6: DB/cache: indexes, hot keys, denormalization, pagination limits.
7-8: CDN/egress: cache policies, compression, media optimization.
9-10: Payments/anti-fraud: routing by providers, reducing failures.
11-12: Bonuses and traffic: personalization, anti-abuse, closing low-ROI campaigns.
After the 12th week - autopilot: showback/chargeback, quarterly target $/unit of value.
15) Dashboards
Exec: OPEX by category, $/RPS, $/transaction, ROI campaigns, savings vs base period.
Those: recycling CPU/Memory/IO, autoscaling events, cache-hit, p95/p99, ingest logs/day, egress GB/day.
Data: cost of queries/scans, retention, size of topics/tables, cost per query.
Payments: authorization conversion, average commission, chargeback share, KYC time.
Bonuses: CPA/RevShare/ARPPU uplift, abuse share, net effect on GGR.
16) Artifact patterns
Cost Playbook (per service): current value, drivers, measures (impact $/complexity), owner, term.
Capacity & Cost Sheet: headroom, $/RPS before and after measures, effect on SLO.
Policy Catalog-Valid instance classes, limits, exceptions, and request process.
Runbook "Night/Weekend": aggressive scale-in/TTL rules, pause of non-critical jobs.
17) Top 20 quick measures ("quick wins")
1. Enable autoscaling and reduce "oversized" bottoms.
2. Cut retention logs and traces to business essentials.
3. Sampling traces + aggregations instead of raw events.
4. Translate heavy reports to night batch windows/matviews.
5. Increase CDN cache-hit due to version/TTL.
6. WebP/AVIF and lazy-loading images.
7. Compress egress through region pinning and compression.
8. Indices to the top 10 slow queries, pagination limits.
9. Cache hot data/lists.
10. Disable unused features/endpoints.
11. Reserved/Savings plans for constant load.
12. Spot/Preemptible for ETL/ML treatments.
13. Remove the duplicate ingest of the same type of metrics.
14. Reduce the frequency of useless crowns/pulling.
15. Route payments to providers with the best "commission × conversion."
16. Personalize bonuses (cap on cost per player).
17. Freeze low-ROI traffic procurement channels.
18. Ephemeral test environments on demand.
19. Auto-closing of idle resources (VM/stands).
20. Politicians for "expensive" instances and log volumes.
18) Antipatterns
Blind savings without metrics and SLO → hidden revenue losses.
Mass rejection of logs/traces → deterioration of MTTR and an increase in incidents.
Universal mouthguards without segmentation → a drop in payment/bonus conversion.
Ignore egress/CDN → invisible accounts grow faster than compute.
19) Roles and Responsibilities (RACI)
Responsible: SRE/Platform, Data/FinOps, Billing/Payments, Risk.
Accountable: Head of Ops/CTO.
Consulted: Product, Marketing, Security/Compliance.
Informed: Support, Finance.
20) Control and improvement
Weekly stand-up savings: playbook progress, blockers, metrics.
Monthly: revision $/unit, comparison with benchmark, reassembly of top 10 measures.
Quarterly: revision of contracts with providers, migration of storage classes/instances.
Total
OPEX optimization is not a one-time "cleaning of accounts," but a continuous system: cost transparency → prioritization of ROI measures → technical optimization without harming SLO → automation and governance. With this approach, $/RPS and $/transaction fall steadily, and the quality of service and the speed of releases grow.