Töleg ssenarileriniň A/B synaglary

1) Näme üçin töleg ssenarilerini barlamak

Tassyklamalary köpeltmek (AR) we şowsuzlyklary azaltmak (DR).
Bahasy azalt: take-rate (interchange/scheme/markup/fixed) we cost-per-approval.
Töwekgelçiligi azaltmak: şol bir tassyklamalarda az çarjbekler/frodlar.
Durnuklylyk: anyk GEO/BIN/usullary üçin üpjün edijini/3DS-strategiýasyny/marşrutyny saýlamak.

2) Synagyň dizaýny

2. 1. Randomizasiýa bölümi

User-level (maslahat berilýär): bir ulanyjynyň ähli synanyşyklary bir şahada düşýär → 3DS/tokenleri "garyşdyrmak" ýok.
BIN-level: haçan synag - emitent boýunça marşrut barada; ulanyjynyň garyşmagy töwekgelçiligi.
Order/Attempt-level: kiçi UI synaglary üçin kabul ederlikli (mysal üçin, ýalňyşlygyň nusgasy), marşrut/3DS üçin islenilmeýär.

2. 2. Stratifikasiýa (randomizasiýa çenli)

Toparlara bölüň: Oýunçynyň GEO, issuer country/BIN6, töleg usuly, kanal (web/app), summa-segment, töwekgelçilik-skor. Bu SRM-iň ýaýramagyny we töwekgelçiligini azaldar.

2. 3. Nämäni barlaýarys

Marşrut/kaskad: PSP_A vs PSP_B, sticky BIN, limit-aware.
3DS syýasaty: frictionless → challenge, BIN/geo üçin mejbury 3DS.
UX flow: ädimleriň yzygiderliligi, ýalňyşlyklaryň/gaýtalamalaryň tekstleri.
Aýlaw parametrleri: Windows we soft-decline kodlary.
Bahalar: IC + vs blended bilen üpjün ediji we ähli bahalara täsiri.

3) Metrikler: maksatly, ikinji derejeli, guardrails

3. 1. Esasy

AR (Approval Rate) = approved/attempted.
Cost-per-Approval = (auth+decline fees)/approved.
Take-rate% (all-in) = fees/volume (hasabat pulunda).
3DS pass-rate; liability shift %.
Latency p95/p99 töleg fly.

3. 2. Töwekgelçilik metrikleri

Chargeback ratio (CBR), refund rate, fraud alerts/1000 trx.
FX slippage (bps) = effective vs reference FX.

3. 3. Guardrails

AR> Y bps güýji ýa-da CBR/Refunds bosagadan ýokary.
SRM (Sample Ratio Mismatch) - garaşylýan traffigiň deňsizligi.
Spikes: gizlinlik, soft-decline surge, 3DS anomaly.

4) Statistika we kuwwat

4. 1. Nusganyň ululygy (paýlara ýakynlaşmak)

n_per_group ≈ 2 (Z_{1-α/2} + Z_{1-β})^2 p(1-p) / δ^2

bu ýerde 'p' - esasy AR, 'δ' - AR-da garaşylýan uplift, α - ähmiýetiniň derejesi, β - II görnüşli ýalňyşlyk.

4. 2. Yzygiderli derňew (irki duralgalar)

Alpha-spending (O'Brien-Fleming/Pocock): barlaglaryň tertibini düzýäris we α tapgyrlara bölýäris.
SPRT/Bayes - amaly çözgütler üçin, ýöne teswirnamany düzüň.

4. 3. Wersiýa-redaksiýa

CUPED: 'Y = Y − θ (X − μ_X)', bu ýerde X - synagdan öňki kowariat (AR/DR/töwekgelçilik-skor), θ - kowariant koeffisiýenti.
Gatlaklaşdyrylan bahalandyrmalar, toparlaýyn ýalňyşlyklar (user/BIN-klasterler).
Take-rate/çykdajy metrikleri üçin Bootstrap (agyr guýruklar).

4. 4. Multivariant synaglary we bandits

MAB (UCB/Thompson): uçuş wagtynda "öwrenmek" we dolanyşygy ýitirmezlik möhüm bolanda.
Gabat gelýän kritiki metrikler üçin (CBR, liability) - guardrails bilen nusgawy A/B-ni saýlaň.

5) Synag platformasynyň arhitekturasy

1. Assignment-service: determinirlenen hash '(user_id, experiment_id, salt)' → bucket.
2. Feature-baýdaklar/Rules-engine: marşruty işjeňleşdirmek/3DS/şahasy boýunça retraý.
3. Wakalar: synanyşyklar/netijeler (authorize/capture/refund/cb) → teker (Kafka/PubSub).
4. Idempotentlik: kaskad üçin umumy 'idempotency _ key'.
5. DWH/Vitrinler: kadalaşan statuslar, fees, FX, töwekgelçilik baýdaklary.
6. Gözegçilik: online-SLI (AR/3DS/latency), alertler, SRM-çek.
7. Teswirnamalar: pre-register gipotezasy, gutarnykly ölçegleri, data-friz.

6) Maglumatlaryň modeli (iň az)

sql ref. experiments (
exp_id PK, name, hypothesis, owner, start_at, end_at,
unit -- USER      BIN      ORDER,
target_metric, guardrails JSONB, design JSONB, alpha NUMERIC, power NUMERIC, meta JSONB
);

ref. experiment_arms (
exp_id FK, arm_id, name, traffic_share NUMERIC, params JSONB, enabled BOOLEAN
);

assignments. buckets (
exp_id, user_id, assigned_arm, assigned_at, salt, hash_key, PRIMARY KEY (exp_id, user_id)
);

events. payments (
attempt_id PK, user_id, exp_id, arm_id,
provider, method, bin, iso2, risk_score,
status, decline_code, three_ds_used BOOLEAN, liability_shift BOOLEAN,
amount_minor BIGINT, currency, latency_ms INT,
authorized_at, captured_at, settled_at, meta JSONB
);

finance. fees (
attempt_id FK, interchange_amt NUMERIC, scheme_amt NUMERIC, markup_amt NUMERIC,
auth_amt NUMERIC, refund_amt NUMERIC, cb_amt NUMERIC, gateway_amt NUMERIC,
fx_slippage_amt NUMERIC, reporting_currency TEXT
);

risk. outcomes (
attempt_id FK, is_refund BOOLEAN, is_chargeback BOOLEAN, fraud_alert BOOLEAN
);

7) SQL şablonlary

7. 1. SRM-çek (eller boýunça traffigiň paýy)

sql
SELECT arm_id,
COUNT() AS n,
ROUND(100. 0 COUNT() / SUM(COUNT()) OVER (), 2) AS share_pct
FROM assignments. buckets
WHERE exp_id =:exp
GROUP BY 1;

7. 2. El boýunça esasy metrikler

sql
WITH base AS (
SELECT e. arm_id,
COUNT()                  AS attempts,
COUNT() FILTER (WHERE status='APPROVED') AS approvals,
AVG(latency_ms)              AS latency_avg_ms,
AVG((three_ds_used)::int)         AS three_ds_share
FROM events. payments e
WHERE e. exp_id=:exp AND e. authorized_at BETWEEN:from AND:to
GROUP BY 1
),
cost AS (
SELECT e. arm_id,
SUM(f. interchange_amt + f. scheme_amt + f. markup_amt +
f. auth_amt + f. refund_amt + f. cb_amt + f. gateway_amt + f. fx_slippage_amt) AS fees_rep,
SUM(e. amount_minor)/100. 0 AS volume_rep
FROM events. payments e
JOIN finance. fees f USING (attempt_id)
WHERE e. exp_id=:exp AND e. settled_at BETWEEN:from AND:to
GROUP BY 1
)
SELECT b. arm_id,
approvals::numeric/NULLIF(attempts,0)             AS ar,
fees_rep/NULLIF(volume_rep,0)                 AS take_rate,
(SELECT COUNT() FROM risk. outcomes r
JOIN events. payments e2 USING (attempt_id)
WHERE e2. exp_id=:exp AND e2. arm_id=b. arm_id AND r. is_chargeback)=0
AS cb_zero_flag,
latency_avg_ms, three_ds_share
FROM base b LEFT JOIN cost c ON c. arm_id=b. arm_id;

7. 3. AR üçin CUPED (mysal)

sql
WITH pre AS (
SELECT user_id, AVG((status='APPROVED')::int) AS ar_pre
FROM events. payments
WHERE authorized_at <:pre_from_end
GROUP BY 1
),
cur AS (
SELECT e. user_id, e. arm_id, (e. status='APPROVED')::int AS ar_flag
FROM events. payments e
WHERE e. exp_id=:exp AND e. authorized_at BETWEEN:from AND:to
)
SELECT arm_id,
AVG(ar_flag - theta (ar_pre - mu_pre)) AS ar_cuped
FROM cur
LEFT JOIN pre USING (user_id),
LATERAL (SELECT AVG(ar_pre) AS mu_pre FROM pre) mu,
LATERAL (SELECT COVAR_SAMP(ar_flag, ar_pre)/VAR_SAMP(ar_pre) AS theta FROM cur LEFT JOIN pre USING(user_id)) t
GROUP BY arm_id;

7. 4. Guardrails barlagy (mysal)

sql
SELECT arm_id,
100. 0 SUM(is_chargeback::int)::numeric / NULLIF(COUNT(),0) AS cbr_pct,
100. 0 SUM(is_refund::int)::numeric  / NULLIF(COUNT(),0) AS refund_pct
FROM risk. outcomes r
JOIN events. payments e USING (attempt_id)
WHERE e. exp_id=:exp AND e. settled_at BETWEEN:from AND:to
GROUP BY 1
HAVING 100. 0 SUM(is_chargeback::int)::numeric / NULLIF(COUNT(),0) >:cbr_threshold
OR 100. 0 SUM(is_refund::int)::numeric  / NULLIF(COUNT(),0) >:refund_threshold;

8) Synagy geçirmek prosesi (end-tu-end)

1. Pre-registration: gipotezasy, metrikleri, dizaýny, ölçegleri, duralgalary.
2. SRM/AA-" boş "täsir synagy (iki gün).
3. Başlamak: assignment freeze, rules-engine/ficheflagda logika.
4. Onlaýn gözegçilik: AR/3DS/latency/health + guardrails.
5. Alpha-spending boýunça aralyk barlaglar (meýilleşdirilen bolsa).
6. Finiş we data-friz: diňe funding/ätiýaçlyk/giç CB/refunds hasaba alnandan soň.
7. Analitika: GEO/BIN/usul/kanal boýunça CUPED/stratifikasiýa, duýgurlyk, geterogenlik.
8. Çözgüt: roll-out, roll-back, ýa-da follow-up synagy; routing düzgünlerini täzelemek.
9. Dokumentasiýa we retrospektiv: sapaklar, bosagalary/agramlary täzelemek.

9) Anti-patternler we duzaklar

Teswirnamasyz peeking/pere-syn → ýalan ýeňişler.
Marşrut synaglarynda Order-level randomizasiýa → elleriň arasynda syzmak.
α düzetmezden köplük (köp metrik/kesim) bilen oýun.
Doly däl bahasy (FX/ätiýaçlyk/refund fees ýatdan çykaryldy) → nädogry take-rate.
SRM çekiniň ýoklugy → göçürilen netijeler.
Indempotent däl retraýalar → goşa awtorizasiýa/ýoýulmalar AR.

10) Howpsuzlyk, laýyklyk we ahlak

Same-method/return-to-source synag arkaly bozulmaly däldir.
Sanksiýalar/ygtyýarnamalar/GEO-syýasatlar - synaglardan daşary.
RG/jogapkär oýun: AR üçin gorag mehanizmlerini ýaramazlaşdyrmaň.
PCI/GDPR: PAN-yň ýerine bellikler, şahsy maglumatlary azaltmak, DPA/SOC2.

11) Synagyň daşbordunyň KPI

AR/DR, uplift we gollar we esasy gatlaklar boýunça ynam aralyklary (GEO/BIN/usul).
Cost-per-Approval, take-rate %, FX slippage (bps).
3DS pass/liability shift, soft-decline share.
Latency p95/p99, ýalňyşlyklar/wagtlar.
CB/Refunds (lag-aware), SRM, traffigiň gurşawy, dowamlylygy.

12) Iň oňat amallar (gysgaça)

1. Ulanyjy derejesinde randomize ediň we stratifikasiýa ediň.
2. Gardrails we SRM barlagyny ulanyň; teswirnamany ýazga alyň.
3. Doly bahany hasaplaň (fees + FX + reserve) we cost-per-approval.
4. Gymmatlyk metrleri üçin CUPED, klaster-ýalňyşlyklary we bootstrap ulanyň.
5. Möhüm töwekgelçilikler üçin - nusgawy A/B; bandits - esasan baha/AR meseleleri üçin.
6. Maliýe/ätiýaçlyk/soňky CB-leri gutarmanka göz öňünde tutuň.
7. Düzgünleri dokumentleşdiriň we wersiýa ediň; post-mortem ediň.

13) Işe girizmegiň çek-sanawy

• Gipotezasy, metrikleri, täsiri, dizaýny, nusgasynyň ululygy, möhleti.
• Randomizasiýa we stratalar, assignment-hyzmat, fiçeflaglar.
• Garawullar/bosagalar, SRM/AA-precheck, aladalar.
• Logi/wakalar, idempotentlik, statuslaryň kadalaşmagy.
• Vitrinler fees/FX/reserve; hasabat walýutasy.
• Duralgalar meýilnamasy (alpha-spending) we data-friz.
• Roll-out/roll-back pleýbuklary; netijeleriň resminamalaşdyrylmagy.

Gysgaça maglumat

Töleg ssenarileriniň A/B synaglary in engineeringenerçilik-statistik düzgün: dogry randomizasiýa we stratifikasiýa, bahanyň we töwekgelçiligiň doly ölçegleri, guardrails we SRM, takyk analitika (CUPED/klaster-berklik/yzygiderli derňew) we "söweşe ukyply" infrastruktura (idempotentlik, telemetriýa, reconciliation). Bu usuly ulanyp, siz AR-ny ýokarlandyrýarsyňyz, ähli tölegleri peseldýärsiňiz we şol bir wagtyň özünde çarjbekleriň we kadalaşdyryjy töwekgelçilikleriň köpelmegi bilen "ýalan ýeňişler" üçin tölemeýärsiňiz.