Wakalary ýazga almak we yzarlamak

Wakalary ýazga almak we yzarlamak

1) Maksady we çarçuwasy

Loglar we söwdalar - syn edilişiň binýady.
Loglar "näme boldy" we "haýsy kontekstde" jogap berýärler.
Treýslar soragyň paýlanan ýolunda "nirede we näme üçin haýal/ýalňyşlyk" diýip jogap berýärler.

• Structured by default (JSON); Trace-first: gyzgyn ýoldaky her bir log 'trace _ id '/' span _ id' -e birikdirilýär.
• Iň az ses, iň ýokary signal: derejeler, sampling, anti-kardinallyk.
• Howpsuzlyk we gizlinlik: gizlemek, redaktirlemek, elýeterliligi kesgitlemek.
• Bloglaryň we wakalaryň wersionirlenen shemalary.

2) Wakalaryň taksonomiýasy

1. Tehniki ýazgylar (runtime, ýalňyşlyklar, tor wagtlary, retralar).

2. Işewürlik wakalary (hasaba alyş, depozit, stawka, netije, KYC-tapgyry) - azyk analitikasy we "pul" ýollary boýunça hadysalar üçin amatlydyr.

3. Audit (kim/haçan üýtgedi: konfigiler, elýeterlilik, baýdaklar, çäkler) - üýtgemeýän žurnal.

4. Howpsuzlyk (autentifikasiýa, artykmaçlyklaryň güýçlenmegi, sanksiýalar/RER-baýdaklar).

5. Infrastruktura (K8s events, autoscaling, HPA/VPA, düwünler/disk/tor).

Her akym üçin - aýratyn retenşn, indeksasiýa we elýeterlilik düzgünleri.

3) Gurluş log (JSON standarty)

json
{
"ts": "2025-11-03T14:28:15.123Z",
"level": "ERROR",
"service": "payments-api",
"env": "prod",
"region": "eu-central-1",
"trace_id": "8a4f0c2e9b1f42d7",
"span_id": "c7d1f3a4b8b6e912",
"parent_span_id": "a1b2c3d4e5f60789",
"logger": "withdraw.handler",
"event": "psp_decline",
"msg": "PSP declined transaction",
"http": { "method": "POST", "route": "/withdraw", "status": 502, "latency_ms": 842 },
"user": { "tenant_id": "t_9f2", "user_key": "hash_0a7c", "vip_tier": 3 },
"payment": { "psp": "acme", "amount": 120.50, "currency": "EUR", "idempotency_key": "u123:wd:7845" },
"safe": true,         // пройдена проверка на секреты
"version": "1.14.2",     // версия сервиса (SemVer)
"build": "sha-1f2a3b4",
"kubernetes": { "pod": "payments-7cbdf", "node": "ip-10-0-2-41" }
}

Talaplar: tekiz shema + domen goýmalary, hökmany meýdanlar ('ts, level, service, env, trace_id, msg'), san bahalary - setirler bilen däl.

4) Derejeler, kardinallyk we göwrümi

Derejeler: 'DEBUG' (bölek däl), 'INFO' (iş faktlary), 'WARN' (anomaliýalar), 'ERROR' (ýalňyşlyklar), 'FATAL' (gyralar).
Kardinallyk: özbaşdak açarlardan/dinamiki labellerden gaça duruň. "Id-açar" ýok.
Log sampling: rate-limit gaýtalanýan habarlar; 'DEBUG' -i diňe skoped we wagt boýunça açyň.
Idempotentlik: 'idempotency _ key' -ni ulanyjylar tarapyndan gaýtalanýan wakalary basmak üçin goýuň.

5) Gizlinlik we howpsuzlyk

PII/syrlary agentlerde gizläň (Fluent Bit/Vector): açarlar boýunça gizlemek kartalary ('email', 'card', 'token', 'authorization').
'user _ key' heşesini düzüň, diňe zerur konteksti saklaň (ýurt, KYC-derejesi, VIP-tier).
Ammarlary bölüň: ýyly (operatiw gözleg) we sowuk (PII/c gysgaldylan kontekstsiz arhiw).
Audit - append-only, WORM-ammar, giriş diňe least privilege ýörelgesi boýunça.

6) Ýollar: standartlar we kontekst

W3C Trace Context: 'traceparent '/' tracestate' sözbaşylary, üstesine-de ygtybarly açarlar üçin baggage (mysal üçin 'tenant _ id', 'region').
Metrleriň we treýsleriň baglanyşygy: Exemplars - 'trace _ id' -ni gistogrammalaryň jemlenen nokatlaryna geçiriň (RCA çaltlaşdyrýar).
Sampling: Esasy sampling 1-5% + dinamiki "ýalňyşlyklarda/haýal p95" problemaly soraglar üçin 100% -e çenli.
Links: Asenkron nobatlar/saglar üçin diňe 'parent' arkaly däl, 'links' arkaly ýatdyňyzy baglaň.

7) Ýygnamak we ugrukdyrmak

Agentler: Fluent Bit/Vector; OpenTelemetry Collector-a OTLP eksport.
Collector: Merkezi şlýuz (batch/transform/filter/routing).

App → (OTLP logs/traces/metrics) → OTel Collector
→ logs: redact → route(security    audit    tech    biz) → hot index / cold archive
→ traces: tail_sampling(errors    p95>threshold) → APM backend
→ metrics: Prometheus exporter (for SLO/alerts)

yaml processors:
batch: {}
attributes:
actions:
- key: env value: prod action: insert filter/logs:
logs:
include:
match_type: strict resource_attributes:
- key: service.name value: payments-api exporters:
otlp/traces: { endpoint: "apm:4317", tls: { insecure: true } }
loki: { endpoint: "http://loki:3100/loki/api/v1/push" }
prometheus: {}
service:
pipelines:
logs: { receivers: [otlp], processors: [attributes,batch], exporters: [loki] }
traces: { receivers: [otlp], processors: [batch], exporters: [otlp/traces] }
metrics: { receivers: [otlp], processors: [batch], exporters: [prometheus] }

8) Instrumentirlemek: SDK mysallary

8. 1 Node. js (Pino + OTel)

js import pino from "pino";
import { context, trace } from "@opentelemetry/api";

const logger = pino({ level: process.env.LOG_LEVEL          "info" });

function log(info) {
const span = trace.getSpan(context.active());
const base = span? { trace_id: span.spanContext().traceId, span_id: span.spanContext().spanId }: {};
logger.info({...base,...info });
}

// пример log({ event: "deposit.created", amount: 50, currency: "EUR", user: { user_key: "hash_0a7c" } });

8. 2 Java (SLF4J + OTel)

java
MDC.put("trace_id", Span.current().getSpanContext().getTraceId());
MDC.put("span_id", Span.current().getSpanContext().getSpanId());
log.info("psp_response status={} latency_ms={}", status, latency);

8. 3 Python (structlog + OTel)

python import structlog from opentelemetry import trace log = structlog.get_logger()

def log_json(event, kwargs):
span = trace.get_current_span()
ctx = {}
if span and span.get_span_context().is_valid:
ctx = {"trace_id": span.get_span_context().trace_id, "span_id": span.get_span_context().span_id}
log.msg(event=event, ctx, kwargs)

8. 4 NGINX → sözbaşylary yzarlamak

nginx proxy_set_header traceparent $http_traceparent;
proxy_set_header tracestate $http_tracestate;

9) Alertler we awto-hereketler üçin signal hökmünde loglar

Nädogry nagyşlar ('psp _ decline', 'fraud _ flag') SLO bilen baglanyşdyryň.
Pattern-rate alertleri: "5xx po/withdraw> 0. 5% 10m üçin" ", fraud_flag spike> + 200% esasy".
Awto-hereketler: 'withdrawals _ manual _ mode = true' loginizde baýdak platformasyndan kill-switch açyň.

rate(count_over_time({service="payments-api", level="ERROR", event="psp_decline"}[5m])) > 5

10) Retenşn, indeksirlemek, saklamak

Gyzgyn: 7-14 gün (operatiw derňew).
Yssy: 30-90 gün (tendensiýalar, RCA).
Sowuk: 180-365 + (arhiw, audit) - gysyş, arzan synplar, belki doly tekstli gözlegsiz.
Indeksleme: kesgitlenen açarlar ('service, env, level, event, trace_id, user. tenant_id'), "yzly-yzyna" indeksine gadaganlyk.
Wakanyň ululygyna çäklendirmeler (mysal üçin, 32KB ≤), aşakdan/aşakdan: "storage-da artykmaç - MTTR duşmany".

11) Audit we üýtgemezlik

Audit wakalaryny gollar/heşler, serwer wagty, "who/what/when/why" bilen aýratyn akym bilen ýazyň.
"DE-de 100% bonus baýdagyny kim açdy?" - jogap haýyşnamanyň 1-2-sinde bolmalydyr.

json
{
"ts": "2025-11-03T14:00:00.000Z",
"actor": "alice@company",
"action": "feature_flag.update",
"target": "bonus.enable_vip",
"old": {"rollout": 10},
"new": {"rollout": 100},
"reason": "campaign_2311",
"ticket": "OPS-3481",
"trace_id": "cf12ab.."
}

12) Iş wakalary we maglumatlaryň modeli

• `event_type`, `event_id`, `occurred_at`, `actor`, `subject`, `amount`, `currency`, `status`, `idempotency_key`.
• Dempotent sarp edijiler bilen "Outbox" we "at-least-once" ulanyň.

13) Kubernetes we pipeline log

Tampon bilen Sidecar/DaemonSet agentleri diske (tor arakesmelerinde).
Ugrukdyrmak üçin podlaryň düşündirişleri ('log. type`, `retention. tier`).
K8s-gözegçileriň ýazgylaryny aýratyn ýygnaň (klaster indeksi).

ini
[FILTER]
Name     modify
Match
Remove    authorization, password, card_number

14) Anti-patternler

"Gerektiği ýaly" setirli loglar, 'trace _ id' ýok.
PII/syr ýazgylarda, dampalar tutuş payload.
Millionlarça täsin açarlar → "partladylan" indeksasiýa.
DEBUG önümde 24/7.
Auditiň, howpsuzlygyň we tehnologlaryň bir indeksde garyşdyrylmagy.
Retrenşn syýasaty ýa-da arhiwden dikeldiş synagy ýok.

15) Giriş çek-sanawy (0-45 gün)

0-10 gün

W3C Trace Context-i gateway/müşderilerde öz içine alyň.
JSON-a terjime etmek, 'trace _ id '/' span _ id' goşmak.
PII/syrlary gadagan etmek (agentde gizlemek), meýdanlaryň sanawyny tassyklamak.

11-25 gün

Akymy bölmek: tech/biz/audit/security/infra, retenşn we ACL bermek.
OTel Collector-y açyň, ýalňyşlyklary/haýal soraglary sampling ediň.
Daşbordlar "Log Rate/Error by route" + Jump-to-trace (Exemplars).

26-45 gün

Wakalaryň şablonlaryna görä alertler we SLO bilen baglanyşyk.
Sowuk bloglar üçin arhiw/dikeldiş (DR-test).
CI-de log shemalarynyň linteri, iş wakalary üçin şertnama.

16) Kämillik ölçegleri

'trace _ id' soraglaryny ýapmak 95% ≥.
JSON ýazgylarynyň paýy ≥ 99%.
"Jump-to-trace" via tapylan hadysalar <15 min (p50) çözüldi.
Bloglarda 0 PII hadysasy (syzdyryş skaneri).
Retenşn ähli akymlar boýunça berjaý edildi (auditi awtomatiki subut edýäris).

17) Goşundylar: mini-snippetler

W3C traceparent generation (psevdo)

txt traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

PromQL - log we SLO baglanyşygy (mysal)

high_error_logs = rate(log_events_total{service="payments-api",level="ERROR"}[5m])
5xx_rate = sum(rate(http_requests_total{service="payments-api",status=~"5.."}[5m])) / sum(rate(http_requests_total{service="payments-api"}[5m]))
alert if high_error_logs > 10 and 5xx_rate > 0.005

OpenAPI - Korelýasiýa sözbaşylary

yaml components:
parameters:
Traceparent:
name: traceparent in: header required: false schema: { type: string }

18) Netijenama

Logirlemegiň we yzarlamagyň güýçli kontury - bu şertnamalar + düzgün-nyzam: gurluş JSON-logleri, bir 'trace _ id', PII-ni howpsuz gaýtadan işlemek, akymlar boýunça marşrut we retenşn, şeýle hem SLO, alerting we yza gaýdyşlar bilen ýakyn baglanyşyk. "Tekst zibilhanasyndan" wakalaryň we ýollaryň şertnamalaryna geçiň we prod-hadysalary anyklamak çalt, öňünden aýdyp boljak we barlap boljak bolar.