Optimizing Network Latency

Brief Summary

Latency = sum of small delays along the chain. Improvement is achieved systemically: reduce "hops" (Anycast/CDN), reduce RTT (routes/QoS), reduce "bolt" (DNS/TLS/TCP), keep queues short (AQM/ECN), minimize bytes (compression/cache), and continuously measure p95/p99. Any optimization without measurements is fortune telling.

Latency Budget: What Delay Consists Of

Let's simplify the decomposition of TTFB (Time To First Byte):


TTFB ≈ t_DNS + t_connect + t_TLS + t_queue + t_routing + t_app + t_first_byte

Ubi:

t_DNS - resolution of the name.
t_connect - L4 establishment (TCP/QUIC).
t_TLS - TLS handshake.
t_queue/t_routing - queues/transitions (buffering, overloads, NAT).
t_app - server processing up to the first byte.

The goal is to set an SLO for each term and regularly check.

SLO/metrics

DNS p95: locally ≤ 20-30 ms, globally ≤ 80-120 ms.
TCP connect p95: ≤ 80-120 ms regionally, ≤ 200-250 ms interregionally.
TLS handshake p95: ≤ 80–120 мс (с OCSP stapling, resumption).
TTFB p95 (static): ≤ 120-150 ms regionally.
TTFB p95 (API): ≤ 200-300 ms.
Jitter p95 (UDP/RT): ≤ 15–30 мс; loss ≤ 0. 1–0. 3%.

Alerts: p95/p99 rise, 'SYN retries' surge, 'resumption' fall, 'ECN CE' or 'packet loss' rise.

DNS Quick Start

Anycast authorities + close recursions (DoH/DoT if necessary).
Short TTL (60-300 s) for dynamic names; negative-TTL under control.
DNS cache next to the application; 'prefetch' hot records.
Exclude unnecessary CNAME → CNAME → A/AAAA moves.

TLS/HTTP: fewer handshakes, faster sessions

Turn on TLS 1. 3, OCSP stapling, CA short chains.
Session resumption (tickets/IDs) и ALPN (`h2`, `h3`).
HTTP/2: multiplexing, fewer connections → lower than p95.
HTTP/3 (QUIC): 0-RTT (idempotent only), better loss/jitter resistance.
Preconnect/Prewarm (edge/client): open connections for hot domains in advance.

Transport: TCP vs QUIC and kernel stack

TCP

Modern Congestion Control: BBR v2 (or CUBIC if conservative).
RACK/TLP for rapid retransmit; enable SACK.
ECN + AQM (CoDel/FQ_CoDel): reduces buffer block and jitter.
TCP Fast Open - avoid for state-changing requests; the benefit is controversial.

QUIC (HTTP/3)

Less "head lock" compared to TCP + TLS + HTTP/2.
Resistant to reordering/losses; Update the implementation (for example, Envoy/HAProxy with H3).
Monitor UDP/443 and MTU/fragmentation.

MTU/PMTUD and fragmentation

Single end-to-end MTU; for tunnels (IPsec/WireGuard/VXLAN), consider overhead.

Enable PMTUD and ICMP "Fragmentation needed."

For QUIC - monitor max datagram and not cut ICMP on the perimeter.

Routing and Path Physics

Anycast for public IP API/edge fronts.
Geo/Latency routing (GSLB) + health-checks.
ECMP and BFD in the factory (Leaf-Spine) to eliminate bad trails in <1 s.
Align aplinks/feasts (IX) in user concentration regions.

Queues and QoS: Keep buffers short

Classes: real-time (RT/VoIP), interactive (API), bulk (backups/ETL).
LLQ/WRR, prioritization of API/payments over bulk.
ECN (CE marking) + AQM (CoDel/FQ_CoDel) on edge queues.

Cut/transfer backups and large bruises from "peak minutes."

NAT, proxy and middleboxes

Minimize NAT/filter cascade.
Stateful middlebox is critical of path asymmetry: balance over 5-tuple, pin-ning threads.
Keepalive support and reasonable idle timeouts for long-lived connections (gRPC/WebSocket).

Cache and data placement

CDN/origin-shield/tiered-cache - fewer trips to origin.
Versioned assets (immutable, 1 year).
Short TTL + 'stale-while-revalidate' for semi-dynamics.

Geo-placement: hot data is closer to the user (read-replica/edge-KV), and writing to the "source of truth."

Application-level optimization

Reduce the number of requests (push bundling/HTTP/2 are no longer in vogue - prefetch/preconnect is better).
Reduce payload: compression (Brotli), web image formats, binary protocols (gRPC).
Request idempotence → secure retrays and aggressive timeouts.
Async/streaming (SSE/gRPC streaming) to reduce TTFB.

Observability: what to measure

Client telemetry (RUM): DNS/connect/TLS/TTFB, Geo/ASN, device.
Network: RTT, loss, jitter, ECN CE/ECT (0/1), interface queues, errors/overflows.
Транспорт: retransmits, out-of-order, cwnd/BBR state, handshake stats, resumption.
L7: p50/p95/p99, error rate, payload size, route histograms/PoP.
Segment metrics by region/ASN/carrier - this is where hotspots hide.

Mini playbooks

1) Quick audit p95

1. Build a latency budget (DNS/TLS/connect/TTFB) from the RUM.
2. Map to network metrics (RTT/loss/ECN) by PoP/ASN.
3. If'connect 'and'TLS' dominate → enable preconnect/resumption/HTTP/3.
4. If'TTFB 'is high → cache/edge/replica and application optimization.

2) Loss/jitter spike

1. Check uplink/interfaces (drops/queues).
2. Enable/tweak AQM (FQ_CoDel), reduce bulk class.
3. Check BFD/ECMP path, exclude flap link.
4. For customers, temporarily raise retrays and reduce the time in between.

3) Regional degradation

1. Switch GSLB to neighboring PoP; withdraw Anycast/32 on degraded.
2. Reduce TTL, activate 'stale-while-revalidate'.
3. Send status to status page, start RCA.

4) Migration to HTTP/3

1. Open UDP/443, enable H3/ALPN 'h3'.
2. Conduct A/B: compare p95 TTFB and error rate.
3. Observe'udp loss '/active clients/cwnd oscillations.

Config crib

Nginx (HTTP/2, OCSP, TLS 1. 3, Brotli)

nginx server {
listen 443 ssl http2 reuseport;
ssl_protocols TLSv1. 2 TLSv1. 3;
ssl_stapling on; ssl_stapling_verify on;
add_header Strict-Transport-Security "max-age=31536000" always;

Compression brotli on; brotli_comp_level 5; gzip on;

Cache static location/static/{
add_header Cache-Control "public, max-age=31536000, immutable" always;
try_files $uri =404;
}

Proxy to API location/api/{
proxy_set_header Connection "";
proxy_http_version 1. 1;
proxy_read_timeout 5s;
proxy_connect_timeout 1s;
proxy_pass https://backend;
}
}

sysctl (Linux network: ECN/SACK/RACK kernel subsystem - approximate parameters)

bash
SACK/RACK/TLP are typically included in modern cores; specify sysctl -w net for your distribution. ipv4. tcp_sack=1 sysctl -w net. ipv4. tcp_ecn=1 sysctl -w net. ipv4. tcp_fastopen=0 # carefully with TFO sysctl -w net. core. default_qdisc=fq_codel sysctl -w net. ipv4. tcp_congestion_control=bbr

QoS (Cisco-style, concept)


class-map match-any REALTIME match dscp ef class-map match-any INTERACTIVE match dscp af31 af21 class-map match-any BULK match dscp cs1 policy-map WAN-QOS class REALTIME priority percent 10 class INTERACTIVE bandwidth percent 50 class BULK bandwidth percent 20 random-detect ecn

Processes and operational practices

Capacity planning: capacity margin ≥ 30% during peak hours.
Change-control: canary changes of routes/PoR/gateways.
Regular PMTUD test and ICMP control.
Path documentation: where is the NAT/proxy/AQM/QoS, which MTUs, who is the owner.
Timeout agreement between L7 and network retraces.

iGaming/fintech specific

Event time (matches/tournaments): warming up CDN/PoP, preconnect to critical domains, temporary increase in the resumption pool, "gray" routes for bots.
Payment steps: dedicated QoS class, Anycast endpoints, short TLS chains and strict versions/ciphers; retreats are only idempotent.
Regional restrictions/PSP: geo-routing + ASN/IP whitelists; fixed egress pools.
Mobile networks: preference for ChaCha20-Poly1305 (with weak AES-NI in customers), aggressive compression and HTTP/3.

Implementation checklist

Latency budget (DNS/TLS/connect/TTFB) and SLO per link.
Anycast/Geo-routing for public APIs/edge, backup PoPs.
TLS 1. 3, OCSP stapling, resumption ≥ 70%, HTTP/2/3 included.
BBR + FQ_CoDel, ECN, SACK; PMTUD works, ICMP is not blocked.
A single MTU along the chain, accounting for the overhead of the tunnels.
QoS: real-time/interactive/bulk, AQM classes on overload interfaces.
CDN/Tiered-cache/Origin-shield; versioned assets and SWR.
RUM + Geo/ASN network metrics; alerts to p95/p99/ECN/losses.
Playbooks: degradation of the region, surge in losses, traffic transfer.

Common errors

ICMP/PMTUD blocking → fragmentation/resending and high p95.
"Thick" buffers without AQM → buffer block and jitter.
Long CA chains and no OCSP stapling → expensive TLS.
NAT stages and asymmetry for stateful filters → retrays/timeouts.
Excessive'Vary '/non-versioning assets → low hit-ratio CDN.
No QoS segmentation → API competes with backups at their peak.

Total

Delay optimization is a combination of network engineering, proper transport, and "bolt" savings on DNS/TLS/cache. Implement Anycast/Geo-routing, TLS 1. 3 + resumption, HTTP/3, BBR + FQ_CoDel/ECN, negotiate MTU, set QoS and SLO, measure p95/p99 and automate playbooks. Then users will receive a quick response even in the most "burning" minutes, and the platform - predictability and margin of safety.