Regional hubs

(Section: Ecosystem and Network)

1) Why regional hubs are needed

• Latency and UX: proximity to the user (RTT↓, TTI/TTF↓).
• Compliance and residency: storage/processing of data within the jurisdiction.
• Stability and capacity: unloading the global core, working with partial isolation of the region.
• Economy: decrease in interregional traffic, local CDN/caches, favorable IX/peering tariffs.

2) Regional hub roles

1. Edge/Gateway - input layer (HTTP/2/3, gRPC, WebSocket, QUIC), rate-limit, QoS, WAF.
2. Reader/API - RPC, indexes, search services, local materialized views.
3. Compute/Stream - event processing, aggregation, anti-fraud filters.
4. Data Plane - TSDB/column showcases, object storage for "warm" data.
5. Compliance/KYC/KYB - local integrations with providers and sanctions catalogs.
6. Payments/PSP - local payment methods and on/off ramps.
7. Bridge/Relay is an interchain message terminal with a local finalization buffer.
8. Observability - metrics/logs/trails, synthetic samples.
9. Governance/Access - directories of roles, keys and limits for regional participants.

3) Deployment topologies

Hub-and-Spoke: central "master hub" + regional spawns with partial autonomy.
Active-Active (Multi-Primary): symmetric operation of several hubs with conflict-free replication (CRDT/leading logs).
Active-Passive: hot reserve with periodic replication and DR rollover.
Edge-Tiered: thin edge nodes (CDN, WebSocket fan out) → thick regional hub.

The choice depends on finalization/consistency requirements, channel costs, and regulatory constraints.

4) Geomarshrutization and residency policy

GeoDNS/Anycast + Latency-Aware LB: we send requests to the nearest healthy hub.
Jurisdiction Routing: data of subjects (EU/UK/TR, etc.) remain in the corresponding hub; interregional transfers - only on white lists.
Traffic SOR (Smart Order Routing) for regions: takes into account RTT, channel cost, compliance flags, quota load and SLO.
Fail-in-Place: When external links are degraded, the hub continues to serve "finalized-only" requests and local operations.

5) Data: directories, replications, storage classes

• P0 - payments/bridge/identification (strict residence, synchronization of "signals" only in aggregates/hashes).
• P1 - product events and aggregates (local view + periodic export).
• P2 - debug/logs (aggressive compression, long retention in the region).

• Events - log-shipping with order within the party (region-scoped keys).
• Storage - asynchronous MMR/CRDT or snapshot backups.
• Residency: DLP/PII policies, tokenization, separate encryption keys per-region.

6) Performance and caching

Caches: edge cache (short TTL), read-through on API, negative cache.
Warm-data: last N blocks/batches, hot indexes on popular methods.
DA/Batch buffers for L2/bridges: local queue of publications with confirmations.
Hardware-Adjusted TPS: $/TPS and $/RPS capacity planning based on regional prices.

7) QoS, queues and backpressure

P0/P1/P2 classes at the bus and gateway level; separate queues and quotas.
Partitioning: 'region' tenant 'topic' key for predicted throughput.
DLQ: quarantine of "poisonous" messages, retrai with jitter.
Admission Control: limitation of "expensive" RPCs (by range, filters, limits).

8) Regional Hub SLI/SLO

• p95 Latency (Edge/API), Success Rate, Queue-Lag p95, Freshness showcases, Finality p95 (bridge/relays), Geo-Hit Ratio (share of requests served in the region), Compliance Pass%.

• Edge/API p95 ≤ 350–450 мс, Availability ≥ 99. 95%.
• Freshness (P1) p95 ≤ 3 min; Queue-Lag P0 p95 ≤ 2 с.
• Geo-Hit Ratio ≥ 85% (without interregional hop).
• DR RTO ≤ 15 min, RPO ≤ 5 min for P0.

9) Observability and dashboards

Ops Core: latency/error/queue-lag/throughput by QoS class.
Geo View: RTT heatmap, Geo-Hit Ratio, interregional traffic.
Compliance: residency, sanctions hits, export logs.
Bridge/DA: p95 finalization, challenge/reorg, publication failures.
Capacity & Cost: TPS_per_$, request CTS/1k, Utilization%.

10) DR and resilience

Backup channels: independent IX/providers, encrypted inter-hub communication tunnels.
Isolated mode: "finalized-only," degradation APIs, local bills, followed by reconcile.
Regular exercises: transatlantic shutdown, loss of DA/provers, "jitter/losses" at borders.

11) Economics and Capacity Planning

CTS (Cost-to-Serve) per 1k ops: channels + calculations + storage + licenses.
TPS_per_$: sustainable bandwidth per $1 of infrastructure.
Peering/IX optimization: local peer points, prefix announcements, compression and butching.
Tier model: T1 (min-set of services), T2 (full-blooded analytics), T3 (full stack + DA/bridge).

12) Reference configurations

12. 1 Routing Policy (YAML)

yaml routing:
geodns:
regions: [eu, uk, tr, la, apac, na]
policies:
prefer_local: true fallback_chain: [nearest_healthy, master_hub]
compliance:
residency:
eu: ["eu"]
uk: ["uk"]
tr: ["tr"]
export_whitelist:
eu: ["anonymized_metrics","hash_anchors"]
slo_gates:
p0_latency_p95_ms: 400 queue_lag_p95_ms: 2000

12. 2 K8s Edge Gateway + HPA

yaml apiVersion: apps/v1 kind: Deployment metadata: { name: edge-gw, labels: { region: eu } }
spec:
replicas: 4 template:
spec:
containers:
- name: gw image: org/edge-gw:2. 7. 0 ports: [{ containerPort: 443 }]
env:
- { name: QOS_CLASSES, value: "P0,P1,P2" }
- { name: DENY_HEAVY_RANGE, value: "eth_getLogs>5000" }
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: { name: edge-gw-hpa }
spec:
minReplicas: 4 maxReplicas: 24 metrics:
- type: Pods pods:
metric: { name: request_latency_p95_ms }
target: { type: AverageValue, averageValue: 350m }

12. 3 Kafka: region/tent partitioning

yaml topic: "events. p0"
partitions: 96 config:
min. insync. replicas: 2 cleanup. policy: delete compression. type: zstd message. timestamp. type: CreateTime

12. 4 Residency and export policies

yaml data_policy:
pii: { tokenized: true, cross_region_export: "deny" }
exports:
anonymized_metrics: { allowed: ["eu","uk","na"], schedule: "5m" }
hash_anchors: { allowed: ["eu","uk","na","apac"], cadence: "15m" }

13) Data schemas and queries

Directory of hubs and links

sql
CREATE TABLE hubs (
hub_id TEXT PRIMARY KEY,
region TEXT, tier SMALLINT, status TEXT,
rtt_ms INT, cost_per_1k_ops NUMERIC,
created_at TIMESTAMPTZ
);

CREATE TABLE interlinks (
src_hub TEXT, dst_hub TEXT,
capacity_mbps INT, cost_per_gb NUMERIC,
encrypted BOOLEAN, health TEXT,
PRIMARY KEY (src_hub, dst_hub)
);

Geo-Hit Ratio и Freshness

sql
SELECT region,
100. 0 SUM(CASE WHEN served_in_region THEN 1 ELSE 0 END)/COUNT() AS geo_hit_pct,
PERCENTILE_CONT(0. 95) WITHIN GROUP (ORDER BY freshness_s) AS freshness_p95
FROM req_stats
WHERE ts >= now() - INTERVAL '24 hours'
GROUP BY region;

TPS_per_$

sql
SELECT hub_id,
AVG(tps_sustained) / NULLIF(AVG(cost_usd_hour),0) AS tps_per_usd
FROM hub_perf
WHERE ts >= now() - INTERVAL '7 days'
GROUP BY hub_id;

14) Operating Regulations

Daily: SLO report (latency/queue-lag/freshness), export/residence audit, status of inter-hub faces.
Weekly: calibration of quotas/QoS and GeoDNS, recalculation of CTS/TPS_per_$, revision of caches and hot indexes.
Monthly: DR exercises (isolated mode, channel switching), DA/bridge check.
Before release: canary rollout by one hub/region, SLO gates and rollback plan.

15) Playbook incidents

A. The fall of the interregional channel

1. Switch to standby IX, enable compression/batches;

2. Hub in "finalized-only" mode;

3. Export queue - to the buffer, with a limit;

4. Communication with participants, post-mortem.

B. Local degradation of API p95

1. Prioritize P0, enable P2-throttle;

2. Increase edge/API replicas;

3. Enable cache for hot methods only;

4. Heavy query diagnostics, deny-rules if necessary.

C. Violation of residency

1. Immediate cross-regional export block;

2. Redaction/export reverse;

3. DPO/Compliance notification;

4. Updating policies and tests.

D. reorg/DA failure peaks

1. Increase Dispute K/Window;

2. Enable "delayed finalization";

3. Notify consumers;

4. Supplement reports.

E. Uneven hub loading

1. Re-tuning GeoDNS/Latency-LB;

2. Quota/price balance;

3. Traffic shaping for affiliates/sources.

16) Implementation checklist

1. Select regions/jurisdictions and target SLOs.
2. Design topology (Hub-Spoke or Active-Active), channels/IX.
3. Post roles: Edge/API/Compute/Data/Bridge/Compliance.
4. Configure residency, directories, and export policies.
5. Enable QoS, queues, caches, and backpressure.
6. Raise observability and Geo/Compliance/Perf/Cost dashboards.
7. Set up DR (RTO/RPO), drills and isolated mode.
8. Enter economic metrics (CTS, TPS_per_$) and budget.

17) Glossary

Geo-Hit Ratio - the share of requests served by "its" hub.
RPO/RTO - data loss/recovery time targets.
Hub-and-Spoke is a central node with peripheral clusters.
CRDT - data structures for conflict-free replication.
CTS per 1k ops - the cost of servicing 1000 operations.
TPS_per_$ - capacity per dollar of infrastructure.

Bottom line: regional hubs turn the global network into a set of locally optimized, compliant and resilient domains. With clear SLO, residency, QoS, and DR procedures, they reduce latency and cost, increase reliability, and enable ecosystem scaling without loss of manageability.