Technology and Infrastructure → Redis: in-memory solutions

Redis: in-memory solutions

1) Where Redis is appropriate

• Cache (read-through/aside, TTL, SWR) and sessions.
• Counters and quotas: rate limiting, anti-fraud, campaign limits.
• Leaderboards/ratings (ZSet), "top N" recommendations.
• Event queues/buses (Streams/PubSub), outbox/inbox, retrays.
• Idempotence (keys with TTL), de-dup webhooks.
• Geo (search for nearest points), Bitmap (flags, DAU).
• Aliases/tokens and short-lived authorization caches.

2) Data structures and when to apply them

String: values/counters ('INCRBY'), idempotent keys.
Hash: aggregates of profiles/configs, storage of "lightweight" objects.
List: simple queues (but without replay/offset semantics).
Set: unique elements, deduplication.
ZSet: sorting by speed (leadboards, TTL calendar - "deferred" events).
Stream: stable queues with consumer groups, 'XREADGROUP '/replay - for webhooks, CDC, retrays.
Geo: 'GEOADD/GEORADIUS' - nearest points/merchants.
Bitmap/Bitfield: series of flags (logins by day, DAU/WAU).
HyperLogLog: Approximate Unique (UU) is cheap on memory.
Bloom/Cuckoo (modules): quick availability checks, reduce "MISS" to source.

• RedisJSON (JSON documents), RediSearch (indexing/searching), RedisBloom (probabilistic structures), TimeSeries (metrics/aggregations).

3) Keys, TTL and memory policies

tenant:{t}:domain:{d}:{entity}:{id}:v{schema}    region={R}    currency={C}    lang={L}

Versioning ('vN'), include only meaningful dimensions (region/currency/language/tenant).
Isolate the per-tenant key spaces.

• Use a TTL matrix (sec/min/hr), add jitter (± 10-20%) to avoid stampede.
• For hot keys - refresh-ahead and single-flight (one leader updates).

• 'allkeys-lru/lfu'is a shared cache without TTL dependency.
• 'volatile-lru/lfu '- only keys with TTL.
• 'noeviction '- write failure on overflow (safer for critical queues/counters).
• Select for the script and always monitor 'evicted _ keys'.

4) Transactions, pipelines and scripts

Pipelines: reduce RTT, group 10-100 teams.
Transactions (MULTI/EXEC) - Do not isolate reads, but execute the batch atomically.
Optimal locking: 'WATCH key' → MULTI/EXEC → check.
Lua scripts: atomic logic on the server side (rate limit, locks, composite operations).

5) Queues and buses: List vs Stream

List + 'BRPOP' - simple, but no consumer groups, offset/replay, weak resistance to drops.
Stream: 'XADD → XREADGROUP → XACK', retry-deadletter (not taken in N minutes), partitioning by key. Recommended for PSP/KYC webhooks, deferred payouts/notifications.

Priority queues: several streams by priority, consumers "suck" out of the high in the first place.
Deferred tasks: ZSet where score = timestamp; periodic'ZRANGEBYSCORE '≤ now → transferred to Stream.

6) High availability and scalability

Replication: master→replica (read-scale).
Sentinel: automatic failover master, discovery, client URIs.
Redis Cluster: 16384-slot sharding, horizontal scale-out. Wrap keys that use multiple structures in '{order: 123}' hash tags.

• For cache/sessions - cluster/replica, 'client-side hashing' SDK supported.
• For queues/streams - minimize cross-slot operations; partition by domain keys.

7) Persistence: RDB, AOF and backups

RDB (snapshots): faster, more economical; risk of loss of last seconds/minutes.
AOF (journal): fewer losses; 'everysec/always' modes. AOF compression and periodic repackaging.
Hybrid: RDB + AOF → fast recovery + moderate losses.
Backups: snapshots and copies of AOF to object storage; check recovery regularly.

For critical queues/idempotency, select AOF'everysec '+ replication.

8) Safety and compliance

AUTH/ACL: roles per-application, prohibition of "dangerous" commands ('FLUSHALL,' 'KEYS').
TLS to client-server and inter-node links; fixed egress-IP.
Network segmentation: private subnets, SG/NACL; access only from the required services/namespaces.
Do not log secrets; PAN/PII in Redis - tokens/derivatives only.
Key commands: avoid'KEYS '- use'SCAN'.

9) Observability and SLO

• Latency (P95/P99), `instantaneous_ops_per_sec`, `connected_clients`.
• Hit ratio, evicted_keys, expired_keys.
• Memory: used, fragmentation ratio, RSS, allocator stats.
• Replication lag, AOF/RDB frequencies and sizes, fork time.
• Streams: PEL (pending entries list), delivery latency, retry count.

• Redis P99 operations ≤ 5-10 ms.
• Evictions ≤ 1 %/hour (cache space).
• Stream delivery P99 ≤ 500 мс, retry rate < 2%.

10) FinOps and Resource Planning

Memory is expensive: measure $/GB-month RAM vs saving requests to origin/DB.
Enable value compression> 1-2 KB (see CPU).
LFU can give a better hit with less volume.
For images/large blobs - not Redis: use CDN + object storage.

11) Patterns for iGaming/fintech

11. 1 Rate limiting (Lua)

Idea: 'INCRBY' in window key + TTL; Lua atomically checks the limit and increases.

lua
-- KEYS[1]=key ARGV[1]=limit ARGV[2]=ttlSec ARGV[3]=inc local cur = redis. call('INCRBY', KEYS[1], ARGV[3])
if cur == tonumber(ARGV[3]) then redis. call('EXPIRE', KEYS[1], ARGV[2]) end if cur > tonumber(ARGV[1]) then return {0, cur} else return {1, cur} end

11. 2 Request idempotence

Key 'idemp: {request _ id}' with TTL 24h, value - result/status. Before performing the operation, we check for presence.

11. 3 Leaderboards

`ZINCRBY leaderboard:game:{g} score user:{u}` → `ZREVRANGE... WITHSCORES`.
For the top N by region/tenant - individual ZSet or prefixes.

11. 4 PSP Webhooks Queue

'XADD psp: webhooks... '→ consumer group' XGROUP CREATE psp: webhooks g1 $ '.
Retrays of "stuck" messages via PEL scanning ('XPENDING' → 'XCLAIM').

11. 5 Deferred payments

ZSet'payout: due '(score = epoch) → the worker periodically transfers finished items to Stream'payout: exec' with deduplication.

11. 6 Antifraud meters

Combination of'PFADD' (unique) + 'INCR' (intensity) + geo/ASN tags; triggers for manual validation.

12) Working with memory and performance

Client connection pools; reduce RTT (keep-alive).
Prefer pipelines to a pack of commands.
Watch for big keys ('MEMORY USAGE', 'SCAN') - it is better to split objects.
Hash with a small number of fields is more economical than many individual keys.
Enable io-threads (read-heavy) if the profit is confirmed by tests.
Avoid frequent 'FLUSHDB/ALL' in the prod; manage via prefixes and'UNLINK 'for safe deletion.

13) Multi-tenant and isolation

Individual clusters/instances or logical DB per-tenant (if the load is small).
Key/memory quotas, split ACLs.
Prefixes in keys and metrics by namespace.

14) Locking and consistency

SET key val NX PX = ttl - simple mutex.
Redlock: use carefully; for distributed critical transactions, it is better to rely on the "source of truth" (DB/ledger) and idempotent operations.
Prefer atomic operations and Lua instead of "long" locks.

15) Anti-patterns

Storage of large blobs/images - overload of RAM and networks.
Financial invariants (balance sheet) only in Redis.
'KEYS'and' scanning the world'in the prod.
No TTL/jitter - dogpile on expiration.
The 'allkeys-' policy on critical queues → data loss at pressures.
Mixing queues, cache and sessions in one instance without quotas and priorities.
Lua scripts that work on the keys of different slots in the Cluster.

16) Implementation checklist

1. Define roles: cache/sessions, queues/streams, counters/limits - post to instances/clusters.
2. Select maxmemory-policy for the task; set limits and monitoring evictions.
3. Key naming, circuit versions, TTL matrix + jitter; single-flight for top keys.
4. For queues - Streams (groups, retrays, DLQ), for deferred - ZSet + transfer.
5. HA: replication + Sentinel or Redis Cluster; check client failover.
6. Persistence: RDB/AOF under script; regular backups and a recovery test.
7. Security: ACL, TLS, private networks, prohibition of dangerous commands.
8. Observability: latency, ops/sec, memory, evictions, replication lag, stream PEL.
9. FinOps: memory profiles, large keys, compression, LFU; avoid Redis for big blobs.
10. Pattern documentation (rate-limit, idempotency, leadboards) and load tests.

Result

Redis is a "multifunctional Swiss knife" of speed: cache, queues, counters, leadboards, geo and probabilistic structures. Its strength lies in the correct choice of data structures, TTL/disability discipline, atomicity of operations, and well-thought-out HA/persistence and observability. Use Redis where milliseconds and high RPS are important, while leaving critical invariants (money, accounting) to the "source of truth" - this way the platform will remain both fast and reliable.