Assembly optimization and caching

(Section: Technology and Infrastructure)

Brief summary

The CI/CD speed in iGaming directly affects the release rate, minute cost, and p99 stability at peak loads. The key is correct caches (dependencies, artifacts, container layers, intermediate compilation results), incremental builds, and determinism. A "good" build is quickly repeated with unchanged input, and cache disability is predictable and controllable.

1) Cache map and what we cache

Dependencies: NPM/pnpm packages, pip wheels/Poetry, Maven/Gradle, Go mods, Cargo crates, NuGet.
Intermediate compilation artifacts: '~/.cache/pip', '~/.m2', '.gradle', '~/.cargo/registry/',' $ GOMODCACHE ',' target/', 'build/',' node _ modules/.pnpm-store '.
Container layers: Docker layer cache (BuildKit/GHA cache), registry-based cache, multi-stage.
Tools and SDKs: toolchains/microimages (JDK, Node, Python, Rustup targets).
Mono/polyrepo-meta: Nx/Turborepo/Bazel remote-cache cache for tasks (lint/test/build).
Test data and e2e fixes: database snapshots compiled by UI bundles.
ML/data: prepared datasets, embeddings, compiled engines (TensorRT/ONNX).

2) Principles of fast and predictable assembly

1. Determinism: fix versions (lockfiles), pin base images, hermetic plugins → reproducible output.
2. Idempotence: the same assembly → the same artifacts and hashes.
3. Incrementality: rebuild only the changed one (DAG/needs/matrix/affected).
4. Locality and "heat": caches next to runners/in the registry, warming up before peaks.
5. Explicit disability: cache keys by lockfile/configuration files and by content hash.
6. Hygiene: TTL/' expire _ in ', auto-cleaning, cache size control and artifacts.
7. Supply chain security: cache ≠ trash for secrets; SBOM/artifact signature remains mandatory.

3) Docker/OCI: quick images without reassembly

Patterns

Multi-stage (builder → runtime).
Minimum runtime (distroless/ubi-micro, only necessary so/ca-certs).
Layer order: first rarely changing (deps), then code.
'.dockerignore': exclude '.git', tests/fixtures, local caches.
BuildKit: 'cache-from/to' → shared cache between jobs and branches.

Dockerfile example (Node + pnpm)

dockerfile syntax=docker/dockerfile:1.7
FROM node:20-bookworm AS deps
WORKDIR /app
COPY pnpm-lock.yaml./
RUN corepack enable && pnpm fetch

FROM node:20-bookworm AS builder
WORKDIR /app
COPY --from=deps /root/.cache/pnpm /root/.cache/pnpm
COPY package.json pnpm-lock.yaml./
RUN corepack enable && pnpm install --offline
COPY..
RUN pnpm build

FROM gcr.io/distroless/nodejs20 AS runtime
WORKDIR /app
COPY --from=builder /app/dist./dist
USER 10001
CMD ["dist/server.js"]

BuildKit в CI (GitHub Actions)

yaml
- uses: docker/setup-buildx-action@v3
- uses: actions/cache@v4 with:
path: /tmp/.buildx-cache key: buildx-${{ github.ref }}-${{ github.sha }}
restore-keys: buildx-${{ github.ref }}-
- uses: docker/build-push-action@v6 with:
push: true tags: ${{ env.IMAGE }}
cache-from: type=gha cache-to: type=gha,mode=max

4) Language ecosystems: what to cache and how

Java/Kotlin (Maven/Gradle)

Remote cache Gradle, concurrency, configuration-on-demand.
Cache key: hash'build. gradle[.kts]` + lockfiles + `gradle-wrapper. properties`.
Publish build cache node to object storage/HTTP.
Incremental compilation and test split by package.

yaml
GitLab CI cache:
key: gradle-${CI_COMMIT_REF_SLUG}
paths: [.gradle/caches,.gradle/wrapper ]
script:
-./gradlew --build-cache --parallel build

Node. js (npm/pnpm/yarn)

Local store cache ('~/.npm', '~/.cache/pnpm'), lockfile key.
Nx/Turborepo remote-cache (S3/Redis) for tasks (lint/test/build).
'turbo run build --cache-dir = .turbo'and "affected" mode for monorepos.

Python (pip/Poetry)

Cache wheels + virtualenv; key by'requirements. lock`/`poetry. lock`.
Assembling wheels in a separate stage, reusing between matrices.
For C extensions - 'pip wheel' + 'auditwheel' in the builder image.

Go

Кэш `GOMODCACHE`, `GOCACHE`; fix'GOTOOLCHAIN '/versions.
Divide the steps: 'go mod download' → a copy of the code → 'go build'.
For large monorails - Bazel/Bazelisk or 'mage' with a layer build.

Rust

`~/.cargo/registry`, `~/.cargo/git`, `target/`; sccache (remote/local).
Sharing cache between branches on 'Cargo. lock`.

C/C++

ccache/sccache + key by compiler flags and SDK versions.
Take out the toolchain in a separate basic image.

5) Bazel/Nx/Turborepo: task and graph cache

Bazel remote cache (HTTP/Cloud) - content-addressable; strict tightness, sandboxes.
Nx/Turborepo - cache of task outputs and "only-affected" execution in monorepos.
Disability: depends on step inputs (files/flags/variables).

6) Cache disability strategies

Key = hash of inputs: lockfiles, compiler configs, manifests.
Mild disability: 'restore-keys' (GHA )/prefixes (GLCI).
Hard disability: rotate namespace/key for critical changes.
Layer separation: deps vs sources - change the code without breaking heavy dependencies.

7) Incremental builds and matrices

DAG/needs: run dependent jabs in parallel, do not wait for sequences.
Paths-filter - Trigger only for affected components.
Shard tests: by directories/seed, align duration.
Warm-pool runners: pre-heated images/caches before peaks (tournaments/campaigns).

8) Artifacts vs cache: how different

Cache: reused inputs/intermediate results, dirty area, TTL short/medium.
Artifacts: final assemblies (images, binaries, charts), unchangeable, signed, with SBOM.
Rules: the cache is cleaned aggressively, artifacts are stored according to the release policy.

9) Observability, KPI and FinOps

Metrics (by pipeline/repo):

Hit-rate of cache (%), Warm-start vs Cold-start time, average/median duration of stages.
Cost per pipeline/job, cache/artifact size, throughput (jobs/hour).
Share of "affected-runs" in monorepo, rebuild unchanged (waste).

Alerts:

Hit-rate falls below the threshold, build image time rises, artifacts inflate, SLO misses.

10) Cache security and supply chain

No secrets in cache/artifacts; mask vars, scans of secrets.
Pin SHA external actions/plugins, only trusted runners.
Signing containers/binaries (cosign), SBOM (CycloneDX/SPDX) and checking for CD.
Isolation: separate cache namespace for 'dev/stage/prod', read-only rights for foreign branches.

11) Practical templates

GitHub Actions - Language + Container

yaml name: ci on: [push, pull_request]
concurrency: { group: ${{ github.ref }}, cancel-in-progress: true }
jobs:
build:
runs-on: ubuntu-latest steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4 with: { node-version: '20' }
- uses: actions/cache@v4 with:
path: ~/.cache/pnpm key: pnpm-${{ hashFiles('pnpm-lock.yaml') }}
- run: corepack enable && pnpm i --frozen-lockfile
- run: pnpm build
- uses: docker/build-push-action@v6 with:
tags: ${{ env.IMAGE }}
cache-from: type=gha cache-to: type=gha,mode=max

GitLab CI — Gradle remote cache

yaml variables:
GRADLE_USER_HOME: ".gradle"
cache:
key: gradle-${CI_COMMIT_REF_SLUG}
paths: [.gradle/wrapper,.gradle/caches ]
build:
stage: build script:
-./gradlew --build-cache --no-daemon build artifacts:
paths: [ "build/libs/.jar" ]
expire_in: 3 days

Jenkins — ccache/sccache

groovy pipeline {
agent { label 'cpp' }
environment { CCACHE_DIR = '/cache/ccache' }
stages {
stage('Build') {
steps {
sh 'ccache -M 10G'
sh 'cmake -B build -S. && cmake --build build -j$(nproc)'
}
}
}
post { always { sh 'ccache -s' } }
}

12) ML/data: speeding up heavy assemblies

Cache of models/embeddings/datasets on local NVMe runners; hash versioning.
Pre-assembly TensorRT/ONNX engines as release artifacts; reuse in inference.
Chunked artifacts (splits) for large models; TTL and deferred cleaning.

13) Implementation checklist

1. Commit lockfiles and base images; Enable BuildKit.
2. Separate Docker layers: deps → code; add '.dockerignore'.
3. Raise remote cache (Gradle/Bazel/Nx/Turbo); start dependency proxy.
4. Configure the dependency cache in the CI by lockfile; include matrices and'paths-filter '.
5. Enter incremental builds and "affected only" in the monorepo.
6. Measure hit-rate, warm/cold time, cost; put up alerts.
7. Enable SBOM/signature, deny secrets in cache.
8. Warm up caches before peak releases; regulate TTL/retention.

9. Document cache disability and runbooks as "cache broken."

10. Regularly clean "eternal" caches and archive heavy artifacts.

14) Antipatterns

Uncompressed versions of tools → "cache is, but not repeated."

Huge Dockerfile with frequently changing steps before installing deps.
Shared "perpetual" cache without keys/TTL → flakes and garbage.
Mixing artifacts and cache; no signature/SBOM.
Matrices "for all" at each PR; lack of 'concurrency. cancel`.
Single runner without warm cache and without dependency proxy.

Results

Assembly optimization is a systematic work with caches, incrementality and determinism. The correct Dockerfile structure, remote-cache for builds, dependency proxy and disability discipline give fast, cheap and reproducible pipelines. Add observability and safety rules - and your releases will be frequent, stable and economical.