Contract Testing

1) Where to apply contracts

HTTP REST/JSON: resources, pagination, filters, idempotency, error codes.
gRPC/Protobuf: message types, statuses, `deadline` semantics, backward compatibility in .proto files.
GraphQL: schemas, non-null, directives, field-level permissions.
Messages/streams (Kafka/Pulsar/SQS): event schemas (Avro/JSON/Protobuf), partition keys, ordering, idempotency keys.
Internal SDKs/libraries: public functions/exceptions/performance contracts.

2) The CDC model: roles and artifacts

The consumer publishes a contract of expectations (sample requests/responses, type matchers, invariants).
The provider runs contract verification against its service/adapter/handlers.
A contract broker (Pact Broker/Backstage/artifact repository) stores versions, tags (`prod`, `staging`, `canary`) and the `consumer@v → provider@v` compatibility matrix.
Release policy: a provider deployment is blocked if any "production-relevant" contract is broken.

3) What to capture in a contract (HTTP example)

• Method/path/parameters/headers (including auth and the idempotency key).
• Body and type matchers (type/format/regex/ranges).
• Error codes and structure; stable `error_code` values.
• Semantic invariants: sorting, uniqueness, monotonic `created_at`.
• Non-functional expectations (optional): p95, size limits, rate-limit headers.

json
{
"interaction": "GET /v1/users/{id}",
"request": { "method": "GET", "path": "/v1/users/123", "headers": {"Accept":"application/json"} },
"matchers": {
"response. body. id": "type:number",
"response. body. email": "regex:^.+@.+\\..+$",
"response. body. created_at": "format:rfc3339"
},
"response": {
"status": 200,
"headers": {"Content-Type":"application/json"},
"body": {"id": 123, "email": "alice@example. com", "created_at": "2025-10-31T12:00:00Z"}
},
"error_cases": [
{
"name":"not_found",
"request":{"path":"/v1/users/9999"},
"response":{"status":404, "body":{"error_code":"USER_NOT_FOUND"}}
}
]
}

4) Contracts for events (event-driven)

Event schema: `type`, `version`, `id`, `occurred_at_utc`, `producer`, `subject`, `payload`.
Invariants: immutable `id` and idempotency by `(type,id)`, ordering within a partition key, monotonic `sequence`.
Schema Registry: stores schema evolution and compatibility rules (backward/forward/full).
Consumer contract tests: replay "golden" events and negative phases (unknown fields, nullable).

json
{
"type":"record","name":"UserRegistered","namespace":"events. v1",
"fields":[
{"name":"id","type":"string"},
{"name":"occurred_at_utc","type":{"type":"long","logicalType":"timestamp-millis"}},
{"name":"email","type":"string"},
{"name":"marketing_opt_in","type":["null","boolean"],"default":null}
]
}

5) Evolution and compatibility

Contract versions: `MAJOR.MINOR.PATCH` semantics (MAJOR means breaking).

• Do not break: do not remove fields, do not change the type/value of `error_code`.
• Add optional fields with defaults; add new endpoints instead of "magic".
• Deprecation: announcement, parallel support, removal based on metrics.
• GraphQL: only add fields, introduce non-null in phases; deprecation directives.
• gRPC/Proto: never reuse field numbers; only add new optional fields.
• Events: `vN` schema; consumers must ignore unknown fields (leniency).

6) Negative and invariant checks

Negative: wrong types, forbidden values, conflicting parameters, exceeded limits.
Invariants: response sorting, `id` uniqueness, correct `next_cursor`, a stable idempotent response on retry.
Time-related contracts: `created_at` in RFC3339/UTC; correct projection onto local calendar days is not part of the transport contract and belongs in business invariants.

7) Stub generation and local development

Provider stubs are generated from contracts for consumer development.
For events: generators of "valid/edge-case" messages based on the schema.
Stubs are labeled with the contract version and build date; publishing them to production is forbidden.

8) CI/CD integration (reference pipeline)

1. Consumer CI:

Lint/build → contract generation → unit/contract tests → publish to the contract broker (tag: `[email protected]`).

2. Provider CI:

Start the service locally/in a container → fetch relevant contracts (`prod`/`staging`) → verify → publish the status to the broker.

3. Release Gate:

A provider deployment is blocked if any contracts are not satisfied.

4. Nightly Matrix:

A `consumer versions × provider versions` compatibility matrix; reports and alerts.

9) Examples of practices by domain

9.1 REST: cursor pagination (contract invariant)

The response contains `items[]`, `next_cursor` (nullable), `limit`, `total` (optional).
Invariants: `len(items) ≤ limit`; a repeated call with the same `cursor` → an idempotent result set.
Error if `cursor` and `page` are both set.

9.2 POST idempotency

The contract requires the `Idempotency-Key` header.
Invariant: a repeated request with the same key returns the same `id`/status.

9.3 Events: ordering guarantees

Partition key in the contract: `partition_key = user_id`.
Invariant: `sequence` increases monotonically within a key; the consumer must handle duplicates.

10) Security and privacy in contracts

Do not include personal data/secrets in examples — synthetic data only.
Capture mandatory security headers: `Authorization`, `X-Signature`, `Replay-Prevention`.
For webhooks: a contract for the signature and the `2xx` response/retries.
Mask sensitive fields in contract test logs.

11) Tools

Pact / Pactflow / Pact Broker — HTTP/message contracts, compatibility matrix.
OpenAPI/AsyncAPI — specifications + test generators (Dredd, Schemathesis).
Karate/REST Assured — scenario-based REST contract checks.
Protobuf/gRPC — `buf`, `protolint`, compatibility tests; Schema Registry for Avro/JSON/Proto in streams.
Conformance tests for GraphQL (graphql-compat), schema snapshot tests.

12) Provider verification pseudocode (simplified)

python def verify_contract(provider, contract):
for case in contract["cases"]:
req = build_request(case["request"])
res = provider. handle (req) # locally/assert match_status container (res. status, case["response"]["status"])
assert match_headers(res. headers, case["response"].get("headers", {}))
assert match_body(res. body, case["matchers"], allow_extra_fields=True)
for neg in contract. get("error_cases", []):
res = provider. handle(build_request(neg["request"]))
assert res. status == neg["response"]["status"]
assert res. json. get("error_code") == neg["response"]["body"]["error_code"]

13) Anti-patterns

"Postman screenshots are the contract": no versions, type matchers or automated validation.
Over-specification: the contract pins exact values instead of types/patterns → false failures.
One shared contract for different regions/channels: ignores variability (flags, geo rules).
Contracts without a broker/matrix: you cannot tell which versions are compatible.
Relying on e2e instead of contracts: slow, expensive, flaky.
No negative/invariant cases: only the "happy path" is tested.

14) Observability and operations

Export statuses to the broker + a "contract health" dashboard.

Alerts: new provider failures against `prod` contracts, growth of "unknown field" in events.
Tracing: `contract_id`, `version`, `decision_id` in verification logs.

15) Deprecation process

1. Add the field/endpoint (non-breaking).
2. Mark the old one as `deprecated` in the specification and announce the timeline.
3. Track consumers via logs/the broker; provide migration guides.
4. Enable a "shadow" deny in staging (dry run), then enforce.
5. Remove once usage drops to zero and compatibility is confirmed.

16) Architect's checklist

1. Are consumers and their owners identified? Are contracts versioned?
2. Is there a broker and a compatibility matrix with environment tags?
3. Does the contract include negative cases and invariants (idempotency, cursors, sorting)?
4. Is a Schema Registry and compatibility mode configured for events?
5. Does the pipeline block a provider release when production contracts are broken?
6. Are the deprecation process and evolution policy documented?
7. Are stubs generated from contracts, and are there local event generators?
8. Are personal data masking and mandatory security headers documented?
9. Are contract metrics/alerts connected, and are there drift reports?
10. Are contracts verified in CI on both sides (consumer and provider)?

Conclusion

Contract testing moves the "source of truth" about interactions into versioned artifacts and makes integrations predictable. CDC, a contract broker and schema evolution discipline replace "breaking surprises" with a managed process: fast checks, clear invariants and transparent version compatibility. This reduces the cost of e2e testing, speeds up releases and improves the quality of the whole platform.