Data Contracts: The Interface Between Producers and Consumers

A data contract is a versioned, explicit agreement between the team that produces a dataset and every team that consumes it — covering schema, semantics, freshness SLAs, and a deprecation policy for change. Like an API contract, it turns an implicit hope ("please don't break my pipeline") into a tested, enforceable interface both sides can build against.

Quick answer: A data contract specifies a dataset's schema, the meaning of each field, freshness and completeness guarantees, and the rules for evolving it over time — agreed upon by the producing team and its consumers. It does for data what an OpenAPI spec does for a service: it makes the interface explicit, versioned, and testable instead of implicit and fragile.

What a Data Contract Actually Is

A data contract is a machine-checkable specification of a dataset's structure and behavior — not a wiki page — enforced the same way an API spec is enforced: with validation, versioning, and a named owner on each side. It exists because "the schema is the contract" quietly stops being true the moment two teams depend on the same table for different reasons.

Most data pipelines break for the same reason most undocumented APIs break: the interface was never made explicit, so nobody noticed when it changed. A backend engineer renames a column during a refactor. A batch job that used to run at 2 a.m. now runs at 6 a.m. An enum gains a new value nobody downstream expected. None of these look like a "breaking change" to the producer, because there was never a contract stating what could and couldn't change.

The API world solved this decades ago with API contracts: an explicit, versioned specification of endpoints, request/response shapes, and behavior guarantees that both provider and caller test against. Data contracts borrow that mental model directly. If you're new to the discipline, this sits inside the broader data PM role, where translating ambiguous upstream systems into reliable downstream products is a core part of the job.

The table above is the whole argument in miniature. Everything a producer team already knows how to do for a public API — version it, document breaking vs. non-breaking changes, register callers, run contract tests in CI — is directly transferable to a dataset. The only thing missing historically was the habit of writing it down.

Why Upstream Teams Keep Breaking Your Pipelines

Pipelines break because the dataset was never treated as a product with real users — it was treated as a side effect of whatever system happened to produce it. A source table that exists to serve one application's internal needs gets adopted, informally, by three analytics teams and a machine learning model, none of whom the producing team knows about.

That gap has a name in the data community: data mesh thinking, popularized by Zhamak Dehghani at ThoughtWorks, argues that datasets should be owned and served the same way software products are — with a defined interface, a named owner, and accountability for consumers' experience. Treating a dataset as a data product with real users rather than a byproduct of some other system is the mindset shift that makes a contract feel necessary rather than bureaucratic.

Three patterns cause most of the breakage in practice:

• Invisible consumers. The producer has no registry of who reads the table, so they can't warn anyone before a change ships.
• Implicit schema. The "contract" is whatever the ORM or ETL job happens to emit today — not a spec anyone reviewed or agreed to.
• No shared definition of "breaking." A producer's "minor cleanup" (dropping a rarely-used column) is a consumer's Friday-night incident.

Chad Sanderson, who popularized the modern data contracts pattern while running the data platform at Convoy and later founded Gable.ai around the idea, frames this as a "shift-left" problem: data quality and interface stability have to be enforced at the point of production, not discovered downstream after a dashboard goes blank. Gartner's research on data quality has repeatedly put the average cost of poor data quality at roughly eight figures a year for large organizations — directionally, the pattern of "cheap to prevent, expensive to fix downstream" holds regardless of the exact number your organization would land on.

The fix isn't heroics in the consuming team; it's making the interface explicit. That's exactly the discipline behind data quality as a feature rather than an afterthought — quality and stability are properties you design into the dataset, not properties you hope for after the fact.

The Anatomy of a Data Contract: Schema, Semantics, and SLAs

A complete data contract has three layers, and most failed attempts only cover the first one. Schema defines the shape (fields, types, keys). Semantics defines the meaning (what does active_user actually mean?). SLAs define the operational guarantees (how fresh, how complete, how available). Skipping any layer just moves the ambiguity somewhere else.

Schema: the structural promise

This is the part every team already half-does: field names, data types, nullability, primary/foreign keys, and allowed enum values. The failure mode isn't that schema goes undocumented — it's that it's documented once, at launch, and never re-validated as the underlying system evolves. A contract makes the schema a living, tested artifact rather than a historical snapshot.

Semantics: the meaning promise

Two teams can agree on a schema — status: string — and still disagree catastrophically about what it means. Does cancelled include refunded orders? Is active_user measured over 7 days or 30? This is precisely the discipline behind building a metrics layer the whole company can trust: one authoritative definition, computed once, consumed everywhere, instead of five teams quietly maintaining five slightly different versions of "active."

A good contract states semantics in plain language next to the schema, not in a separate doc nobody reads at query time: "order_status = cancelled means the order was voided before fulfillment; refunds post-fulfillment use order_status = returned."

SLAs and freshness guarantees

This is the layer most data contracts skip entirely, and it's the one borrowed most directly from Site Reliability Engineering. Google's SRE book popularized the SLI/SLO/error-budget model for service reliability; the same structure maps cleanly onto data: an SLI might be "minutes since last successful load," an SLO might be "95% of days, freshness stays under 30 minutes," and an error budget tracks how often that's allowed to slip before it's treated as an incident.

Before you write a freshness SLA, understand the consumer's actual job to be done. A nightly executive dashboard and a real-time fraud model consume the same order data but need wildly different freshness guarantees — writing one SLA for both either over-promises to one or under-serves the other.

Schema Evolution and Backward Compatibility

Not every schema change is a breaking change, and treating them all the same way — with a blanket "never change anything" policy — is how contracts die of their own bureaucracy. The useful distinction is the same one API designers have used for years: does this change require every existing consumer to update their code to keep working?

What counts as backward-compatible

The practical rule: additive changes are usually safe by default; anything that removes, renames, retypes, or restricts a field needs a version bump and a migration window. This is the same heuristic behind semantic versioning for software, formalized publicly at semver.org — additive changes bump a minor version, breaking changes bump a major version, and consumers can safely automate their reaction to either.

Versioning the contract itself

Version the contract, not just the table. In practice this means: a contract file (often YAML or JSON) checked into source control, a version number that increments on any breaking change, and automated tests that fail a build the moment a producer's change would violate the currently-published contract. dbt has built this natively as contracts on its models — schema tests that fail CI before a breaking change ever reaches a consumer's table. Stripe's API versioning approach is the model worth studying even outside the API world: every account is pinned to a version at signup, and Stripe maintains backward-compatible shims rather than forcing a synchronized migration — the data equivalent is maintaining a compatibility view for a deprecated column while consumers migrate off it.

Who Owns Breaking Changes: Governance and Deprecation Policy

The producing team owns the schema; the contract is what gives consumers a vote without giving them a veto over every change. In practice, ownership works best as a triangle: the producer proposes changes, a shared contract test suite enforces what's allowed without explicit sign-off, and a lightweight review catches anything the tests can't (semantic drift, a change that's technically additive but practically confusing).

A deprecation policy is the part most teams skip until the first incident forces one into existence. A workable minimum:

1. Announce the deprecation with a concrete sunset date, not "soon" — typically 30, 60, or 90 days depending on how embedded the field is.
2. Dual-write the old and new shape simultaneously so consumers can migrate on their own timeline rather than a forced cutover.
3. Flag usage — instrument the deprecated field so the producer can see, not guess, who's still reading it.
4. Escalate directly to any consumer still active at 50% of the runway remaining, rather than waiting for the deadline to surprise them.
5. Remove the field only after usage hits zero or the sunset date passes, whichever the policy states explicitly.

Mapping a consumer's move from the old contract version to the new one is its own version of a customer journey — an awareness stage, a migration stage, and a cutover moment, each of which benefits from the same deliberate design you'd apply to any other user-facing transition, not just an email and a hope.

A contract without a deprecation policy is just a schema with extra steps — it describes today's shape but gives consumers no guarantee about tomorrow's.

A Sample Data Contract You Can Adapt

A usable data contract is short enough to read in five minutes and specific enough that a test suite can enforce every line. Below is a stripped-down example for an orders dataset, covering fields, types, freshness, and the deprecation policy in one artifact rather than three disconnected documents.

contract:
  dataset: orders
  version: 2.1.0
  owner: checkout-platform-team
  consumers:
    - billing-reconciliation
    - finance-dashboard
    - fraud-model-v3
  freshness_sla:
    metric: minutes_since_last_load
    target: "< 30 minutes, 95% of days"
  completeness_sla:
    target: "99.9% of expected daily orders present"
  deprecation_policy:
    notice_period_days: 60
    dual_write_required: true
    escalation_trigger: "consumer active at 50% of notice period"
  breaking_change_definition:
    - field rename
    - field removal
    - type change
    - nullability tightening

Notice what this artifact does that a spreadsheet schema never does: it names an owner, lists known consumers, states the SLA numerically, and defines "breaking" in terms a CI pipeline can check automatically. That's the difference between a contract and documentation.

Where a tool like this fits

Treating an interface as something you specify once and generate consistent artifacts from — rather than something every team writes down slightly differently — is the same instinct behind Prodinja's API Designing tool, which lets you define fields, types, and guarantees for an interface and generate a concrete spec from them. It's built for API endpoints, but the underlying discipline is identical to what a data contract needs: state the interface explicitly enough that it can be tested, versioned, and handed off, rather than reverse-engineered from whatever the producer happens to ship.

Key Takeaways

• A data contract is the API-contract mental model applied to datasets — schema, semantics, freshness SLAs, and a deprecation policy, agreed upon by producer and consumer, not just documented by the producer alone.
• Most pipeline breakage traces back to invisible consumers and implicit schemas, not malicious changes — the producer genuinely didn't know who was reading the table or what "breaking" meant to them.
• Schema evolution needs a shared definition of backward-compatible vs. breaking: additive changes are usually safe; renames, retypes, removals, and nullability tightening require a version bump and a migration window.
• Freshness and completeness guarantees borrow directly from SRE's SLI/SLO model — pick the guarantee based on the consumer's actual job to be done, not a one-size-fits-all number.
• A deprecation policy with a stated sunset date, dual-write period, and usage monitoring turns "we changed something" from a surprise into a scheduled, low-drama migration.
• Ownership works as a triangle: the producer owns the schema, a contract test suite enforces the rules automatically, and a lightweight review catches what tests can't.
• A usable contract is a single, versioned artifact — fields, types, SLAs, and deprecation rules together — not three disconnected documents that quietly drift out of sync.

Frequently Asked Questions

What is the difference between a data contract and a data schema?

A schema describes structure — field names and types. A data contract wraps the schema with semantics (what fields mean), operational guarantees (freshness, completeness), and change-management rules (what counts as breaking, and the deprecation process for it) — the schema is one section of the contract, not the whole thing.

Who should own a data contract, the producer or the consumer?

The producing team owns and maintains the contract, since they control the underlying system, but consumers should have a formal way to register their dependency and be notified of change. Ownership without a consumer registry just relocates the invisibility problem rather than solving it.

How do data contracts relate to data quality monitoring?

Data contracts are the preventative layer; data quality monitoring is the detective layer. A contract with automated tests catches a violation before a breaking change ships; quality monitoring catches drift that happens despite the contract, such as an unexpected spike in null values within an allowed schema.

Do I need a data contract for internal analytics tables, or only for external data APIs?

Internal tables need contracts more urgently, precisely because they lack the natural friction of a public API (auth, rate limits, a support team) that forces external interfaces to be explicit. Any internal table with more than one consuming team benefits from the same explicit interface treatment as a public API.

What tools enforce data contracts in practice?

dbt's native contracts feature validates model schemas in CI before deployment; Great Expectations and similar frameworks validate data quality and distribution rules at runtime; and schema registries (common in Kafka-based architectures) enforce compatibility rules on streaming topics. None of these replace the governance decisions above — they enforce whatever contract your team has actually agreed on.