Why context layers exist (and how they differ from semantic layers)
Semantic layers won the argument. Snowflake organised its entire Summit around governed enterprise context. dbt, Cube, AtScale and the warehouse vendors all ship one, and in late 2025 Snowflake, Salesforce and dbt Labs launched the Open Semantic Interchange to make them interoperate. The reason has little to do with humans: an agent that guesses what “revenue” means is worse than useless, so every serious data team now governs its definitions.
Then the agents went into production, and a second gap appeared. An agent on a well-governed semantic layer gets the number right and the explanation wrong. Ask it what churn was last month and it answers correctly. Ask it why churn spiked and it speculates, fluently, about pricing and seasonality, while the analyst sitting next to it knows the spike was a billing migration that wrote false cancellations. The agent cannot know that. The migration was never data.
That gap is what the industry has started calling the context layer.
Yali Sassoon at Snowplow draws the line by tense and address: a semantic layer describes the past consistently for whoever asks, while a context layer assembles what one agent needs for one decision, now. Snowflake’s engineering team draws it as six layers of trustworthy agency, and names the missing middle plainly: warehouses capture state, not the decisions and debates that made state true.
Most of the market still confuses the semantic layer for context. Our framing is shorter: the semantic layer is the governed what, the context layer is the governed why, and until the why is written somewhere an agent can read, agents will keep producing confident answers that are subtly but disastrously wrong.
The encouraging part is how cheap the fix appears to be. In the same post, Snowflake describes an internal experiment: they gave an agent a plaintext ontology file, a few hundred lines of join keys, table grains and cardinality hints, on top of a best-practice baseline. Answer accuracy rose 20 percent, tool calls fell 39 percent, latency fell 20 percent. Internal figures, so hold them loosely, but they match what we see in client work.
Written-down knowledge beats a bigger model, and it costs an afternoon rather than a procurement cycle.
Why you cannot buy a context layer
Vendors have noticed the category, and most of their definitions converge on the same shape: a context layer is a semantic layer with an agent on top, or a catalogue with an agent on top, or a knowledge graph with an agent on top. Yali’s warning applies to all three. A definition the platforms already meet by construction settles the category the moment it is named, and it settles it in favour of whatever the vendor was selling last year. The same vacuum showed at a dinner we hosted a few weeks ago, where we found a massive split between context layer theory and practice.
The deeper reason no product fits is that the content does not exist yet. The reasoning behind a metric definition, the caveat a senior analyst attaches before a number reaches a board deck, the knowledge that the March numbers carry migration artefacts: all of it lives in someone’s head, or in a Slack thread that scrolled away. No retrieval system, however clever, can retrieve what was never recorded. Building a context layer is mostly the work of getting that knowledge written down, and writing things down is a workflow problem. Tooling can host the result; it cannot produce it.
The knowledge is also heterogeneous, which rules out a single store. A static warning about a column, a dated event that moved a metric, a two-page decision record, and a set of known-correct query answers are four different shapes of knowledge. Force them into one wiki and the agent retrieves prose when it needed a joinable fact. Force them into one table and the reasoning gets truncated to a comment field. Each shape has a natural home, and the context layer is the assembly of those homes behind one interface.
The build, at a glance
Let’s make this a lot more concrete and split context into four kinds of knowledge. We think an agent should be reading them all, from four “homes” accessible via MCP.
Our experience having done this is that there are seven rules from building this with clients, each expanded below:
- Demand the why. The semantic layer alone answers what; only context answers why.
- Ship caveats with the model. Static warnings belong in dbt YAML, version-controlled, agent-readable.
- Make events joinable. Events are dated facts; store them where dated facts live.
- Version your definitions. A definition change looks like a business change until someone records it.
- Write reasoning in plain files. Markdown in git beats any wiki for agent-priority knowledge.
- Pin correct answers. Verified queries anchor trust and double as your eval set.
- Wire capture into the workflow. Force the writing-down into steps that already happen, or watch it rot.
To make it concrete, and to start our work like tradition demands, we started a coffee company. Not made up at all, of course 😉
Introducing Kelder Coffee
Kelder Coffee sells single-origin coffee on subscription from Amsterdam, plus one-off retail. Around 40 people, Series A, one data analyst, no data engineer. Shopify, Recharge, Klaviyo and the ad platforms flow through Fivetran into Snowflake, modelled in dbt. Notion holds the docs, Slack the chat, Linear the tickets. No BI tool in the base case; we cover the Omni variant at the end.
Six things happened at Kelder in the first half of 2026. The analyst carries all six in her head, nothing in the warehouse records any of them.
One question runs through the rest of the piece: why did subscriber churn spike in March?
Rule 1: Demand the Why
Kelder’s semantic layer is in good shape. Churn is defined once and computes the same for anyone who asks. The same two questions, asked either side of the context line.
The second answer needs three things the semantic layer does not carry: the event, its expected effect on the metric, and the reasoning trail. The next four rules give each one a home.
Rule 2: Ship caveats with the model
Anything a competent analyst would warn a colleague about before they query a table belongs in dbt YAML, next to the model, version-controlled, and exposed to agents through the dbt MCP server. At Kelder, the subscriptions mart carries its own warnings:
- name: churned_at description: Timestamp the subscription entered a terminal cancelled state. meta: caveat: > The 2026-03-12 Recharge backfill wrote churned_at for ~1,900 paused subscriptions. These rows carry is_migration_artifact = true and are excluded from churn metrics as of v2026.03.2.
| Column | What it is | What the agent needs to know |
|---|---|---|
| churned_at | Timestamp of terminal cancellation | ~1,900 false values from the 12 Mar backfill, flagged and excluded from churn |
| mrr_eur | MRR attributed to the subscription | Zero for gift subscriptions by design |
| status | Unified subscription status | Shopify and Recharge use different source values; unified in one intermediate model |
One limit: descriptions and meta fields describe the what and the watch-out. They carry no timeline and no reasoning. Teams that stop here have a well-documented semantic layer and still no context layer.
Rule 3: Make events joinable
Events that explain anomalies are dated facts, so they live where dated facts live: in the warehouse. A business_events table turns “everyone on the data team knows about the March migration” into a row the agent joins against any metric query. Logging a row takes a minute. Skipping it costs an hour of confused analysis per person per anomaly, indefinitely.
create table context.business_events (
event_date date,
event_type string, -- migration | promo | outage | definition_change | logistics
title string,
description string,
affected_metrics array,
expected_effect string, -- up | down | gap | none
owner string,
source_link string -- decision record, Linear ticket, or Slack permalink
);
Kelder’s table after six months:
| event_date | event_type | title | affected_metrics | expected_effect | source_link |
|---|---|---|---|---|---|
| 2026-03-12 | migration | Shopify to Recharge billing migration | subscriber_churn_rate, mrr | up (artificial) | context/decisions/0007 |
| 2026-03-28 | promo | Pause-instead-of-cancel campaign | subscriber_churn_rate, pause_rate | down | Linear KEL-412 |
| 2026-04-09 | outage | Fivetran Klaviyo connector down 31h | email_attributed_revenue | gap | Slack permalink |
| 2026-05-01 | definition_change | Churn v2: 30-day dunning grace | subscriber_churn_rate | down (definitional) | context/decisions/0009 |
| 2026-05-19 | logistics | PostNL courier strike | refund_rate, delivery_days | up | Linear KEL-467 |
| 2026-06-14 | promo | Father’s Day gift bundle | aov, new_customers | up | Notion campaign page |
You can now register the table in the Snowflake semantic view so Cortex Analyst can join it. Any “why did X move” question then resolves against recorded events before the agent starts speculating.
Rule 4: Version your definitions
A metric that changed definition mid-year produces a trend break that looks like a business change. Kelder’s changelog is a two-row table, and it is doing more work than most dashboards:
| metric | version | valid_from | definition | reason for change |
|---|---|---|---|---|
| subscriber_churn_rate | v1 | 2025-01-01 | Cancelled in month / active at month start. Failed payments count as churn on day one | Initial definition |
| subscriber_churn_rate | v2 | 2026-05-01 | As v1, with a 30-day dunning grace before a failed payment counts as churn | Recharge dunning recovers a material share of failed payments; day-one counting overstated losses |
Asked to compare churn year on year, an agent with this table flags that pre-May figures use v1 and offers both a restated and an as-reported series. A senior analyst does that unprompted. Now the agent does too.
Rule 5: Write reasoning in plain files
Reasoning does not fit in a YAML field. Decision records hold it: short, numbered markdown files in the dbt repo, one per decision, immutable, superseded rather than edited. Plain files in git beat a wiki for agent-priority knowledge for one reason: agents read them natively, with no connector, no export and no permission model in the way. The same logic makes an Obsidian-style vault a stronger agent-facing knowledge base than Notion or Confluence; the file format does the integration work. Notion stays for prose that non-technical colleagues own.
kelder-dbt/
├── AGENTS.md # agent instructions, < 300 lines, hand-written
├── models/
└── context/
├── glossary.md
├── metric_changelog.md
├── decisions/
│ ├── 0007-recharge-migration-churn-artifacts.md
│ └── 0009-churn-v2-dunning-grace.md
└── quirks/
└── gift_subscriptions.md
Decision 0007 in full, because the unit of narrative context should look boring:
# 0007: Recharge migration wrote false churn events
Date: 2026-03-14 · Status: accepted · Owner: Sanne (data)
## Context
The 12 March billing migration backfilled Recharge subscription
statuses. Recharge has no "paused" state in the legacy import path,
so ~1,900 paused subscriptions arrived as "cancelled", writing
churned_at timestamps that never happened.
## Decision
Flag affected rows with is_migration_artifact = true. Exclude
flagged rows from churn metrics. Report both raw and adjusted
churn for March 2026 in any board-facing number.
## Rejected alternative
Deleting the rows. Finance reconciles against Recharge exports,
so the raw record must match source.
## Consequences
March raw churn (9.4%) and adjusted churn (3.3%) will both exist.
Anyone quoting March churn without the adjustment is wrong.
Two sizing rules. Keep agent-instruction files hand-written and under roughly 300 lines; research from ETH Zurich found hand-curated AGENTS.md files improved agent task completion by around 4 percent while LLM-generated ones made agents slightly worse at a 20 percent higher token cost [source link to add before publication]. And once the context directory outgrows always-on loading, serve it through retrieval, Cortex Search over the markdown or a filesystem MCP, rather than stuffing the prompt.
Rule 6: Pin correct answers
The context stack tells the agent what is true. Verified queries pin down what correct output looks like, question by question, and double as the eval set: rerun them after any model or definition change and diff the SQL. On Snowflake this is native; Cortex Analyst semantic views take a verified_queries block used as few-shot grounding, plus custom instructions for business logic.
| Verified question | Pins down |
|---|---|
| What was subscriber churn last month? | v2 definition, artefacts excluded |
| What was real churn in March 2026? | Raw and adjusted both returned, adjustment named |
| Which channel has the best CAC? | Attribution window and channel grouping |
| How did the Father’s Day bundle perform? | Bundle SKU list, promo date range from business_events |
Start with the top 10 to 20 questions your business asks every week. Every correction a stakeholder makes to an agent answer is a candidate verified query.
Rule 7: Wire capture into the workflow
Every artefact above rots unless writing it is a forced step in work that already happens. Kelder wires capture into two moments: the pull request and the incident. No new tools, no reliance on goodwill.
## What changed
## Why (business context, not implementation detail)
## Metric impact
- [ ] No user-visible metrics move
- [ ] Metrics move: metric_changelog updated
- [ ] Metrics move: business_events row added
- [ ] Decision record added or superseded (link)
The incident runbook gets one added step: before closing, log the business_events row with a Slack permalink as source_link. The difference in practice:
| Before | After | |
|---|---|---|
| PR description | “Fix churn logic” | “Exclude migration artefacts from churn. The 12 Mar backfill wrote ~1,900 false cancellations. Changelog updated, decision 0007 linked, business_events row added.” |
| Where the reasoning ends up | A Slack thread that scrolls away | Three queryable artefacts, written in the two minutes the author still remembers why |
Wiring the context layer to an agent
We should now set up an agent and connect it to three well-scoped MCP servers. The Snowflake managed MCP server exposes Cortex Analyst over the semantic view (metrics, verified queries, the registered business_events table) and Cortex Search over the markdown. The dbt MCP server exposes model descriptions, sources and the semantic layer tools. A filesystem MCP reads the context directory and AGENTS.md directly. Notion, Slack and Linear connect on demand for human-owned prose and incident threads. Keep the server count low: tool selection degrades as servers multiply, and three scoped servers beat eight overlapping ones.
This wiring also answers the small-company case with no BI tool at all. Snowflake Semantic Views (bootstrapped with Semantic View Autopilot, then hand-corrected) become the metrics surface, Cortex Analyst with verified queries sits on top, and the agent is the only interface anyone queries. Nothing in the stack beyond Snowflake requires a paid seat.
The Omni variant: context inside the semantic model
Larger organisations running Omni get first-class homes for most of this inside the semantic model itself. ai_context fields exist at model, topic and field level, sample_queries play the verified-query role, and the Omni MCP server exposes the governed model, context attached, to any external agent under the asking user’s permissions. Budget the roughly 200k characters of context: broad rules at model level, domain facts at topic level, field level only where a metric is easily confused.
| Level | Scope | What Kelder writes there |
|---|---|---|
| Model ai_context | Every query | “Kelder sells coffee subscriptions plus one-off retail. Currency EUR. Churn follows the v2 definition since 2026-05-01. March 2026 churn contains migration artefacts; prefer adjusted figures. Check business_events before explaining any anomaly.” |
| Topic ai_context (subscriptions) | One domain | “MRR excludes gift subscriptions. Pause and cancel are distinct states since 28 Mar; a rising pause rate is expected and healthy.” |
| Field ai_context (churned_at) | One column | “Excludes rows where is_migration_artifact = true. Raw values exist for Finance reconciliation only.” |
The warehouse tables and the decisions directory stay. Omni becomes the surface where semantics and context meet the business user, and dbt descriptions pipe in so caveats written once serve both stacks.
The payoff
Rerun the opening question against the finished stack.
Q: Why did subscriber churn spike in March 2026?
Raw churn in March was 9.4 percent against a trailing average of 3.1 percent. The business_events table shows a billing migration on 12 March that wrote cancelled statuses to around 1,900 paused subscriptions (decision record 0007). Excluding flagged artefacts, adjusted churn was 3.3 percent, in line with trend. One caution for year-on-year comparisons: the churn definition changed on 1 May 2026, so pre-May figures use the old definition.
Every clause traces to one home: the number to the semantic layer, the event to rule 3, the reasoning to rule 5, the exclusion logic to rule 2, the definition caution to rule 4. None of it required a smarter model. All of it required someone at Kelder spending roughly an hour, spread across four months, writing things down in places an agent can read.
Where to start
Metrics layer first. Context earns nothing while the data foundations beneath it are shaky. Then the two warehouse tables, backfilled with the last 12 months of events you already know about; that backfill session is the highest-value hour in the whole build. Then the first five verified queries. The PR template and the incident step come last, because forcing functions only stick once feeding them has a visible payoff.
If you run this and agent accuracy still stalls, the bottleneck is almost never the model. It is a why that nobody wrote down yet. The teams that win the next two years of AI analytics will be the ones whose institutional knowledge exists somewhere an agent can read it. Getting there costs a few tables, a directory of markdown, and a habit.