'Nobody Trusts AI Analytics Yet,' Say London Data Leaders

Last week we hosted a dinner for 15 data leaders at Galvin La Chapelle in London, co-hosted with our partners at Omni. The guest list was a mix of Heads of Data, CTOs, and analytics leads from companies at different stages. Some with mature data stacks, and also some still wrestling with their first platforms.

No slides, no pitches. Just great food and an honest conversation about AI analytics, and why nobody trusts it yet.

The theme was “After the Dashboard” and our thesis was that while most teams have shipped an AI analytics pilot by now, the interesting question isn’t whether it works. It’s why nobody trusts it.

Here’s what we heard.

After the Dashboard: What We Learned Hosting a Room Full of Data Leaders Event Summary

Why People Ask AI Instead of Their Data Team

This one surprised us quite a bit. End users are now more comfortable asking an AI chatbot for data than asking their own analysts.

Not because the AI is better, but because they don’t want to look foolish. They don’t want to reveal they have forgotten how to use the dashboards. They don’t want to ask for an insight they feel they should already know. And they’re in Claude all the time anyway.

This reframes the entire “self-serve analytics” story. For a decade, data teams assumed the bottleneck was access: give people dashboards, teach them SQL, build self-serve tools. But the bottleneck was never access. It was social friction: people were embarrassed to ask questions.

AI removes that embarrassment. You can ask a chatbot the same question 5 times without anyone judging you.

The opportunity is real. If you track what questions people ask through AI chat, you get an unfiltered view of what insights the business actually needs, not what stakeholders tell you they need in a quarterly planning meeting. That’s a goldmine for any data team willing to listen to it.

The risk is just as real. Those questions are being answered by an AI that doesn’t understand your business. It doesn’t know that “revenue” changed definition last quarter. It doesn’t know your attribution model broke after the iOS update. It sounds confident. It’s almost certainly wrong on the things that matter most.

Zendesk’s 2026 CX Trends Report found that this “shadow AI” usage has increased 250% year-over-year in some industries. The same dynamic is playing out in analytics. People are already asking AI their data questions. The question is whether they’re getting answers from a governed system or from ChatGPT with a CSV pasted in.

The Demo-to-Production Gap Is a Context Problem

We know, every AI analytics demo looks incredible: you type a question in natural language, a chart appears, that decision seems so easy now.

Then you try to put it in production and it falls apart within weeks. Andreessen Horowitz published a piece last month that walks through why that is (and why it’s a pretty old problem in the first place).

Consider deceptively simple question: “What was revenue growth last quarter?”

First problem: how does the agent know what “revenue” means? It’s a business definition, not a database column. The semantic layer was last updated by someone who left six months ago. It doesn’t include 2 new product lines.

Second: where’s the right data? Raw data is split across multiple tables and warehouses. The agent doesn’t know which source is authoritative. Most orgs haven’t really centralised their data properly yet (loading it into Snowflake is alas not enough), let alone documented a single source of truth.

Third: what context is missing? A one-off refund, a seasonal pattern, a set of tests. Or a strategic decision to recognise revenue differently for one product line. None of this is encoded anywhere machine-readable. Maybe deep in a Notion page somewhere, but that’s if you’re lucky.

The agent writes technically perfect SQL against bad data or deprecated metric definitions. And tells you with high confidence the exact wrong conclusion. The board sees a number that doesn’t match anything else. Trust drops another notch.

We’ve seen this pattern across 60+ client engagements. The data is usually fine, the models are fine, the strategy is fine. The problem is that nobody wrote down why the metrics are defined the way they are.

Benn Stancil put it well: you can’t onboard an analyst by giving them logins to a bunch of tools, and you can’t make a good AI bot by granting it access to the same sources. You have to teach both how the business actually works.

The External Context Problem in AI Analytics

Most of the conversation about AI analytics context focuses on internal definitions: what does “active user” mean, which date field to use, how churn is calculated. But one of the most interesting threads at dinner came from a completely different angle.

A travel company can see booking trends. It can apply internal factors: seasonality, promotional calendar, channel mix. But what about cost of living changes? Currency movements? A conflict that makes a destination unappealing overnight?

These factors are critical to interpreting what’s actually happening. But they don’t live in your data warehouse. They don’t live in your semantic layer. They exist in the outside world.

The architectural question this raises is hard. Do you need separate agents? E.g. one for BI, operating inside your governed semantic layer, and one for external context, pulling from news feeds, economic data, market signals, that you merge when needed? How do you maintain the accuracy and governance of the BI agent while enriching it with messy, unstructured external data?

Nobody at the dinner had cracked this. We haven’t either. But it’s the next frontier after the internal context debate settles.

Walled-garden tools like Amplitude or GA4 don’t even attempt it. They’re closed ecosystems optimised for their own data. If you want to combine internal analytics with external intelligence, you need a modular stack that lets you plug components in and swap them out.

Three Layers of Trustworthy AI Analytics

Over the course of the evening, a simple framework kept emerging. There are 3 layers you need before AI analytics becomes trustworthy. Most organisations are still stuck on the first one.

Clean, centralised data — a single source of truth
Clear definitions — a semantic layer governing what metrics mean
Clear context — the why behind the numbers (the layer nobody’s cracked)

Layer 1: Clean, centralised data

A single source of truth. Data flowing from your systems into a warehouse where it can be queried consistently. This is table stakes and most teams haven’t finished it. Data is still scattered across Salesforce, HubSpot, Stripe, Shopify, GA4, and a dozen internal databases. Reports still contradict each other. Without this, nothing else works.

Layer 2: Clear definitions

A semantic layer that governs what metrics mean, consistently, across every tool in your stack. This is what Omni, dbt’s metrics layer, and Cube are solving. The semantic layer captures the WHAT: how revenue is calculated, what “active user” means, which joins are valid. It’s necessary. It’s not sufficient.

Omni‘s approach is interesting here because it embeds the semantic layer inside the BI tool itself. For scale-ups trying to avoid adding yet another component to the stack, that matters. You get governed definitions as a by-product of doing BI properly, not as a separate implementation project. Their ai_context metadata lets anyone add instructions like “when someone asks about revenue, they mean net revenue excluding refunds”, directly inside the semantic model, no code required.

Layer 3: Clear context

This is the one nobody’s cracked. Not just what a metric calculates, but why it exists. What decision it was built to support. What changed last quarter that makes the historical comparison misleading. Who defined it and why they made the choices they made.

Benn Stancil called this the context layer. Hex’s State of Data Teams 2026 report confirms the pattern: data trust is the number one concern around AI adoption, and teams are investing in a range of techniques beyond semantic models alone. Atlan is building “context graphs” that try to capture this. Margaret-Anne Storey’s work on cognitive debt frames it from the academic side.

The problem is incentives. Nobody’s motivated to write this down. It’s like documentation: everyone agrees it matters, nobody does it.

But AI changes the economics. For the first time, there’s a real business cost to not capturing context. Your agent will make expensive mistakes. That might be the forcing function that finally makes organisations take institutional knowledge seriously.

What Data Leaders Should Do About AI Analytics Trust

If you’re a Head of Data at a scaling company, here’s the practical takeaway.

Look at what people are asking the AI. If you’ve deployed any kind of AI analytics, even just Claude or ChatGPT with access to your data, the questions people ask are the clearest signal you’ll ever get about what insights actually matter. Most data teams build dashboards based on stakeholder requests. The AI chat log tells you what people actually want to know when nobody’s watching.

Be honest about which layer you’re actually on. Most organisations are still on layer 1: centralising data, getting a warehouse running properly, making reports consistent. That’s fine. But don’t bolt an AI chatbot onto a foundation that isn’t solid. The chatbot will expose every inconsistency you’ve been quietly living with.

Start capturing context now, even if it’s just a wiki. Write down why metrics are defined the way they are. Document what changed and when. Record the decisions behind the data model. It doesn’t need to be a sophisticated tool. It needs to exist. Your next hire, human or AI, will need it.

The Closing Thought

We spent a decade building the modern data stack. Then we plugged an LLM into it and discovered nobody had written down why any of the metrics exist.

Without all 3 layers (clean data, clear definitions, clear context) letting AI loose on your analytics is like handing a brilliant but brand-new hire the keys to your board deck.

They know all the tools. They’ve read every textbook. They’ll sound completely convincing.

They don’t know that your biggest client changed payment terms last month. They don’t know the marketing team redefined “qualified lead” 3 times this year. They don’t know the CFO doesn’t trust any number that comes from the CRM as he’s been seriously bitten by bad data entry in the past.

They’ll almost certainly get something disastrously wrong. And you won’t know until the board meeting.

We’re hosting more of these dinners later this year. If you’re a data leader at a European scale-up and this resonates, get in touch.

This post is based on a dinner we hosted with Omni at Galvin La Chapelle, London, March 2026. Thanks to everyone who joined, and to Cillian, Rob, and Jordan for co-organising.

News & Views

‘Nobody Trusts AI Analytics Yet,’ Say London Data Leaders