How data teams earn a seat at the leadership table

From the founder: This article is part of a LinkedIn series by Nabil Jallouli, Rollstack’s co-founder and CEO, on the future of BI and data teams. Read the original article on LinkedIn.

In the previous article, we got to dive into why having a strong centralized semantic layer with good documentation helps create a trusted layer for the whole organization, whether for its own workflows or for the vendors it uses. In the next 2 articles, we'll get to dive into 2 important areas for any data team:

1. Being better trusted advisors to leadership
2. Informing leadership about the health of the business

In this article, I'll dive into a topic that is dear to my heart: how can a data team be a close partner to leadership, i.e., someone leadership involves early in a decision, trusts to make or challenge an assumption, and expects to bring a recommendation?

Across all the organizations I worked at (Groupon, Pinterest, Deel), the involvement of data teams in decision-making varied a lot, and the role of people involved with data varied significantly. But a straightforward split was:

1. People working on data infrastructure
2. People working on leveraging data for business problems

The first group is in charge of architecting the systems and data flows, and building the pipelines to ingest data and make it easy for business functions to analyze.

The second is in charge of understanding a department's business problems, framing hypotheses, and validating or refuting them by building analyses and models, whether backward-looking or forward-looking. These are business analysts working within each of the business functions. The second group extends well beyond people with "data," "analyst," or "scientist" in their title, but the techniques they use in their day-to-day work tend to be similar:

• Financial analytics teams
• Marketing analytics teams
• Strategy & Operations teams
• Customer Success Operations teams
• etc.

This article focuses on the 2nd group. The dynamics with the first group, and how these might evolve, will be further explored in the 3rd article.

What ultimately determines how good a partner a business analyst is (I'm using "business analyst" as a generic term here for all business-function analysts: strategy & operations managers, finance analysts, marketing analysts, etc.) is their ability to:

1. Help business leadership understand performance. What happened last week? How does it compare with goals and expectations? What needs attention?
2. Investigate a business question. Why did conversion fall? What explains the difference between 2 customer segments?
3. Evaluate a decision. What might happen if we change pricing, increase marketing spend, or enter a new market?

This means that the nature of the work revolves around:

1. Standardizing the business metrics and views for the recurring reporting on business performance across departments: WBR, MBR, QBR, board deck, etc.
2. Building ad hoc analyses to answer specific business questions or test hypotheses
3. Modeling to understand and forecast the potential impact of business initiatives

For all streams, good business analysts need to be able to run the data work end to end properly, including asking the right questions, scoping, converting the question into a set of technical tasks, and finally presenting it. Because historically it was very rare to find people able to run things end to end, from the technical analysis to presenting the recommendation and communicating it to leadership, teams would segment this work, leading to things being lost in translation as well as slow iteration cycles.

With the technical data work becoming easier, the scoping part is the key here, meaning that business experts and domain knowledge experts are increasingly important. But technical work becoming easier doesn't necessarily mean you don't need technical skills. You can't manage or check work performed by AI if you don't have a good technical understanding as well. I believe the winning profiles will be people in technical roles venturing to understand the business problems more deeply, and the business folks venturing to understand the technical stack more deeply.

Let's take a concrete example: marketing leadership asks the marketing team: Can we invest more in Meta Ads without hurting ROI? It sounds like a straightforward question, but it isn't until we agree on what ROI means. First, are we looking at revenue or profit? The first purchase or the CLTV? Which customers and time periods should be included? Are we comparing Meta with another channel, or just with its own past performance? And what investment increase are we considering?

The context and reasons why the question is coming up matter too. Maybe another channel is becoming too expensive. Maybe we have budget left at the end of the quarter. That meaningfully changes what we are trying to learn. Only after clarifying these questions can we start the analysis. We can look at what happened as the budget went up, but we also need to think about the next dollar. The customers we reach next may be less profitable. Some of the purchases may have happened anyway. We need to model the diminishing returns, etc.

What leadership is looking for ultimately is also a plan to test this. Maybe we should increase the budget by a limited amount, agree on what success looks like, and check the result before committing to more.

This is also where handoffs become expensive. If one person defines the question, another one runs the analysis, and a third presents it, the person in the room likely doesn’t know which assumptions were made or have all the context of the analysis. Any follow-up question from leadership sends the work back to whoever did the analysis. Running the whole thing from analysis to recommendation is the job.

A tempting way to bridge this gap between technical expertise and business acumen is to put AI on top of the data warehouse and let everyone ask their own questions. Not a single organization is ready for that. It requires first educating teams on what doing proper data analysis means. If someone doesn’t know the difference between correlation and causation, how are they supposed to review work coming out of AI? It also requires a major organizational transformation to review the teams’ roles and scopes. Finally, it requires evolving the tooling. The models might be ready, but all the inputs needed to make them work and, especially, to review their work are not.

On public benchmarks, the numbers look great, but on real company data it's a different story. There’s this study done by researchers at MIT and Harvard who built a benchmark called BEAVER from private enterprise data warehouses, the type of environments that no model has seen in training. The best agentic text-to-SQL system, ReFoRCE, gets 11.4% of the queries right on BEAVER. The same system gets 62.9% on the public enterprise benchmark Spider 2.0. Even when the model is given assistance for every step, accuracy only climbs to 30.1%. The gap right there is not data quality. It's quirky column names, joins nobody documented, and business definitions that live in people's heads. This is what we covered in my previous article and what the semantic layer is supposed to partially fix.

At the same time, data teams are drawn to this vision because they're drowning in requests. The question then becomes: who's accountable in case of errors? Is it the data team? Is it the business analyst? Is it the model? When presenting a deck to the board or an analysis to your company's CFO, even 95% accuracy is not an option. So how can you reconcile a self-serve culture with a culture of data accuracy? Not by opening the raw tables to everyone. Self-serve should happen on top of vetted datasets and approved analyses, and the data team's job becomes building and maintaining that layer:

1. First, it starts with the basics: getting your data right. You've heard it multiple times: garbage in, garbage out.
2. Build a library of vetted datasets organized into team data marts: this is old-school BI best practice that is even more important now.
3. Invest in documentation: documentation is context for the agents.
4. Appoint domain experts as an analysis review board.
5. Make vetted and approved analyses an official artifact the whole organization can trust, reuse, and build on for its own questions.
6. Make analyses recurring and deterministic by default: once an analysis is vetted, it reruns on new data with the same logic instead of being regenerated each time.

Now, technically, how can you leverage LLMs while achieving 100% accuracy in your data reporting, including high-stakes materials such as board decks or client reporting?

It's mainly about balancing deterministic and probabilistic approaches.

At Rollstack, we believe LLMs are great for the cold start, but deterministic approaches still win at getting the right data into the hands of leadership and teams. Being a trusted partner to leadership starts with providing the right data and owning it. When the CRO interjects, "This pipeline number is wrong," you can't say, "Let me check with ChatGPT how it came up with it." Being a trusted partner means owning the methodology, the execution, and the number. I believe the winning strategies will combine humans in the loop with breaking problems down into small pieces that can be verified, while adding checks and balances throughout the process. If we take the factory example and how it gets automated, nobody automates whole lines from production to delivery to the customer on day one. You automate one step at a time once the machine does it reliably, and the person who used to do that moves to checking the output, then to handling the exceptions, then to the next station. Humans never leave the loop. Their tasks change as automation and accuracy improve. I strongly believe data work is going in a similar direction: teams leverage LLMs for data transformation but ultimately check the code, introduce tests, and then lock in the pieces that are proven so they run deterministically from then on, allowing teams to spend their time on the judgment calls and on whatever hasn't been automated yet. Attempts to automate the whole data value chain in one shot, from the data warehouse straight to the board deck, are bound to fail due to a lack of auditability, explainability, trust, governance, and repeatable determinism.