These days, most leaders expect their teams to use data to assess problems and make decisions. But how do you develop this practice?
In this article we look at why simply training everyone on data and analytics basics, so-called data literacy, is not enough. We explain why you need to develop better thinking habits, as much as data skills.
In our follow-up article, 7 Steps for getting your teams to make data-informed decisions, we then describe how to systematically change how your teams work with data.
Why do many data literacy efforts fail?
Most data literacy efforts focus on teaching techniques. Techniques for analyzing a problem, making predictions, or visualizing data and communicating insights. These are important. However, without showing teams how to ask the right questions all the data brought forward may not lead to better decisions.
Data theater
When you use data without understanding how it will inform your decision, it often looks like this:
Your team meets to decide if they should build a new product feature. At the outset, your UX analyst describes meeting a customer who asked for exactly the feature in question. He gives a great example of why this feature really matters.
But there’s more. He also collected data from Customer Service to see if customers actually ask for this feature. While no one mentioned the feature directly, several customers complained about problems the new feature may solve. On the other hand, the financial analyst is concerned that the product will become more costly to support.
More opinions and observations follow. The financial analyst does a rough calculation on support cost right in the meeting.
In the end though they all agree to build the product feature. It feels like there are more reasons for than against it.
Did this team look at data? Yes. Did it affect their decision? Possibly. It is unclear what difference the data made.
Perhaps, just looking at data gave everyone the sense that they were not simply making a gut-level decision. You could call this “data theater.” Everyone has been made to believe that data was considered, even though the logic isn’t clear.
Think back again to how they presented the data:
It’s not clear why the service call analysis was a good data point to confirm the feature would solve the customer problem. And while they looked at support cost, they didn’t put it in useful terms. How many additional units they would need to sell to cover new feature development and ongoing support cost – that could have been useful.
Data-informed decision making
Contrast data theater with data-informed decision making: When you make data-informed decisions you are clear which role data will play in your decision process.
It means first framing the problem or decision. You will use your expertise, intuition, experience and previous analyses to:
- State the specific questions you need to answer.
- Describe what you still need to learn because you don’t know enough today.
- Clarify why and how your answers and learnings will matter for your decision.
- Identify which information you will need to answer your questions sufficiently.
Only after clarifying the key questions and considering what information you need, will you go off to get and analyze data. In short, you gather data for specific purposes, which will then lead to data-informed decisions.
The challenge with structured decision making
As logical as the approach above sounds it doesn’t come easy to us. It doesn’t because our brain doesn’t go there first.
Evolution has shaped how the human brain collects, processes and responds to data. And for most of humans’ time on earth our brain’s primary task was to help us stay alive.
Let’s look at where the brain goes first. You will see how our instincts are helpful in the wild, but less helpful for making thoughtful organizational decisions:
- Humans heavily rely on rules of thumb, “heuristics,” to simplify, and assess and decide quickly. It’s efficient. The brain manages our body’s energy budget and it’s stingy. This works great if the situation we are assessing conforms to our earlier experiences.
- We seek out confirming evidence. This has mostly served us well. Most of the time, if we see something that doesn’t fit our prior experience or knowledge, we would be right to question it.
- People are wired to fill information gaps, which to survival is just as critical as food. However, we prefer information that makes us feel good. Conversely, we try to avoid negative information.
These thinking habits are with us all the time and affect how you frame and solve a problem or decision: They affect what data you ask for. You skip potentially uncomfortable questions. You likely over-index on recent experiences for explaining what the data shows, etc.
In short, our brain’s default operating mode defeats the purpose of why we look at data in the first place – to get a more objective read on a matter.
Develop better decision-making and thinking habits
What to do? Brain scientists, psychologists and behavioral economists appear to agree on one recommendation – develop good decision-making habits. How?
(1) Recognize your mental traps.
(2) Develop practices that help avoid those traps.
(3) Make it easy to apply those practices regularly.
Overcoming confirmation bias
Let’s look at one of the most common traps: confirmation bias. Confirmation bias is our tendency to favor information that confirms our beliefs. Or rather, favor information consistent with our prior experiences, which in turn have shaped our current beliefs. Needless to say, it’s an enemy of objective analysis.
Here’s how you may apply the three steps outlined above to counter confirmation bias:
Recognize confirmation bias
The first line of defense is to recognize this mental trap. Start with examining personal examples. How did you go about collecting information when buying a new car or signing up for an expensive training class?
- What were the key assumptions you made that were important to your decision?
- When you looked for information to back those up, where did you look?
- What pieces of information in support of your assumptions did you find?
- Did you specifically look for anything that would contradict your assumptions?
If you are like most people, you will find that you went to familiar sources for information. Did you spend much less time seeking out critical reviews? Likely. Did you look for data that would prove the opposite of what you were hoping to learn? Likely not.
Now recall a recent decision or discussion within your organization. You’ll likely find situations where you grabbed data to confirm what you already believed to be true. Or where someone gathered data to support something already decided. Or a situation where someone glossed over data that may have pointed in a different direction.
In short, you want to develop your intuition. Your intuition that you or someone else may take a too narrow view, ignoring the broader picture. Not even bothering to seek out data that could prove the theory wrong.
Create practices to avoid bias
Once you know how confirmation bias rears its head, you can establish some simple practices to avoid it:
- Don’t allow anyone to pull data before stating the question to be answered.
- Make people express upfront what results would make them comfortable reaching certain conclusions.
- When you interpret analysis results, try to make a case for an alternative interpretation.
Make it easy to adopt practices
Finally, create incentives or remove barriers to turn practices into habits. For example:
- Make sure people have sufficient time to prepare analyses. Sufficient means as much time as needed given how much is at stake.
- In meetings, preface any review of data with a brief discussion of the analysis approach.
- Reward a sound case made against a decision as much as a case as made in favor.
Raise your team’s data acumen
If you want better decision making, not just data literacy, you have to take a comprehensive approach to upskilling your teams. An approach tackling thought patterns and problem-solving practices along with data skills.
Your goal is to have your teams use data to assess performance, set goals, improve processes, or develop innovative ideas with more rigor and objective understanding.
In 7 Steps for getting your teams to make data-informed decisions, we build on the insights from this article. We describe how you can systematically improve how your business and operations teams work with data.
We can help
See examples of how Data Brave has taught teams to become more data-savvy. if you’d like to brainstorm how to help your teams to embrace data-driven decisions and use data to scale processes.
