On Designing a Decision Intelligence System for Uncertainty
Part 1: High-Level Observations
This is a multi-part post to discuss strategies for enhancing the design of a decision intelligence system to accommodate uncertainty. In this post series, I am thrilled to impart the knowledge and insights we’ve garnered from the design and execution of DAOSoft, our all-encompassing decision intelligence system.
I share several broad observations in this first post of the series to help establish design criteria around uncertainty. In subsequent posts, I will discuss the impact of specific uncertainties in the decision intelligence system design.
Creating a decision intelligence system can be difficult because decision-making in intricate situations involves various factors and facets, even without uncertainty. But uncertainties do exist and cannot be ignored. On the other hand, it’s also true that these uncertainties can make the design process even more challenging.
A potential strategy for steering the design process begins with crafting the system without accounting for uncertainty and establishing a robust design base. After laying down this solid foundation, you can incrementally incorporate elements of uncertainty, adjusting and fine-tuning the design to tackle each emerging obstacle. Following this methodology, you can guarantee the creation of a resilient and equipped system to manage any uncertain situation.
Humans in the Loop
Human engagement in a decision-making process supported by a decision intelligence system is vital not only because, ultimately, it is humans who make decisions but also to handle uncertainties that machines are incapable of. By working collaboratively, humans and machines can make better decisions and achieve superior outcomes.
By incorporating human experts’ intuition and critical thinking, the decision-making process can be adaptive and make judgment calls even under uncertain or unfamiliar circumstances. However, it’s essential to remember that when humans are involved, additional uncertainties may come into play in decision-making.
Grasping the Realm of Uncertainty
“In an increasingly uncertain world, we need to ensure our decisions are resilient to the most plausible future uncertainties.” — Ram Charan
Uncertainty is a pervasive aspect of decision-making processes. Whether buying a house, predicting a weather pattern, designing an engineered system, or deciding to switch jobs, uncertainty often dictates the decision, action, and outcome (DAO) dynamics. The question is: How can we genuinely incorporate this seemingly nebulous realm of uncertainty in our design of the decision intelligence system?
Here are some essential points to consider while formulating requirements for handling uncertainty:
- Uncertainty is not a monolith
- Tolerance of Uncertainty is a viable strategy
- Handling one type of uncertainty may introduce another
- You may not know what you don’t know
Let us elaborate on each of these points further.
Uncertainty is not a monolith.
Uncertainty manifests itself in various forms and at various times during the decision-making and decision-implementation time horizons and beyond. For example, uncertainties in a supply chain decision system could emerge from fluctuating demand, supply interruptions, or volatile pricing.
It’s impractical to account for every possible uncertainty; hence, pinpoint specific uncertainties you plan to integrate into your design. To do so, you will necessitate sketching out various scenarios, pinpointing areas susceptible to uncertainty, then categorizing the identified uncertainties.
Tolerance of Uncertainty is a viable strategy.
Eliminating uncertainty is not always possible. Even if it is possible, it may be unnecessary or unaffordable. Tolerance of uncertainty can be a viable strategy. We can work to reduce uncertainty to a level where its impact on the outcome is manageable. You must identify these acceptable uncertainty thresholds upfront.
Consider launching a new product in a highly competitive and fast-changing market, such as the technology industry. In this situation, the company faces a high level of uncertainty. It’s impossible to know how well the users will receive the product, what the competitive landscape will look like at launch, and how market trends will evolve.
In such a case, one viable strategy for handling this uncertainty is to develop a high tolerance for it. Instead of trying to eliminate or reduce the uncertainty — which might not be possible due to the inherent unpredictability of the market — the company embraces it as a part of the decision-making process.
Handling one type of uncertainty may introduce another.
Fixing or mitigating one type of uncertainty can often lead to the introduction of another. This observation points out the inherent uncertainty trade-off. For instance, a technology firm decides to use a more sophisticated machine learning model to reduce ‘Model Uncertainty’ in their customer prediction system. While the more sophisticated model might improve prediction accuracy, it could simultaneously introduce ‘Operational Uncertainty.’ The new model may require more computational power, specific system configurations, or specialized expertise. These new demands could introduce fluctuations in system performance or create challenges in system maintenance.
Similarly, the model’s complexity might introduce ‘Human Uncertainty’ as stakeholders find it harder to understand and trust the model’s predictions. Hence, while addressing the initial uncertainty around the model’s predictive accuracy, the company introduces new operation and human interaction uncertainties. This example shows that in complex systems, uncertainties often interplay, and managing them can be a delicate balancing act.
You may not know what you don’t know.
The statement “you may not know what you don’t know” pertains to the concept of unknown unknowns in decision-making. This refers to potential outcomes or factors that are not just currently unknown, but we are also unaware that they exist as possibilities. The danger lies in their unpredictability and the potential for surprising, high-impact events.
For example, let’s consider the case of a pharmaceutical company developing a new drug. They’ve conducted extensive research and trials, addressing known variables like efficacy, side effects, production costs (known knowns), and potential issues like regulatory changes or competitor’s actions (known unknowns). However, there may be factors entirely outside their current knowledge and anticipation, such as an unforeseen global event causing supply chain disruption or the emergence of a novel virus that dramatically shifts healthcare priorities and resources. These are the unknown unknowns. They were not only unforeseen but were not even considered potential uncertainties. Awareness of this level of uncertainty underscores the importance of resilience, adaptability, and ongoing learning in decision-making processes.
What is next?
In part 2 of this post series, I will discuss the uncertainty that arises from a lack of information. Take, for instance, the decision to purchase a house. It is essential first to understand the size of the mortgage for which you’re eligible. This type of uncertainty is called Epistemic Uncertainty.