Executive pre-work can manufacture consensus before the workshop begins. Five leaders may submit different-looking summaries built on the same customer anecdote, founder forecast or AI-generated interpretation. Counting those submissions as independent evidence exaggerates what the team knows.
Design the assignment around three separations: observation from inference, independent sources from repeated sources, and disagreement about the world from disagreement about objectives. The output is a map of the assumptions capable of changing the decision.
Define the uncertain quantity before requesting confidence
“Can we serve enterprise customers?” combines purchasing demand, implementation effort and delivery capacity. Split it into claims that could resolve differently: who will purchase, how much work delivery requires, and whether the resulting contribution is acceptable.
For each consequential forecast, specify the population, event, horizon and conditions. For example: “Of the ten named eligible prospects receiving the standard offer this month, how many will sign within 45 days, assuming no custom development?” A prediction based on bespoke features answers another question.
Give every executive the same factual brief and condition set. Ask them to identify which condition is unrealistic before estimating. Apparent forecast disagreement often disappears once people discover that they assumed different staffing or payment terms.
Collect judgments before circulating the sponsor's position
Lorenz and colleagues' estimation experiment found that social influence could reduce diversity and increase confidence without improving collective accuracy. The tasks were numerical estimates; the relevant workshop adaptation is preserving an independent starting judgment.
Use separate Teams Facilitator customer selection, SWOT or Business Model Canvas sessions around the common question. Have participants return their reviewed conclusions with an accompanying evidence sheet. You compare the sessions manually.
Request the original observation, its date and source, the inference drawn from it, and what would change the inference. If someone uses an AI to summarize interview notes, retain the underlying notes. Generated wording does not create an additional observation.
Specify the sharing audience before submission. Separate sessions provide independent working space; they do not establish anonymity or a confidential assessment process.
Audit dependence between sources
Ask each executive to label where the evidence originated. Sales, finance and the founder may all cite “three interested hospitals,” while all three opportunities came through one reseller using the same introductory pitch.
That creates two dependencies: the leaders share a source, and the prospects share a recruitment channel. The team has evidence about one route into a market, with correlated opportunities, rather than three independent demonstrations of general demand.
Use this comparison sheet outside the product:
| Claim | Original evidence | Shared dependency | Decision affected |
|---|---|---|---|
| Hospitals can buy quickly | Reseller's three introductions | Same intermediary; approval unverified | Whether a short pilot is feasible |
| Delivery needs two weeks | Estimate for one existing integration | Assumes identical customer systems | Capacity commitment |
| Cash arrives before setup | Draft commercial model | Contract actually says acceptance first | Financing the pilot |
Preserve both corroboration and dependence. Two functions independently observing the same failure can strengthen a diagnosis; two functions quoting one email cannot increase the sample size.
Elicit ranges without averaging away scenarios
O'Hagan's expert-elicitation guidance begins with private plausible bounds, then develops more detailed judgments and facilitated discussion. Borrow the ordering when preparing numerical forecasts: explore what could make the outcome unusually low or high before settling on a central estimate.
For implementation effort, ask what is included: engineering, customer configuration, waiting for credentials and rework. Distinguish elapsed days from person-days. An estimate of five engineer-days and a forecast of 20 elapsed days may both be correct.
If sales estimates one week and implementation estimates six, ask for the assumptions behind each. Perhaps they describe standard deployment and a migration project. Their midpoint describes neither scenario. Retain conditional estimates and determine which scenario the proposed contract actually permits.
This is preparation for judgment, rather than a full SHELF probability elicitation. Formal distribution fitting requires the corresponding definitions, training and checks.
Find the assumption at which the recommendation switches
Move from “what do you believe?” to “which belief changes the choice?” Suppose the team prefers enterprise if implementation stays below 15 engineer-days and prefers smaller accounts otherwise. Compare each leader's estimate with that switching point.
If everyone's plausible range lies below 15, refining effort estimates may have little immediate decision value. If the ranges straddle 15, scoping a representative account could change the recommendation. Direct pre-work toward that uncertainty.
Repeat this for price, buying cycle and capacity. Check combinations: an attractive price may coincide with expensive customization, so combining the highest price estimate with the lowest effort estimate could create an implausibly favorable case.
Return the comparison to participants for factual corrections before the workshop. Preserve initial and revised judgments with the reason for revision. The live agenda should identify the remaining switching assumptions and the person authorized to resolve value tradeoffs.
Retain forecasts so confidence can become accountable
For recurring binary predictions, record a probability and an unambiguous resolution rule. “At least three standard pilots signed by 30 November” can be scored; “strong customer interest” cannot.
Gneiting and Raftery's scoring-rule paper explains why proper scoring rules reward honest probability assessments in expectation. A simple binary Brier loss is the squared difference between the forecast probability and the outcome, coded one or zero. A 70% prediction that occurs scores 0.09; if it fails, 0.49. Lower loss is better under this convention.
Review many comparable resolved forecasts, including uncomfortable misses. One accurate call establishes little about calibration. Avoid comparing leaders' raw averages when they predict events of different difficulty, or turning scores into a compensation contest that encourages selective forecasting.
Preserve the conditions attached to the forecast. If the team adds two engineers after predicting delivery under existing staffing, the outcome no longer tests exactly the original conditional claim. Record both the forecast and the intervention rather than rewriting the prediction. Otherwise a leader who correctly identifies a risk and helps prevent it can appear less accurate than one who never exposed the risk. That would reward the wrong contribution to the workshop.
Bring the unresolved claims into the fractional CXO workshop. If the team shares the same forecasts but still prefers different outcomes, the remaining work is a strategic tradeoff. A guided session can structure that conversation; collecting more confident prose will not settle it.