0:00
And I also don't like consensus decisions because those tend to be um in other situations, the consensus is fairly risk-averse, unlike with The Martian.
Search this video
0:18
And people want to have something that everybody can agree with.
0:21
And you lose lost your opportunity for somebody who's had an insight, somebody has a new idea uh of of trying to put that in practice.
0:33
With the consensus is to try to do um the safe thing that's comfortable that everybody can can live with rather than to do something that's bold and innovative.
0:46
So, I have I have problems with consensus decisions.
0:57
Gary, so glad to talk to you today.
0:59
Thank you very much for inviting me, Shane.
1:01
So, you're a cognitive psychologist who has spent your entire career observing how other people make decisions.
1:06
But you took an interesting angle to this, rather than focus on how to make better decisions directly, you approached it from you how we can develop expertise to make better decisions, we can sort of reduce errors or have better insight or preferably both.
1:23
And yet these often seem in conflict with one another.
1:25
I thought a good place to start this would be what sparks insight and what prevents us from putting our insights into use.
1:34
Is it because they're often contradicted uh they often contradict the beliefs we hold?
1:39
Uh an insight definitely is going to be incompatible with beliefs we hold.
1:44
That's why we're surprised.
1:46
Uh I did a study of uh 120 examples of insights.
1:50
I wonder I wondered where insights came from.
1:54
And I did the study out of frustration.
1:58
Because I would give talks and I would say if you want to improve performance, there's two things you can do.
2:03
And there would be like a down arrow, what you want to reduce, and an up arrow, what you want to increase.
2:09
And to improve performance, what you want to reduce is errors.
2:13
And that's what most organizations do.
2:14
They try to cut down on errors.
2:19
But they're missing the up arrow of improving increasing insights.
2:21
And so most organizations only focus on reducing your errors.
2:27
So, I would present this slide and people in the audience would say, "Yes, that makes sense. That's my organization.
2:32
All they care about is reducing errors."
2:34
But then they would ask me the question, "So, what do you know about insights?"
2:41
And I would answer, "I don't know anything about insights.
2:43
I just have this one slide. I I haven't studied it."
2:46
And once I I gave a presentation I used the slide at a at a talk I gave in Singapore and I got asked that question.
2:54
And and then I flew home afterwards and it's a 17-hour flight.
3:00
I had a direct flight to the East Coast.
3:02
That's a long time to be stewing over the fact that I felt like an idiot not being able to tell them what insights are about.
3:08
So, I decided to to do a study.
3:10
So, I collected 120 examples of insights to see what's going on and and uh hopefully find out how to improve them. Increase them.
3:20
And I found I and I tried to find a common theme and I couldn't.
3:26
Because it turned out that there were three different pathways.
3:27
One pathway is a connection pathway where we put things together.
3:33
So, there we're not contradicting anything.
3:34
I'm going back to your original question.
3:36
We're not we're not setting our our mental model.
3:41
We're just putting different ideas together.
3:43
Um like uh the example I use is Darwin, you know, knowing that there was uh change in species, but what's driving the change?
3:53
And then he reads Malthus's book about um uh population growth and competition for scarce resources.
4:03
He read the sixth edition.
4:06
And he puts that together and he said, "That's what's driving evolution is a competition for scarce resources."
4:13
So, that that's a a connection type of insight.
4:15
A second type of insight is a contradiction insight where something happens that doesn't make sense.
4:25
And there you do have to change what you believe or wonder what's going on.
4:28
And the example I use there is a story I heard from a police officer who was driving around with a partner who is uh uh in his first year.
4:42
And they're stuck in traffic.
4:42
They're the there's a red light.
4:44
And the partner, this young guy, looks at the car ahead, which is a new BMW.
4:52
And he sees the driver take a deep drag on his cigarette and then flick the ashes.
5:00
And he says, "Who flicks the ashes in a brand new BMW? That doesn't make sense. Something is off here."
5:07
So, they light them up, pull the car over.
5:10
Sure enough, it was a stolen car.
5:12
So, that's uh a second type of of insight which is a contradiction insight where something happens that you didn't expect.
5:22
Now, this didn't force anybody to revise their mental model or their thinking.
5:26
It just uh allowed them to to investigate further.
5:33
Now, your question is where's the resistance coming from?
5:35
And that and that's the third type of pathway that we identified, the correction pathway, where you're stuck because you have a flawed belief.
5:47
And that's the problem and that's what's hanging you up.
5:51
And that that's the kind of insight most researchers study because it's easy to study in a laboratory.
5:56
And the way we get we get past that flawed belief, uh we found 20 I found about 27 cases of of that kind of uh pathway.
6:08
And usually what happened was there was a hint, something happened that made them investigate further.
6:14
And so, instead of dismissing the anomaly, which is what we usually do, they became curious about it.
6:22
So, in terms of insight, in terms of changing our beliefs, that's the secret sauce for this kind of pathway, or even practically for any of the pathways, is to become curious about things that don't make sense.
6:38
Now, what blocks insight in organizations?
6:43
And this was the really discouraging part of my research on insight.
6:46
What blocks insight is that organizations say they want insights and they want innovation and they really don't.
6:57
But they're they're being serious.
6:57
They think they want insights.
7:02
But insights are disorganizing.
7:07
As you point out, insights make you change the way you think, make you change all kinds of things, and you know, they may not be right.
7:14
And so, most organizations actually inhibit insights.
7:20
When when when people bring up new ideas, they they tend to stifle them.
7:26
And so, that's that's a a big problem.
7:28
Is that because organizations are mostly focused on the error-reducing side versus the gaining insight side?
7:36
And that that that's the tension between these things.
7:38
They don't like variance.
7:40
They don't like things that are outside of the norm.
7:41
Yes, they like predictability.
7:44
To be a good manager, you want things to run smoothly.
7:49
And insights are not ways of running smoothly.
7:51
Insights are disorganizing and disruptive.
7:56
And so, that's that's a a major reason that that organizations um without even intending to block the insights that come that come their way.
8:08
So, so with that said, on an individual level, if I'm working in an organization, what can I do within the the confines of only what I control, so myself, to maximize um the insights that I observe and that I can put them into use within that environment?
8:25
And then I guess the second question would sort of be what can an organization do to facilitate uh people gaining more insight and putting those insights into use?
8:37
Okay, so what can you do?
8:37
What you can do is to um you can be deliberate.
8:42
Like you're you're describing now, you can say, "I want to have more insights."
8:46
Which means taking an insight stance rather than a a stance that recoils at something new and unexpected and jarring. You can become curious.
8:57
You can start to celebrate your own insights.
9:00
Because if you go through the day, we often have very small insights, things we didn't know that and we make these discoveries.
9:09
And we just sort of, you know, cruise right past them and use them.
9:11
But if we make a mistake, we beat ourselves up. Right?
9:15
At least I know I'd beat myself up.
9:18
Like, how could I have been so stupid?
9:20
But I'm less likely to say, "Gee, that was that was really clever of me or of people that I'm working with."
9:26
So, you can you can start attuning yourself to the insights you have and the insights people around you have, so you're more sensitive to that.
9:35
You can you can't be curious about everything because then you you just waste tons of time.
9:42
But you can be a little curious for a few seconds and say, "I wonder what that means."
9:47
And think about it and imagine it.
9:49
And just get yourself into that kind of a mindset of wondering and speculating rather than recoiling at something that's unfamiliar.
10:01
So, that's what you could do as an individual.
10:04
What can you do as an organization?
10:04
They are unless enthusiastic.
10:09
Um organizations really If you have an idea that's a bad idea and you make a mistake, that's the down arrow.
10:20
Everybody can see it, right? It's public.
10:24
So, there's a lot of cost for making a mistake.
10:28
If you fail to make an insight, nobody will know.
10:33
So, the reward the reward structure is always favors the down arrow.
10:40
Organizations, if they're really serious about this, can try to create a mechanism so that right now, if I have an idea, I bring it to my boss. I bring it to you.
10:55
And you bring it to your boss and it goes up the chain.
10:56
It only takes one person in that chain to say, "No, we better not do it."
11:01
And then the idea is rejected.
11:05
And good ideas are fragile and precious and and and easily uh discarded.
11:12
So, they they So, they could give you or they could give me an option for a review if I think the idea has been rejected prematurely.
11:22
I can I really appeal I don't want to make trouble, but there there could be somebody there could be uh uh some organ- some part of the organization that could re-examine re-examine the idea.
11:37
Because let's take an organization that wants to um come up with new ideas for whatever it is that they're doing.
11:50
Who's running the organization?
11:50
People with lots of experience.
11:51
And we I found in my research that experience is essential for for coming up with insight.
11:58
However, people with with experience have lots of scar tissue of things that that got tried before and failed.
12:08
And if you look at innovations, usually there's a bad track record.
12:10
People trying it and failing and then somebody else trying it and failing until eventually it succeeds.
12:19
But until you get to the point where it succeed, you're going to have people in the experienced people, the senior people in the organization, what are they going to say?
12:27
They're going to say, "You know, we tried that and it didn't work."
12:34
Because in the past it didn't work.
12:34
This may be the time for it to work.
12:36
Maybe the technology is mature.
12:39
Maybe, you know, the world situation has changed. Who knows?
12:46
But the people with who are calling the shots can remember times when this was tried in the past and it failed and often that's a good enough reason to squelch the idea.
12:57
I think that's a really interesting point, especially when it comes to experience.
13:00
Because if the experience is is typically the people at the the um top of the organization have the most experience and they probably worked in a job very similar to the one that you're working in now, the problem is their experience in that role is 20 years ago or 15 years ago.
13:18
And I would imagine that the environment has changed a lot.
13:23
Whereas the experience of your manager or your team leader or whatever might be more recent and they would have a more accurate view uh of to what the the lay of the land is right now. Right. Right.
13:34
So, so experience I I I still found that most of the insights in in the sample that I studied would not have arisen unless people had experience.
13:43
I mean, Darwin didn't didn't just sit in the in the study and imagine. He he was on the Beagle.
13:51
He was he was watching these various species.
13:53
He was looking at the variation.
13:55
He was building his experience base.
13:57
So, experience is important.
14:02
Um but there are other things there are other factors that that are operating and and experience is not infallible here.
14:11
and and and and the the people who are calling the shots are risk averse and that's another issue in in organizations.
14:20
The people at at the top who are calling the shots don't want to rock the boat because they're getting ready to retire.
14:27
They're getting ready to to draw a pension.
14:33
Uh my sense I I don't have data on this is they tend to be risk averse.
14:36
They want to keep things going smoothly until until they reach the end of their career at the organization.
14:44
I want to come back to something you said about experience.
14:48
Is there a difference between experience and expertise?
14:52
We all know somebody who's got 20 years of experience, but it seems to be the same year repeated 20 times. Right.
14:59
There there is a difference.
14:59
And so, one of the things I've examined is how do you know who's an expert?
15:06
And I identified about seven or eight criteria.
15:10
And the discouraging part is none of them are foolproof.
15:13
There is no gold gold standard for who's an expert.
15:15
But uh years of experience certainly contributes to to expertise, but it isn't the same thing because there are people who just don't reflect on what happened.
15:29
They don't learn um about what happened.
15:33
Uh one of the other criteria I use if I'm going to find uh gauge somebody's expertise is I'll ask them, um "Tell me the last mistake you made. Let's talk about that."
15:46
And if the person says, "I can't think of any mistakes."
15:51
To me, that means this person may be competent, but he certainly is not an expert because my uh you know, what what I've observed is that experts are well aware of their mistakes and their mistakes eat at them until they can they can sort of imagine, "What I should have done is this."
16:10
And and figure out a way around it.
16:12
So, experts are highly aware of mistakes.
16:17
But people who are journeymen, many of them stay as journeymen because they they want to move on and forget about their mistakes.
16:26
I want to come back to what you said about sort of how do we identify the difference between people who know what they're talking about and not.
16:35
And And to this point, one of the most important decisions I think that we make um and we don't even recognize that we're making a decision is who we listen to and who we decide is credible and who we decide is not credible.
16:48
And we we don't recognize that we're making a decision in that moment, but how do we sort out who is credible and who's not or who's who knows what they're talking about or who's faking it. Right.
16:58
That's that's a really important question.
17:01
And I think most people do a mediocre job of determining who they who could who they can rely on.
17:11
And um a lot of people go on surface characteristics.
17:16
So, if I carry myself, I stand up straight and usual, and I carry myself with confidence, uh people are going to listen to me.
17:27
Um and if if I express reservations and I use qualifiers like, "I think this is the case, but I'm not sure."
17:34
I'm essentially telling people I'm not entirely an expert here and so I'm uh disqualifying myself.
17:42
I'll give you an example.
17:45
Um Uh a friend of mine was excited about uh a scenario-based technique for for training.
17:56
Uh using uh uh decision-making exercises.
17:59
And that's a precursor for our the shadow box method that we're using now.
18:04
And so, he had uh some uh exercises that that that he had developed for him.
18:12
He worked in a petrochemical plant.
18:16
And he had about, oh, 30 years of experience there.
18:19
And he said, "Let you know, let this would be great.
18:21
Let's use this for uh for people who are just have been working outside checking valves and climbing ladders and things like that.
18:32
Now they're inside working the panels, so they they've been promoted.
18:35
Uh and there were two people who had just been promoted.
18:38
He said, "Let's get these people up to speed more quickly and let's have them do these these decision-making exercises."
18:44
And one of the people was somebody everybody felt good about.
18:49
He just carried himself with great confidence.
18:52
And and and people were were were excited about about working with him.
18:59
And the other one was was sort of uh tentative about things.
19:03
And they they worried about him.
19:05
And they ran them through the decision-making exercises. What did they find?
19:11
The first person didn't really get it.
19:14
He didn't really understand critical dynamics of the plant.
19:19
And the second person, for all of his uh caveats, really did get it.
19:24
He understood what the relationships were and the different cause causal factors and how they work together.
19:32
Now, the story has a bad ending because they decided the first person, despite all their their their enthusiasm for him, he wasn't ready.
19:41
He wasn't going to be safe operating a panel of a petrochemical plant that could blow up.
19:48
I mean, there there is a volatile compounds and lots of heat and that's a dangerous environment.
19:53
They said, "He's not ready.
19:55
He's going to have to continue outside for for another year before he we can use him on a panel."
20:01
And the second person was somebody that they were ready to use.
20:07
And after they did that, nobody wanted to engage with the decision-making exercise.
20:14
They realized if you don't do well, you could have consequences.
20:17
And And they weren't intending the consequences to be to be used to to to evaluate people.
20:27
But because they used it in that way, nobody wanted to do any more of that training.
20:31
That's the sort of like begs the question, I guess.
20:33
How do we evaluate people's decision-making? Yeah.
20:39
How do you evaluate their decision-making?
20:40
You run them through a scenario.
20:44
And um see what choices they make, find out what their rationale is, find out the reasons that they're picking one thing over another, and use that to determine if if if you think that their mental model is rich enough.
21:01
So, that's one thing that you can do uh to evaluate whether somebody has expertise.
21:08
Uh another I I I told you is ask people tell me tell me about the last mistake you made.
21:14
Or or you can you can should use years of experience just don't take your old that seriously.
21:20
So, there's a variety of criteria that you can use.
21:22
You can use their track record.
21:25
Have they had a Have they had a a track record of successes?
21:30
But none of these are foolproof.
21:32
I may have I may be an investor.
21:34
I have a track record of calling the market right in the last five times.
21:39
People say, "Wow, he really knows what he's talking about."
21:43
But there are lots of people who are trying to predict which way the market is going to go.
21:47
And some of them are going to get it right by chance.
21:52
And people assume it's because they know what they're what they're doing, and in fact they're just the lucky ones.
21:56
So, you can't rely on performance.
22:02
And you can't rely on years of experience.
22:04
And you can't rely You can ask peers, you know, who who do you go to?
22:09
That's often a good question.
22:09
Of course, the the people who work with with us have a better sense of whether they can whether they can rely on us or not.
22:16
But none of these are foolproof.
22:19
There's not a foolproof answer about who to trust, but you can use several of these and get a better answer than you might have by looking at how how somebody carries themselves.
22:33
To your point about explaining rationale and sort of why you made a decision and the variables that you consider and maybe how those variables interact over time, I I think organizations partly get in the way of this because they don't they
22:46
might write a summary of why they're doing something, but it's so high-level it doesn't actually contain much thought or reflection in in the thinking, which also prevents people from learning from other people how they're making decisions and developing expertise. You're you're exactly right. So, what
23:01
You're you're exactly right.
23:01
So, what you're you're what you're saying is that we we need to go beyond a simple surface explanation of why people did things.
23:10
And we want what what all my colleagues and I talk about as cognitive interviews.
23:17
To get beyond the surface explanation to find out how were you sizing this situation up? What were you noticing?
23:26
And you know, what what inferences were you drawing?
23:29
And we can we can be asking people those kinds of questions to get into their head more deeply rather than I figured the market was due for a correction or something like that.
23:41
This is a little bit of a tangent, but not too too much.
23:45
I thought about an idea called the decision scorecard for an employee.
23:53
One of the dreaded rituals of working in organizations is evaluating employees.
23:59
And and the people the supervisors don't want to do the evaluation cuz it's it's it's not a lot of fun.
24:06
And the employees are being evaluated, so it's not fun for them.
24:10
So, I said, "Instead of evaluating their have you know, I meet my objectives."
24:15
Cuz look most of the time I I state to you what my objectives are a year in advance.
24:19
And after about three or four months I have different goals.
24:23
And And now you're going to evaluate me on objectives from a year ago, and I've totally forgotten what those objectives are.
24:30
Uh what you can do and what what I've I've done on one occasion just trying this out is use a a decision scorecard.
24:39
I sit down with the employee, and I actually I've used it several times.
24:44
And I say, "Let's go back in the previous year.
24:46
What were some of the major decisions you made?"
24:51
And we we to you know, and I come to the meeting with what I think were the decisions, the employee comes with their decisions, then we compare notes, and then we look at the decisions and say, "Which ones worked and which ones didn't?"
25:07
Now, some decisions may have been good decisions that didn't work through no fault of the employee.
25:11
And some may have been bad decisions that worked because the employee got lucky.
25:19
So, you can't just look at the outcome, but you have to look at what What was the person thinking about when they made the decision?
25:27
And what can we learn whether it was a success or a failure?
25:32
And my experience is that this decision scorecard method of evaluating people um is much less stressful.
25:39
It It And And the employees and and I enjoy it because we're we're we're all learning a lot as we go through it.
25:48
Evaluating those decisions requires you to have knowledge of what you what you knew at the time, what you were thinking at the time, not retrospectively going back and trying to piece it together cuz now you have new information.
26:01
And we've written a lot at Farnam Street on the concept of decision journals.
26:03
I'm curious as to what information, if you were going to create a journal that an employee had to fill out every time they made a decision, what information would you put in that decision journal to then use at these meetings where you're evaluating their decisions later on?
26:22
I think the idea of a decision journal is a great idea.
26:24
And I hadn't heard of that before.
26:27
And so, what can I What can I suggest?
26:32
I would like to know when the employee is making a decision, um what is the decision?
26:42
What are the goals that the employee wants to achieve?
26:45
The primary goal, but there may be other goals that the employee is aware of.
26:51
Um what are the prime pieces of information the employee is using to make the decision?
26:59
What's the what what's being noticed?
27:01
What What's driving the decision?
27:05
Um I'd like to know um Who are the other people or or teams that are going to be affected by this decision?
27:16
Those are the things that I'd like to examine.
27:18
And then in retrospect, you know, we we can see what were there Were there Were there cues that I should have been paying attention to that I wasn't?
27:31
Are there goals that I should have been thinking about that I wasn't?
27:34
But you have it in the decision journal, you have a record of what was the person thinking about at the time.
27:40
I think it's a great idea.
27:44
And for For those of you listening, if you go to fs.
27:46
blog/dj for decision journal, DJ, you can you can get access to the template that we have online.
27:53
Jerry, I'll send that to you after. Okay. Thank you.
27:57
I also I also think it's really important that people write it down in their handwriting and not on a computer because it's so easy to look at things on a computer and convince ourselves that we didn't write that.
28:07
And you're going to try to you know, somebody else wrote that.
28:11
I was clearly thinking way better than you know, what it looks like.
28:15
But when you when you're confronted with your own handwriting and your own thinking, it does two things.
28:20
One, one when you write it down, it's actually a bit of reflection.
28:23
So, you're taking this complex decision and you're distilling it, you're reflecting on it.
28:29
And then you often realize through writing it that you don't understand it as well as you thought or you know where to go look for new different piece of information to to more effectively complete your understanding of it or to offer more value.
28:41
And the other benefit to doing it is that you can look back in the past and you can sort of like figure out patterns in your decisions. Right.
28:51
Um both of those are are are important issues.
28:55
Uh so, the decision journal the act of journaling is is a way of helping you inquire better and be curious about things that you hadn't thought about.
29:07
Yes, all of those are are are all of the above.
29:09
I I I would like to see the decision journal when we're finished here. Awesome.
29:15
I think we're actually going to publish one that people can buy on Amazon and write at some point this year.
29:20
Um maybe I'll get you to look that over before we we do that.
29:22
I would love your insight into that. Happy to do it.
29:28
One One of the things we're we're down a few rabbit holes here and I want to come out of this, but you used the term mental models.
29:32
What do you mean when you use the term mental model? What is that to you?
29:36
This could be another rabbit hole, but we we a few years ago my colleague Joseph Borders and Ron Westrum and I did a study of mental models in petrochemical plants.
29:46
And going into this study, I thought that a mental model was a set of beliefs about how something works, whether it's a a piece of equipment, a piece of machinery, it's an organization.
30:02
Here Here's how it works.
30:02
Here's Here's what happens.
30:04
Here's the kinds of of components and how they fit together.
30:09
So, that was my belief going in.
30:09
And we spent a a week at a petrochemical plant running people through a challenging scenario to to capture their mental model.
30:19
And that was part of their mental model, but only part.
30:21
We found that there was another part of the mental model, um not just how does it work, but what are its limitations, what are its boundary conditions, where can it go wrong.
30:35
And people who are experienced are well aware of these boundary conditions.
30:39
And so, they have that's part of the richness of their mental model.
30:43
And then we found another component, which is, um what are the workarounds that I that I can use if we run into one of those limitations.
30:52
How can I recover from it?
30:57
And then we found that there was a fourth component.
30:58
We have actually a matrix of of this a 2 by 2 matrix.
31:02
The last component is somebody with a good mental model can also anticipate where people might get confused.
31:12
What kinds of mistakes they might make, what kinds of uh mis- flawed beliefs they might have, so that you need to be more careful as you describe what they're supposed to do.
31:25
So, um my idea of a mental model started out with how something works.
31:31
And now it includes all four of these components, how it works, what its limitations are, how to work around them, and how to anticipate confusion on the part of people who I'm going to be working with.
31:44
Do you think I relate to the learning process?
31:45
So, I have a a I I think people learn with a learning loop and I'll just explain this briefly here.
31:54
It You have an experience, you reflect on the experience, that reflection gives you an abstraction, and the abstraction turns into action, and so you have this loop of learning.
32:06
When you learn a concept from somebody else, you're you're learning their abstraction.
32:13
So, you're putting an abstraction into use as your action.
32:18
And if you take the time to reflect, well, reflection is how we learn those edge cases, how we encode it in our mind where it's more likely to work in this scenario or not.
32:28
And so, in the first case, when you're learning from somebody else, I think you have like pattern matching, but it's exact pattern matching.
32:35
It's like searching Google with quotes.
32:38
Whereas in the second case, you're doing pattern matching, but it's associative pattern matching.
32:41
So, you're searching Google, but it's finding things that Oh, this is like this, or this is like that.
32:49
What do you think about that?
32:52
That That feels reasonable.
32:52
I think I think, well, to do that kind of learning, you're you're you're sort of exposing a a problem where I I get your abstraction.
33:09
But I haven't lived your experience.
33:13
So, your abstraction I might draw the wrong lessons from your abstraction. Right.
33:17
I don't know where it's coming from.
33:20
And I could ask you to explain it.
33:24
And language is a is a means for communication, but it's also a means for miscommunication, because there's all kinds of ambiguity in the words that we use.
33:34
And people use the same words and mean different things, and then believe that they mean the same thing.
33:43
And can get themselves in in trouble that way.
33:46
So, that's a limitation of having you explain your abstraction.
33:52
A way around that limitation is to ask you about stories.
33:59
Where, you know, where did you come up with this abstraction? What happened?
34:04
Tell me what what what went on that changed your mind.
34:08
And then I can ask you about about that event.
34:10
So, you were in this situation, and something happened that surprised you. What happened?
34:17
And why were you surprised?
34:17
And how did you notice it?
34:19
And what What what uh what sense did you make of it?
34:26
And um stories are are are not uh hampered by the limitations of language, because now we have an incident accounts that that we can derive.
34:39
And many organizations don't take advantage of stories.
34:44
And we we've just finished a project for a petrochemical group about using stories to try to get at these kinds of of expertise and communicate them.
34:56
Can you expand more on what role stories play?
34:59
Are they just compressions of ideas?
35:01
Are Are Are they sort of highlighting certain variables or details or omitting certain variables or details or getting us to anchor to something?
35:12
Stories are doing all of those things.
35:15
And they're doing it in a in a way that's that's really engaging.
35:20
Like, uh here's what happened, and I didn't know what to do about it, and then you want to know, what did I do about it? How did it end?
35:29
And and and um I know when when people have um recounted stories in a couple of groups that we studied when we were doing this project, we'd say, "Okay, so what's the takeaway from the story?"
35:42
And everybody had a different takeaway.
35:45
So, that's the richness of stories is what what have you learned from the story?
35:51
What's What are the implications?
35:53
And then we, as as a small team, just compare notes and realize that, because we have different mental models, we have different interpretations of the story and the importance of the story.
36:09
What makes for an effective story?
36:09
Um first of all, it's uh there's got to be some sort of mystery about something happened that wasn't expected.
36:20
And and then people listening to the story want to know how it how it uh wound up.
36:25
Second is the storyteller uh should be somebody who likes to tell stories.
36:33
And I've encountered some people who don't like to tell stories.
36:38
And uh and I'll ask them about stories and they'd say, "I can't remember any stories."
36:42
So, either they're a a avoiding the issue or they their their brain isn't wired to to be able to access stories.
36:51
Um or some people think they're telling a story when they just give me a narrative.
36:56
This happened, and that happened, and we go through what happened, but there's no story. It doesn't move. It doesn't evolve.
37:02
There's no transformation.
37:06
And the good the good stories result in an insight.
37:09
At the end of the story, you come to a realization you didn't have at the beginning.
37:15
And that's probably the most important part of a good story is that it it it it leads to an insight, at least one insight, that people uh are struck by.
37:27
That reminded me when you were saying that about the cop in the ashes, right?
37:31
So, he's looking for the contradiction.
37:33
So, he he's got a story in his mind in a way about the world works, and we all do when we're driving, cuz we're not consciously paying attention to the road, but we notice anomalies.
37:45
We notice something that is unexpected.
37:48
And in in a vehicle, that that's our cue to pay attention. Right.
37:53
As a driver of a vehicle, for whatever reason, we get encoded with this after years of driving.
37:57
It's like, "Okay, now I I'm switched on.
37:59
Maybe my heart races a little bit, but like I react.
38:01
I get out of my trance, and my body awakens.
38:06
It's like I'm in low power mode, and then I wake up."
38:08
And at work, but at work, we don't do this.
38:11
We We We get woken up, and we we have a surprise or a contradiction, and then we ignore it, or we explain it away, or we move on to the next task, instead of going from low power mode to sort of like full power mode, "What's going on here?"
38:25
because the surprise would indicate the world is not working the way that I I thought that it should work. Right. Right.
38:32
That um so, what you want to have is is a way to harness the power of curiosity when something didn't happen the way it was supposed to.
38:41
And our natural tendency is to dismiss it, to ignore it, and to explain it away.
38:50
Um and you think, "Well, why can't people think like scientists?"
38:53
Cuz scientists are careful.
38:55
But the researchers who have investigated this find that scientists do the same thing.
38:59
They have something that's called knowledge shields, the way all of us do, that that a a a variety of techniques for explaining away inconvenient data and anomalies.
39:12
But what we're effectively doing is we're holding on to our story.
39:14
What do you mean by that?
39:16
So, so we have a story about how the world works, and then we notice an anomaly, and rather than try to fit that anomaly in or explore the anomaly, we we it's easier, it's low power mode for us just to continue with our story the way that we're we're telling it to ourselves, and ignore or dismiss the anomaly.
39:35
That's why we we use these knowledge shields, and scientists use the knowledge shield.
39:42
Scientists don't want to challenge have anything challenge their theories.
39:46
And so you bring up a a a data point that's going to make my my my my uh pet theory look a little bit sketchy, and I'm going to say, "Well, um how did the how did you collect those data?
39:59
Or um I'm not really sure that you interpreted the data correctly."
40:05
I'm going to find all kinds of ways of of dismissing it. Yes. That's a problem. It leads to fixation.
40:13
Because one of our strengths when we encounter a situation is we we quickly make sense of it.
40:17
We we use the patterns that we've built up.
40:21
And uh usually works, but it doesn't always work.
40:25
And so some people say we should discourage people from coming up with immediate reactions.
40:29
But that's ridiculous because that's not the way we think, and it would us.
40:36
So instead you want us to come up with a a quick reaction, but if something if we're wrong, there's going to be an anomaly, and we want to be able to to revisit it.
40:47
So we we want people, that's the way we break out of fixation, is we notice the anomalies.
40:56
The way we get stuck in fixation and making fixation errors is we explain away the anomalies, hold on to the original wrong impression until it's far too late.
41:06
Now for your decision journal, one of the things that you could capture is um what do you think is going on in the in the situation now?
41:17
You know, describe what's what's going on.
41:21
And then as time occurs, maybe people can jot down uh data that they they that they receive uh that are inconsistent with that belief.
41:34
So you you can sort of track when is it that you first started to lose faith in your initial impression.
41:40
And could you have questioned it earlier?
41:46
Did you wait longer than you needed to looking at back at it with hindsight?
41:52
I I like that approach a lot.
41:54
Um and you you can do more than just decision journals.
41:56
You can uh my friend Adam Robinson has the idea of a surprise journal, which is just a little notebook you carry around in your pocket, and every time you're surprised, you just make a note of it.
42:06
You don't actually have to think about it or reflect on it in the moment.
42:09
You just make a note that something didn't go as you thought it would.
42:12
And one, it shows you the world is way more sort of unpredictable than we think in in these little minute ways.
42:19
And two, you can always look back over it later, and then go through it and be like, "Oh, I want to dive into this one.
42:25
I want to skip this one." And I do this one.
42:29
I I want to switch gears just a little bit.
42:31
How how do you think about the difference between people that stagnate at say an intermediate skill level and people who reach a mastery of the same skill? Right.
42:40
So this gets into the issue of expertise and how do people get to to the higher levels of expertise?
42:50
And uh some models of expertise say that that people just acquire more and more information, and that's how expertise builds.
42:57
But that doesn't work because that's like having uh uh unexamined uh experiences.
43:08
And and you're not you're not coming up with a richer mental model.
43:10
What we've we've my speculation working with uh a colleague Holly Baxter is that people get up to a certain level of performance, and then they start to stagnate.
43:24
And they start to plateau.
43:27
And people who who break through and move to the next level um are the ones who have to who engage in unlearning.
43:39
Who realize there are certain uh conventions they've bought into or beliefs they hold that are either wrong or are limited and and don't apply as broadly as as they imagine.
43:53
And so the the the issue of unlearning becomes important for expertise, and that ties in you said you know, get away from the idea of a surprise journal, but it does tie into the into this notion of a a surprise journal.
44:06
What happened that surprised you so that now you can maybe engage in some unlearning and move to a higher a richer mental model.
44:15
That's what experts have.
44:17
Is is there any uh sort of practical things that you've seen work particularly well in the wild for people developing mastery in uh less time say than people would expect?
44:31
So let's imagine two people starting out a job.
44:33
We'll say person A and person worker A and worker B.
44:36
Let's say worker A has uh they're both going to be on the job for a year.
44:43
They both started the same job for a year.
44:45
And worker A has a lucky year.
44:50
And everything goes smoothly.
44:53
And there's no disruptions, no problems.
44:56
It's just a really very very uh low stress year.
45:01
Worker B has had a turbulent year.
45:05
All kinds of things have gone wrong.
45:05
All kinds of adaptations have been necessary.
45:11
Um there were you know, mistakes made that had to be identified, diagnosed, and recovered.
45:18
And one thing is like the whole year has been filled with all of this.
45:24
Who is the one that you would like to have working with you after that first year? Worker A or worker B?
45:32
I would rather I I don't know how you're going to answer, so rather than Oh, I would choose person B because the surface area of their learning is so much richer and detailed. Exactly.
45:45
So worker the second worker with a turbulent year has learned much more.
45:50
Has gained much more during that year.
45:53
What can you do for worker A who's had the smooth year?
45:59
You can take some of these um crises that worker that the second worker had and encapsulate them in story or a scenario, and you can present them to worker A just to to give them vicarious experiences to to get them to move forward.
46:17
So that's something that that you can do.
46:22
And and and uh we've seen organizations doing such things like that, and that that's actually one of our decision training methods.
46:29
It's to provide those vicarious experiences.
46:32
That's really interesting when you you think about organizations as as sort of like trying to reduce the variability or reduce errors, and then at the same time acknowledging that errors create surface area for reflection, experience, expertise, and learning.
46:53
Yeah, but where people spoil it is is they then take it one step too far and say, um "The more errors you you make, the the more you're going to learn." Which which is true.
47:05
And and so we want to encourage people to make errors. And they don't mean it.
47:09
Organizations don't want to encourage people to make errors, especially some of the industries that I work in, where errors can be very dangerous.
47:16
And the fact is I don't like to make mistakes.
47:19
And if I do something, if I put on a workshop that has gone poorly, um my immediate reaction is I never want to put another workshop on again.
47:32
I mean, this is just too painful.
47:35
And it takes me a few days to get out of that emotional funk and to think about it and say, "You know, if I had done it this way, I could have used that problem, and it could have been much better."
47:48
And so by the end of that kind of rumination, that kind of reflection, my reaction is I can't wait to do another workshop cuz now now I want to try that again.
48:00
So but but the immediate reaction is one of being uh pretty devastated by by a mistake.
48:10
And and so it's going to be a a hard case if you tell me, "Gary, you know, try to try to make mistakes, you know, and that's how you're going to learn."
48:17
It's not in my emotional makeup.
48:17
Maybe it's in your shame, but it's not in my emotional makeup.
48:22
Oh, I don't know if I intentionally make them, but I certainly have more than my statistical fair share of them.
48:28
What is cognitive flexibility theory?
48:28
So cognitive flexibility theory is the the notion of trying to help people achieve expertise by um preventing them from locking in to routines and standard ways of doing things so that they can become more naturally adaptive.
48:55
And um you can give them vicarious experiences that can't be handled by the usual routines just to force them out of their comfort zone and to get them into a mindset of um not only being prepared to be adaptive, but enjoying being adaptive.
49:15
Cuz that's another thing that I've noticed about experts.
49:22
If they're doing the same job over and over again, they get bored.
49:26
And then something happens, and they can't use the same routines, and the journeyman, they get really frustrated.
49:34
Like, "This is not supposed to happen. I don't know. Who do I call?
49:39
And the experts, their eyes light up.
49:42
Okay, we can't use the the techniques that have been successful for us up to now. What can we do? What can we uh invent?
49:52
And so, the eyes lighting up is like uh for me a key to cognitive flexibility theory is to get people into that mode where they're excited when things don't go as planned and then they're going to have to improvise.
50:06
Coming back to the beginning, we've talked about sort of a a lot about gaining better insights.
50:13
Let's talk about some of the tools that we can use to reduce errors.
50:14
Maybe we can start with a pre-mortem. What is it? Why is it useful? How do we conduct one?
50:21
Uh and I'm not as as excited about tools for reducing errors because I that gets all the play.
50:29
And there's tons of techniques like checklists and uh procedure manuals and things like that.
50:37
So, so there's lots of uh techniques like that.
50:39
But what there isn't is a method like the pre-mortem.
50:44
So, let me tell you how a pre-mortem works. Um we didn't need one.
50:50
I just sort of invented it for when I for a company that my the company I was running back in the late 1980s.
50:59
And we we mostly had successful projects, but not always.
51:02
Sometimes our projects would fail.
51:03
And then we would do an after-action review and say, "What went wrong?"
51:10
And we'd realize, "Oh, if we'd just done this differently, um it might not have have ended so so badly."
51:18
And I said, "Why don't we We always have a kickoff meeting when a project starts.
51:23
Why don't we move that after-action review to the front? Like a a post-mortem.
51:31
A post-mortem is something that you do like in a hospital when a patient has died.
51:37
And a physician does a a post-mortem to find out why did the patient die?
51:42
And the advantage of doing a post-mortem and and discovering what was the real cause of death or the causes of death is that the physician gets smarter, now has a richer mental model.
51:52
So, it helps the physician.
51:54
And now the physician can tell the family, "Here's why your your loved one died."
52:00
Cuz they're they're they really want to know.
52:03
And now you you can you can help reduce their pain by giving them an an answer and they can work with that.
52:10
Everybody benefits from a post-mortem.
52:14
Except for the patient cuz the patient is dead.
52:17
So, we said, "Instead of doing a post-mortem for projects that fail after they fail, let's move it to the beginning."
52:23
And that's why it's called a pre-mortem.
52:26
And the way it works is if we're on a team, you take everybody on the team, we're all sitting around a table, and usually we do it at a at a kickoff meeting.
52:35
Here's Here's how we're going to carry out the plan.
52:39
Now, we've spent an hour, hour and a half just going through the kickoff meeting. What are we going to do? Who's doing what?
52:45
What are the roles and functions?
52:47
We've got all that nailed down.
52:48
Now, we've got about 20 minutes that left and we have enough time to do a pre-mortem.
52:55
And I say, "Okay, now we're going to do the pre-mortem. Everybody just relax. Sit back in your chairs.
53:02
Now, I'm looking in an imaginary crystal ball.
53:07
I actually do have a crystal ball, but I don't carry it around with me all the time.
53:11
I'm looking in an imaginal imaginary crystal ball and it's now 6 months from now.
53:18
Uh and this project has gone off the rails in a major way. It's been a disaster. It's been a failure.
53:27
Could be 6 months, could be a year.
53:29
Pick Pick your time frame.
53:32
What What happened is the crystal ball is showing failure.
53:38
Now, the crystal ball isn't showing us why it failed.
53:42
It's just showing us that it has failed.
53:44
That much is certain, and the crystal ball is infallible. It never lies.
53:50
Now, everybody around the table, they have a pad of paper in front of you and a pen.
53:55
Take the next 2 minutes and write down all the reasons why this project failed.
54:04
And I I have a timer on and I start the timer, and then everybody's writing like mad for the next 2 minutes.
54:12
And then I say, "Time's up."
54:15
And I I've got usually a whiteboard or on Zoom I'll I'll have a a virtual whiteboard.
54:24
And I'll say, "Now, I'm going to go around the the room.
54:28
Um and I'll start if I'm the project leader.
54:30
Here's the top of my list.
54:33
Here's the thing that I mentioned."
54:36
And usually it's something that never came up during the meeting.
54:40
But I'm if I'm the leader, I I the leader and the facilitator could be different people, but in this case we'll say it's the same person.
54:46
I want to set a good example that I'm just not just coming with something that's absurd, but something that's real.
54:56
And then I go to the next person.
54:56
Shane, what do you have on top of your list?
55:01
And then, Dave, what do you have on your list that neither Shane or I have mentioned?
55:05
And we go around the room and I'm writing these down.
55:07
Usually we go around the room a second time. Um sometimes a third.
55:11
And we have a long list of things that would explain why this project failed.
55:19
And most of them are things that people hadn't thought about.
55:25
And now we've sort of are are harvesting the collective wisdom of the people and the scar tissue of the people in that room.
55:34
And we're we're seeing all the ways that this plan that we were so confident so overconfident in, now our overconfidence has been diminished.
55:46
Now, some people have worried that maybe the confidence level got too low.
55:49
So, then we have a we added a last part of the exercise.
55:53
Look at everything on the whiteboard.
55:56
Look at all of those items.
55:58
Now, let's take another 2 minutes and write down what each of us What can I do personally to try to reduce the chance of some of these happening?
56:11
And then people are writing down what they can do.
56:13
And that's a way of trying to uh reduce the risk.
56:18
And so, that's that's how we we run a pre-mortem.
56:21
What we find is it surfaces ideas and and and and and flaws that people hadn't considered, which is extremely important.
56:34
But it also creates a culture of candor in the in the team where people are used have are starting to get used to expressing problems rather than covering them up.
56:48
And it creates a sense of trust that I can say something and I'm not going to get criticized for it.
56:57
The way the pre-mortem works is it reverses the usual dynamic of these meetings.
57:03
Often at the end of a meeting somebody will say, "All right, we're done.
57:04
We're just about done with the meeting.
57:06
Does anybody see any problems?"
57:11
Nobody wants to identify a problem.
57:11
You just spent the last hour and a half coming up with discussing the plan.
57:19
Nobody wants to uh admit that there's a problem.
57:21
People aren't even thinking about problems.
57:23
They're all in a go mode. Let's get started.
57:26
Let's They're all impatient to start.
57:30
Um and there you know, there could be consequences of of exposing problems.
57:36
With a pre-mortem, we we reverse that dynamic.
57:39
The way you show you're smart in a pre-mortem is the quality of the of the items that you generate.
57:48
And now you're you're you're you're uh you're you're demonstrating your um your competence by raising like surfacing issues that people hadn't thought about.
58:02
And there's a competition cuz I want to come up with good problems that nobody else has come up with.
58:09
And so, people are in this competition to surface problems that in an ordinary meeting nobody would ever say.
58:17
I think you hit on something that is hiding in plain sight that most people miss and and we've spent quite a bit of time on our ourselves is sort of how do you run more effective meetings by changing what people signal uh as value.
58:31
And and you can do this by asking people for unique insights into the problem that nobody else in the room has as one way.
58:37
Instead of everybody coming in and summarizing um the executive summary, which everybody's read, but everybody needs to paraphrase cuz they want to signal that they they've done the work and they're they should be in the room and they know what the problem is, even though they're just regurgitating the same terms over and over again.
58:54
I I think it's really uh a key uh key factor to the pre-mortem that you're you're changing you're signaling expertise to your coworkers through coming up with something valuable and uniquely insightful into the problem. Right.
59:10
So, it's sort of like uh it's almost theatrical.
59:13
I mean, there there there's a certain uh performative quality to this.
59:18
And And And we try to I I try to do it quickly.
59:21
I I I just want to get everybody's entry very quickly so that the energy level stays high.
59:27
And I want to get one from each person because often like one of the senior members of the team might want to give the entire list and then everybody just sort of in a you receive mode.
59:38
And I want to get out of that mode. I want everybody ready.
59:42
What are the common mistakes that people make in running a pre-mortem?
59:45
One of the mistakes is that they just frame it wrong from the beginning.
59:50
They'll say, "Okay, we should do a pre-mortem. So, what can go wrong?"
59:56
It's not an issue of what can go wrong.
59:58
We're dealing with a crystal ball that has showed that the plan has failed. We know that it failed.
1:00:06
And our job is is to explain why.
1:00:08
So, saying what can go wrong is too tentative. It's too vague.
1:00:14
And so, that's a that's I think maybe the most common mistake that that that people make is they don't frame it that way.
1:00:20
Another mistake is they let individuals talk too long.
1:00:27
And and everybody else is waiting.
1:00:27
And this way we're we're going around the room at a pretty rapid clip.
1:00:34
And everybody just stating one thing at a time.
1:00:38
That's the second mistake that that people make.
1:00:40
And a third mistake might be that um the leader gets cold feet and says, "I just want to hear what everybody else has."
1:00:56
And so, instead of creating a culture of trust and candor, the leader is is creating a dangerous environment where people can get in trouble.
1:01:06
And the leader is is being shielded and protected.
1:01:09
When we think about reducing errors, one of the the common things that sort of comes up with people is cognitive biases.
1:01:18
I'm curious as to what you've seen, what you think about trying to limit cognitive biases.
1:01:22
We've talked about one, which is fixation error.
1:01:26
And I have my own theory on this, but I'd like to hear your your views. All right.
1:01:30
So, I have to be careful about this.
1:01:34
And just caveat what I'm about to say that I'm an outlier.
1:01:39
That the whole issue of decision and judgment biases is a major line of research.
1:01:46
And the community, the heuristics and biases community has been extremely effective at the in the research they've done and the applications that they've put this work to in the judgment decision community.
1:01:59
So, I'm um I I want listeners to understand that the dominant view is that these these biases are important and need to be taken into account.
1:02:14
Um Uh having said that, I don't agree.
1:02:20
I don't like the idea of decision biases.
1:02:24
I know my sense of the literature is that there's been little if any success in debiasing people.
1:02:29
And I think that's a good thing.
1:02:32
Because the biases that get identified are essentially related to the heuristics that we've learned and are part of the experience that we have.
1:02:46
And they only look like biases in hindsight once one of them doesn't work. But it's a heuristic. It's not an algorithm.
1:02:52
So, sometimes heuristics don't work.
1:02:55
And the way the research has been done is you put people you give them a task where the heuristic that they're used to apply is not going to apply in this situation.
1:03:07
And you show that people people fall into the trap that you've set to them.
1:03:15
And you say, "You see that their heuristics are flawed and yet people many people most people fall prey to it."
1:03:25
Um but nobody has done research on the positive side of these heuristics.
1:03:32
And so, I think that's a bias on the part of the decision bias researchers that they're only looking at the down arrow.
1:03:40
How do these heuristics get us in trouble?
1:03:44
And they're not looking at the strength of these heuristics.
1:03:46
And you know, Danny Kahneman and Amos Tversky who developed these ideas, they themselves stated that these heuristics are generally useful even though they're not perfect.
1:04:00
And what's happening is the community is setting up because it's easy to set up studies where the heuristics get you the wrong answer.
1:04:07
And so, there's lots of these studies performed.
1:04:11
And so, that's that's the message that gets gets compiled without looking at the upside.
1:04:17
So, I've written about positive heuristics.
1:04:19
That all the major heuristics that get criticized are extremely valuable for coming up with insights and making decisions. So, availability bias.
1:04:37
Of course, that's going to be helpful if I want to draw on my experience in order to know how to work in a situation.
1:04:43
It's not going to be perfect.
1:04:46
But that's what my experience advises me.
1:04:48
The same thing for representativeness bias. Anchoring bias.
1:04:55
I'm going to anchor on a number.
1:04:58
Now, sometimes people do foolish things if you just give them a random number, they anchor on that.
1:05:02
So, people are capable of foolishness.
1:05:04
But but generally I've got to start somewhere.
1:05:08
And so, I draw on my experience to see what's this like and I'll I'll anchor on that and I'll adjust from that because my initial impression if I have if I have a reasonable amount of expertise is going to be accurate.
1:05:26
And so, the the heuristics that get criticized we would be crippled without those heuristics.
1:05:35
And I've created a thought experiment.
1:05:40
Um which is you know, people worry about emotion crowding our intellect and getting in the way of of of good decision making.
1:05:52
And as it turns out, there's a part of the brain, I forget what what the structure is called, where emotion and decision making come together.
1:06:03
And uh there are some people who have a lesion in this part of the brain.
1:06:07
And this is work done by Damasio about 15 years ago or so.
1:06:13
And he studied people with lesions in that part of the brain where they were unable to draw on their emotions to make decisions.
1:06:24
In other words, they were purely rational. They were like Mr. Spock.
1:06:30
And their intelligence didn't suffer.
1:06:34
Their IQ was as high as it ever was before the the lesion.
1:06:39
But their lives became miserable.
1:06:39
It would take them 45 minutes to decide what restaurant to go through. They got divorced.
1:06:46
They lost their jobs because they could not use their emotions because our emotions are a way of drawing on on the patterns we've built and on our experiences.
1:06:57
So, here's here's the demonstration the thought experiment I have.
1:07:02
If you are a strong believer in the idea that emotions crowd our thinking and interfere with our with our decisions and our judgments.
1:07:18
Um We know where this area is.
1:07:22
We can do laser surgery to triangulate different laser beams to create a lesion in this area.
1:07:28
So, you will no longer be troubled by emotions when you make decisions.
1:07:37
And my challenge is to any of the judgment decision researchers who worry about emotions crowding our decisions, would you have the surgery done?
1:07:49
I have yet to find a backer.
1:07:49
And I've said I will pay for the surgery myself.
1:07:55
Now, this would be a great experiment a great opportunity.
1:07:57
Nobody has taken me up on this offer.
1:08:01
Talk to me a little bit about making decisions in teams of people versus an individual making a decision where boards or committees are are are sort of asked to come to a decision.
1:08:13
What are the pros and cons of a team-based approach?
1:08:15
And what have you seen as more effective ways to get teams to make decisions?
1:08:21
Team decision making, the teams use different kinds of strategies.
1:08:26
And there's different formats.
1:08:28
You can have an autocratic team where a leader just makes a decision.
1:08:35
You can have teams voting.
1:08:35
You can have the vote be anonymous if people are afraid of repercussions.
1:08:44
There's all kinds of of techniques for for getting teams to uh uh to to work together more effectively.
1:08:54
And um there are situations like Let's take a pirate ship.
1:09:05
In the old Caribbean days when you had pirate ships around, pirate ships were extremely democratic.
1:09:13
There was nobody in charge.
1:09:13
They voted who was going to lead them.
1:09:15
They they voted about what targets they wanted to attack.
1:09:20
But once they were in battle, there was a captain.
1:09:25
And the captain had the captain's orders had to be obeyed because under that time pressure with that kind of urgency and those kinds of stakes, you needed to switch your decision strategy to something more autocratic.
1:09:37
So there's there's no general rule for how a team should make decisions.
1:09:42
It's going to depend on uh the situation and the context.
1:09:50
Uh so what can we say about having tried to help teams make better decisions?
1:09:57
Um I can give you an example of what not to do.
1:10:01
There was a movie with Matt Damon a number of years ago called uh The Martian.
1:10:09
And Matt Damon was part of a team that went to Mars and something happened and they needed to depart and they gathered everybody together but they couldn't find Matt Damon.
1:10:21
He was undevelopable and it was urgent that they depart otherwise they everything would be uh endangered.
1:10:29
And so they departed without him.
1:10:32
They assumed that he had gotten injured or killed.
1:10:33
That's why he wasn't there.
1:10:36
So now they're taking off from Mars.
1:10:38
They're heading away and Matt Damon somehow finds his way back to the station and radios them and said, "Where are you?"
1:10:46
And they realize he's still there.
1:10:49
So the decision is do we go back or not?
1:10:54
And the people on the spaceship know that if they go back it's going to be risky.
1:10:59
So they said let's come up with let's see if we can come to a a consensus about what to do.
1:11:09
And I don't like the idea of consensus decisions and I don't like the idea of a consensus decision in this kind of a dangerous environment.
1:11:18
Because as the people on the spaceship went around there was enormous pressure on everybody to go along with the with the the the consensus which was we should go back and rescue him.
1:11:35
And it was public and they went back and they they risked all of their lives in order to save him.
1:11:43
What they should have done is have people vote in a way that was secret.
1:11:48
And I see this with people who are cross country skiing in the back country and you don't want to just cross country ski where it's level.
1:11:57
You want places where there's hills but if the hills are too great then you have a risk of an avalanche.
1:12:03
At a certain point they'll look ahead and say "This could be risky.
1:12:08
Do we want to continue or not?"
1:12:10
And it's the same dynamic.
1:12:13
Uh where there's a lot of pressure on people to um inhibit their fear and go along with the with the consensus rather than than finding a way to to vote anonymously and realizing that one or two of the members don't want to do it.
1:12:31
So uh um having votes that are anonymous is important and I also don't like consensus decisions because those tend to be um and in other situations the consensus is fairly risk averse unlike with The Martian.
1:12:55
And people want to have something that everybody can agree with and you've lose lost your opportunity for somebody who's had an insight.
1:13:04
Somebody has a new idea uh of of of trying to put that in practice but the consensus is to try to do uh the safe thing that's comfortable that everybody can can live with rather than to do something that's bold and innovative.
1:13:23
So I have I have problems with consensus decisions.
1:13:26
I have problems with group decisions.
1:13:29
I think groups as a rule like a heuristic make terrible decisions.
1:13:34
Individuals should make decisions.
1:13:36
Groups should provide input.
1:13:36
Um but there should be one sort of person making the decision.
1:13:42
Um but that's my personal sort of experience is that I I've lived through through all of the uh different roles that I've had in life. I would agree with that.
1:13:52
Uh I was trying to answer your question how should how should teams make decisions?
1:13:57
No, this is too important though right because not everybody in in when people a lot of people agree with it but they can't change it.
1:14:04
it but they can't change it. So they're part of a committee and the committee's making a decision even though everybody on that committee might think it'd be more effective if one person uh sign their name to the decision which also would increase the accountability of the process believe it or not because
1:14:19
often what I found with group decision making is that when the decision is correct everybody in the room was responsible for it and when it was wrong everybody in the room tried to convince them not to do it and so there's no sort of like learning that happens from these but in in in the real world a lot of organizations still insist on making decisions as a group. So the question
1:14:38
So the question was sort of like how do we make more effective decisions as a group and and I think that you answered that beautifully and the second question is how do we how do we illuminate the insights in a group setting when we are making a decision as a group?
1:14:55
How do we find ways to um surface unique insights into the problem that we might not know?
1:15:03
Do you have any experience with that?
1:15:07
I will the pre-mortem method. Right.
1:15:09
is that to to a great extent.
1:15:13
Um And I think what's important about the pre-mortem method that could also be used generally by uh in in in group and team environments is to have people um craft their ideas and their interpretations and their assessments individually and privately before they surface it to other people.
1:15:36
And I the the research on like brainstorming where all brainstorming together the research shows that brainstorming does not produce more ideas or more innovative ideas.
1:15:46
And I think teams would be and groups would be better having individuals generate their own concepts and then after they've done it independently and privately sharing it with the others.
1:16:01
And and that would be a good way to generate insights that are hard to surface about why the project might fail.
1:16:08
Is there a way to even bring those forward into how we're proceeding with the project?
1:16:14
Like how do we get the insights on the table even if those insights are defining the problem because before you can solve it you have to define it.
1:16:22
And often we have very different If you were to This is a fun experiment.
1:16:27
I suggest everybody who who does this in group settings do this.
1:16:32
When you go to a decision to make or go to a meeting to make a decision have everybody in the room write out the problem on a piece of paper that they think they're solving with this decision and then compare how different and how much variance there is in those problem statements.
1:16:48
And so often when somebody has a more unique what what tends to happen in the real world is you go to a meeting somebody surfaces what the problem is it sort of clicks the minimal standard in everybody's mind and then being all type A individuals we jump on solving the problem and we forget to sort of go back and say like do we have the right insight into the problem?
1:17:09
Have we defined the problem?
1:17:11
And this is where if you have a sole decision maker instead of a group you can acknowledge that the responsibility of that decision maker maybe to listen to other people's definitions of the problem but ultimately they're responsible for defining the problem for the group and for themselves.
1:17:24
Okay, so I have a bunch of reactions cuz you've said a lot of very interesting and exciting things just now.
1:17:30
So first of all I agree with you.
1:17:33
Better to have a single leader.
1:17:38
Uh and that will relax the group and the team and they the leader is somebody that they that they trust.
1:17:44
They don't have to think that the leader is infallible but they they just know that the leader is going to be effective at at uh making the most of of the team's resources.
1:17:58
Um but one of the traits that you want in a leader is somebody who can honestly query the team members and see what what they think.
1:18:07
And uh and and a lot of a lot of times that querying is done in a rote way that doesn't accomplish much.
1:18:19
So I know that I'm the leader but I'm not supposed to say, "Here's what we're doing."
1:18:24
I'm supposed to have some some guise of of being open to other ideas.
1:18:31
So I say, "So let's go around the table and and see what other people think. What do you think? Okay, good. What do you think?"
1:18:36
So we're just checking the boxes so that I can say, "But I I went around the the table and I learned what people had had in mind."
1:18:46
That's not the way to do it.
1:18:46
Instead of just checking the boxes you want to be curious about what do they think?
1:18:55
What are your thoughts about this? Hmm.
1:18:58
Uh in what ways do you think um do you have a different sense of what's going on here or what our goals can be? Now let's capture that.
1:19:06
So that's the way to go around the table not in a rote fashion.
1:19:14
Now I'm I'm uh you're you're talking about you're starting to get into the area of wicked problems and and difficult problems.
1:19:23
And what I'm I'm doing a project on that right now with two colleagues, John Schmidt and Shawn Murphy, on on wicked problems.
1:19:30
And how to proceed with wicked problems.
1:19:35
And we're we're finding that um the notion of the goals is going to get richer and it's going to change as we proceed as we go along.
1:19:51
And you want to capture that.
1:19:51
You don't want to lock in to our original ideas because almost certainly our original ideas are going to be inadequate.
1:20:04
Now there's also way to do project reviews that could try to capture more insights.
1:20:13
In many organizations, if we're doing a project, well, I'll say, "Okay, Shane, you're doing this project.
1:20:19
Let's have a review every 3 months."
1:20:23
And now it's standard review how much what resources have you expended, what did you project that you were going to expend, what tasks did you accomplish, where are we in the milestone chart, and we go through all of those things.
1:20:40
And you need to go through those things. Those are important.
1:20:44
But I can also ask you in that review meeting, "Shane, in the last 3 months since the our last meeting, has anything happened that surprised you?"
1:20:57
And if we're dealing with a complex situation, with a wicked problem, that's different from something that's sort of like painting a room where everything is predictable.
1:21:08
So I say, "Has anything happened in the last 3 months that has surprised you?"
1:21:13
And if you say, "No, no, nothing nothing has surprised me." You don't have to worry.
1:21:22
That's when you have to worry.
1:21:26
Because I want to know what surprised you because that's like your surprise journal you talked about.
1:21:32
And now we can we can start to examine that and see and be curious about it and see what it might mean and how we can change our notion of the problem or the goals or any any of the important parameters.
1:21:50
I want to switch gears a little bit.
1:21:52
Often when we we are making a decision, there's two really similar options that are I wouldn't say equally but near equal um in terms of their effectiveness or our anticipated effectiveness about how they'll solve the problem.
1:22:06
And we spend a lot of time, money, and resources trying to make a perfect decision.
1:22:12
Can you explain the zone of indifference to us? Right.
1:22:17
I love the idea of the zone of indifference.
1:22:21
That's just the name I gave it. It's not my idea.
1:22:25
Uh it was popularized by by others who who noticed this exact phenomenon.
1:22:28
And the phenomenon the the way it the phenomenon works is if I've got two choices, a terrible option and a wonderful option, quick, which one do you pick?
1:22:41
Okay, that's not a hard decision.
1:22:45
As we move them closer together, and so now the the terrible option has some positive features and the great option has some negative consequences, and we get to this point where it's really hard because uh the strengths and advantages of of one versus the other are almost totally balanced.
1:23:13
And we get to this point, these are the hardest decisions people ever wrestle with.
1:23:19
And the paradox is if the advantages and disadvantages of the two options are almost perfectly balanced, it doesn't matter which one we pick.
1:23:32
And yet we will spend days and weeks going around in a funk, which one am I going to choose?
1:23:38
Committees spend hours going over it.
1:23:42
And I if there's anything that that can be valuable and and efficient to what I'm discussing, it's this.
1:23:53
When you think you're in the zone of indifference, to recognize, "I'm in the zone of indifference.
1:23:58
I'm never going to be able to tease them apart.
1:24:03
So I'm just going to pick one and spend my time in more fruitful ways.
1:24:08
Even if it means flipping a coin."
1:24:11
I I I like that a lot because I think we do spend so much time often we get paralyzed with perfection.
1:24:20
And I have a heuristic called stop, flop, or no that I use to sort of like get myself out of this paralysis, which is you stop gathering useful information.
1:24:29
Flop is the first lost opportunity, so you're about to lose an option or opportunity, or you know you know what to do.
1:24:36
And so if you stop, flop, or no, it it can get you out of this sort of paralysis. Okay.
1:24:42
And so that works for you. Yeah.
1:24:45
And it sound sounds like a reasonable strategy. Yeah, it's not perfect.
1:24:50
It's it's a heuristic again.
1:24:52
But um talk to me about shadow box.
1:24:54
Let me give you a little history.
1:24:59
Uh in 1985, um my colleagues and I came up with a description of how people actually make life-and-death decisions under extreme time pressure and certainty.
1:25:12
And we called it the recognition prime decision model.
1:25:14
And it's a blend of intuition plus analysis.
1:25:20
And we found we studied firefighters.
1:25:20
We found that most fire that accounted for almost 90% of the tough decisions firefighters made.
1:25:30
And we studied other domains and we found that it was also accounting for that.
1:25:34
Other people have replicated it.
1:25:37
So this strategy has been replicated fairly widely to explain how people can make really tough decisions even when they're not dealing with with time pressure.
1:25:49
And so we're we're very happy with this strategy.
1:25:51
It seems to uh it seems to to hold up.
1:25:54
It's held up now for over 35 years.
1:25:57
I keep expecting it to to be cast off by something better, but it hasn't happened yet.
1:26:05
Um but we never could figure out what do we do with it. How do we train people?
1:26:09
What we could we could train people by saying, "Here's the recognition prime decision model.
1:26:16
Now, you know, follow this."
1:26:16
And that would be useless.
1:26:20
Because of uh telling people the strategy isn't going to buy them anything.
1:26:24
The intuition part is a reflection of the patterns they've built up through experience.
1:26:30
And so there's no shortcut for that.
1:26:33
And uh uh analysis part is by using imagination and imagining uh how this decision how this option would play out.
1:26:45
And it really frustrated us that we could never find a way to provide some training.
1:26:52
And then about a little over 10 years ago, I met up with this guy, Neil Hines, who at that time was a battalion chief.
1:27:02
He was a firefighter with the New York Fire Department.
1:27:05
And he had was just finishing a master's degree um on helping people to handle new situations and tough situations.
1:27:20
And I he told me a little bit about it and it kind of sounded interesting. I wasn't sure.
1:27:24
Then I got invited to put on a workshop at the Seattle Fire Department.
1:27:29
And I it was a like a two- or three-day workshop.
1:27:31
And I said I would do it, but I asked Neil if he would come with me so we could do it together.
1:27:36
And he said he'd be happy to come with me.
1:27:37
Could he have a half hour in the workshop to to use his method that he had developed in you know, for his master's project.
1:27:47
And the way the what happened in the master's project, and this was in the aftermath of the 9/11 attacks, was they said, you know, they have all these firefighters encountering a situation they hadn't seen before and were really stymied about what to do.
1:28:04
So Neil created scenarios that would be unusual scenarios and he gave those scenarios to people and to see how what they would action they would choose and what their reasoning was.
1:28:17
And he also gave the scenarios to experts so that now people could compare what they would do to the experts.
1:28:26
So I didn't know hardly any of this when I invited Neil to work with me.
1:28:29
I just thought I would enjoy working with him.
1:28:33
So he puts on this half-hour stint in the Seattle Fire Department at the workshop.
1:28:37
And I watched the audience. They ate it up. They loved it.
1:28:42
And I said, "This may be what I've been looking for for decades as a way of training decision-making skills."
1:28:49
So here's how shadow box works.
1:28:49
It's primarily a scenario-based approach.
1:28:57
I create a tough, challenging scenario.
1:29:02
And we we go through the scenario. Usually it's in text.
1:29:08
And I stop it part of the way through and I say, "Here's decision point one, Shane.
1:29:15
At this point, here's four different options.
1:29:19
There are things you can do, courses of action.
1:29:22
Rank order which you would do from top to bottom, and then write down your reasons.
1:29:29
Sort of like your decision log.
1:29:33
And then we continue the scenario, and then we stop it again, decision point two.
1:29:40
Here are three goals that you might pursue.
1:29:45
Rank order the importance of each of these goals, and write down your reasons.
1:29:49
And then we continue with then we stop it again.
1:29:51
Here's decision point three.
1:29:54
Here's five pieces of information that you can pursue that would help you.
1:29:59
Rank order the importance of each one of them, and write down your reasons.
1:30:01
And it could be any anything like that, and we just go through the scenario that way.
1:30:08
At the same time, I've had a panel of maybe three experts.
1:30:13
They've gone through the same scenario that I'm showing you.
1:30:17
They've written They've done their ranking.
1:30:23
They've written down their reasons.
1:30:26
We've combined their rankings.
1:30:29
We've synthesized the reasons.
1:30:32
So now when you go through the scenario, and we get to decision point one, here's four different courses of action, and you you rank order them, and you write down your reasons, and then we say, let's see what the experts picked.
1:30:49
And then immediately we show you how your ranking compares to the experts.
1:30:54
And you want to match the experts.
1:30:56
You're really trying to get your ranking to match theirs.
1:31:00
But the So that's the game part.
1:31:00
You want You want to make get that to match.
1:31:05
But the real learning is Look at what the expert wrote down as their reasons.
1:31:10
Here are the things they noticed.
1:31:12
Here's the things that they inferred.
1:31:14
Here's the things they were worried about.
1:31:15
Here was the opportunities they saw.
1:31:17
And you look at what you wrote down and what they wrote down.
1:31:24
And now you are seeing the world through the eyes of experts.
1:31:29
But the experts don't have to be there cuz that would be a bottleneck.
1:31:33
They've done their work, and now we just have what we recorded from them, and we can provide it to you as feedback, so you're seeing the world at least this scenario through their eyes.
1:31:45
So that's how ShadowBox works, and in a half-day of training, we show that people's match to the experts increases by about 25% in just a half-day of training.
1:32:01
Now there's another version of ShadowBox, which is video-based, where I'll have like a one or two-minute video of something, and I'll say, here's a situation.
1:32:16
It could be a police officer dealing using body cam camera footage, or could be something that's staged in a hospital.
1:32:29
And we have uh like if it's a hospital, you know, tell me you know, what's happening in the situation.
1:32:37
And you watch the scenario from beginning to end, and then we say, now let's start again.
1:32:47
And as we go through the scenario, you can pause it at any point you want, and then there's a little circle, and you can put the circle around any cue that you think is important to be watching.
1:33:02
And you do that as successively as you go, and you write down why you why you you you highlighted that cue, and we go through the scenario.
1:33:11
And then after after you finish that, we say, now let's go through the same scenario, and we've had experts do the same thing you did.
1:33:17
Let's see what the experts were focusing on, and when they were focusing on it compared to you.
1:33:26
And then what reasons they they gave.
1:33:26
So this really is seeing the world through the eyes of experts.
1:33:32
So that's how ShadowBox works.
1:33:32
It's like shadowboxing with these tough situations, but in a way that's that's not risky.
1:33:40
I I I love that idea, and I think it's really effective, and we're exploring using it with some of our stuff.
1:33:48
When you mentioned the police officer in the body cam, a question that came up for me is it's one thing to watch in a room with air conditioning, and watch this police officer approach a vehicle.
1:34:00
It's a whole 'nother thing for the police officer to be approaching the vehicle, and be be in a different environment.
1:34:09
And that sort of brings me to the question of what role does environment play in our decision-making?
1:34:13
And are there things that we can do to improve our environment that improve our decision-making?
1:34:19
That that's a very important observation, because this has come up in some of the work that we've done with use-of-force decisions, where you're If you're a police officer, you might be in a situation, should I use force or not?
1:34:38
And sometimes there's That's just part of the job.
1:34:39
That's a horrible part of the job for police officers.
1:34:44
But we find that there may also There may be sometimes ways to to avoid getting into that decision.
1:34:57
The way you structure it, the way you frame it, the way you manage it can possibly diffuse the situation so that you don't necessarily have to get into a confrontation.
1:35:10
That you might be able to to avoid it.
1:35:13
So yeah, structuring the environment is a much better approach.
1:35:17
And this is something we have done a little bit with, not as much as we would like, about And this When I say we, this is my my friend and colleague John Schmidt, how do you get voluntary compliance rather than intimidation?
1:35:36
And for many a number of police officers, they only know one dominant way of getting civilians to comply with them, which is to threaten them, to intimidate them.
1:35:52
But we did a project a number of years ago.
1:35:57
My wife and I My wife Helen and I did the project to try to see what are the techniques for gaining voluntary compliance, for avoiding these kinds of direct confrontations.
1:36:10
And the nickname for this project was funded by DARPA, Defense Advanced Research Projects Agency.
1:36:14
The nickname was the Good Strangers Project.
1:36:16
And we studied military uh personnel, and we studied police officers who were known to be good at diffusing situations, at creating an environment that was a benign rather than a high-conflict environment.
1:36:37
And one of the things we learned, so I remember one one one police officer that we interviewed.
1:36:47
It was a It was a small office.
1:36:51
He was a large police officer.
1:36:51
He was hard to to fit into this office.
1:36:53
Not only was he large, he was extremely strong.
1:36:58
And like his arms were bigger than than my legs, okay?
1:37:01
He was a really strong police officer.
1:37:07
And he said, said, "I joined the police because I liked the action.
1:37:13
I liked, you know, I I I just liked it, and uh and and it was exciting, but I noticed as my career was going on, if I would sometimes get in fights with people, and I almost always won the fight.
1:37:27
But I always There was damage.
1:37:30
I was accumulating damage.
1:37:33
And then I looked at one of my colleagues.
1:37:34
He could get people to do things for him and just by the way he carried himself, and the way he talked to them.
1:37:45
And I said, I I can't continue much longer in the mode that I am. What's he doing?
1:37:52
What What's his technique?
1:37:52
And he said, ever since then I've been trying to come up with golden words, and I've been jotting them down when I see other police officers use them.
1:38:01
And I'm trying to build a repertoire for how to gain voluntary compliance.
1:38:07
One of his rules of thumb, maybe the most important one from my consideration, is he said, whenever he's in an an encounter with a civilian, or even with a lawbreaker, he said, "I want that person to trust me more at the end of the encounter than at the beginning."
1:38:34
And that governs the way he carried himself.
1:38:39
That he was carrying himself in a way that would move the needle to create greater trust rather than diminish trust.
1:38:50
And in terms of mindsets, that's such an important mindset shift.
1:38:54
His previous mindset was, I want to get people to do what I tell them, and so I've got to be tough, and I've got to be threatening.
1:39:04
And his new mindset was, how can I get them to do what I want, and um and have faith in me.
1:39:14
And, um, we heard great stories from police officers who are able to accomplish that.
1:39:20
I think that's a great place to end this, um, conversation, Gary.
1:39:25
Thank you so much for your time.
1:39:27
Thank you very much for the opportunity.
1:39:29
I've really enjoyed the conversation, Shane.