Annie Duke: The Framework of Decision-Making and Play Calling

Football is one of the world’s most strategic sports, with each play marking a new opportunity for either team to prevail, unlike other free-flowing sports. The driving force behind a team’s success, or lack thereof, is often the head coach or the play-caller. Decision-making on the front or end of plays called during the game is tantamount to the execution of the play.

Annie Duke, a poker champion and decision-making scholar, joined The 33rd Team’s Wednesday Huddle this week to share how to effectively evaluate situations where you do not have all the information but need to make a critical decision.

“In poker, we would say that poker is one long game,” said Duke. “But football is one long game, too, right?”

Duke emphasized that, in football, like most things, you needn’t win at every juncture, play, or quarter, but if you win more than you lose, then you will win the long game. An oft-cited “poor decision” in NFL history was Pete Carroll’s decision to run a passing play on second down within the one-yard line with 26 seconds remaining in Super Bowl XLIX, but was it a poor decision? Decisions leading to poor results tend to elicit a judgment that the decision was flawed—a phenomenon called “resulting.”

“Resulting is looking at the quality of the outcome and mapping that back to the quality of the decision as if they are perfectly correlated,” she said. “The problem is that one cannot actually draw that conclusion from the result exclusively.”

Duke’s logic is sound. For example, you cannot deduce someone is a poor driver if they were involved in an accident. It could be that they were not at fault in the collision, or someone hit them due to negligence on their part. Nevertheless, Duke asserts that mistakes are a natural and inevitable byproduct of making decisions without all the information.

“Pete Carroll can call a pass play, and USA Today can write a headline reading ‘Worst Play in Super Bowl History,’” she said. “Pete Carroll is a pretty good coach. Maybe it was not the worst play call in Super Bowl history, so we want to be really careful of this problem.”

The quality of decisions truthfully lies in the probability that it will yield a favorable outcome, argues Duke. Yet the mounting challenge is calculating probabilities without all of the information. When uncertainty enters the equation, or even dominates it, then what?

“Could Pete Carroll have known the ball would be intercepted beforehand? Obviously, the answer is no,” she said. “Could he have known that [Bill] Belichick was practicing the particular defense he ran on that particular play? It turns out, the answer to that is also no, because Belichick had never run it before; it was not on tape.”

What should the decision-making process look like with everything riding on the line? Moreover, what should it look like when many things are (1) out of your control; and (2) when you are dealing with a great deal of uncertainty, as coaches often are in a football game?

“Let’s think about what he did know at the time. He has three downs available: second, third, fourth, and he is down by four at the one-yard line, and he only has one timeout—there are 26 seconds left,” she said.

“So, he knows that if he hands the ball off to Marshawn Lynch, he will score about 50% of the time from that distance, but the other half of the time, he will have to use his last timeout to stop the clock.”

Duke’s thought experiment reveals an obvious but revelational concept. Decisions, especially in a football game, are not within a vacuum. Past or future implications always factor into a coach’s decision-making or play-calling process beyond that of the next play’s success only.

“Returning to Carroll, let’s just say he does hand the ball to Lynch on second down, and Lynch does not score, and he has to burn a timeout,” she said. “If he hands it off to Lynch again, he does not get any more time, but we also know that if he attempts to pass, the clock will stop without using a timeout if the ball is incomplete. That is the only way to give yourself a chance to run three plays in his position, to pass at least once.”

Duke’s perspective is insightful. The probability of a poor outcome (losing the game) due to Carroll’s decision to pass it to Ricardo Lockette on a slant route was likely lower than the probability of a poor outcome if he handed the ball off to Lynch.

“The chance of interception is the cost of getting that third play, and the interception rate is nearly 1%,” she said. “In that situation, which is a meager price to pay to get three attempts at the end zone as opposed to two against the Patriots defense.”

Although the outcome was tragic, as we all know, it should not change the simple fact that handing the ball off to Lynch in those closing seconds had a higher probability of yielding an unfavorable outcome. Considering what Pete Carroll knew when he called the play, it was likely a sound football decision. Yet, the poor result of that good decision has given teams and coaches around the league pause about running a similar play in similar circumstances.

Duke explained that culture could often be an obstacle to optimized decision-making on a micro-and macroscale basis. The culture within an organization may demand firm answers rather than educated guesses or reports on the probability of something happening or not. The culture around football at large, in the national media, among fans, etc., can disincentivize quality decision-making. Pete Carroll’s Super Bowl blunder is an excellent example of that.

“If Pete Carroll had lost the game by handing it off to Lynch twice, we all know that USA Today would not have said that was the worst play call in Super Bowl history, right?” she said. “The title would have been “Bill Belichick Does it Again,” it would have had nothing to do with Pete Carroll; it would have been just like, ‘wow, that defense is really good.’”

It shows that intuition, albeit an indispensable decision-making tool, is far from perfect. While it may seem that there is an obvious choice at first glance, sometimes, the obvious choice is not necessarily the best. Duke preaches that quality decisions are made within quality decision-making processes, wherein decision-makers attempt to reduce uncertainty. Duke used the early-to-mid 2010s Houston Rockets as an example of a major sports team diving into the uncertainty and reaping the benefit.

“The math on the three-point shot has been known since the time of Larry Bird, but it was not until somewhere around 2014 or so that it was implemented,” she said. “Now, the teams that caught onto it like the Rockets really early started beating everyone.”

The NBA’s movement toward the three-point shot mirrored a decision-making process, particularly from a coaching perspective, that prioritized the mathematically correct decision despite its departure from traditionally long-standing basketball principles.

“Now in the NFL, we have had something similar [happen] with fourth-down conversions,” she said. “Okay, so that brings us back to Pete Carrol, right? Pete Carroll made the mathematically right choice, despite some career risk associated with it.”

Innovation in modern sports is a double-edged sword. As Duke said, there is inherent career risk with pioneering the x’s and o’s to uncharted places. Pioneers in the strategic ins and outs of the game can have significant upside or drastic, career-ending consequences. A novel idea that proves successful will frame the innovator as a sort of savant, while a failing idea may spell the end of someone’s pro tenure.

“Now, the fact is that people have different time horizons, and a coach does have career risks,” she said. “The owner wants to win tomorrow, and the coach wants to win over the long run, so these things are very, very sticky culturally.”

It is a philosophical dilemma. Should a coach push the boundaries and, perhaps, practice an innovative take on the game in hopes of being “ahead of the curve” and pleasing ownership with gridiron success, or should they play it safe?

“The one absolutely sure way you can protect yourself in that situation is to win a couple of Super Bowls, right?” she said. “Belichick can be so creative because he has won so much. It is settled that if you see Belichick do something really bizarre, you may think, ‘oh, it must be right because it is Belichick.’”

So, who is to say what is a correct decision or not? That question will likely echo throughout sports for the rest of time. The dynamic between trust, hierarchy, innovation, and decision-making will puzzle NFL front offices for years to come. Meanwhile, this is an excellent place to start. Admit that there is uncertainty, attempt to reduce it, and choose the mathematically correct choice. After all, the law of averages will be on your side because football is a long game.