There has been some discussion of this in the Goat thread, but I figured it was interesting enough to warrant its own thread. With so much uncertainty in the probabilities of the various events, attempting to pin down a specific win probability differential really serves little purpose, other than to give the 538 guys a chance to show off their math chops (or for Bill Barnwell to try to do this and fail) - but sometimes you can get a pretty good sense of the correct answer just by looking at a rough outline of the equation.
Unfortunately, this is not one of those cases - it is simply too close to call. Here is how it looks:
Win probability following the fourth down conversion attempt:
(1-p(Success)) * p(TD) * p(2-pt) * p(OT) + p(Success) * {(p(TD1) * [p(2-pt) * p(OT) + (1-p(2-pt)) * p(FG)] + (1-p(TD1)) * p(TD)}
Win probability following field goal attempt:
p(FG make) * p(TD) + (1-p(FG make)) * p(TD) * p(2-pt) * p(OT)
where
p(Success) = probability of converting the 4th down
p(TD1) = probability of scoring a TD on the drive following the successful 4th down
p(2-pt) = probability of making the 2-point conversion (duh)
p(OT) = probability of winning the game after the game has been tied, even if there is regulation time left.
p(TD) = probability of outscoring the opponent by a TD over the remainder of regulation
p(FG) = probability of outscoring the opponent by a FG over the remainder of regulation
p(FG make) = probability of making the current field-goal attempt
Obviously, we have to make a bunch of simplifying assumptions even to get to this point. (For example, we are omitting the fact brought up in the Goat thread that in some cases, two subsequent field goals are enough for a win.)
The variables that are most difficult to estimate are the ones that involve the total remainder of regulation time, so rather than make a direct estimate, I like to solve for these at the break-even point, and see if the value seems clearly too high or too low. (And then see how the breakeven point changes as the values for the other input probabilities are changed.)
For example, if we estimate that p(Success) = 0.6, p(TD1) = 0.8, p(2-pt) = 0.4 (because of the shitty OL play), p(OT) = 0.4 (because in some scenarios, the opponent will end up with a greater number of possessions in regulation after the game has been tied), p(FG make) = 0.9, and p(FG) = 2 * p(TD), then the break-even point for p(TD) is approximately 0.3 - values higher than this favor the FG attempt, lower values favor the 4th-down conversion attempt. If this breakeven value had been something like 0.8, we could be pretty confident that going for it was the right call; if it had been 0.05, it would be strong evidence that Belichick was wrong. As it is, it falls squarely in the range of plausibility, so given the coarse nature of this approach, we can't say much one way or another.
The basic intuition is that you get some small amount of win equity from going for the 4th down (probably around 10%) thanks to the possibility of tying the game on this drive. This is offset by the fact that kicking an immediate FG makes a potential TD on a subsequent drive more valuable. The probability of this subsequent TD is low, meaning that this overall pro-FG effect is low, but then again so is the threshold that has to be cleared.
Please let me know if I made any egregious mistakes in setting this up (or if you get a significantly different result using different probability estimates). Even though this analysis doesn't give us a definite answer, it does suggest (IMO) that those who had a strong opinion one way or the other were probably incorrect.
Unfortunately, this is not one of those cases - it is simply too close to call. Here is how it looks:
Win probability following the fourth down conversion attempt:
(1-p(Success)) * p(TD) * p(2-pt) * p(OT) + p(Success) * {(p(TD1) * [p(2-pt) * p(OT) + (1-p(2-pt)) * p(FG)] + (1-p(TD1)) * p(TD)}
Win probability following field goal attempt:
p(FG make) * p(TD) + (1-p(FG make)) * p(TD) * p(2-pt) * p(OT)
where
p(Success) = probability of converting the 4th down
p(TD1) = probability of scoring a TD on the drive following the successful 4th down
p(2-pt) = probability of making the 2-point conversion (duh)
p(OT) = probability of winning the game after the game has been tied, even if there is regulation time left.
p(TD) = probability of outscoring the opponent by a TD over the remainder of regulation
p(FG) = probability of outscoring the opponent by a FG over the remainder of regulation
p(FG make) = probability of making the current field-goal attempt
Obviously, we have to make a bunch of simplifying assumptions even to get to this point. (For example, we are omitting the fact brought up in the Goat thread that in some cases, two subsequent field goals are enough for a win.)
The variables that are most difficult to estimate are the ones that involve the total remainder of regulation time, so rather than make a direct estimate, I like to solve for these at the break-even point, and see if the value seems clearly too high or too low. (And then see how the breakeven point changes as the values for the other input probabilities are changed.)
For example, if we estimate that p(Success) = 0.6, p(TD1) = 0.8, p(2-pt) = 0.4 (because of the shitty OL play), p(OT) = 0.4 (because in some scenarios, the opponent will end up with a greater number of possessions in regulation after the game has been tied), p(FG make) = 0.9, and p(FG) = 2 * p(TD), then the break-even point for p(TD) is approximately 0.3 - values higher than this favor the FG attempt, lower values favor the 4th-down conversion attempt. If this breakeven value had been something like 0.8, we could be pretty confident that going for it was the right call; if it had been 0.05, it would be strong evidence that Belichick was wrong. As it is, it falls squarely in the range of plausibility, so given the coarse nature of this approach, we can't say much one way or another.
The basic intuition is that you get some small amount of win equity from going for the 4th down (probably around 10%) thanks to the possibility of tying the game on this drive. This is offset by the fact that kicking an immediate FG makes a potential TD on a subsequent drive more valuable. The probability of this subsequent TD is low, meaning that this overall pro-FG effect is low, but then again so is the threshold that has to be cleared.
Please let me know if I made any egregious mistakes in setting this up (or if you get a significantly different result using different probability estimates). Even though this analysis doesn't give us a definite answer, it does suggest (IMO) that those who had a strong opinion one way or the other were probably incorrect.