It's the Little Things: Pick Your Favorite Unsung Effort From SB 51

[DennyDoyle'sBoil]

Why is going for two down 19 obvious? I get it if there are ten minutes left and you have virtually no chance of winning if the other team scores. In that scenario, down 17 is better than down 18. But in the third quarter, an 18 point lead means a 21 point deficit if you give up a field goal, which can be managed without a two point conversion. It seems like a really close question to me before the fourth.

 

[Stitch01]

You can keep setting up new straw men, but none will change what the poster was actually saying in the first place which was much simpler.

As noted before, I would be pretty surprised if the Pats did not have their own model, and if the assumptions were not a lot more nuanced than 'treat all games from forever the same' or 'assume 50% 2 pt' as Barnwell did. Like others here I look at those kinds of tools to try and get a rough sense of things, but the Pats are likely looking a lot deeper than that. For me, as we have done on the Main Board for years and years, we do best when we try to figure out what they might be doing and looking at rather than just assuming they were right and wishing away any complexity. I'd be surprised if they aren't self-evaluating decisions about situational stuff they same way they self-evaluate the offense and defensive tendencies. That doesn't mean any given game changes their model completely---and it also doesn't mean that they ignore the past when thinking about the next decision. Their assessment of their own decisions is a lot more relevant than some game 20 years ago that didn't involve Belichick and I suspect their model reflects that.

Not strawmen at all.

So what specifically do you think they learned from the Seattle game and how specifically do you think a model should properly adjust because of the result of the Seattle game?

They have a model of some sort. Whether its a detailed mathmatical model or just a hueristic model, they are modeling out their 4th down decisions (math models have an advantage of being less prone to common cognitive errors such as recency bias).

My suggestion is 1) pretty much nothing, they were evaluating all the correct factors when they made their decision against Seattle as they did on Sunday and 2) by zero or basically zero because its three data points among numerous short yardage and they can build conversion rate assumptions from the numerous goal line and short yardage plays they ran this year and tens of thousands of such plays league wide. So it shouldn't change their model.

Similarly, what should they have learned from going for it in the Denver game? 1) pretty close to nothing, they were evaluating the correct factors both in Denver and on Sunday 2) zero or pretty close to zero because the two conversion attempts and two defensive possessions shouldn't change the underlying probability assumptions for conversions or defensive stops in their model given they have numerous plays and defensive possessions of their own and a league wide baseline to drive the probability assumptions in their model.

How much did the AFCCG drive their decision on Sunday? Close to zero. It wouldn't affect their model, see above.

How much did kicking versus going for it matter Sunday in terms of win equity? Not very much either way, but open to mathematical assumptions that would show otherwise.

In fact, learning that BB kicked Sunday in large part because of what happened last year in Denver would make me (very marginally) think less of BB as a coach, not more.

 

[Bellhorn]

Why is going for two down 19 obvious? I get it if there are ten minutes left and you have virtually no chance of winning if the other team scores. In that scenario, down 17 is better than down 18. But in the third quarter, an 18 point lead means a 21 point deficit if you give up a field goal, which can be managed without a two point conversion. It seems like a really close question to me before the fourth.

Perhaps I did word it too strongly. The assumption as I was thinking about it in the abstract was that any extra scoring drives by the Falcons would eat up too much time for the comeback to be completed, but obviously the actual game events make that questionable.

The general principle, as has been brought up many times in these pages, is that when requiring multiple TDs plus a two-point conversion, there is an informational advantage to attempting the two-point as early as possible. What is often overlooked is that when three TDs are required, there is an absolute probabilistic advantage as well, given that a failure on the first attempt can be made up for with successes after the next two. If you assume 100% XP success and 50% two-point success, you complete the comeback 62.5% of the time if you attempt the two-point after the first TD, 50% otherwise. That's far from trivial, and when combined with the informational advantage, my hunch is that it outweighs the downside of requiring an extra two-point in the event of an opposition FG. But you're right that this downside needs to be accounted for as well.

 

[PedroKsBambino]

So, the point tims4wins made was that in retrospect the Patriots would have been better off kicking FGs in Denver. That is simply true as a retro-analysis matter; you can add up the points and see (at least, based on any realistic set of assumptions about FG%). However (and I think this is where you are getting lost) that observation is different than saying that since they would have been better off in Denver kicking field goals looking back on the game (with different knowledge than when the decisions were made) that must mean they change huge numbers of decisions going forward. Also different than saying "the decisions they made last year were wrong". If I hit on 20 in blackjack it is a bad decision---but if I get an ace, I can also say that in retrospect I was better off hitting than not.

No one is saying the decisions in Denver were terrible. No one is saying that they must have changed their model radically. No one is saying that the sole reason they left time on the clock Sunday was because they tried to run out the clock against Seattle and they lost.

What I think is being said is that they look at a ton of data, and they likely are self-scouting and trying to figure out whether they have calbrated their decisions correctly. They are likely constantly adjusting. The Patriots have access to a ton more research and information and analysis than we do, so its worth reflecting on what that might be and how (if at all) it might change things. Everything I am aware of about how they scout, cross-check, research, and make decisions suggests to me that they are very self-critical and thus they are, in fact, looking at these past games and evaluating them rather than just assuming they were right. My personal suspicion is that they'd do the Denver game the same way, and that they'd focus more on scoring than time against Seattle. I think Sunday's game is wholly consistent with the second of those, though we don't have enough info to say that's causal of course.

I'm just speculating, because I haven't looked at the data anywhere near closely enough to have a meaningful point of view (nor have you, though you seem to think you have). I would guess they spend time trying to figure out all sorts of things as part of that self-analysis, like what sample to use, how to adjust it for their own strengths and context, etc. rather than just taking NFL-wide assumptions. I would guess they have a way to think about higher scoring games as opposed to lower scoring games. And lots of other complexity. I personally think it's very likely that "they were evaluating the correct factors both in Denver and Sunday" as you conclude, but I also recognize that I don't really know what they looked at, and thus if or how they might have adjusted their thinking.

I like Barnwell and win expectancy tools and think they add to the discussion. But I also recognize that if people treat those things as truths, it's not so...and the guys who build the models will tell you the same.

 

[Stitch01]

Someone is lost. It is for sure not me. No, they were not better off kicking field goals in Denver. Or, using the same logic, Atlanta made a mistake and would have been better off going for two after their fourth touchdown. Treating Barnwell as truth is a strawman.

Also Tim wrote

This is not a specific play, but the coaching staff / BB for learning and adapting from two recent experiences that partially resulted in losses:

So his point was that the team adapted and changed as a result of what happens last year and did better this year. I disagree. Changing decision making based on that would be a clear error. BB is smarter than that.

 

[SamK]

I'll take Pink Stripes for $1,000 Alex

Love this.

And this small detail got me: On White's direct snap conversion, I thought Andrews might have had his bell rung. (Rewatching now I think he caught White's hat in the back of the head.) He seemed slow getting up, but TV cut away to the scoreboard or something. Then they showed a medium shot of BB on the sideline. What was coach doing? Taking a semi-long look into David Andrews' eyes as he came off the field. A long look -- because we still needed Andrews (and Coach may have a soft spot for smart, undersized centers.) But only semi long to avoid telegraphing his concern to 'protocol' game officials -- because we still needed Andrews.

 

[Morgan's Magic Snowplow]

It had no effect on the outcome but I loved the fake kneel down play, which definitely could have worked with a bit better execution and a slightly less aware opponent. They actually had too many guys out in front of Lewis. If one or two had blocked the backside pursuit, Lewis could have been a bit more patient following the lead blockers and then potentially cut it back and made something happen in the open field. They apparently copied the play from a similar play run by Western Kentucky (therefore "Hilltopper"), who executed it much better and with more time on the clock. Having a little time seems key. Its going to be hard to score on this play but its not that hard to envision ripping off 40 yards and moving from a "kneel down" at your own 30 with 20 seconds left in a half to a FG attempt at the opponent's 30 after you run the play and then call a TO. Unfortunately, every opponent will probably now be looking for this so it might be a few years before we could try it again. But I love that BB and the staff seem to always be looking out for weird trick plays run by college teams.

 

[DeadlySplitter]

Yikes. Just watched the last three plays in OT. (Son was re-watching the 4th Q and OT and I just popped in for the finale.) Didn't realize how close the fade to Bennett was to being an intercepted.

It really wasn't that close

 

[TFP]

I tried to read through the thread and didn't see it. Remember the punt Chung caught over his shoulder? Yeah that was a misdirection/trick return that Atlanta covered beautifully. The Pats really did pull out all the stops.

 

[Bellhorn]

Brady quick-snapped with :20 left on the White TD at the end of regulation. Looked like he wanted to catch the Falcons nodding off.

I think in situations like that, you do whatever you need to do to score the TD (or prevent it on the other side) and don't worry about the clock.

This makes no sense. You still lose the game if your opponent scores, and their probability of scoring varies (perhaps significantly) depending on how much time you leave them. Ergo, any remotely coherent evaluation of clock management in this particular endgame scenario needs to make some attempt to balance the probability of scoring against the probability of allowing an opposing score. In this case, those 20 extra seconds were potentially huge - it is a heck of a lot easier to drive 40 yards for a FGA in 57 seconds than it is in 37. If the coaching staff believed that this was offset by the greater probability of scoring via a quick snap, then more power to them - it seems hard to believe, but we're not really in position to answer it definitively, so might as well defer to the experts on this one. Credit to the D for stepping up and making it a moot point. Regardless, the "just score and forget about the clock" mentality - completely ignoring one side of the trade-off - is definitely wrong.

Regarding the question of leaving time for the Patriots to attempt a FG drive of their own following a failed two-point and a successful onside kick, the key point here is that the expected starting field position would be around midfield. So there is a sweet spot - probably around 30 seconds or so - where the clock would be much more of an obstacle for the opponent than it would for you in that scenario. But of course, you can't pinpoint the exact scoring moment ahead of time, and since the drive following the onside kick is much less likely to come into play in the first place, it is better to aim for leaving too little time on the clock rather than too much.

 

[DennyDoyle'sBoil]

I think they wanted to score as fast as possible. Right up to the last play, they still had enough time to get a stop on defense if they failed to make the two point conversion and didn't recover an onside kick. White got tackled at the one with 1:22 left. If he scores there, with two timeouts, you can still potentially get the ball back with 30 seconds if you make a stop. You might not even onside kick.

I think that was the mindset right up to the end, which is why they rushed to get the last play off. They wanted a second chance no matter how slight. By the time they got the play off, they had gone too far and would have had barely any time to try to get the ball back. But even so, they probably could have at least forced a punt to try a punt block. Once the clock got to 1:05 or so, I agree the smarter play is just to let it run since the amount of time you would have left is no more than ten seconds or so using both timeouts on defense. But by then, they are at the line and to just use the whole play clock puts false start by a confused player on the table.

 

[mulluysavage]

When I first saw that throwaway in the 4th quarter 3:36 to go, I was waiting for the flag to drop for an intentional grounding foul occurred in end zone safety.

This was very very close to both a sack in the end zone and intentional grounding. Either one effectively ends the game.

Troy and Joe don't pick up on it. It was actually huge. At the time I thought Brady's toss was too risky, both from an interception and IG perspective, but now I think the high risk was certainly worthwhile. The SB was on the line.

 

[tims4wins]

Someone is lost. It is for sure not me. No, they were not better off kicking field goals in Denver. Or, using the same logic, Atlanta made a mistake and would have been better off going for two after their fourth touchdown. Treating Barnwell as truth is a strawman.

Also Tim wrote

This is not a specific play, but the coaching staff / BB for learning and adapting from two recent experiences that partially resulted in losses:

So his point was that the team adapted and changed as a result of what happens last year and did better this year. I disagree. Changing decision making based on that would be a clear error. BB is smarter than that.

BB has flat out admitted he screwed up in the 2013 AFCCG by not kicking a FG and failing on a 4th down, and they ended up losing by 10, when they could have been down 7 with the ball.

 

[speedracer]

This makes no sense. You still lose the game if your opponent scores, and their probability of scoring varies (perhaps significantly) depending on how much time you leave them. Ergo, any remotely coherent evaluation of clock management in this particular endgame scenario needs to make some attempt to balance the probability of scoring against the probability of allowing an opposing score. In this case, those 20 extra seconds were potentially huge - it is a heck of a lot easier to drive 40 yards for a FGA in 57 seconds than it is in 37. If the coaching staff believed that this was offset by the greater probability of scoring via a quick snap, then more power to them - it seems hard to believe, but we're not really in position to answer it definitively, so might as well defer to the experts on this one. Credit to the D for stepping up and making it a moot point. Regardless, the "just score and forget about the clock" mentality - completely ignoring one side of the trade-off - is definitely wrong.

Fine.

PFR's win probability calculator (no idea how robust it is, but it's the only one I have access to right now) gives a 53.3% chance of winning given 1st and 10 at the 23 yard line and 0:37 left in a tie game. With 0:57 it goes up to a whopping 53.61%. (The calculator doesn't incorporate timeouts, but I'm guessing in the fixed scenario with 0 timeouts, the delta doesn't rise to more than 1% or so.) The WPR peaks at 55.68% with 1:39 left, then starts to go back down as the probability of a comeback drive increases.

Looking a bit further, the PFR model seems considerably less empirical than Brian Burke's models (which I think I have on my computer and will try to get working if I get a chance), so the probabilities are definitely off, though I wouldn't be able to say which way (although you could argue that with the relative O-D mismatches on both sides, both percentages go up). But on the Patriots side, you shouldn't be passing up edges greater than that size in order to control the clock. My guess is that the quick snap bought them more than a half-percent chance of scoring on the drive, and the 5-6% edge that the other team would have with a full clock wouldn't be worth, say, passing up an attempt at an open deep pass.

 

[jablo1312]

A detail I picked up after a 3rd re-watching...Logan Ryan came on a corner blitz on the 3rd and 6 following Atlanta's spike to stop the clock w/ ~20 seconds left in the game. It looked to me like Freeman was close to once again not recognizing the blitz/not doing his assignment, because Ryan almost gets by him before DF chips him at the last fraction of a second. LR would have been running at Matt Ryan with a shot at a strip sack, although admittedly MR was facing towards the side that the blitz was coming from instead of facing away from it like on the Hightower. Still seems like we were pretty close to hearing only about how Devonta Freeman blew the Super Bowl.

Also: Hightower clearly got held (by a TE, not by Matthews) on Flowers sack on 2nd and 11 with just under 4 minutes left. 100% could have been called, didn't end up mattering luckily. Kinda hilariously, he looks over at Flowers, registers that he got the sack, and immediately walks over to the referee to complain about it.

Finally, for all the grief the Falcons have gotten for the sack and subsequent holding penalty, the decision by Ryan on 3rd and 33 is arguably just as bad. Why throw a ball that even if caught will lead the receiver immediately out of bounds? Why not try to create a higher percentage throw, or a dump off to the RB to get a guaranteed 6-8 yards and a potential shot at the FG with the block still moving?

 

[heavyde050]

This was very very close to both a sack in the end zone and intentional grounding. Either one effectively ends the game.

Troy and Joe don't pick up on it. It was actually huge. At the time I thought Brady's toss was too risky, both from an interception and IG perspective, but now I think the high risk was certainly worthwhile. The SB was on the line.

I think Troy and Joe didn't pick up on it because that would have been a terrible IG call if made.
The pass was definitely in the area of a receiver.

 

[Hoya81]

I think Troy and Joe didn't pick up on it because that would have been a terrible IG call if made.
The pass was definitely in the area of a receiver.

Brady was also knocked off balance as he was throwing.

 

[PedroKsBambino]

Someone is lost. It is for sure not me. No, they were not better off kicking field goals in Denver. Or, using the same logic, Atlanta made a mistake and would have been better off going for two after their fourth touchdown.

There is a difference between ex ante and ex post analysis---I shared the blackjack example earlier to help highlight this distinction for you. The Patriots no doubt understand this truth and how to use each type of analysis--unfortunately, you do not and no amount of attitude in your posts is going to cover up that gap.

Whether or not we have radically different takes on the impact of the Denver and Seattle games on their decision making Sunday, we do have radically different takes on whether they look back on games and think about their own choices in a rigorous way. Oh well.

 

[Stitch01]

The fact that you threw out a strawman accusation earlier in the thread amuses me greatly right now although its clearly from ignorance rather than malice on your end.

The Pats think about their own choices in a rigorous way. They dont always make perfect choices in the heat of the moment, but they've shown they are pretty much always considering the right factors. I think they treat the results of the Denver and Seattle games like pulling a bad card in blackjack. Pulling a bad card doesnt change strategy in future hands. I also dont think they look at situations at a superficial level, so they understand that 4th and 16 with 11 minutes left in the game down 19 against the Falcons isnt really similar in any way to 4th and 1 down 8 with 5:30 minutes left (go for it but its close and I wouldnt have liked a field goal but there's at least an argument for it) or 4th and goal down 8 with 2:30 minutes left against the Broncos (kicking is really dumb). Or that being down 7 with 1:30 left in a game your defense is playing with some new personnel and is playing terrible (killing the down was smart football) in is similar to having an advantageous look in a hurry up down 8 with 1:00 left in a game your defense has been pretty good in.

I think looking and saying "well, BB must have learned and adapted (because it didnt work, therefore we need to adapt) from situation A to do make a different decision in situation B" is lazy analysis. Are they thinking about their analysis? Sure. Do they put any weight on the fact that kicking three field goals would have made the game 21-10? Zero or so close to zero it doesnt matter.

But still curious what you think BB learned from the Seattle game. Im pretty confident he understood that he was balancing decreasing the chances of scoring with decreasing the chances of Seattle scoring when they got the ball back and trying to maximize the chances of winning the game. Pretty sure he was doing the exact same thing Sunday. Pretty sure he understood that the situation on Sunday was not identical to the situation in the Seattle game for reasons Ive laid out elsewhere. Pretty sure he understood that if the Pats didnt score a TD in Seattle they lost. Pretty sure he understood scoring a TD wasnt automatic. Pretty sure he understood the chances of scoring a TD went down by giving up first down. Pretty sure he understands that adjusting whatever mathematical our heuristic "model" he uses to make game decisions based on one dice roll is flawed.

 

[Stitch01]

BB has flat out admitted he screwed up in the 2013 AFCCG by not kicking a FG and failing on a 4th down, and they ended up losing by 10, when they could have been down 7 with the ball.

The closest they were with possession that game in the fourth quarter was 16 points. I guess you could assume if the Pats kicked a FG they'd have been able to stop the Broncos and get the ball back down 7, but given the Broncos were a historically good offense and the next three Broncos possessions after the missed 4th down were FG, FG, end of game deep in Pats territory that seems like a stretch.

So I dont really see how they get to down 7 with the ball and even if we just look at how the game played out there's not a lot of evidence kicking to play for the TD/TD/FG combo would have "worked' given the defense never really held the Broncos scoreless after that.

http://www.espn.com/nfl/playbyplay?gameId=340119007

 

[PedroKsBambino]

No one will ever say you lack for attitude and confidence, Stitch, so I guess you have that going for you! I am sure you believe you know how the Patriots assessed the Denver and Seattle games. I am equally sure you do not actually know how they assessed it, or even (as we've seen in this thread) how they think about doing the analysis. I don't know what they might have concluded from reviewing those games either, though I do have hypotheses about it.

I don't think anyone has claimed either the Denver or Seattle game is identical to Sunday---I do imagine the Pats looked at the Seattle game and evaluated whether delaying the plays to try and burn the clock actually maximized their chances of winning. My suspicion is that they determined it may not have, because it ended up limiting their choices offensively in a way that kept them from tying things. They did so (I suspect) in the Seattle game in part becuase the defense was so awful that day that they placed more weight on running down the clock than they normally would. That may or may not be different than they felt Sunday, though there are some similar elements that I doubt they missed or oversimplified to 'the score is different'

What happened in Seattle, and how they broke it down after the fact, for me is an input into a much more complicated set of choices on the last drive Sunday, where (as others have noted) they likely were trying to keep alive both a one-drive and a two-drive option until pretty much the TD play). The fact the Seattle game is an input does not in and of itself mean it changed any conclusions (since we don't know whether it did or did not), but it turns out the reality of a good model is it tends gets better if you add data to it. Trying to make all these things into binary choices and absolutes isn't how they do things (I do not believe) and I don't think it helps the discussion here either.

 

[Stitch01]

OK, ignoring the piling strawmen for now..

Though there are some similar elements that I doubt they missed or oversimplified to 'the score is different'

Ive explicitly noted the 1/2 score option, differences in the defense, differences in the opponents, and the fact the Pats were in the hurry up and may have had an advantageous look in prior posts. Thought it would be obvious "down 8" was a shorthand for most of that in the context of the conversation. Apologies if you didnt pick up on that.

because it ended up limiting their choices offensively in a way that kept them from tying things.

Not really, other than having 3 downs instead of 4. What do you think they couldnt do offensively? After giving up 1st down to kill 40 seconds and burn the last Seattle TO the whole playbook was open. The 2nd down play was a Blount run up the middle. Sunday, James White scored on a run up the middle. I personally think BB understood that giving up a down reduced the chances of scoring a TD before the Seattle game. I also agree they should score TD's on their runs up the middle rather than not scoring TDs.

but it turns out the reality of a good model is it tends gets better if you add data to it.

Yes. I explicitly addressed this the last time you brought it up. The obvious reality of a good 4th down decision model is that three plays or one situation isnt going to materially change the assumptions that drive the model given there are thousands of situations and plays in the data set. The Patriots are a smart group, you are right, I am very confident they understand this.

It probably shouldnt bug me just "they adapted from this experience that resulted in losses" is 1) lazy analysis and 2) undervalues that they have been consistently thoughtful when it comes to game management 3) has way too much of a "it worked, therefore, good decision it didnt work, therefore, bad decision" flavor. I dont always agree with what they do game management wise, I certainly think they have been too conservative on end of half drives, but Im not going to wait until the next time they are aggressive at the end of a half and score and say "they must have learned and adapted from the 2015 Jets game".

 

[PaulinMyrBch]

I tried to read through the thread and didn't see it. Remember the punt Chung caught over his shoulder? Yeah that was a misdirection/trick return that Atlanta covered beautifully. The Pats really did pull out all the stops.

Here it is when it worked. Rams, Stedman Bailey.

 

[DennyDoyle'sBoil]

Here's a small but interesting factoid that I don't know where else to post. In the playoffs, kickers were an amazing 40 for 41 including 7-7 on kicks over 50 yards and 11-12 on kicks 40-49. The only miss was Crosby against Atlanta.

Yet in extra points, the same group of kickers was 47-53. It's a weird phenomenon but of course a small sample. Still, if I have time in the next couple of days I may try to compare extra point kicking to 30-39 yard field goal kicking in the league.

 

[kelpapa]

Here it is when it worked. Rams, Stedman Bailey.

Sherman did it, too:

 

[tims4wins]

Fantastic write up by Greg Cosell including screen shots and video of key plays

http://sports.yahoo.com/news/greg-cosells-super-bowl-review-inside-the-play-that-flipped-the-patriots-super-bowl-win-195345367.html

 

[jablo1312]

1 thing that occurred to me..how often do you see a team make up a 20+ point deficit in the 4th quarter without recovering an onsides kick or using all of their timeouts?

 

[DJnVa]

Here it is when it worked. Rams, Stedman Bailey.

Punt coverage has to "trust" the punt return team on direction and not where his punter was supposed to kick it.

 

[Super Nomario]

One sneaky play: with 32 seconds left and the Falcons driving to try to kick a go-ahead FG at the end, Patricia called for a double slot blitz (Chung and Ryan), dropping the rest of the D into a pretty vanilla Cover 2. It didn't get home, but forced Ryan to react quickly. He hit Hooper for a four-yard gain, then Atlanta had to spike it, putting them in a 3rd-and-6 with only 18 seconds left. An unusual playcall, well-timed, effectively forcing overtime.

 

[steveluck7]

Not an unsung effort but a little break. I just saw another replay of the Hightower strip sack, the ball pops up and it looks like it went through the hands of an ATL o-lineman (#65).

 

[brandonchristensen]

Not sure where to post this...

http://www.nepatriotslife.com/2017/02/check-film-how-patriots-finally-solved.html