The analytics of NBA coaching challenges

kazuneko

Member
SoSH Member
Nov 10, 2006
2,818
Honolulu HI
Thought I’d post this topic as I’m interested in what SOSHers more knowledgeable than me might think about this.
It seems pretty clear to me that Udoka (and other NBA coaches) overvalue the end-of-game use of coaching challenges. Obviously a foul is a foul and a point is a point regardless of if the bad call that leads to it is in the 1st or 4th quarter. So why do so many coaches, and Udoka in particular, save those challenges until the end of the game? Making it even worse is the fact that the big risk of a failed challenge (a lost timeout) is clearly a bigger risk in the very quarter that NBA coaches preserve their challenges for: the 4th quarter.
I wasn’t able to find much online about this but I’m sure there is stuff out there for people who are better at finding such things. With all the close games that these Cs have lost, can they really afford to be giving up potential points because of Udoka’s preference for keeping his challenge until the end of the game?
 

Attachments

Last edited:

DJnVa

Dorito Dawg
SoSH Member
Dec 16, 2010
53,854
Obviously a foul is a foul and a point is a point regardless of if the bad call that leads to it is in the 1st or 4th quarter.
Is that obvious?

A bad call in the first quarter leading to 2 points gives you 40 or so minutes to make it up. A bad call in last minute of game leaves you 40 or so seconds.
 

Eddie Jurak

canderson-lite
Lifetime Member
SoSH Member
Dec 12, 2002
44,481
Melrose, MA
Thought I’d post this topic as I’m interested in what SOSHers more knowledgeable than me might think about this.
It seems pretty clear to me that Udoka (and other NBA coaches) overvalue the end-of-game use of coaching challenges. Obviously a foul is a foul and a point is a point regardless of if the bad call that leads to it is in the 1st or 4th quarter. So why do so many coaches, and Udoka in particular, save those challenges until the end of the game? Making it even worse, is the fact that the big risk of a failed challenge (a lost timeout) is clearly a bigger risk in the very quarter that NBA coaches preserve their challenges for: the 4th quarter.
I wasn’t able to find much online about this but I’m sure there is stuff out there for people who are better at finding such things. With all the close games that these Cs have lost, can they really afford to be giving up potential points because of Udoka’s preference for keeping his challenge until the end of the game?
I think the NBA should get rid of coaches challenges because they affect one play out of hundreds and are just not that important.

Coaches save them for late because of the leverage of the situation.

What is the impact of a single basket on win probability in any of the first 3 quarters? Miniscule. Late in the fourth it is a different story.
 

Jimbodandy

Member
SoSH Member
Jan 31, 2006
11,408
around the way
Way to solve this problem is to give you the challenge back if it's successful. Then people would use them.

I'd be in favor of getting it infinitely. If the refs keep fucking you, and the calls keep getting overturned, you keep getting challenges.
 

wade boggs chicken dinner

Member
SoSH Member
Mar 26, 2005
30,506
Way to solve this problem is to give you the challenge back if it's successful. Then people would use them.

I'd be in favor of getting it infinitely. If the refs keep fucking you, and the calls keep getting overturned, you keep getting challenges.
Are you ready to watch 4 hour basketball games?
 

Jimbodandy

Member
SoSH Member
Jan 31, 2006
11,408
around the way
Are you ready to watch 4 hour basketball games?
I know...the TV argument.

Are the refs so bad that we'd be adding 90 minutes to games because of a dozen successful challenges? It hasn't killed the NFL. They get one back, I believe.

If the standard is something like "clear and obvious", then most shouldn't get overturned anyway.

I'd rather give teams the chance to overturn obvious errors and not just 1 of them.
 

slamminsammya

Member
SoSH Member
Jul 31, 2006
9,193
San Francisco
The idea that things are more important in the 4th quarter than in the 1st quarter seems like obvious fallacious thinking. If a game was close, you should in retrospect be totally indifferent to gaining 2 points in the 1st or 4th.
 

DeJesus Built My Hotrod

Well-Known Member
Lifetime Member
SoSH Member
Dec 24, 2002
48,215
The idea that things are more important in the 4th quarter than in the 1st quarter seems like obvious fallacious thinking. If a game was close, you should in retrospect be totally indifferent to gaining 2 points in the 1st or 4th.
This is incorrect. If you characterize a possession as "clutch" it mathematically changes the value of each type of scoring opportunity.

We can prove this with anecdotal data
 

benhogan

Granite Truther
SoSH Member
Nov 2, 2007
20,127
Santa Monica
How long would it take before a challenge is denied? I'm guessing not very long.
I'd guess half would lose their challenge before halftime, so it would probably extend games on average 5-10mins (there should be an off-court/field video replay ref that is staring at an HD screen to make quick calls for all sports)

According to the chart above 60% are successful in Q1. A Head Coach has to be pretty damn positive to challenge early now yet they are wrong 40% of the time. So I'd estimate they would be right half the time on every challenge if the rule was changed
 

DJnVa

Dorito Dawg
SoSH Member
Dec 16, 2010
53,854
The idea that things are more important in the 4th quarter than in the 1st quarter seems like obvious fallacious thinking.
I don't think "more important" is the right phrasing, but there's a time value involved. You can make a decision to not use the challenge with 2 minutes elapsed because you have 46 minutes to make up those 2 points.

If the time remaining wasn't a factor wouldn't every 2 point lead lead to exactly the same probability of winning?
 

wade boggs chicken dinner

Member
SoSH Member
Mar 26, 2005
30,506
The idea that things are more important in the 4th quarter than in the 1st quarter seems like obvious fallacious thinking. If a game was close, you should in retrospect be totally indifferent to gaining 2 points in the 1st or 4th.
Right but a coach has no idea whether a game will be close in the 1Q.

EJ has it right. If a game isn't close, it doesn't matter when you take the challenge. If a game is close, a coach can have his greatest impact on win probability by changing a call late in the game as opposed to early in the game.

For example, according to this win probability calculator, a team that is up 6 with possession of the ball with 2 minutes to play in the 1Q has a win probability of 63.7%. Without possesion of the ball, the win probability of 61.6%.

Same scenario with 2 minutes left to play in the 4Q, win probability goes from 97.3% to 94.6%.
 

slamminsammya

Member
SoSH Member
Jul 31, 2006
9,193
San Francisco
Right but a coach has no idea whether a game will be close in the 1Q.

EJ has it right. If a game isn't close, it doesn't matter when you take the challenge. If a game is close, a coach can have his greatest impact on win probability by changing a call late in the game as opposed to early in the game.

For example, according to this win probability calculator, a team that is up 6 with possession of the ball with 2 minutes to play in the 1Q has a win probability of 63.7%. Without possesion of the ball, the win probability of 61.6%.

Same scenario with 2 minutes left to play in the 4Q, win probability goes from 97.3% to 94.6%.
This is a misuse of "win probability". I can explain more but forward-looking leverage is not the right metric for thinking about this. Busy in a meeting at the moment.
 

SteveF

Member
SoSH Member
Jul 14, 2005
1,936
Thought I’d post this topic as I’m interested in what SOSHers more knowledgeable than me might think about this.
It seems pretty clear to me that Udoka (and other NBA coaches) overvalue the end-of-game use of coaching challenges. Obviously a foul is a foul and a point is a point regardless of if the bad call that leads to it is in the 1st or 4th quarter. So why do so many coaches, and Udoka in particular, save those challenges until the end of the game? Making it even worse is the fact that the big risk of a failed challenge (a lost timeout) is clearly a bigger risk in the very quarter that NBA coaches preserve their challenges for: the 4th quarter.
I wasn’t able to find much online about this but I’m sure there is stuff out there for people who are better at finding such things. With all the close games that these Cs have lost, can they really afford to be giving up potential points because of Udoka’s preference for keeping his challenge until the end of the game?
The psychology of regret and the tyranny of the remembering self. We regret an action taken more than we regret an action not taken even if they result in identical consequences (status quo bias). We also tend to place disproportionate importance on the tail end of an experience compared to the weighted average of that experience because it's what we remember. Taken together, you'll remember and regret not having a challenge at the end of a game more than you will regret not having used the challenge at all, even if the impact on expected win percentage is the same.

This is not simply my opinion, but is backed up by experimental data.
 

kazuneko

Member
SoSH Member
Nov 10, 2006
2,818
Honolulu HI
The psychology of regret and the tyranny of the remembering self. We regret an action taken more than we regret an action not taken even if they result in identical consequences (status quo bias). We also tend to place disproportionate importance on the tail end of an experience compared to the weighted average of that experience because it's what we remember. Taken together, you'll remember and regret not having a challenge at the end of a game more than you will regret not having used the challenge at all, even if the impact on expected win percentage is the same.

This is not simply my opinion, but is backed up by experimental data.
Well said.
I think you can see this psychology in the Chicago game. I thought there were multiple first half plays that Udoka could have challenged and won. Instead, he kept his challenge with the idea -I assume- that there might be some late game scenario that would be more important. In the end he didn’t use his challenge at all in a game that came down to 4 tense free throws by the team’s worst free throw shooter. Would it have been nicer to have had an extra 1 or 2 points? Of course.
This points to another flaw in this approach to using coaching challenges: holding them until the end of the game increases the chance that they never get used at all. This -inevitably- costs the teams points.
 

NomarsFool

Member
SoSH Member
Dec 21, 2001
8,160
It’s not the same thing, of course, but there are some parallels with the idea of using your best closer early in a game vs. saving him for a late game situation which may not be high leverage anyway.
 

reggiecleveland

sublime
Lifetime Member
SoSH Member
Mar 5, 2004
27,958
Saskatoon Canada
Two things
1. Challenges take too long. The ref at the game shouldn't watch it. They should take 90 seconds to watch it. Then correct it remotely and tell the scorers table..

2. I don't like the OOB challenge. Refs do a good job of avoiding foul calls by just giving the team that got fouled the ball. If a guy blocks out and they fight, he gets the ball if the guy comes over his back. On challenges, the wrong team gets the ball at times.
 

Eddie Jurak

canderson-lite
Lifetime Member
SoSH Member
Dec 12, 2002
44,481
Melrose, MA
Two things
1. Challenges take too long. The ref at the game shouldn't watch it. They should take 90 seconds to watch it. Then correct it remotely and tell the scorers table..

2. I don't like the OOB challenge. Refs do a good job of avoiding foul calls by just giving the team that got fouled the ball. If a guy blocks out and they fight, he gets the ball if the guy comes over his back. On challenges, the wrong team gets the ball at times.
I'd get rid of them except perhaps in late game situations. I just can't see how one missed call in the second quarter can have such a big outcome on the game that a weird remedy needs to be sort of available to fix it.
 

themuddychicken

New Member
Mar 26, 2014
80
2. I don't like the OOB challenge. Refs do a good job of avoiding foul calls by just giving the team that got fouled the ball. If a guy blocks out and they fight, he gets the ball if the guy comes over his back. On challenges, the wrong team gets the ball at times.
This drives me nuts, because in my mind the refs calling out of bounds instead of calling a a weak foul at the end of a game is a good thing. Refs have a way to do the right thing (reward the proper team the ball) without handing out decisive points via a foul. But OOB challenges change that for one play a game, not enough to change what the refs do but enough to determine outcomes.

I think the solution is to allow for foul calls on OOB challenges if the foul influenced the OOB. It will suck when teams challenge an OOB, just to get a foul called against them, but at least it'll be fair. The current system where a team is rewarded with possession due to a foul the ref purposefully chose not to call is not fair.
 

Cellar-Door

Member
SoSH Member
Aug 1, 2006
34,467
I think the best use of the challenge is to prevent a guy getting to a key number of fouls early, especially on offensive fouls.
Outside of that, I think it's a toss-up between saving it for late and risking not using it, and getting a call that might get you into double digits on a lead and maybe rest guys.
 

slamminsammya

Member
SoSH Member
Jul 31, 2006
9,193
San Francisco
I think the best use of the challenge is to prevent a guy getting to a key number of fouls early, especially on offensive fouls.
Outside of that, I think it's a toss-up between saving it for late and risking not using it, and getting a call that might get you into double digits on a lead and maybe rest guys.
Here's a heuristic - a star player is gonna be worth 3-5 net points per 100 possessions. A reversed foul call will get you an extra 10-15 minutes, let's say his backup is a -1 player so that is netting 4-5 points per 100, 10-15 minutes will be roughly a 1 to 1.5 ish swing.

Contrast that with a call that reverses a turnover for your team (or otherwise gives you possession) - 1.1 points ish. A call that reverses free throws for the other team will net about 0.6 ish points I would guess based on the difference between 2 free throws and a normal possession.

So yeah, I can see the argument for saving a guy from foul trouble, especially if its an offensive foul since that is a turnover too.
 

Marceline

Well-Known Member
Lifetime Member
SoSH Member
Sep 9, 2002
6,441
Canton, MA
I'd get rid of them except perhaps in late game situations. I just can't see how one missed call in the second quarter can have such a big outcome on the game that a weird remedy needs to be sort of available to fix it.
Well if a team is down 4 with 30 seconds left, and they opted to not challenge a call in the 1st Q with a high likelihood of being overturned that would have netted 2 points, then it did in fact have a huge impact on win probability. You just didn't know it at that moment.
 

Eddie Jurak

canderson-lite
Lifetime Member
SoSH Member
Dec 12, 2002
44,481
Melrose, MA
Well if a team is down 4 with 30 seconds left, and they opted to not challenge a call in the 1st Q with a high likelihood of being overturned that would have netted 2 points, then it did in fact have a huge impact on win probability. You just didn't know it at that moment.
That presumes that the changed score has no impact on the behavior of either team.
 

wade boggs chicken dinner

Member
SoSH Member
Mar 26, 2005
30,506
Here's a heuristic - a star player is gonna be worth 3-5 net points per 100 possessions. A reversed foul call will get you an extra 10-15 minutes, let's say his backup is a -1 player so that is netting 4-5 points per 100, 10-15 minutes will be roughly a 1 to 1.5 ish swing.

Contrast that with a call that reverses a turnover for your team (or otherwise gives you possession) - 1.1 points ish. A call that reverses free throws for the other team will net about 0.6 ish points I would guess based on the difference between 2 free throws and a normal possession.

So yeah, I can see the argument for saving a guy from foul trouble, especially if its an offensive foul since that is a turnover too.
Interesting analysis, thanks for posting. I know you know this but just wanted to point out for the sake of discussion that reversing the 1st foul is not the same as reversing the 5th foul.

If keeping a star player on a court is the most important thing a coach can do with his challenges, isn't this an argument for keeping the coaches challenges to the end of the game?
 

slamminsammya

Member
SoSH Member
Jul 31, 2006
9,193
San Francisco
Interesting analysis, thanks for posting. I know you know this but just wanted to point out for the sake of discussion that reversing the 1st foul is not the same as reversing the 5th foul.

If keeping a star player on a court is the most important thing a coach can do with his challenges, isn't this an argument for keeping the coaches challenges to the end of the game?
Again, the 10-15 minutes you gain throughout the game is more valuable than 3 minutes in the 4th quarter. I think the notion of win probability swings is confusing people about the basic fact that points have equal value throughout the game. And the hypothetical was avoiding foul trouble (e.g. 2nd foul early 1st quarter, or 3rd foul before halftime), a 1st foul isn't challengeable probably (unless the play itself is good value).
 

Eddie Jurak

canderson-lite
Lifetime Member
SoSH Member
Dec 12, 2002
44,481
Melrose, MA
Again, the 10-15 minutes you gain throughout the game is more valuable than 3 minutes in the 4th quarter. I think the notion of win probability swings is confusing people about the basic fact that points have equal value throughout the game. And the hypothetical was avoiding foul trouble (e.g. 2nd foul early 1st quarter, or 3rd foul before halftime), a 1st foul isn't challengeable probably (unless the play itself is good value).
Not really. I do agree with the foul thing. That is a way for the challenge to have a more substantial impact on a game.
 

wade boggs chicken dinner

Member
SoSH Member
Mar 26, 2005
30,506
Again, the 10-15 minutes you gain throughout the game is more valuable than 3 minutes in the 4th quarter. I think the notion of win probability swings is confusing people about the basic fact that points have equal value throughout the game. And the hypothetical was avoiding foul trouble (e.g. 2nd foul early 1st quarter, or 3rd foul before halftime), a 1st foul isn't challengeable probably (unless the play itself is good value).
I don't know if reversing a second foul in the 1Q is going to give a player an extra 10-15 minutes. A lot of time that just moves the player's rest around, particularly if the player doesn't get the third before halftime.

I agree with the third foul before halftime and most (all?) coaches would be on board challenging a borderline third foul to a star player before halftime.

But I definitely agree that waiting until a player is in foul trouble is better than using the challenge on, for example, a borderline first foul.
 

HomeRunBaker

bet squelcher
SoSH Member
Jan 15, 2004
30,096
That presumes that the changed score has no impact on the behavior of either team.
This part is crucial to understand and I’d imagine a part of DeJesus’ initial response which I agree with. Teams play to the score in the final 5,4,3,2 and final 60 seconds…..the closer we get to the final buzzer the more the score dictates the scheme of both teams a The claim that 2 pts in the 1Q is equal value to 2 pts late in the game is a flawed argument.
 

slamminsammya

Member
SoSH Member
Jul 31, 2006
9,193
San Francisco
This part is crucial to understand and I’d imagine a part of DeJesus’ initial response which I agree with. Teams play to the score in the final 5,4,3,2 and final 60 seconds…..the closer we get to the final buzzer the more the score dictates the scheme of both teams a The claim that 2 pts in the 1Q is equal value to 2 pts late in the game is a flawed argument.
You are correct that the impact is not identical, but it is close enough to true for the purpose of analyzing the best challenge strategy.
 

HomeRunBaker

bet squelcher
SoSH Member
Jan 15, 2004
30,096
You are correct that the impact is not identical, but it is close enough to true for the purpose of analyzing the best challenge strategy.
There are more reasons to challenge a play simply bc it’s early/late in the game. Take the identical play in the 1Q and 3-4Q. If it’s the foul on a key player early that would limit his minites than challenging early would seemingly be correct……say Tatum’s 3rd midway through 2Q or his 2nd early 1Q. Without any such factor I’d prefer saving it for a situation later in the game that has other factors involved aside from the call being right/wrong especially if there is any chance that the early challenge fails.
 

kazuneko

Member
SoSH Member
Nov 10, 2006
2,818
Honolulu HI
Udoka again didn't use his challenge tonight. That is not that unusual for him and a clear result of his"save the challenges to the last quarter" strategy. Tatum missed minutes due to foul trouble, and there did seem to be a clear miscall early. It probably wouldn't have changed the result, but -in retrospect- not using the challenge in the first half may have yet again cost the team.
 
Last edited:

HomeRunBaker

bet squelcher
SoSH Member
Jan 15, 2004
30,096
Udoka again didn't use his challenge tonight. That is not that unusual for him and a clear result of his"save the challenges to the last quarter" strategy. Tatum missed minutes due to foul trouble, and there did seem to be a clear miscall early. It probably wouldn't have changed the result, but -in retrospect- not using the challenge in the first half may have yet again cost the team.
I said this exact same thing the other day about 1H challenges being beneficial in removing a foul from a key player. Add one more flaw to the Ime/Brad leadership team.
 

kazuneko

Member
SoSH Member
Nov 10, 2006
2,818
Honolulu HI
So Udoka finally used a challenge (after multiple games without using one) and for some reason decides that he should risk losing a timeout (in a close game with 3 minutes to go in the 4th quarter) in order to ask the refs to reconsider a clear shove by Robert Williams - and it wasn’t even a shooting foul. He loses the challenge and the timeout, and three minutes later the Cs are down 3 with 3.5 seconds to go and are forced to bring the ball up the length of the court…:(
Who’d have thought that his best use of the coaches challenge in the last three games would be the two games he didn’t use it at all..:(
Of course, in the same game Billups showed that this isn’t an issue unique to Udoka, failing to use his challenge at all, despite a clear miscall in the first half that would have taken 3 points off the board (when Tatum got an and-1 on a play that should have been an offensive foul)..
 
Last edited:

Eddie Jurak

canderson-lite
Lifetime Member
SoSH Member
Dec 12, 2002
44,481
Melrose, MA
So Udoka finally used a challenge (after multiple games without using one) and for some reason decides that he should risk losing a timeout (in a close game with 3 minutes to go in the 4th quarter) in order to ask the refs to reconsider a clear shove by Robert Williams - and it wasn’t even a shooting foul. He loses the challenge and the timeout, and three minutes later the Cs are down 3 with 3.5 seconds to go and are forced to bring the ball up the length of the court…:(
Who’d have thought that his best use of the coaches challenge in the last three games would be the two games he didn’t use it at all..:(
Of course, in the same game Billups showed that this isn’t an issue unique to Udoka, failing to use his challenge at all, despite a clear miscall in the first half that would have taken 3 points off the board (when Tatum got an and-1 on a play that should have been an offensive foul)..
My guess is that he as about to call a time out anyway.
 

kazuneko

Member
SoSH Member
Nov 10, 2006
2,818
Honolulu HI
My guess is that he as about to call a time out anyway.
Then he should have just taken a timeout. There was zero chance that the call was going to be overturned - as was obvious by Scal and Gorman’s reaction and readily apparent to anyone who watches the replay. The only logic to keeping the coach’s challenge until the end of the game is to save it for an end-of-game big play in a close game.
By the time Udoka used the challenge he clearly knew that it would be a close game, yet rather than saving it for a big play he wasted it on an unimportant, non-shooting foul that didn’t seem to have any chance of being overturned. Whether he “was going to take a timeout anyway” or not it was not a good decision.
 

NomarsFool

Member
SoSH Member
Dec 21, 2001
8,160
I forget the rule, but aren’t you allowed to only keep 2 timeouts for the last 2 minutes? It wasn’t a particularly adept challenge, but I don’t think it cost them anything