Math in Basketball: Garbage or Not Garbage?
- Thread starter slamminsammya
- Start date
One general note in favor of the utility of these various "all in one metrics" is that they're pretty strong at predicting future team performance. Essentially every NBA gambling group I know uses some flavor of these metrics. You can even use simple RAPM, add a basic aging curve, and combine with basic minutes projections and get pretty good looking preseason win projections.
Basketball is not a closed sport with seperate events like baseball at bats, so the stats are less valuable in comparing players, who is the best etc.
The analytics are getting better. For example I have heard at countless coaching symposiums that the midrange shot is terrible, useless, etc, that the pullup is the worst shot in basketball, etc. Yet I often hear Kobe and MJ are the GOATs. I always argued these conclusions since my eyes told me most pullups were at the end of the shot clock. More advanced metrics now consider this. Kemba being able to make 40% (just recalling this from somewhere, maybe different this year) of contest pulls in the last 3 seconds of the clock is actually pretty good. I saw a stat at a coaching thing once about how Bird, Reggie Miller, MJ, had whole seasons where they were insanely better than the league in late shot clock situations, but these were all shots that hurt their overall efficiency.
They are valuable for a coach trying to improve a team or exploit weaknesses of another team. I use HUDL a video streaming and sharing service with my high school team, but the stats feature was even more useful. I believe if I had these stats a year earlier I may have been n the championship game in 2019, nt a cancelled 2020 game.
What the stats showed me was my bias, for guys that are tough athletic on the ball defenders. I was playing two players guy like this too much, and more importantly their negative value was on offence and could be mitigated. The stata also allowed me to show the players that if they never, ever shot they would be more valuable, but the solution was to stop taking particular shots, and recognize their strengths and weaknesses.
The analytics are getting better. For example I have heard at countless coaching symposiums that the midrange shot is terrible, useless, etc, that the pullup is the worst shot in basketball, etc. Yet I often hear Kobe and MJ are the GOATs. I always argued these conclusions since my eyes told me most pullups were at the end of the shot clock. More advanced metrics now consider this. Kemba being able to make 40% (just recalling this from somewhere, maybe different this year) of contest pulls in the last 3 seconds of the clock is actually pretty good. I saw a stat at a coaching thing once about how Bird, Reggie Miller, MJ, had whole seasons where they were insanely better than the league in late shot clock situations, but these were all shots that hurt their overall efficiency.
They are valuable for a coach trying to improve a team or exploit weaknesses of another team. I use HUDL a video streaming and sharing service with my high school team, but the stats feature was even more useful. I believe if I had these stats a year earlier I may have been n the championship game in 2019, nt a cancelled 2020 game.
What the stats showed me was my bias, for guys that are tough athletic on the ball defenders. I was playing two players guy like this too much, and more importantly their negative value was on offence and could be mitigated. The stata also allowed me to show the players that if they never, ever shot they would be more valuable, but the solution was to stop taking particular shots, and recognize their strengths and weaknesses.
Measuring how well a metric predicts individual player performance is hard and can be circular. The simple answer is that if you want to predict a player's RAPM next year, you take their trailing 3-year RAPM, add a simple aging adjustment, and then regress to the mean by about 25%. If the player has changed teams, you regress their RAPM to the mean by an additional ~20%. So that gives you a sense of scale of how much RAPM is tied to team.
That's somewhat circular however, since predicting future RAPM isn't the same as predicting future player performance. But it's one datapoint here.
As you can imagine, they're somewhat secretive, but generally the various NBA gambling consortiums I'm familiar with all work the same way. They have their own version of RAPM, with a fairly sophisticated prior, which they combine with daily minutes projections to get an estimate of each team's rotation strength. They blend this with a top-down estimate of team-strength (which is not dependent on minutes), to capture the value of coaching/scheme. And then the rest is adjusting for rest/travel/etc... where the adjustments can get pretty sophisticated. Everyone has their own flavor of adjustments here, and nobody bets "blind", but that's the basic outline of what the groups I know of are/were doing.
FWIW, the group I'm most familiar with had a significant losing stretch in 2018, and disbanded after however, so it's possible the market has caught up with this basic approach however.
You can make RAPM perfectly predictive of itself by making it a constant for every player.
Yeah - that's my point. The value is in predicting future team performance, where it performs well.
Long-form article on the evolution of, and current status, of defensive stats in NBA: https://www.theringer.com/nba/2021/5/11/22423517/nba-defense-analytics-nikola-jokic.
I don't think it mentions anything that hasn't been said before in this thread or other pop-up discussions on the topic but I thought it was interesting to see it all compressed into one article.
I was interested to know that the biggest problem with tracking data is that there is too much information to make organize sensibly.
I don't think it mentions anything that hasn't been said before in this thread or other pop-up discussions on the topic but I thought it was interesting to see it all compressed into one article.
I was interested to know that the biggest problem with tracking data is that there is too much information to make organize sensibly.
That was an excellent article, thanks.
"The best advice overall might be to remain humble when assessing player defense, with numbers, film study, or ideally a combination of the two. Analysts have to approach the project as “knowing that you’re not really going to get it perfectly right,” says FiveThirtyEight’s Paine. “You’re just trying to get close.”
Of course the real question is how close they come.
As I think about it some, I wonder if the issue is trying to figure out a metric to talk about defense overall instead of thinking about defense as component parts. It may be one instance where raw stats could be more descriptive than trying to pin an overall number. What I mean by this is that a guy with a lot of deflections, is good at deflections, which is important. Same goes with steals and FG% allowed at the rim. And it may well be that a guy who has a low FG% against is good at contesting shots. And to have a great defense, you need all of these in different players.
It will be interesting to see how these stats evolve. It will be really interesting to see if anyone is ever going to be able to figure out who is the best at being in the correct spot on defense - so he can help cut off the lane on a drive or get to 3P shooters on rotation or the like.
Great post.
If we had average distance from the shooter when he receives the pass, it would provide some insight into how well defenses rotate (and perhaps particular guys). A lot of the meaningful data on defenses simply isn't being collected yet, I think.
Great points. Some players excel in specific components of defense while have glaring weaknesses in others. Take TJ McConnell......among the best in the league, if not the best, at deflections yet can be exploited in switches due to size. How can you put a numerical value on the latter when schemes would result in a variety of doubles and/or soft helps? Then you have his shot contesting which is tricky as he can be quick enough to “contest” a shot but isn’t long enough to truly contest it......which is of greater value, a hard contest by McConnell or a soft contest by say a Thybulle or Covington?
Thanks. I think the data you describe is being collected by the Second Spectrum cameras but as the article notes, a lot of it isn't making its way to the public.
It must be interesting to see what's really going on in front offices with respect to metrics.
P.S. someone is going to figure out a way to capture and analyze data from old game films and that person is going to make a name for himself or herself.
You - and to its credit, the article - raise the issue about scheme. If McConnell is playing in an aggressive trap the PnR defense, he's going to have better "metrics" than if he's playing in MIL's heavy drop coverage. And it's super tough to figure out scheme from play-to-play.
Similarly, if McConnell is told not to contest Marcus Smart and he happens to go off one night, maybe he's executing the scheme correctly but just unlucky.
I would be super interested in knowing what the Cs internal metrics said about Jaylen's defense this year. It hasn't been graded very well either by metrics or eye test.
You would think that NBA teams spend enough time watching film that they'd have a pretty good idea if a guy is a good fit defensively for their schemes or not.
You'd think. In reality, it's probably not much different then when someone takes a hitters spray chart and superimposes it on Fenway. This dude is going to hit 80 HRS!
I always wonder how much an NBA coaches/scouting meeting resembles the scene in Moneyball complete with the eye rolls.
The thing about basketball more than a lot of other sports is that it seems like the one where effort can be called into question on nearly every play.
To use baseball, the most analytically inclined sport we have, as an example, it is really hard for effort to come into question during the typical machinations of a game. A pitcher can't half-ass his pitches, he is going to throw as hard and as accurate as possible on every pitch. A hitter doesn't just give up when he steps into the box. Yeah a player might not hustle down the line once and awhile, but in general you rarely see anything but maximum effort during a baseball game. In addition, because position players only have a few times per game to influence the outcome, they are always going to try their hardest during those infrequent moments.This is why baseball has the best All Star Game, because it's unrealistic for the players to not put in the maximum effort when they are playing the game.
Compare that to basketball. The game is so free-flowing and individual players have so much influence over the game that effort can come and go at a moment's notice. Even the best defenders will fall asleep several times per game and get beaten by more-engaged offensive players. Some players may be more prone to lackadaisical effort than others, but that doesn't inherently mean they are worse defenders when the game really counts. LeBron has coasted as a defender for years during the regular season, but most people would still say that in the final two minutes of a close game, he is an excellent defensive player. So who cares if he is getting beat on a backdoor cut in January if you know he will be locked in in June?
Then you throw in all of the additional variables that come from a sport where effort can be turned on and off so easily, which can lead to statistics that are disingenuous when trying to assess the quality of individual players for situations when effort really does matter. In the NBA we talk about things like "load management," "schedule loss" and "tanking teams" in a way that is different than other sports. I don't know how you can come up with accurate statistical analysis that can take into account the variability that comes from the nature of the sport.
To use baseball, the most analytically inclined sport we have, as an example, it is really hard for effort to come into question during the typical machinations of a game. A pitcher can't half-ass his pitches, he is going to throw as hard and as accurate as possible on every pitch. A hitter doesn't just give up when he steps into the box. Yeah a player might not hustle down the line once and awhile, but in general you rarely see anything but maximum effort during a baseball game. In addition, because position players only have a few times per game to influence the outcome, they are always going to try their hardest during those infrequent moments.This is why baseball has the best All Star Game, because it's unrealistic for the players to not put in the maximum effort when they are playing the game.
Compare that to basketball. The game is so free-flowing and individual players have so much influence over the game that effort can come and go at a moment's notice. Even the best defenders will fall asleep several times per game and get beaten by more-engaged offensive players. Some players may be more prone to lackadaisical effort than others, but that doesn't inherently mean they are worse defenders when the game really counts. LeBron has coasted as a defender for years during the regular season, but most people would still say that in the final two minutes of a close game, he is an excellent defensive player. So who cares if he is getting beat on a backdoor cut in January if you know he will be locked in in June?
Then you throw in all of the additional variables that come from a sport where effort can be turned on and off so easily, which can lead to statistics that are disingenuous when trying to assess the quality of individual players for situations when effort really does matter. In the NBA we talk about things like "load management," "schedule loss" and "tanking teams" in a way that is different than other sports. I don't know how you can come up with accurate statistical analysis that can take into account the variability that comes from the nature of the sport.
Hitters definitely have lazy plate appearances.
I'm also not sure how the NBA would be much different than the NHL or NFL in regards to effort on every play.
I'm also not sure how the NBA would be much different than the NHL or NFL in regards to effort on every play.
Measuring things accurately - or as accurately as possible - is difficult but it doesn't mean it shouldn't be done. Regardless of whether the eye rolling is going on, there is far too much money at stake to not keep pushing for better metrics.
I think that everyone agrees on this.
The effort is worthwhile. And evolving it takes time and bodies. Less black box would help.
Weren't you around for the JD Drew era?
But to extend your point even further, there is a difference between "effort" and "peak performance." I hate criticizing guys' effort in any sports because they all want to win and it looks like to me they are trying as hard as they can.
Not to sidetrack this thread, but it's my understanding that a lot of teams are spending a lot of money trying to figure out why someone performs differently on one day versus another (you can tell I'm interested in this subject because of my screen name). Figuring that out is probably more useful - and maybe even farther off - than figuring out good metrics for analyzing defense.
The black box complaint was addressed upthread for the metrics we widely discuss.
The important take away for me is that fans really aren't equipped to have a good faith discussion about defense. If people associated with the teams are telling you how tough it is, its highly unlikely that someone sitting in their mom's basement will have a better handle on this aspect of the game. I know we will have people that are pretty confident in their eye-test so I don't doubt we'll be discussing it. However given the complexity of measuring defense (e.g. how do you account for one player looking lazy or lost on D when it was really the function of another player failing to help or switch properly) it seems like a fruitless task.
Defensive complexities exist in football and hockey as well where it is difficult to capture how a teammates actions affect another player. This is one part of the game where I’d find the eye test to be most effective if you have experience in these schemes. If you don’t know what the players are supposed to be doing you can watch until you’re blue in the face and it won’t matter but for a coach or a trained eye you should be able to recognize whether each players actions were correct/incorrect aa well as fast/slow.
This just goes down a rabbit hole because it gets into biases even with experienced professionals (e.g. what if the player a person is evaluating is someone they cannot stand? We see this all the time in this forum where posters are really fixated on one player as being a problem and then every wrong thing they do is dissected in great detail and sometimes it gets a bit irrational. It a human trait though and afflicts the best of us). The eye test certainly has some value but imo, its fairly low and you then have to apply a filter for the "tester" which just complicates things.
In short, it seems like an ideal defensive metric would essentially remove any subjective analysis but, alas, we aren't even close. As such, we will continue to get evaluations from everyone including people who have never experienced a lick of organized basketball as well as those people you reference who have experience in the schemes.
I share your frustration, believe me.
There are some folks who definitely don't value defense as much as they should. And there are a lot of definitive statements being made here and elsewhere based on numbers that even their creators caveat sometimes as directional (among a million other caveats).
I do take issue with the mom's basement stuff, as that's precisely where advancements often come from in this space. The history of baseball numbers is littered with guys whose names we know only because of their contributions to the science. They published in a scholarly way and welcomed critique. Others built on their work. It took decades to get where we are with baseball analysis, and it's outstanding (and still weakest on defense imo lol).
We know more now about the game of basketball and its players because of the work that these cats do, some of whom contribute to this forum. Some run their own numbers, some analyze others' numbers. It's all valuable and building on itself.
It's just really weak still on defense. But so was baseball once, and it's pretty damn good now.
Bill Belichick often criticizes football metrics that are based on analysis of each play using the all 22 film on the grounds that the evaluators don't always know what a player's assignment was on a given play. So maybe the player blew something and the analyst missed it, or maybe the player did exactly what he was supposed to do and the analyst, not knowing this, dinged him for a missed assignment.
The bottom line here is that the metrics don't tell the whole story and can't be relied on as if they do. But they do tell a part of the story in as fair and objective a way as is possible, so they have some use.
This is an interesting look from 538 on how Defensive Metrics look at Rudy Gobert's season. Bottom line, they don't agree on much but that Gobert is a great defender.
https://fivethirtyeight.com/features/rudy-gobert-is-having-one-of-the-best-defensive-seasons-in-modern-nba-history/
https://fivethirtyeight.com/features/rudy-gobert-is-having-one-of-the-best-defensive-seasons-in-modern-nba-history/
Really interesting piece. When even 538 says that the defensive metrics don't agree on anything (except Gobert), it's pretty telling. It's also not the first time that 538 has mentioned blending the numbers ("filling in the gaps" here) to produce results that make more sense and espoused that as a postive thing, not the sloppy corner-cutting technique that it is.
I want to keep supporting the effort, but if this was a reference for a candidate that I was thinking of hiring for a role on my team, I'd consider it lukewarm at best.
I want to keep supporting the effort, but if this was a reference for a candidate that I was thinking of hiring for a role on my team, I'd consider it lukewarm at best.
Caveats that they don't provide much context here about volume or the significance of this difference, but I found this pretty interesting:
