NBA Draft Game Thread (Spoilers allowed)

[Deathofthebambino]

Holy shit guys, turns out Ainge didn't want to give 5 fucking picks!  What an incompetent asshole he is!

 
I actually just got an alert on my phone from Yahoo Sports that said at one point, Ainge had offered 6 picks, including 4 first rounders, to move up to No. 9 and Charlotte declined, because they really wanted Kaminsky.  That's crazy if true.  I think I'm glad they said no. 

 

[bowiac]

An easier adjustment would be to benchmark the value of each pick through a representative swath of recent history (~20 years or so). I know that 82games has a chart that sort-of does this, and basketball-reference keeps track of the win shares for each player drafted (in the draft table each year). So there are at least two ways to compile that data very quickly. The trick is, as bowiac mentioned, taking trades into account.

My rankings do this already to adjust for this baseline (using VORP rather than win shares), but I think Swedgin's point is that the gap between a hit with a top pick the average expectation there is so big that it dominates things. I'm not sure that's accurate, but it's a possibility. The best "value" according to this model is Chris Paul, and LeBron is second. Then again, there's real value to taking Chris Paul (#4), rather than Marvin Williams (#2 that year). And Marc Gasol, Kawhi Leonard, Paul George, and Chandler Parsons all rank higher than Durant does for instance.
 
The correlation between how much "value" a pick provided and their draft spot is -0.00056 overall. I'm not sure this is a big effect all in all.
 
One question is what is the definition of a good pick however. For instance, every Minnesota fan will be happy if Andrew Wiggins turns into Paul George in a couple years, and nobody will much care that he was not so good this year. Right now, I'm adding up the value of a player's first four seasons (rookie contract length), but that might not be a good measure. Bad teams don't care what you do as a rookie, and good teams basically never have impact rookies, so maybe I should just toss rookie year performance. Etc...

 

[RG33]

The ESPN report says that too, but it says "potentially 4 1st round picks" which I read to mean it included the picks the Celts have the could be #1s but are highly likely to be #2s based on where the team finishes (Minnesota pick is one I think?).




EDIT: Meant to quote DOTB's post

 

[Eddie Jurak]

My rankings do this already to adjust for this baseline (using VORP rather than win shares), but I think Swedgin's point is that the gap between a hit with a top pick the average expectation there is so big that it dominates things. I'm not sure that's accurate, but it's a possibility. The best "value" according to this model is Chris Paul, and LeBron is second. Then again, there's real value to taking Chris Paul (#4), rather than Marvin Williams (#2 that year). And Marc Gasol, Kawhi Leonard, Paul George, and Chandler Parsons all rank higher than Durant does for instance.

What about year to year variation in draft strength? 

 

[southshoresoxfan]

Yeah any system where Chandler Parsons is more valuable than Durant needs tweaking.

 

[PedroKsBambino]

What makes this so hard is that the life span of so many GMs is short. So Ainge has few contemporaries to compare to.

 
Always good to look at data.  That acknowledged, there's a few major flaws that seemed to have been ignored in the design.  One is what Devizier notes---14th is not really 'average' because the other GMs are changing a bunch in there, and comparing "Ainge" to "three other guys" with some other team doesn't really show much of anything.   The worst guys tend to disappear, and I suspect the best teams tend to have a few 'wins' that probably need to be smoothed out.
 
Also, there's a bunch of no-brainer high-VORP guys relative to slot in the last 14 years---Lebron, Paul, Davis, etc.---which will give some teams a very high score that really says nothing about their ability to draft (it's true that Marvin Williams instead of Paul is bad---but that's caught in the current design.  Paul being an apex-star and generating a lot of VORP doesn't change that he was a pretty clear pick when he was selected, imo.  Certainly, we know Ainge would have taken him there!)  If those aren't corrected for or smoothed some, there's a pretty material false-positive problem for 'good drafting'.  As noted subsequently, that is a real consideration.  Suspect the totals are not that big, so a couple 'hits' skews the data a lot (this is the 'Kawhi Leonard' problem).    Not sure the Durant ranking below lower-picked guys who became stars is a problem, though---to me, that's the model working reasonably well and setting a high bar for top picks.  
 
One that is not an obvious problem, but may well speak to the challenges of evaluating drafting records, is I suspect variability is higher as you get deeper into the round.  Using the average VORP for each spot over a set of years assumes (implicitly) variability is the same in all 'tiers' of the draft, but I doubt that is actually the case.  Would need more study to be sure on this one. As an aside, this is what I suspect a deep analysis would show is most spectacular about the Spurs record---they did it at end of round, when many picks don't even really ever play meaningful minutes.
 
I get it was quick and dirty; as I said, always good to put data into the picture...and also to step back and think about the problem, too.   Also, just to be clear, I'm not sure that Ainge is a great drafter and think challenging that conclusion is quite reasonable; I also think we need to be rigorous in thinking through the data before we draw conclusions,too

 

[bowiac]

What about year to year variation in draft strength? 

I have nothing in there for that. I could adjust for that pretty easily, but I'm not sure it's a good idea really. I'm using a regression over the same 12 seasons Ainge has been GM for, so it evens out for this analysis. In other words, if I do it year-by-year, some guys might look better, some guys might look worse, but it's fundamentally zero sum. I'll check to confirm it doesn't change anything however.
 

Yeah any system where Chandler Parsons is more valuable than Durant needs tweaking.

I'm not sure this is right. Getting Chandler Parsons for a 2nd round pick takes a lot more skill than getting Kevin Durant with the second overall pick, no?

 

[bowiac]

Always good to look at data.  That acknowledged, there's a few major flaws that seemed to have been ignored in the design.  One is what Devizier notes---14th is not really 'average' because the other GMs are changing a bunch in there, and comparing "Ainge" to "three other guys" with some other team doesn't really show much of anything.   The worst guys tend to disappear, and I suspect the best teams tend to have a few 'wins' that probably need to be smoothed out.

This is certainly fair. Ainge has probably been the 14th best GM of the 60 or so we've seen in the NBA in that span. I'm not sure how important that is for our purposes however, but maybe.
 

Also, there's a bunch of no-brainer high-VORP guys relative to slot in the last 14 years---Lebron, Paul, Davis, etc.---which will give some teams a very high score that really says nothing about their ability to draft (it's true that Marvin Williams instead of Paul is bad---but that's caught in the current design.  Paul being an apex-star and generating a lot of VORP doesn't change that he was a pretty clear pick when he was selected, imo.  Certainly, we know Ainge would have taken him there!)  If those aren't corrected for or smoothed some, there's a pretty material false-positive problem for 'good drafting'.  As noted subsequently, that is a real consideration.  Suspect the totals are not that big, so a couple 'hits' skews the data a lot (this is the 'Kawhi Leonard' problem).    Not sure the Durant ranking below lower-picked guys who became stars is a problem, though---to me, that's the model working reasonably well and setting a high bar for top picks.  

With respect to the Kawhi Leonard problem, I don't think that's an issue. I'm not using the average value of each pick to generate the baseline, precisely because of the Kawhi Leonard situation. You'd get some weird results, like 60th overall being pretty strong (Isaiah Thomas), and 2nd overall being worse than 3rd overall (Darko), etc... Rather, what I'm using is a non-linear regression between the draft spot, and the value. This results in a smoother chart that I think distributes the value pretty well (~0.8 R^2). This also deals with the issue of a single outlier in the second round screwing things up too much.
 
With respect to the no-brainer high VORP guys, that's not dealt with, aside from the fact that there's a pretty high baseline for those picks in the first place. Cleveland doesn't rate all that well by these rankings for instance, in spite LeBron. New Orleans grades out very well however, but mostly because of Paul (4th pick) rather than Davis. I'm not sure what else there is that can be done about that however. Insofar as you're going to try and objectively grade these things, I don't know how to avoid giving New Orleans credit for drafting Paul. Maybe just cross-slice the data without the top 4 picks or something (that's about where the dropoff gets steep). I'm open to suggestions however.
 
I will try and start a thread and post the actual data on this tomorrow however. There are some design questions I'm interested in fleshing out. For instance, just in terms of this Winslow/Kaminsky situation, I took a look just at the likelihood of ever getting a single superstar season out of a player based on draft pick (using the same sort of regression), and it showed that the 9th pick is about four times better than the 16th pick in that respect. That's in contrast to a straight expected-VORP analysis, which has the 9th pick something like twice as good as 16. It's possible that using a higher baseline would give a better picture of the actual value of a pick in this way, since VORP is above replacement, so any value above Phil Pressey counts. 

 

[radsoxfan]

 
I get it was quick and dirty; as I said, always good to put data into the picture...and also to step back and think about the problem, too.   Also, just to be clear, I'm not sure that Ainge is a great drafter and think challenging that conclusion is quite reasonable; I also think we need to be rigorous in thinking through the data before we draw conclusions,too

 
Agree with everything you wrote, and too think the analysis is difficult.
 
On sort of a tangent, the Celtics actually know much more about how well they rate players than we do.  I heard someone (maybe Ainge?), talking about how it's important that they go back over their entire draft board to see if they were doing a good job, and where they can improve,  not just look at the players they picked. The guys you end up getting is just a fraction of your draft board, no reason to artificially shrink the sample just to the guys who they end up taking. I guess this seems obvious, but for whatever reason I didn't really think about it this way before.
 
In 2012, they Celtics might have done a very good job in player analysis, but they ended up with Fab Melo as one of their picks just because one of their major misses happen to come up when they had a selection.  Similarly, it's entirely possible Ainge had a bad draft the years he picked Rondo and Al Jefferson/Tony Allen, but was bailed out by the fact that a few of his "hits" on the board ended up on his team. Clearly Ainge really liked Winslow this year, but isn't going to get any credit or blame for that when people look back at his drafting skills.
 
The more years you draft, the more this should even out of course.  But until I heard it talked about, I didn't really think about how much more these guys know about their own ability than we do.

 

[bowiac]

On sort of a tangent, the Celtics actually know much more about how well they rate players than we do.  I heard someone (maybe Ainge?), talking about how it's important that they go back over their entire draft board to see if they were doing a good job, and where they can improve,  not just look at the players they picked. The guys you end up getting is just a fraction of your draft board, no reason to artificially shrink the sample just to the guys who they end up taking. I guess this seems obvious, but for whatever reason I didn't really think about it this way before.

This is the "iceberg" issue, and it's well taken. However, it's not "artificially" shrinking the sample to the guys who they end up taking. That's the only sample visible to us - there's nothing artificial about that. However, there's only so much you can do about that, and it affects all forms of analysis equally. I think people are by and large cognizant that we only see a small part of what's going on, but that's not limited to attempts to model things statistically. (Nor is it limited to basketball - this issue applies to all aspects of life really).
 
To some degree, you need to work past it unless your approach is going to just be "in Ainge we trust." And that's fine if that's the approach, but I don't see how you can then distinguish Ainge's drafting from David Kahn's.

 

[radsoxfan]

This is the "iceberg" issue, and it's well taken. However, it's not "artificially" shrinking the sample to the guys who they end up taking. That's the only sample visible to us - there's nothing artificial about that. However, there's only so much you can do about that, and it affects all forms of analysis equally. I think people are by and large cognizant that we only see a small part of what's going on, but that's not limited to attempts to model things statistically. (Nor is it limited to basketball - this issue applies to all aspects of life really).
 

 
Maybe artificial isn't the perfect term, I'm not saying its something we have any control over or can fix in the analysis. I don't think there is any reason to believe the sample we see suffers from some systematic bias that needs to be accounted for.
 
Our data just lacks the power it could have and the power it does have within the Celtics front office. It's just interesting (at least to me) to think about how much more Ainge and Co. know about themselves than we ever will, barring some leaks of their draft board. 

 

[moondog80]

Here's an alternative analysis: I found a website (below) that lists redrafts back 2013-2004.  I have no idea how serious the methodology behind it is, but on first glance the results seems as valid as any.  It's at least a starting point.  The difference between a player's actual draft slot and their redraft slot can be a crude measure of surplus value, that is reflective of the talent that was actually available in the draft that year.  For most years, they only re-draft 30 picks, so if a player was not redrafted, how do we handle it?  I decided to assign a draft slot of 31, which I think is reasonable; you can more or less get the 31st best player in any draft or his talent equivalent off the street, so whether Fab Melo was really the 35th or 350th best player from his draft doesn't really matter.  Let's penalize them out of the first round and be done with it. 
 
Rondo + 19
Sullinger + 11
Allen + 10
Bradley + 10
Jefferson + 10
Olynyk +3
Delonte -1
Green - 11
Melo  -9
Jajaun Johnson - 4
Giddens -1
 
That's a surplus of 37 slots.
 
Then there's the second rounders.  Big Baby was huge a surplus at +27.  None of the rest made the re-draft, though it's worth noting that Leon Powe helped win a title and Ryan Gomes was useful for a little while.  I'm not sure how we should quantify this -- it doesn't seem fair to completely ignore the rest of the second round picks.  Maybe we can just ding them one point for each second rounder who didn't make the re-draft?  So that's plus 27 for Baby, and minus 7 for the rest.  Net gain of 20 for 2nd round, 57 overall.
 
I sort of made up my methods on the fly, but I tried not to bias things in favor of my prior notion that Ainge has been that he's a good drafter.  And maybe the draftsite people are secretly Celtics fans.  But this seems like at least a first step toward showing that he's pretty good at this.
 
http://www.draftsite.com/nba/redraft/2013/

 

[bowiac]

I sort of made up my methods on the fly, but I tried not to bias things in favor of my prior notion that Ainge has been that he's a good drafter.  And maybe the draftsite people are secretly Celtics fans.  But this seems like at least a first step toward showing that he's pretty good at this.

I don't particularly like this method for a couple reasons. First, it values all moves up a draft spot equally, which isn't correct. This results in someone like Avery Bradley (+10) being worth almost as much as drafting Kawhi Leonard (+13). This is why it makes sense to pay attention to the magnitude of a pick using something like VORP (or whatever measure you want). Getting the 10th best player with the 20th pick is not worth nearly the same as getting the best player with the 10th pick. These things are not linear, but using this method treats it as such.
 
Second, the "floor" creates problems. For instance, the 30th pick is all upside (can only go up essentially), while the 20th pick can go +19 or -11. In reality, there's not too much difference between the 20th pick and the 30th pick in terms of value. Using VORP or something reflects this (the difference between 20 and 30 using VORP is about the same as the gap between 6 and 7 by itself).
 
Third, ignoring second round picks is a problem, largely because there have been some star/very good players taken in the second round (Draymond Green, Marc Gasol, Chandler Parsons, Isaiah Thomas).
 
I also have concerns about the redrafts themselves (MCW ranks above Gobert or Dieng for instance), and it does seem to cause issues for this analysis with respect to Ainge in particular, but that's somewhat less salient.

 

[moondog80]

I don't particularly like this method for a couple reasons. First, it values all moves up a draft spot equally, which isn't correct. This results in someone like Avery Bradley (+10) being worth almost as much as drafting Kawhi Leonard (+13). This is why it makes sense to pay attention to the magnitude of a pick using something like VORP (or whatever measure you want). Getting the 10th best player with the 20th pick is not worth nearly the same as getting the best player with the 10th pick. These things are not linear, but using this method treats it as such.
 
Second, the "floor" creates problems. For instance, the 30th pick is all upside (can only go up essentially), while the 20th pick can go +19 or -11. In reality, there's not too much difference between the 20th pick and the 30th pick in terms of value. Using VORP or something reflects this (the difference between 20 and 30 using VORP is about the same as the gap between 6 and 7 by itself).
 
Third, ignoring second round picks is a problem, largely because there have been some star/very good players taken in the second round (Draymond Green, Marc Gasol, Chandler Parsons, Isaiah Thomas).
 
I also have concerns about the redrafts themselves (MCW ranks above Gobert or Dieng for instance), and it does seem to cause issues for this analysis with respect to Ainge in particular, but that's somewhat less salient.

I agree that this isn't the end score I came up with is flawed, but I still think this tells a pretty good story, qualitatively. He's had relatively few total whiffs, and when he has, it's been in the part of the draft where whiffs are pretty common. Most of the time, he does better than slot. And 30th pick, that should be all upside, right? The cost of failing to get the 30th best player is pretty small, because that's just a guy off the street. But if you get a stud, you've almost created something from nothing.

Could you post the data you came up with? I'm curious to see just how much the Melo and Giddens picks are weighing him down.

Seems that one problem with VORP is that it's really hard to draft someone who will massively outperform his slot VORP where Ainge is drafting. If you're good, you can get the Avery Bradleys and Tony Allens, but you have to be either really, really good or really, really lucky to get Marc Gasol. Maybe the thing could be to look at VORP relative to VORP available at that point in the draft?

On another note, we're probably measuring developmental ability in addition to drafting ability. Maybe Kawhi Leonard isn't so good if the Kings draft him? Going to be tough to separate the two though.

 

[bowiac]

I agree that this isn't the end score I came up with is flawed, but I still think this tells a pretty good story, qualitatively. He's had relatively few total whiffs, and when he has, it's been in the part of the draft where whiffs are pretty common. Most of the time, he does better than slot. And 30th pick, that should be all upside, right? The cost of failing to get the 30th best player is pretty small, because that's just a guy off the street. But if you get a stud, you've almost created something from nothing.

30 is all upside, I agree, but so is 20 really. That's a pretty key problem here. Those picks are barely different in terms of actual value, but they're as different as picking 11th to picking 1st in your scoring.
 

Seems that one problem with VORP is that it's really hard to draft someone who will massively outperform his slot VORP where Ainge is drafting. If you're good, you can get the Avery Bradleys and Tony Allens, but you have to be either really, really good or really, really lucky to get Marc Gasol. Maybe the thing could be to look at VORP relative to VORP available at that point in the draft?
 

There isn't much of a relationship between over-performing VORP and where you're drafting in my analysis. The reason is because the baseline is so much lower there, that getting an actually good player counts for a lot. I'll take a look at using VORP relative to VORP available however (player drafted VORP divided by best player remaining VORP? not sure).
Your last point (Kawhi on the Kings) is well taken, but pretty much impossible to separate for these purposes.
 
I'll do a post when I have time giving the actual data, but Fab Melo and Giddens are not weighing down Ainge much. It's mostly assumed you'll miss that late in the draft. What is weighing him down however is that VORP hates Avery Bradley - i.e., doesn't credit that as a good pick. I think VORP is correct about that (I don't think Bradley is a particularly useful NBA player), but that's the subject of some disagreement obviously.

 

[Cellar-Door]

30 is all upside, I agree, but so is 20 really. That's a pretty key problem here. Those picks are barely different in terms of actual value, but they're as different as picking 11th to picking 1st in your scoring.
 
There isn't much of a relationship between over-performing VORP and where you're drafting in my analysis. The reason is because the baseline is so much lower there, that getting an actually good player counts for a lot. I'll take a look at using VORP relative to VORP available however (player drafted VORP divided by best player remaining VORP? not sure).
Your last point (Kawhi on the Kings) is well taken, but pretty much impossible to separate for these purposes.
 
I'll do a post when I have time giving the actual data, but Fab Melo and Giddens are not weighing down Ainge much. It's mostly assumed you'll miss that late in the draft. What is weighing him down however is that VORP hates Avery Bradley - i.e., doesn't credit that as a good pick. I think VORP is correct about that (I don't think Bradley is a particularly useful NBA player), but that's the subject of some disagreement obviously.

I personally don't like VORP at all, and I think Bradley is a perfect example of where in particular Defensive VORP is broken. It takes things that already favor big men (DRtg) and then for no good reason position adjusts them to favor big men EVEN MORE. It's dumb, and it deeply discounts the value of defensive wings, which is stupid, as defensive wings who are capable of even decent offense are really valuable in the league.

 

[bowiac]

I personally don't like VORP at all, and I think Bradley is a perfect example of where in particular Defensive VORP is broken. It takes things that already favor big men (DRtg) and then for no good reason position adjusts them to favor big men EVEN MORE. It's dumb, and it deeply discounts the value of defensive wings, which is stupid, as defensive wings who are capable of even decent offense are really valuable in the league.

I don't believe this is true. As you can see in the FAQ behind the development of BPM, this was left out of BPM (VORP):
 

Height would have been significant (it helps the R2 by 0.01) but was not chosen based on principle. It does not directly reflect any performance on the court, and could bias results – Yao Ming could be a problem for a regression including it!

 
Further, some defensive wings are given a lot of credit by BPM. The stat loves Tony Allen and Kawhi Leonard for instance. It's obviously not perfect, and especially for defense, I prefer true adjusted plus/minus metrics. I'm using here for ease of calculation, but will cross-check with in RAPM stats as well. That said, while VORP may underrate Bradley defensively, I think he's a broadly overrated defensive player generally. My dislike of him long predates the existence of VORP.

 

[Cellar-Door]

 
I don't believe this is true. As you can see in the FAQ behind the development of BPM, this was left out of BPM (VORP):
 
 
Further, some defensive wings are given a lot of credit by BPM. The stat loves Tony Allen and Kawhi Leonard for instance. It's obviously not perfect, and especially for defense, I prefer true adjusted plus/minus metrics. I'm using here for ease of calculation, but will cross-check with in RAPM stats as well. That said, while VORP may underrate Bradley defensively, I think he's a broadly overrated defensive player generally. My dislike of him long predates the existence of VORP.
 

Ah, they've changed it. (or it is a different VORP than the one started by Kevin Pelton).

 

[bowiac]

Ah, they've changed it. (or it is a different VORP than the one started by Kevin Pelton).

This is completely unrelated to Pelton's work (his stat is also called WARP, not VORP. Annoying, I know).
 
I've posted about this before, but to summarize: BPM (which is the basis for VORP) is an attempt, via box-score stats, to replicate adjusted plus minus. What this means is you take a large dataset of adjusted plus/minus data (covering 14 years in this case), and then you regress box-score statistics onto that. It's a somewhat fundamentally different approach than WARP, or win shares, or PER, in that it's entirely "top down." It doesn't attempt to estimate the value of a steal, block, etc..., and add all that up. It just tries to build a stat that replicates RAPM as well as possible. You end up with terms in there like "sqrt(AST%*TRB%)", which don't mean anything by themselves. There is no inherent "value" to the square root of your assist rate multiplied by your total rebound rate. The only value comes in that this ends up modeling something about RAPM that other box score stats fail to do.
 
The upside of this approach is that it works very well at predicting how teams will perform going forward, in particular when players change teams. The proof is in the pudding so to speak, even if the terms don't make sense by themselves.

 

[PedroKsBambino]

Here's an alternative analysis: I found a website (below) that lists redrafts back 2013-2004.  I have no idea how serious the methodology behind it is, but on first glance the results seems as valid as any.  It's at least a starting point.  The difference between a player's actual draft slot and their redraft slot can be a crude measure of surplus value, that is reflective of the talent that was actually available in the draft that year.  For most years, they only re-draft 30 picks, so if a player was not redrafted, how do we handle it?  I decided to assign a draft slot of 31, which I think is reasonable; you can more or less get the 31st best player in any draft or his talent equivalent off the street, so whether Fab Melo was really the 35th or 350th best player from his draft doesn't really matter.  Let's penalize them out of the first round and be done with it. 
 
Rondo + 19
Sullinger + 11
Allen + 10
Bradley + 10
Jefferson + 10
Olynyk +3
Delonte -1
Green - 11
Melo  -9
Jajaun Johnson - 4
Giddens -1
 
That's a surplus of 37 slots.
 
Then there's the second rounders.  Big Baby was huge a surplus at +27.  None of the rest made the re-draft, though it's worth noting that Leon Powe helped win a title and Ryan Gomes was useful for a little while.  I'm not sure how we should quantify this -- it doesn't seem fair to completely ignore the rest of the second round picks.  Maybe we can just ding them one point for each second rounder who didn't make the re-draft?  So that's plus 27 for Baby, and minus 7 for the rest.  Net gain of 20 for 2nd round, 57 overall.
 
I sort of made up my methods on the fly, but I tried not to bias things in favor of my prior notion that Ainge has been that he's a good drafter.  And maybe the draftsite people are secretly Celtics fans.  But this seems like at least a first step toward showing that he's pretty good at this.
 
http://www.draftsite.com/nba/redraft/2013/

 
There's a little bit of a 'what problem are we assessing' question here.  If the problem is 'who drafts best' then it is really only fair to evaluate a GM based on the players available at their slot each year---in other words, it isn't Ainge's fault that Kawhi Leonard wasn't available to him, he can only pick the guy each time that is available to him.
 
On the other hand, if we're dealing with the problem "whose drafts have the most impact" then it matters how they manage the luck of the players available to them.  So if Ainge picks best guy available and it's Avery Bradley, and Spurs do too and it's Kawhi, that is a big win for Spurs because their guy is more valuable---even if they were each best guy at the specific slot.
 
I do think part of the thing with Ainge's draft is that he's had a lot of small to medium wins (guys like Big Baby) and really no huge wins (Rondo is closest to that).   That's certainly not a bad thing, but I'm not sure it makes him great either.

 

[PedroKsBambino]

This is completely unrelated to Pelton's work (his stat is also called WARP, not VORP. Annoying, I know).
 
I've posted about this before, but to summarize: BPM (which is the basis for VORP) is an attempt, via box-score stats, to replicate adjusted plus minus. What this means is you take a large dataset of adjusted plus/minus data (covering 14 years in this case), and then you regress box-score statistics onto that. It's a somewhat fundamentally different approach than WARP, or win shares, or PER, in that it's entirely "top down." It doesn't attempt to estimate the value of a steal, block, etc..., and add all that up. It just tries to build a stat that replicates RAPM as well as possible. You end up with terms in there like "sqrt(AST%*TRB%)", which don't mean anything by themselves. There is no inherent "value" to the square root of your assist rate multiplied by your total rebound rate. The only value comes in that this ends up modeling something about RAPM that other box score stats fail to do.
 
The upside of this approach is that it works very well at predicting how teams will perform going forward, in particular when players change teams. The proof is in the pudding so to speak, even if the terms don't make sense by themselves.

 
As several of us noted last summer, the challenge here is that who is on the court at any time is a variable, and this method doesn't really have a way to meaningfully address that variable....because it's trying to approximate a stat that tries to do that, and does it quite imperfectly.
 
This is not to say that it is not a useful quick way to assess many players, only that we need to be aware of gaps and outliers too.

 

[bowiac]

Agreed. I'll emphasize, there's not much reason other than use of use of use VORP/BPM for any active player. BPM is just trying to model RAPM. It does a pretty good job, but it is essentially just worse than RAPM (the very best it can do is model RAPM perfectly).
 
The point of BPM/VORP is really for players before the 2001 season - the kind of play by play data needed to generate RAPM didn't exist before then. It gives a way to compare Jordan to LeBron that's more useful than WS/48 or PER, etc... For active players however, RAPM is the way to go (just less convenient). I used VORP as a starting point, since this was intended to be quick and dirty, but it's not the end game.

 

[moondog80]

 
I do think part of the thing with Ainge's draft is that he's had a lot of small to medium wins (guys like Big Baby) and really no huge wins (Rondo is closest to that).   That's certainly not a bad thing, but I'm not sure it makes him great either.

 
 
Again, it's really, really hard to have a huge win when you draft so low.  In the past 10 years, there have been 150 selections to the All-NBA teams (first, second, third team).  You know how many times one of them was a guy Ainge had a chance to draft?  Seven times.  And two of those were Al Jefferson and Rajon Rondo, guys he did in fact draft (essentially draft in the case of Rondo).  The other 5 were David Lee twice, Marc Gasol twice, and DeAndre Jordan -- guys lots of people missed on.

 

[PedroKsBambino]

 
 
Again, it's really, really hard to have a huge win when you draft so low.  In the past 10 years, there have been 150 selections to the All-NBA teams (first, second, third team).  You know how many times one of them was a guy Ainge had a chance to draft?  Seven times.  And two of those were Al Jefferson and Rajon Rondo, guys he did in fact draft (essentially draft in the case of Rondo).  The other 5 were David Lee twice, Marc Gasol twice, and DeAndre Jordan -- guys lots of people missed on.

 
Agreed, and that is why this whole thing is tough to assess---lots of noise, and a small sample.    A sample which is influenced by the luck of who is available when you pick, too.

 

[bowiac]

Again, it's really, really hard to have a huge win when you draft so low.  In the past 10 years, there have been 150 selections to the All-NBA teams (first, second, third team).  You know how many times one of them was a guy Ainge had a chance to draft?  Seven times.  And two of those were Al Jefferson and Rajon Rondo, guys he did in fact draft (essentially draft in the case of Rondo).  The other 5 were David Lee twice, Marc Gasol twice, and DeAndre Jordan -- guys lots of people missed on.

I mostly disagree with the bolded, at least for these purposes. Getting someone like Chandler Parsons is a pretty huge win, even if he's nowhere near All-NBA. If you're grading based on relative value, where you draft isn't going to inherently make or break you.
 
There's a lot of randomness certainly, but the best picks by this metric haven't overwhelming been high picks.

 

[moondog80]

I mostly disagree with the bolded, at least for these purposes. Getting someone like Chandler Parsons is a pretty huge win, even if he's nowhere near All-NBA. If you're grading based on relative value, where you draft isn't going to inherently make or break you.
 
There's a lot of randomness certainly, but the best picks by this metric haven't overwhelming been high picks.

 
 
I said it was hard, not impossible.  How often does a second round pick turn out to be Chandler Parsons?  By my quick eyeball test he's one of the top five 2nd round picks 2009-2013 (I wanted a five year period, too soon to look at 2014...him, Draymond Green, D'Andre Jordan, Khris Middleton, Isiaih Thomas).  That's out of 150, so the top 3%.  That's a pretty high bar.

 

[bowiac]

I said it was hard, not impossible.  How often does a second round pick turn out to be Chandler Parsons?  By my quick eyeball test he's one of the top five 2nd round picks 2009-2013 (I wanted a five year period, too soon to look at 2014...him, Draymond Green, D'Andre Jordan, Khris Middleton, Isiaih Thomas).  That's out of 150, so the top 3%.  That's a pretty high bar.

Super quick and dirty, 6 year VORP minus expected VORP. Average player drafted 18th.
 

 
Yes, Ainge couldn't get LeBron, but many of these guys were either available when Ainge picked, or close to it (i.e., if you like someone, you can trade up, like the Spurs and Kawhi).

 

[Eddie Jurak]

I do think it would be interesting to evaluate each drafting GM against the best available talent for each selection. The problem with that is that a late round longshot (side note: my autocorrect wanted to turn longshot into longshoreman) pans out ends up making every GM look bad.

 

[bowiac]

I do think it would be interesting to evaluate each drafting GM against the best available talent for each selection. The problem with that is that a late round longshot (side note: my autocorrect wanted to turn longshot into longshoreman) pans out ends up making every GM look bad.

Yeah - this latter concern is why I'm not sure about this sort of method, but I'm coding that now to see what it looks like, since a number of people have suggested that method. Marc Gasol causes big issues here however for instance.

 

[moondog80]

Yeah - this latter concern is why I'm not sure about this sort of method, but I'm coding that now to see what it looks like, since a number of people have suggested that method. Marc Gasol causes big issues here however for instance.

There has to be a way to dilute the effect of one outlier. What if you compared a guy to say, the mean of the 10 or so players picked below him? I mean, the T-Wolves will score terribly for picking Corey Brewer 7th when Gasol went 48th, but there is likely not a GM in the world who would have taken Gasol 7th. He just wasn't a realistic option. I don't know, maybe it's too hard to define, but I have a conceptual issue with assigning a huge penalty for not making a move that absolutely nobody would have made.

 

[bowiac]

There has to be a way to dilute the effect of one outlier. What if you compared a guy to say, the mean of the 10 or so players picked below him? I mean, the T-Wolves will score terribly for picking Corey Brewer 7th when Gasol went 48th, but there is likely not a GM in the world who would have taken Gasol 7th. He just wasn't a realistic option. I don't know, maybe it's too hard to define, but I have a conceptual issue with assigning a huge penalty for not making a move that absolutely nobody would have made.

I don't like this, because 1) It's pretty arbitrary (10 players? why not 12, why not 6, etc...); 2) Some GMs do take totally off the beaten path guys (like last year). Hell, Rozier wasn't on some team's top 50, and Ainge just took him 16th.
 
But yes, this is why I don't like this method. Guys like Gasol are going to cause problems regardless (the Lakers get a ton of credit for drafting him, but if they knew he was going to be this good, they wouldn't have waited until 48 to get him). But there's major value in these moonshot guys, and it doesn't seem sensible to cap it. In real basketball teams, one Marc Gasol is worth a lot more than 10 Big Babies, even if there's more luck involved.

 

[PedroKsBambino]

Super quick and dirty, 6 year VORP minus expected VORP. Average player drafted 18th.
 

 
Yes, Ainge couldn't get LeBron, but many of these guys were either available when Ainge picked, or close to it (i.e., if you like someone, you can trade up, like   the Spurs and Kawhi).

 
You have Rondo coded for Phoenix---is that how you ranked things?  Leonard for Ind, so perhaps you've flipped both
 
Many of them were available when Spurs picked, too.  Saying 'you needed to pick the outliers' is a standard exactly zero teams can meet.
 
If we smooth out the two categories of outliers suggested above (and validated by what you shared) there's little chance Ainge is outside of the top 10.  I do not come into this believing he's a great drafter, but the data is telling that story, frankly..or at least that he's above-average at worst.   But still not sure what that really means, since being pretty good at mid-to-late first rounders gets a team nothing in the big picture absent the stars
 
In terms of dealing with outliers, I wouldn't cap it, but I do think we should be cognizant that with a sample size of 15 guys, allowing one significant outlier to drive the score likely isn't a fair description of true talent level (though it is, of course, of actual value delivered).  That's part of why just leaving the outliers feels problematic to me....it's a fair way to describe current value, but not true talent.

 

[bowiac]

You have Rondo coded for Phoenix---is that how you ranked things?  Leonard for Ind, so perhaps you've flipped both

No - they're not flipped. That's column there is just what team actually drafted these guys. When running rankings, I've adjusted for trades by crediting a team with "drafting" a player if he first plays a game for that team. This works for almost everyone (including Rondo/Kawhi), but does miss guys like Royce White (drafted, cut, later played for the Kings, so he gets credited to the Kings). I've manually adjusted guys like that, but may have missed a couple. It's rare for them to be impact players (i.e., I can't find any - even Whiteside played a few games for the Kings).
 

Many of them were available when Spurs picked, too.  Saying 'you needed to pick the outliers' is a standard exactly zero teams can meet.

That's not a standard I've set. The ranking I'm talking about is just relative to other teams.
 

If we smooth out the two categories of outliers suggested above (and validated by what you shared) there's little chance Ainge is outside of the top 10.  I do not come into this believing he's a great drafter, but the data is telling that story, frankly..or at least that he's above-average at worst.   But still not sure what that really means, since being pretty good at mid-to-late first rounders gets a team nothing in the big picture absent the stars

Can you explain this? I don't follow.

 

[PedroKsBambino]

Agree that guys like White won't really matter.
 
On the other point, I explained it above.  The two categories I speculated would be overly-impactful were Lebron types (high VORP guys who were also obvious high picks) and Leonard types (low picks who have been pretty valuable).  Those seem to be borne out by the list you posted.
 
Since the N we're talking about for draft picks isn't that high for most of the GMs in the sample, having 1-2 of those guys will lead to a high ranking, but that may or may not mean they were actually good---because it's just 1-2 guys.  Again, it's a question of whether you're trying to say 'actual impact' or 'ability to draft'  If the latter, we probably need to regress or otherwise adjust the impact of the outlier guys.
 
If Ainge was 14th without any adjustments for this, it's very unlikely he won't move up if we do reasonable adjustments.  Obviously I don't know without seeing the data, but instinct and knowledge of the sample sizes suggests that's the case.

 

[bowiac]

I see. Well, that raises an interesting question. I think you're almost certainly correct that Ainge would move up if we "flattened" the curve by capping the impact of particularly strong picks. And I am mostly interested for these purposes in "ability to draft." The question I have however is why "flattening" things like that would increase our ability to spot good drafting? I'm mostly fine doing that for LeBron, but really just him. I don't see why we should do it for Kawhi, Gasol, Millsap, Batum etc...
 
It seems like doing that sort of flattening would mostly reward GMs who took low-upside talent, rewarding guys for small "hits", but not crediting people with bigger wins for taking bigger chances. It seems to me that this actually is what Ainge is doing by the way - with the exception of Fab Melo, he's not chasing upside, but rather chasing competent low-end production. 

 

[PedroKsBambino]

I see. Well, that raises an interesting question. I think you're almost certainly correct that Ainge would move up if we "flattened" the curve by capping the impact of particularly strong picks. And I am mostly interested for these purposes in "ability to draft." The question I have however is why "flattening" things like that would increase our ability to spot good drafting? I'm mostly fine doing that for LeBron, but really just him. I don't see why we should do it for Kawhi, Gasol, Millsap, Batum etc...
 
It seems like doing that sort of flattening would mostly reward GMs who took low-upside talent, rewarding guys for small "hits", but not crediting people with bigger wins for taking bigger chances. It seems to me that this actually is what Ainge is doing by the way - with the exception of Fab Melo, he's not chasing upside, but rather chasing competent low-end production. 

 
Because the sample size is small and those guys are outliers who will skew---it's pretty much just a math issue, and understanding the difference between true talent and a smaller sample size variance.

Put it this way:  if we soften the impact of the huge value later picks and a team has actual skill at picking impact guys late it will still show up strongly---but if they only have a single guy like that, they may well not have the skill and just got lucky in hitting it once.
 
I'd guess Spurs don't lose any ground in such an adjustment, but a couple others who don't have that clear skill will. 

 

[bowiac]

Because the sample size is small and those guys are outliers who will skew---it's pretty much just a math issue, and understanding the difference between true talent and a smaller sample size variance.
Put it this way:  if we soften the impact of the huge value later picks and a team has actual skill at picking impact guys late it will still show up strongly---but if they only have a single guy like that, they may well not have the skill and just got lucky in hitting it once.
 
I'd guess Spurs don't lose any ground in such an adjustment, but a couple others who don't have that clear skill will. 

I'm sympathetic with the idea of regressing results to the mean, but it's more complicated than being "just a math issue." This has the result of systemically rewarding GMs (possibly Ainge) who eschew drafting impact talent in favor of low upside guys. I'm not sure how sensible an approach that is in the NBA, but I'm willing to be convinced.

 

[PedroKsBambino]

I'm sympathetic with the idea of regressing results to the mean, but it's more complicated than being "just a math issue." This has the result of systemically rewarding GMs (possibly Ainge) who eschew drafting impact talent in favor of low upside guys. I'm not sure how sensible an approach that is in the NBA, but I'm willing to be convinced.

 
Nothing I've suggested is about rewarding low upside guys at all.  What I've said is that we want to separate true talent from small-sample outliers, and if we don't smooth the impact of Leonard/Gasol etc. we will be unable to determine if someone is actually talented at picking guys late, or just got lucky a time or two.  This is why i said (and am confident it is) just a math issue once one understands the concern we're trying to manage.
 
As I noted above, if you don't care about true talent and solely want to measure actual impact, then your approach makes sense.

 

[bowiac]

Nothing I've suggested is about rewarding low upside guys at all.  What I've said is that we want to separate true talent from small-sample outliers, and if we don't smooth the impact of Leonard/Gasol etc. we will be unable to determine if someone is actually talented at picking guys late, or just got lucky a time or two.  This is why i said (and am confident it is) just a math issue once one understands the concern we're trying to manage.

Your "confidence" aside, can you explain why this won't result in rewarding low upside guys?

 

[PedroKsBambino]

Your "confidence" aside, can you explain why this won't result in rewarding low upside guys?

 
As I've said, it is a math problem---if you aren't going to do the math I kind of can't help that much, since you're the only one with the data.  
 
As I and others have noted, there's reason to wonder whether true talent in drafting is about the best guy in the slot you pick (which might mean some low upside guys rather than misses) rather than a bunch of complete misses and one hit that happens to be a large hit because of the relatively luck of the right pick that year being a guy who is great (rather than just 'ok').  if you model those both out and do the math on each given your methodology I think you may see the point that's been made....but without the data it's just an educated guess.
 
As noted, you seem interested in solving for 'greatest impact' across all picks by a GM which is fine---I think a different question is 'best drafter year to year' which you don't seem that interested in.  Which is obviously your choice.

 

[bowiac]

As noted, you seem interested in solving for 'greatest impact' across all picks by a GM which is fine---I think a different question is 'best drafter year to year' which you don't seem that interested in.  Which is obviously your choice.

You are mistaken. I'm interested in answering the "best drafter" question. What I don't understand is why you don't think smoothing the impact of guys like Leonard/Gasol won't have the impact of rewarding drafting low-upside (but high likelihood of being useful) players. That's what I keep asking, and you haven't given me an answer.
 
I also don't understand what "do the math" means in this context.

 

[PedroKsBambino]

You are mistaken. I'm interested in answering the "best drafter" question. What I don't understand is why you don't think smoothing the impact of guys like Leonard/Gasol won't have the impact of rewarding drafting low-upside (but high likelihood of being useful) players. That's what I keep asking, and you haven't given me an answer.
 
I also don't understand what "do the math" means in this context.

 
I've given you the answer multiple times, though...you just get stuck on your own coming in assumptions in some of these discussions.
 
If you care to figure out best drafter, rather than highest impact, I think you need to have a way to deal with the quality of all of their picks given the small sample...an average is generally not the way to do this, because of the impact of high-value outliers.   A huge number of statistical models operate with this premise, and you either get that application to this discussion or you don't.    I think others have noted the same, and you can agree or not.

 

[bowiac]

I give up. I don't understand your claim, and you don't seem interested/capable of explaining it.

 

[moly99]

Throwing out the outliers works in most statistical models because we are looking for the ability to deliver typical output. For the 38th pick in the draft, all we care about are the outliers like Chandler Parsons. Outperforming the average 38th draft pick by 3% still doesn't equate to an NBA level player. Avoiding busts and delivering steals is the job of a GM. You can't throw that out.

 

[Eddie Jurak]

I wonder if part of the right model would be looking at the odds of a big hit or a big miss.  
 
Drafting Tractor Traylor when there are 2 first ballot Hall of Fame talents left on the board is a level of "miss" that is not adequately captured by looking at the average value of a #6 pick.  So was the Celtics' decision to use two draft picks on players not named "Tony Parker" in 2001.  On the other hand, how much credit should Rick Pitino really get for having the good fortune to have The Truth fall into his lap.   
 
And I think year to year context matters - it doesn't make sense to ding GMs for drafting in a year with a weak draft class or reward them for a strong one.  
 
Ainge (and most GMs) passed on Draymond Green to draft Fab Melo.  That's a miss, but it would be a much bigger miss if there had been five Draymond Green caliber players on the board and Ainge passed on all of them to draft Melo.

 

[bowiac]

Ainge (and most GMs) passed on Draymond Green to draft Fab Melo.  That's a miss, but it would be a much bigger miss if there had been five Draymond Green caliber players on the board and Ainge passed on all of them to draft Melo.

I've thought about this, but can't think of a coherent way to make it work. Plus, you can only draft one player anyway.
 
a) Player Value minus value of the best player drafted after where you picked. (i.e., Melo's value, 0 VORP, minus Draymond's value, 6.3 VORP. = negative 6.3 VORP).
 
b) Player Value minus total value all better players drafted after where you picked. This ends up really killing some players however, as you might expect. Melo and Bosh look like bad picks because of Wade, but Kirk Hinrich looks as good as LeBron (best player available). Some of this has to do with the quirks of VORP, which likes Hinrich, and doesn't care for Melo. The best players here are guys like LeBron, but also Remon Van de Hare. This method does a pretty plausible job of picking up the worst picks however:
 

 
While here's the list of worst picks just taking VORP minus expected VORP at that drop spot (using a regression):
 

 

[bowiac]

One question I'm interesting in getting thoughts on is about what constitutes "value" from a draft pick. I don't mean the question of VORP vs. RAPM etc... I mean, what are you aiming for. Possibilities:
 
1) Total value seasons 1-4. These are the rookie contract years.
 
2) Career value. Bird rights, RFA, and other cap quirks mean there's a lot of value to drafting a good player because of "incumbency", and this lasts beyond the rookie contract.
 
3) Peak value (best season, average of best 3 seasons, etc...). RAPM hates Durant as a rookie for instance, like one of the worst players in the entire league. Lets say that's correct, and he was terrible. Who cares? Insofar as you're trying to win a title, you mostly just care about peak performance. The lottery exacerbates this of course - if Durant hadn't been so terrible, maybe Westbrook is somewhere else. More generally, almost nobody's rookie season matters. This isn't baseball or football, where impact rookies happen all the time. NBA impact rookies are super rare.
 
My current preference is to look at seasons 3-7 I think - that's about when you hope to get good seasons out of a player, both max guys, and sub-max guys. But I'm interested in other thoughts.

 

[wade boggs chicken dinner]

I give up. I don't understand your claim, and you don't seem interested/capable of explaining it.

I'm not sure I understand what either of you are saying but I'll take a guess.

I think PKB is asking - with respect to which GM is a better drafter - GM A who takes guys five or six times who represent a bonus over slot but doesn't get an impact player or GM B, who takes a whole bunch of flyers and hits on one or two just out of sheer luck?

Wouldn't your model reward the latter - although one could argue that the former is actually a "better drafter"?

 

[MainerInExile]

I'm not sure I understand what either of you are saying but I'll take a guess.

I think PKB is asking - with respect to which GM is a better drafter - GM A who takes guys five or six times who represent a bonus over slot but doesn't get an impact player or GM B, who takes a whole bunch of flyers and hits on one or two just out of sheer luck?

Wouldn't your model reward the latter - although one could argue that the former is actually a "better drafter"?

 
Right, but we have no idea what is luck and what is skill.  Small sample sizes may make 1 the expected value of hits even with skill.  But for all we know, the GM is throwing darts at a list of players.  Compromise: 0.5*(regressed model) + 0.5*(unregressed model).

 

[bowiac]

Wouldn't your model reward the latter - although one could argue that the former is actually a "better drafter"?

So, first, all those flyers are going to count against him here - they're not "free" as it were. But in the abstract, lets say you draft LeBron in the second round, and he's such a great talent that he overwhelms all the misses. Then yes, that guy would rate better than GM A.
 
The issue is:
 
1) We don't know to what degree that's sheer luck. Some of it probably is, but there's also a skill element. We regress something like BABIP to the mean because we have a pretty good idea what degree its luck, and what degree its not. I have no clue to what degree drafting Kawhi Leonard is luck/ Nor has anyone suggested a good way to find out. There's no way to use "math" to solve this problem without figuring out a method to figure out what we're regressing towards and how much.
 
2) By regressing to the mean (lets say 50%), you're going to end up rewarding GMs who consistently draft low upside talent. By doing so, they are more likely to get a series of small "hits", but losing out on the big wins. Now you're taking away part of the main downside of that strategy (the lack of impact talent). There is both skill in picking an NBA quality player, but also skill in having a good draft strategy in the first place. 
 
I think both these issues are pretty important. Issue #2 is particularly salient with respect to Ainge I suspect (as it seems he does draft a lot of low-upside talent). Given the importance of impact talent in the NBA, I'm not sure how smart that is.