The predominant input in my determination is subjective scouting report data. It tends to produce our strongest indicators for player performance. It's not gospel, certainly -- players who have been regarded as reaches have turned out to be good and worth as much as more than their pick value. But it happens less often, to a substantial degree, than to players who have been picked near the consensus or are regarded as "steals" by the consensus. I looked at Wallace's holistic athletic profile to see what my loose athletic model has to say because I was curious given his mediocre (but not poor) agility numbers. You're correct, my model regards him as an elite position-specific athlete at offensive tackle. These players tend to succeed more than poor athletes. But that I think only moves the needle a little bit. Athleticism matters, but not enough for me to jump a player's round grade by more than one even for astounding athletes at positions where it dispositively matters, much less for a position where it matters less than at EDGE or 3-Tech.
For example, Wallace's athletic score is next to 2021 Consensus Board reach Dan Moore (picked 128th, ranked 194th). He has had, according to Steelers fans and PFF, a very underwhelming career. Steelers fans seem to be more down on him than PFF, even. They both have long frames, good size and experience playing left tackle. Moore had better agility scores and Wallace had better explosion scores and the model is crude enough to not make that distinction (and this could be crucial to the LT vs RT discussion you are having) but the example makes the point -- you can reach for athletes and be wrong. Other high-level offensive tackle athletes who have struggled include Kaleb McGary, Andre Dillard, Jedrick Wills, Lydon Murtha, Nick Zakelj, Marcus McKethan, etc.
As an interesting sidenote, I looked through the 2021 draft class to see which reaches made the Pro Bowl but found that every single player from that draft class picked after the second round who made a Pro Bowl was regarded as a substantial steal by the Consensus Big Board (Landon Dickerson, Creed Humphrey, JOK, Amon-Ra St. Brown, Talanoa Hufanga). That leaves the first round to produce any Pro Bowl reaches and there was only one: Mac Jones.
I've found that substantial reaches do better when it's an elite athlete vs a non-elite athlete, that part is true. The effect is somewhat small but likely to be statistically significant. The evidence that a player with a lot of visits but ranked low on a board is underrated is somewhat compelling to me. But other elements of his profile do not suggest the profile of a player who generally overperforms -- things like large disagreements about his playing ability (he has low variance in rank), a substantial absence from many boards (suggesting an overlooked player, he has been ranked by 50 different analysts) or pre-draft reports of NFL favorability -- e.g. the anonymous scouting tidbits we get from Bob McGinn (McGinn's survey of scouts ranked him outside of the top 12 tackles), Jeremy Fowler, Bruce Feldman, etc. Indeed, the reporting has suggested that scouts have pushed back against the Wallace pick in the instances where reporters have checked in on their feelings about the pick.
Do I think that Wallace is ultimately better than his consensus ranking of 186? Yea, I do. Do I think he is so much better than 186th to be justified as a selection at 68th? Not particularly. His absolute highest ranking was 80th. Among the 25% most positive rankings he received, his average rank was 117th.
These are ultimately an aggregation of other people's opinions. These people have, in aggregation, largely demonstrated their accuracy. You don't have to trust them or believe them, they can and have been wrong.
Typo. It's been fixed to a B.
Yes, I've been concerned about this and I think about it every time I add a new board to the pile. Largely, I "recruit" new boards either by looking for organizations whose boards I've pulled from in the past (for example, the Huddle Report added five new boards this year, primarily to house successful analysts whose previous stops have been shuttered) or who have demonstrated some level of success (to bring up THR again, I look at the top performers in predicting the top 100 to see if any new names pop up that I did not include). I do let people email or DM me submissions, but I don't just include them willy-nilly, there is some level of rigor that I need to appreciate happening before they are included. That either means scouting reports or YouTube videos or even tweets demonstrating some level of familiarity with the players being discussed. I added Matt Fitzgerald's board last year after some hesitation because he has scored well in the past. I'm not confident many people have heard of him and his first big board was originally published at a publication not traditionally known for its rigor -- Barstool Sports. But Fitzgerald scored well in THR's Top 100 every year and looking through what he's published, it's been generally items that suggest he's familiar with these players.
I've noticed that as I've added more boards, the Consensus Board gets better at the various tasks I demand of it, whether it's to have a comprehensive database of players so that there isn't a single player called who I don't have data for (I don't want a repeat of missing out on Justin Rohrwasser or Quincy Williams) to get closer to the NFL's valuation of players or to improve in predicting player outcomes. Moving from 40 to 60 and from 60 to 80 has improved the board's overall accuracy.
