With the 3GML draft still in our rearview mirror, I thought I’d take a stab at providing some draft grades. Tyler recently evaluated the 3GML team drafts within the context of the 3GML world, looking at team need, scheme fit, and value, with in-depth interviews of the 3GML general managers. There is certain insight that can only be provided through a qualitative analysis of each team’s situation in detail, and Tyler did an excellent job with his analysis. To complement his work, I thought I’d take a step back and look at the 3GML draft in the context of real NFL teams using a broader analytical approach.
There was a multitude of directions I could take this – I opted for the perhaps the simplest, being still a novice in the analytics world. I decided grades would be based on draft pick value. Specifically, I chose to look at the value of a team’s pick versus the value of a player’s estimated draft pick (EDP), or the pick where the player is expected to be drafted at / valued at versus the rest of the draft class. For instance, if Player A’s EDP is 15 overall, and he is drafted by a team at 20 overall, this is good value. But it had to be more than that, because if Player B is valued at 145 overall, and is drafted at 150 overall, that value is should not be equivalent to Player A’s. Thus, I weighted each pick to its respective trade value based on the Jimmy Johnson (JJ) table (we can argue the validity of that table another time, but at the moment, we still use that table for 3GML trades).
There are pros and cons to such an approach. At minimum, it provides a baseline by which to grade every team’s draft using the same metrics. However, such an approach is not situationally biased. For instance, team needs are not factored in here. If Chicago wants to take a TE, and the selection is good value based on EDP, it doesn’t matter if they already roster 8 TE’s – the value is in the pick. Introducing team needs into the equation is difficult, and adds a tint of subjectivity to the analysis (who’s to say Chicago doesn’t need 9 TE’s? … that’s a joke I promise). This approach also doesn’t currently factor in positional value; it is something that I hope to work more on in the future, but at present, I don’t have the historical draft and player performance data needed to objectively weight each position. So for now, grades are determined by EDP value vs. draft pick value.
The Consensus Big Board
While EDP is a familiar term, it isn’t universal. Someone has to establish a player’s EDP. We all have Big Boards, but those are subjective. I needed something objective. Thankfully, the folks at The Athletic did the work for me. They created a “Consensus Big Board,” aggregated from the rankings gathered across NFL Draft media analysts. The last version they updated contains the work of 68 different evaluators and forecasters. I use those terms specifically, as they state, “one type of analyst, the ‘forecasters,’ tend to have much more access to NFL and college personnel, which gives them information about injury concerns, character, off-field issues and behind-the-scenes information that could change our understanding of a player one way or the other. The other group, the ‘evaluators,’ rely on public data — primarily college game film and advanced statistics.” The Athletic creates a board for each. They suggest the Forecaster Board better reflects the actual draft than the Evaluator Board, which should be no surprise given the information they are gathering. However, the Evaluator Board, they claim, is better at projecting NFL success. The Consensus Big Board is a combination of both the Forecaster and Evaluator Boards – ideally the best of both worlds. Whether that is the case, only time will tell, but for the present, we can compare the performance of all 3 boards (Consensus, Evaluator, and Forecaster) to the actual draft position of players to see which is the best predictor of draft positions.
Establishing Board “Performance”
The first thing I looked at was the ranking of every drafted player on each board vs. their actual draft position. The results, shown above, are perhaps unsurprising. All boards were better at predicting players draft positions earlier in the draft than later, where there is significant scatter in the data. I broke this down further by looking performance per round by board.
The figure above compares the distribution of each board’s EDP vs actual draft position by round using box plots. Draft rounds are indicated by the dashed horizontal lines. Unsurprisingly, the 1st and 2nd rounds are pretty tight. The distribution of EDP versus actual draft locations largely fall within their respective rounds, although the distributions are skewed. The long tales at lower picks suggest several players were taken in the 1st and 2nd round with lower relative EDP (and yes, those outliers in Rd 1 are indeed Seattle’s new LB Jordyn Brooks). By Round 3, the EDP spread from 1st round to 6th round value for all 3 boards (I don’t know if that is an indictment of the boards or the NFL front offices…). Despite the spread, the median EDP remain within their respective rounds until the 6th round, at which time the Consensus Board appears to be the least accurate, while the Forecaster Board the most accurate at predicting pick location. However, this is purely a qualitative assessment.
To put numbers on this, I fit linear models for each boards EDP vs. actual draft location. Looking strictly at their ability to predict the actual draft location, the Consensus Big Board was best overall (R2=0.65, RMSE 40.6), followed closely by the Forecaster Board (R2=0.648, RMSE=40.9). The Evaluator Board lowest (R2=0.625, RMSE 42.2), but that should be no surprise. As we’ve established, the Forecaster Board is based on what people are hearing from teams, so, in theory, it should better reflect what teams will do on draft night. It may seem odd on the surface that the Consensus Big Board performed the best then, since it is a combination of both the Forecaster Board and the Evaluator board. However, think of it as the best of both worlds, averaging out the weaknesses in both the Forecaster and Evaluator Boards. While I include all 3 boards in my following analysis, final grades are determined by the better preforming Consensus Big Board (CBB).
NFL and 3GML Overall Draft Performance
With an EDP board in hand, team picks can be compared to EDP to establish value. Luckily, the folks at The Athletic also kept track of the 255 picks in the 2020 NFL Draft, and included it on their board, making my job easier. All I had to do was establish the value for each pick based on JJ trade chart. Now, Jimmy Johnson may have been a good coach, but he was no mathematician. His chart follows an exponential decay curve, but he rounded them out to make them clean numbers. Thus, they don’t follow a nice equation that I could use to ascribe pick value easily – I would have to enter 255 values in by hand, since the teams decided that they weren’t going to draft in order based on The Athletic Consensus Board and the actual picks were spread all over the board. I wasn’t going to do this, and although there was a workaround that maintained the JJ values exactly (involving coding up a script to locate the picks and create a new array of pick values rearranged in the order of the actual picks on the Consensus Big Board), I took the easy way out and just fit a function to the JJ value chart (at R2=0.998 I think we can all live with that decision). So pick values you see below won’t perfectly match what you had written down. Blame Jimmy…and my laziness.
I compared the player selected at each team’s pick (with corresponding JJ value) against the players’ respective EDP (and corresponding JJ value) on the 3 Big Boards to establish the value of each pick. I totaled this up for each team, both in total points and percentage of original pick value. To establish a grade, I repeated this, but then normalized the value difference between original pick value and player EDP value by the number of picks in the draft (255). What this does is set a baseline: if a team uses Pick 125 to pick a player with an EDP of 125, the score is zero, it is neither good nor bad value. If they pick a player with an EDP above or below the actual pick value, the grade will be positive or negative and scaled based on the JJ value difference of the pick and EDP. For example, the Miami Dolphins drafted OT Austin Jackson at Pick 18, but his Consensus Big Board EDP was 43, yielding a difference of 25 picks. They went on later to pick G Solomon Kindley, a player with an EDP of 163, at pick 111, yielding a difference of 52 picks. Their grade for the Jackson pick was -1.7 (bad), while their grade for the Kindley pick was only -0.2 (not that bad). While the Kindley pick could be viewed as more of a reach by pick number, the value between Picks 111 and 163 (JJ = 47) is much less than the value between Picks 18 and 43 (JJ = 430). Each team’s picks were graded as such and averaged to establish an Overall Draft Grade. I repeated this entire process for the 3GML draft picks and compiled all of the grades and values into the table below to show how 3GML stacks up against the actual NFL front offices.
Teams are ranked by Overall Consensus Grade. Since these numbers are not easily interpretable, I as established a GPA grade based on the Consensus Grade, and curved all GPA to the top grade (AZ Cardinals). Again, the Consensus Grade encompasses all other aspects shown on this table (forecaster grade, evaluator grade, pick value above / below estimated draft position), but I split them out for your viewing pleasure. One thing to note is the value that Buffalo got out of their draft with as little capital as they had (215.67% original value, vs. the poor value Seattle got out of their near league-average capital (32.78% original value). In comparison, all 3GML teams got 96% or greater value from their picks, with the 3GML Texans maximizing their draft capital up to 162.2% of original value. The table is fun to look at, so I won’t spend much time analyzing it here.
3GML Breakout
Instead, let’s look at each 3GML GM’s picks and draft grade. For each team, I broke out the board by pick and for fun, threw in the first 5 UDFA to see if we got any value there (all UDFA were ascribed pick 256 to keep it even between teams). Each team is discussed below in alphabetical order, with best value, worst value, and overall grade discussed. For simplicity, I did not factor in value for trades, assuming that the move up or down accounted for the value of the new pick. Therefore, when I say “original pick value,” I mean the value of the final picks for each team.
Green Bay Packers
Best Value: Edge Zach Baun
The Packers waited until pick 57 to draft the CBB’s 32 ranked player. Baun had to wait a little longer in real life to hear his named called (pick 74), but the Pack gladly snatched him up in the bottom of the 2nd. Edge was also a need for the 3GML Packers, so the Packers found that sweet spot where value = need. Baun turned out to be the best value of the 3GMLs draft filled with good value throughout.
Worst Value: QB Tua Tagovailoa RB Clyde Edwards-Helaire
I know the numbers say Tua Tagovailoa was the worst value, but I can’t get on board with that conclusion. The picks are weighted by the Jimmy Johnson Trade Value chart, which means the jump from 3 to 5 is significant. No one in their right mind would think it was “Bad Value” to take the 2nd best quarterback and 5th overall player at 3. The value of the QB position would negate this, but as I discussed above, I haven’t yet determined how to weight these grades by positional value. So instead, I’m choosing to ignore the data and look at what many analysts would say is bad positional value – drafting RB Clyde Edwards-Helaire at pick 34. The Chiefs snagged him at pick 32 in real life, prompting the 3GML Packer’s GM to trade up grab him. Turns out just picking him at 34 was poor value. Regardless, the 3GML Packers GM got his guy in Edwards-Helaire, who will contribute early, so the Pack can live with the value.
Overall Grade: B-
The Packers draft class netted a 2.6 GPA, equivalent to a B- on the curved grade scale. Retaining 99.2% of their original draft value is impressive, especially considering they had the 2nd most original draft capital (Miami was 1st and only maintained 74.5% value). This grade has them in a five-way tie for 16th with the Rams, Colts, 3GML Chargers, and Titans, or in other words, around league average grade.
Houston Texans
Best Value: WR Denzel Mims
Many had Mims projected in the first round, but he managed to slip into the waiting arms of the 3GML Texans at pick 56. Mims wouldn’t have to wait much longer in real life, being picked by the NY Jets at 59. The CBB had Mims ranked at pick 33, just outside of the 1st round, so snagging the 33st ranked player at 56 yielded excellent value. The 3GML Texans GM will also tell you that his value was truly better than indicate here.
Worst Value: DT James Lynch
The 3GML Texans GM identified James Lynch as one of “his guys,” so I doubt he cares that, analytically, this was his worst value pick. At only -0.2 CBB grade, the pick wasn’t much of a reach. The Texans took Lynch at pick 88, when he was valued 13 picks lower by the CBB. Either way, Lynch was viewed as a back of the 3rd round player, and that’s where the 3GML Texans snagged him.
Overall Grade: B
The 3GML Texans had an excellent draft, make no mistake, but entering with such little draft capital put them behind the other teams to begin with. By increasing their draft capital by 162.2%, the Texans secured a 3.1 GPA, good for a solid B and tied for 8 overall with the Carolina Panthers.
Miami Dolphins
Best Value: OT Josh Jones
Everyone was confused by OT Josh Jones’s tumble down the draft boards, but the 3GML Dolphins GM didn’t mind. He snagged the CBB’s 29th ranked player at pick 42, yielding the best value in their draft class. Jones would wait until the Cardinals snagged him at pick 72 in real life, but the 3GML Dolphins don’t mind. They got great tackle prospect for good value, and he landed in a great situation in real life.
Worst Value: QB Jalen Hurts
We are beginning to see a common theme amongst the 3GML draft picks. The worst value for each team seems to be the “my guys,” and QB Jalen Hurts was the guy tabbed as the future starter of the 3GML Chargers by their GM. The 3GML Dolphins picked Hurts at 31, but he was the 71st overall player on the CBB. Hurts ultimately went earlier than that, as a surprise 2nd round pick (53) by the Eagles, suggesting that his value to NFL teams may be closer to where the 3GML Dolphins valued him than previously thought. Either way, Hurts is in the new face of the 3GML Dolphins franchise.
Overall Grade: B-
The Dolphins’ GM always drafts his own way, but his own way was still good for a 2.6 GPA, tied for 16th with the Rams, Colts, Titans, and 3GML Packers. Retaining 96% of their original draft capital, the Dolphins register a league average B- grade.
San Francisco 49ers
Best Value: CB Kristian Fulton
Perhaps a surprise, but the 3GML 49ers pick of CB Kristian Fulton at 54 was better value than WR Ceedee Lamb at pick 14 according to the CBB. Fulton was the 28th ranked player by the CBB. Fulton fell to pick 61 in real life, where the Titans gladly drafted him to replace the loss of Logan Ryan. The 49ers didn’t necessarily need another CB, but Fulton was valued even higher than his CBB value by the 3GML 49ers GM, who gladly moved up to snag him at 54.
Worst Value: QB Jordan Love
This wasn’t a “my guy” pick by the 3GML 49ers GM, who made it clear he did not value Love enough to take him at pick 13. Instead, the need to bring in young QB talent prompted the 3GML 49ers GM to gamble on the talent of Love at pick 26. The CBB ranked Love at Pick 35, although he would go at pick 26 to the real Green Bay Packers, and the 3GML 49ers GM could not be happier with Love’s landing spot.
Overall Grade: A-
The 3GML 49ers managed to land a good value draft according to the CBB, ending up with a 3.6 GPA (A-) good for 4th in the league. The 3GML 49ers GM isn’t shy about saying he approaches the draft similar to the CBB, accruing observations from many sources (although certainly not 68 sources like The Athletic!) and modifying his board based on scheme fit and personal preference. According to the CBB, the 3GML 49ers approach paid off, at least in the short term.
Conclusions
This was my first foray into an analytical draft grade for the real and 3GML teams. The results yielded interesting findings and in large part corroborate observations by many sources (particularly on the worst draft classes). The shortcoming of the approach are also apparent, as positional value is required to temper the negative grading of QBs for instance. I hope to have a method in place when the 2021 NFL Draft rolls around next year. Regardless, the analytical draft grades highlight the 3GML GMs' draft acumen in the 2020 NFL Draft!
