Advanced NFL Stats

An Underdog Wins with Aggressive, Risky Football

2009-06-25T15:17:00.009-04:00

No, not that kind of football.

A couple weeks ago I wrote a post about how underdogs can increase their chances of winning by employing a high-risk, high-reward strategy. It seems that’s just what the US soccer team did in their recent upset against the globe's top team, Spain.

According to this analysis by the Journal’s Carl Bialik, the American team uses long aggressive passing, looking for fast-break scores, instead of using a more typical ball control offense. This opens up opportunities for a quick goal, but usually results in the opponent controlling the ball on the US side of the field (or pitch, if you’re a ‘football’ aficionado). As long as the goalie has a good game, and the defense gets some breaks, the strategy works.

It makes sense because the US team has nothing to lose. No one expects them to go very far in World Cup play, so they can afford to use a risky gameplan without being humiliated if they end up losing 4-0.

Why There Is So Much Holding

2009-06-24T08:00:00.005-04:00

Nothing upsets a coach more than a offensive holding call in the middle of an otherwise productive drive. The play, usually a good one, is nullified, and the penalty moves the line of scrimmage back 10 yards. What was a promising 2nd and 5 becomes a difficult 2nd and 15.

Yet holding calls are frequent, which suggests there's obviously something useful about holding. For passing plays, the alternative is often a sack, which is bad in all kinds of ways. Plus, not all instances of holding are called. I'm sure if you polled defensive lineman, they'd say less than 10% of holds are actually flagged.

So I wanted to know, "for passing plays, what's the break-even detection rate for a hold which would make it worthwhile?"

It's a complex question with lots of variables, so let's isolate some. First, let's define the utility of "worthwhile" as based on the probability of converting a first down. Consider a general 2nd down and 5 situation. Typically, an offense in that situation that calls a pass will convert for a 1st down 71% of the time. We'll note this as P_1D = 0.71.

An offensive holding penalty negates the play and penalizes 10 yards from the previous spot, forcing a 2nd and 15. That makes the chances of a 1st down considerably lower. The probability of a 1st down "given a hold" is P_1D|Hold = 0.20.

For all 2nd and 5 pass plays in which there was no sack, the probability of conversion is P_1D|NoSack = 0.73. But for all 2nd and 5 plays that resulted in a sack, the probability of conversion is P_1D|Sack = 0.30.

In order of preference, you'd rather have neither a sack nor a hold (0.71), then a sack (0.30), and lastly a hold (0.20). But not all holds are called. I'm not sure what the detection rate really is, but we can solve for what detection rate would make a hold worthwhile.

For now, let's assume that if the pass rusher beats his blocker, he will cause a sack 100% of the time. And let's call the ref's holding detection rate "x." The break-even detection rate could be found with a simple linear equation:

P_1D|Hold(x) + P_1D|NoSack(1-x) = P_1D|Sack

Solving for x, we get:

.20(x) + .73(1-x) = .30
.20(x) + .73 - .73(x) = .30
-.53(x) = -.43
x = .81

So assuming that a defender that beats his blocker would always sack the quarterback, the blocker should hold him whenever he believes the probability of detection is lower than about 0.81. In other words, he'd get away with it 1 out of 5 times. It's understandable why a blocker would intentionally hold a pass rusher in this situation.

But pass rushers who beat their blockers don't sack the QB 100% of the time, so let's generalize the equation. Call the probability of a sack given the defender beats his blocker "y." The break-even equation now becomes:

P_1D|Hold(x) + P_1D|NoSack(1-x) = P_1D|Sack(y) + P_1D|NoSack(1-y)

Simplifying, we get:

.20(x) + .73(1-x) = .30(y) + .73(1-y)
.20(x) + .73 - .73(x) = .30(y) + .73 - .73(y)
-.53(x) = -.43(y)
x = .81(y)

The bottom line is that the probability of detection at which committing holding is worthwhile is when it is about 4/5 the chance a pass rusher will get a sack if he beats his blocker. For argument's sake, say that a pass rusher in the backfield gets a sack half the time. The probability of detection would need to be below 0.4 for the hold to make sense. It all boils down to the graph below:

So all a blocker needs to do is quickly solve the equation above immediately after the snap, given his estimate of...I'm just kidding. Of course I don't expect anyone to use math to make decisions in the heat of battle, but this analysis does explain one reason why we see so much holding. There are other complicating considerations too. A pass rusher could miss the sack but hurry the pass, causing an incompletion or worse. There are all kinds of possibilities. But ultimately, despite the apparent harshness of the penalty, the infraction is not always called, and in many cases can be worth the cost.

Note: Data is from the 1st quarter of all NFL games 2000-2008. Other quarters are excluded to eliminate the effect of "end-game" plays--hurried plays at the end of the halves, desperation plays by trailing teams, and clock-burning plays by leading teams.

Best Games of the Decade

2009-06-22T08:00:00.011-04:00

Now that all NFL games since 2000 have been added to the Win Probability Archive, we can step back and take an inventory of some of the more special games in recent years. I've created a simple search tool for finding many of the most compelling games of the decade.

There are many things that make a game special. Any game with playoff implications is more interesting than one between mediocre teams, and playoff games themselves are obviously critical. But many of those games are just plain boring. They're sometimes duds, decided by the end of the 1st quarter.

I wanted to know what the most exciting games were purely between the sidelines, what were the biggest comebacks, and which teams played the most dramatic football. That's why I created two new indices--Excitement Index (EI) and Comeback Factor (CBF). Admittedly, these stats are purely from a spectator's perspective, and would have very little application to the game itself. But hey, it's fun.

Comeback Factor

The comeback index was easy. For any given game, the 'CBF' is based on the lowest win probability at any point for the ultimate winner. To make bigger comebacks have bigger CBFs, I made CBF be the inverse of the lowest WP.

For example, if a team is down by 10 with 10 minutes left in the 4th quarter, they'd have around a 0.13 WP. This means the trailing team has a 1 in 8 chance of winning, and the CBF is therefore 8. A team that comes back from a 0.01 WP, would have a CBF of 100, the largest possible.

You might be tempted to say that CBF should factor in the lateness of the comeback. Certainly, a comeback in the final minutes is more dramatic than one staged in the 3rd quarter. I agree, however WP already factors that in. A 17-point lead in the 2nd quarter has an equivalent WP as a 2-point lead late in the 4th quarter.

Excitement Index

"Excitement" was harder to measure. Unlike measuring comebacks, there is no single true measure of excitement, and different people can have different definitions. I tested a few different methods, including several suggested by commenters, and ultimately chose a method that I think is both effective and straightforward.

EI is simply the sum of the WP graph's movement throughout a game. That's it. Despite the simplicity, this method captures much of what makes a game interesting. Games with large swings in WP will end up with large EIs, while blow-out games where the WP quickly climbs to 0.95 for one team will have smaller EIs. That same blow-out, but where the trailing team climbs back into contention will have a larger EI. (Credit goes to eje100, JMM, and NeilC for first suggesting similar methods.)

What about measuring closeness? The closeness of a game is obviously an important part of how compelling it is. And EI captures that too. The closer the game is to a 0.50 WP, the more magnified the WP movement becomes for any given play. For example, a 40-yd pass to the 10 yd line when the score is 30-6 is going to move the WP by barely 0.01. But that same play when the score is tied will move the WP by 0.25 or so, depending on the time remaining.

But games with more plays will obviously have a higher WP. Shouldn't EI account for the number of plays by using the average WP movement? I say no. A fast pace helps make a game exciting. Offenses furiously trying to score as quickly as possible is fun to watch. Pace counts. Plus, overtime games would tend to have the most plays, and therefore the higher EIs. And that's what we'd expect from an OT game. If 'sudden death' is anything, it's exciting to watch.

The Best Games

Below is the search tool, and here is its permanent home. Just enter a year, a team, and whether you want to rank games by excitement (EI) or comeback (CBF). Or you can select 'any year' or 'any team' to find the most interesting games for the entire league in any year, or for the entire decade.

The most exciting game of the decade? Would you believe a meaningless 13-10 game in December 2000 between the Bills and Patriots? Neither would I, until I saw the graph. Happy clicking...

Win Probability Graphs: 2007 Playoffs

2009-06-02T11:11:00.003-04:00

+0.19 for Tyree’s catch.

+0.41 for the TD pass to Burress.

Sadly, no one will remember the 2-yd gain by Jacobs on 4th and 1 to keep the drive alive, but that play had a Win Probability Added (WPA) of +.21. If Tyree doesn’t make the catch, the drive is still alive--it was ‘only’ 3rd down. If Jacobs is stuffed—that’s all she wrote.

Of course, there’s no good way to quantify the style points for Tyree’s miraculous grab or Manning’s escape from the sack.

One of my major goals this off-season is to create win probability graphs for every NFL game since 2000. I'm starting with the 2007 playoffs, one of the most improbable championship runs ever. The New York Giants defied the odds in four consecutive games, never once favored to win. Yet somehow they slayed the dragon, the sport's most formidable offense in its history.

I'll be rolling out more than 2000 games over the next several days. Each graph has complete play-by-play descriptions. Just roll your cursor over the graph.

Also included are some new statistics. Comeback Factor (CBF) is simply the odds against the team that ultimately wins at their darkest moment. Excitement Index (EI) [boy, does that need a better name--I'll take suggestions] is how exciting the game was. Think of it as an EKG or Richter Scale for a game. It's the sum of all the movement in the graph. Blowouts are flat-lines and have relatively no movement, while close, high scoring games are the most exciting. Close, but low scoring games will be right behind.

In the play-by-play descriptions you might notice a stat labeled "LI." That's the Leverage Index, a concept borrowed from the sabermetric community and Tom Tango in particular. LI measures how crucial a particular game situation is toward the outcome. This should be an interesting new way to look at each play, and I'll explain it fully in a forthcoming article.

For now, keep the year menu on 2007. The playoff teams that year were the Colts, Pats, Giants, Jags, Titans, Steelers, Packers, Seahawks, Redskins, Cowboys, Bucs, and Chargers.

There are still a few hiccups with the graphs, usually due to errors in the NFL gamebooks I use to create them. Comments and suggestions are more than welcome.

wp.advancednflstats.com/nflarchive.php

Injury Rates and An Extended Season

2009-06-01T20:50:00.007-04:00

At the recent owners meeting, the NFL disseminated a study that concluded an increase in the season schedule from 16 to 18 games would not increase injury rates. The report caught a lot of criticism as a halfhearted attempt to obscure the toll a longer season would take on the players. Judy Battista of the New York Times and Mike Reiss of the Boston Globe both point to flaws in the study.

But I suspect there is a fundamental misunderstanding about what the report says and how it's being interpreted. All I really know about the report is that it says, "the NFL's injury rate doesn't increase at the end of the season." There is no doubt a longer season would result in more total injuries. The bigger question is how many more injuries--does the injury rate itself increase?

Much of the criticism of the study focuses on the use of team injury reports, well known for their deceptive omissions. In an excellent article, Bill Barnwell at Football Outsiders found an additional flaw in the study. It left out players who go on the IR. Before you consider players on the IR, it appears that the injury rate peaks at week 10 before it decreases for the remainder of the season. Barnwell explains why this isn't really the case.

Since team injury reports are notoriously unreliable, the best information is actual games missed. Thankfuly, Barnwell provides that data in his article, and it's very interesting stuff. When you factor in the IR, the number of games missed climbs steadily. He concludes, "the data looks totally different, and in a bad way for the NFL..."

The way I see it, however, is that the NFL report is right, no matter what the intent was of its authors. There is no increase in injury rates toward the end of the season. The injury rate is effectively linear. Of course, as the season wears on, the number of players unable to play due to injury will accumulate, creating an upward climbing injury total. Once you go on the IR, you don't come off. This cuts to the heart of the debate about whether players become increasingly susceptible to injury as the season, along with the number of cuts and collisions, wears on.

Here is a graph of the data included in the Barnwell article.

The blue line is the games missed by roster players (those not on the IR). Except for the uptick on the final week, when playoff bound players nurse their wounds and everyone else has their bags packed for the Caribbean, it's very steady. The green line is the number of games missed by IR or physically unable to perform (PUP) players. Note how its slope steadily increases. The red line is the combination of the injured roster players and IR/PUP players.

Here's what I take away from this data. Players on the IR increase at a (very) slightly exponential rate--specifically it's:

#IR = 0.006w² + 0.1w + 1.6, where w=week.

That .006 term is extremely small, and when combined with the negative camber of the blue line, results in a very linear total, (especially when week 17 is thrown out, although you don't need to.) [Note: By the way, the slight non-linearity of the increase is evidence, however tiny, for the notion that players become more susceptible to injury as they endure the season.]

Ultimately, the total number of players who miss games due to injury is indistinguishable from a linear line (r-squared of .97). Its increase is exclusively due to players going on the IR, which is a one-way check valve.

So will there be more players missing games at the end of the season if the NFL adds two more games? Of course. But it won't be Iwo Jima out there. No explosion of wounded players with "cascading" injuries. It will be a demanding, grueling, even cruel extra two games for the players, but it would barely be noticeable to the fan and to the game itself. I suspect that's what the NFL report is trying to spell out. Even counting the uptick in the final week, each team would average an extra half a missed player by a potential week 19.

Personally, I'm against lengthening the season for a lot of reasons. The nerdiest is that there is a mathematical elegance to 2 conferences, 4 teams per division, 8 divisions, 16 games, 32 teams, and 256 games per season. Please, no 17th game or 33rd team--I'd have to redo all my algorithms and equations! Actually, I just think 16 is plenty. The fewer the number of games, the more unpredictable the season, and I like that.

Michael Vick Was a Better QB Than You Think

2009-05-28T08:00:00.007-04:00

Before Michael Vick's run-in with the law, he was widely considered an exciting and elusive quarterback with below-average throwing ability. Usually, his poor completion percentage is cited as evidence of his lackluster passing ability. It seems that reputation is still with him. Case in point is this excerpt from Peter King's most recent post:

Well, as a quarterback, Vick was decidedly mediocre in his four full seasons starting for the Falcons. His completion percentages when he started at least 15 games -- 54.9, 56.4, 55.3 and 52.6 -- were poor; he had 36 fumbles and 38 interceptions in his last 46 starts.

One thing is certain--Vick is (or was) an unconventional QB. He was a breakaway runner without peer, and the normal rules of quarterbacking don't apply. The vast majority of Vick's 529 career runs were scrambles on pass plays. When a conventional QB progresses through his reads, he looks at WR#1, WR#2, TE, and then dumps off to a RB. But Vick, on the other hand, doesn't bother with the dump off. He is the dump off.

Most of Vick's 52 running yards per game can really be credited as passing yards. They were yards gained on pass plays in passing situations. He averaged 7.1 yards per run. Compare that to the NFL's 5.0 average net passing yards per attempt and 4.1 rushing yards per attempt. And remember, there is never a risk of interception if he tucks the ball and runs. If you factor in his sacks into his rushing average, it becomes 5.1 net yards per run, which is still good. But if we do that, it would make his net passing yards per attempt much higher. After all, we can only count his sack yards against him once. Otherwise, it's double jeopardy.

So Vick's pass completion stats aren't padded by lots of rinky-dinky dump offs. David Carr actually led the NFL in completion percentage in 2006 with a gaudy 68.3%, only to lose his job the following off-season (to Vick's backup nonetheless).

To get an idea of how deep Vick was throwing compared to his league counterparts, we can look at yards per completion. In 2006, the most recent year Vick played, the league's other top 30 QBs averaged 11.4 yards per completion, while Vick averaged 12.1.

Vick's receiver corps in Atlanta was never known as particularly talented. If we remove receiver YAC from the equation, the numbers are even more favorable for Vick. His Air Yards, the yards a complete pass travels in the air forward of the line of scrimmage, is impressive. The top 30 other QBs in 2006 averaged 6.3 Air Yds per completion, while Vick averaged 7.7.

Yes, his interception and fumble rates are higher than you'd like, but those too should be considered in the context of the depth of his throws and the frequency with which he runs. So although Vick's passing stats, especially completion percentage, appear sub-par, that should be expected, and they are somewhat offset by greater gains for each completion. I'm not claiming he's great, or even above average, just better better than his conventional stats suggest.

Are NFL Coaches Too Timid?

2009-05-26T08:00:00.007-04:00

Risk is at the heart of football strategy. Aggressive, risky gameplans should result in boom-or-bust high-variance outcomes, sometimes scoring lots of points but sometimes scoring very few. Conservative gameplans result in relatively consistent low-variance outcomes. Teams would more likely score close to their average score.

In this post, I’ll look at what high and low variance strategies would look like in terms of point totals and how they affect each team’s chances of winning. I’ll also compare the theoretical strategies to the actual distributions in the NFL. We'll see why NFL coaches should be more aggressive when they're the underdog.

High Variance Strategy in Basketball

Some time ago, I came across an article posted by basketball researcher Dean Oliver that analyzed high and low variance strategies for the NBA. Oliver calculated the win probability of each opponent according to the mean and standard deviation (SD) of each team’s scoring tendencies. SD represents the degree of variance. The more aggressive and riskier the strategy, the higher the SD will be. For example, a basketball team that shoots lots of 3-pointers would have a high variance.

The key to accurately modeling basketball is realizing that each team’s score is correlated with that of its opponent. The pace of a basketball game ties each team’s score together, and there is a high level of covariance. When one team scores a high number of points, the other team will tend to score more too. Game scores are interdependent.

In Football

Recently the Smart Football blog illustrated the advantage of high variance strategies for underdogs. A high variance strategy increases an underdog’s chances of winning but comes with the cost of also increasing its chances of being blown out.

In the NFL as a whole, visiting teams average about 19 points with a SD of 10 points while home teams average about 23 points with a SD of 10 points. But unlike basketball, football opponent scores are negatively correlated. This makes intuitive sense because the better one team does, the worse the other should do. If one team gets lots of first downs and doesn’t commit turnovers, its opponent will usually start drives with poor field position, and vice versa. The covariance between NFL opponent scores is -1.9 points-squared.

If NFL scores were normally distributed, this is what the typical score distribution would look like. The visitor scores are in red and the home scores are in blue.

We can calculate each team's chances of winning by summing all the probabilities with these distributions and factor in the covariance using Dean Oliver’s method. This estimates that the home team wins 56.5% of the time, which happens to be exactly the NFL actual home field advantage.

Disclaimer

There’s one problem. NFL scores are not normally distributed, primarily due to its unique scoring, which typically comes in chunks of 3 or 7. Here is what the actual distribution of scores looks like.

The good news is, if we group the scores into bins of 7 points, we get a quasi-normal distribution. (Technically, it may be more of a gamma or Poisson distribution.) I’m going to stick with normal distributions to simplify the math and to better illustrate the concepts I want to convey.

Demonstration

Here’s why underdogs should play aggressive and risky gameplans. Take an example where one team is a 7-point favorite over its underdog opponent. Say the favorite would average 24 points and the underdog would average 17 points. With a SD of 10 points for each team, the underdog upsets the favorite 31.5% of the time. The favorite’s scoring distribution is blue and the underdog’s is red.

But if the underdog plays a more aggressive high-variance strategy, increasing its SD to 15 points, it would upset the favorite 35.3% of the time.

Note that I haven’t increased the underdog’s average score in any way, just its variance. The increase in its chance of winning results due to more of its probability mass moving to the right of the favorite’s mean score of 24. In fact, the higher the variance, the wider the probability mass will be spread. Consequently, more mass will be to right side of the favorite’s average score. But more mass will also be to the left, meaning there is a higher risk of an embarrassing blowout.

Even if employing a high-variance strategy is non-optimum, it can still help an underdog. In other words, even if an aggressive gameplan results in an overall reduction in average points scored, it often still results in a better chance of winning.

The next graph plots the scoring distributions of just such a scenario. Like before, the favorite’s average score is 24 with a SD of 10. But this time the underdog’s average is reduced from 17 to 16. The increase in variance still results in a slightly better chance of winning despite its overall reduction in average points scored. In this case, it's 33.2% for the underdog.

What about the favorite? Should it increase its variance in response to an aggressive underdog? No. Ideally it should play as consistently as possible. The lower the variance the better for the favorite. The next example shows a favorite playing a low-variance game with an average of 24 points and a SD of 5 points. The underdog is playing conventionally with a 17 point average and 10 point SD. The result is an increase in the favorite’s chances of winning from 69.5% in the original example to 73.0%.

And if the underdog plays an aggressive high-variance game, the low-variance strategy is still better for the favorite. In this case the favorite still improves its chances of winning from 64.7% to 67.8%.

In Practice

So what does any of this mean in the real world? Simply put, to win more often underdogs should employ a high-variance strategy from the beginning of the game. It shouldn’t wait until the 4th quarter and become desperate. Go for it on 4th and short, run trick plays, throw deep, and blitz more often. Roll the dice from the get-go.

The real question is, what is the optimum level of risk? I’m not sure, but I do know NFL coaches are operating far from it.

Looking at games from the ’02 through ’06 seasons (a total of 1280), underdogs do not increase their variance. For example, for games in which the point spread is between 6 and 7.5 points, the underdog’s SD is 9.8 points, slightly less than the overall league average. Ideally, it should be higher. The favorite’s SD is 10.4 points when ideally it should be lower.

The table below lists the SDs of points scored for the favorite and underdog according to the most common point spreads.

Spread	Favorite SD	Underdog SD
0 - 1.5	9.6	10.5
2 - 3.5	9.8	9.4
6 - 7.5	10.4	9.8
10 - 11.5	10.5	8.7

If anything, there appears to be slight trends in the exactly wrong directions. The bigger the spread, the smaller the underdog’s variance and the bigger the favorite’s variance. It appears underdogs may get less aggressive while favorites may get more aggressive.

Conclusions

This is more evidence coaches do not coach to maximize their team’s chances of winning. My theory is coaches are delaying elimination until the latest point in the game—that is, trying to “stay in the game” for as long as possible. Underdog coaches minimize risk all game long hoping for a miracle along the way. They seem to be reducing the chances of being blown out, but this is not consistent with giving their team the best chance to win.

But if you think about it, this kind of approach might be good for the NFL as a whole. It keeps games entertaining as long as possible, and keeps viewers tuned in.

Coaches of favored teams could be accused of the same crime. They might be playing with too much variance. But there is certainly a limit to just how consistent a team can be, no matter how hard it tries. There will always be random variation in team performance. I suspect a SD of 10 points may be near that limit, and that coaches of both favorites and underdogs simply play the least risky game they can consistent with accepted conventions.

Thoughts on the Apparent Unimportance of Run Defense

2009-05-21T13:12:00.003-04:00

John Morgan of the Seahawks blog Fieldgulls.com asked me recently about why run defense appears relatively less important in terms of winning than do other facets of the game. John writes:

You state defensive rushing yards per attempt allowed is the least important component to winning, but I wonder if that factors in game situation. Losing teams are likely to reduce their yards per attempt allowed when winning teams are running out the clock.

John makes a good point, and indeed the team-stat regression model I used when I made that conclusion did not take game situation into account. John goes on to point out that teams that are already behind may face a large number of predictable run-out-the-clock runs, which would make their run defense appear better. Plus, the game theory aspects of running and passing should enable a good run defense to make stopping the pass easier.

I think this is an interesting point, and I soon should be able to test how much late-game runs affect a team’s overall defensive efficiency after some improvements to my play-by-play database. A preliminary look indicates it probably doesn’t make much of a difference. Still, I think John raises good points about the game theory aspect and predictability.

In theory, good a run defense should make a pass defense better. And the stats suggest it does. The correlation between defensive running efficiency and passing efficiency is 0.20. Some of that correlation has to do with the fact that superior defensive athletes are superior against both the run and the pass. But some of it should also be due to the game theory aspect.

(In comparison, offensive running and passing efficiency correlate at 0.13. I think the difference is that there is a lot more variance in offensive passing than in other facets due to the focus on a single player’s ability. The quarterback’s contribution is so critical to passing in ways that aren’t applicable to the other aspects.)

According to game theory principals, defensive running and passing would only be equally important if offenses and defenses were operating at the game equilibrium point-- that is, they’re playing the optimum mix of passing and running. But I think they might not be.

My theory is that this may be why run defense appears so unimportant. Say teams are not operating at the game equilibrium, and passing is, on balance, a more lucrative strategy than running. In other words, there really is a considerable passing premium where the payoff for a pass is generally higher than a run, all things considered.

Having a good run defense would therefore be somewhat self-defeating. Take the 2007 Vikings defense that gave up only 3.1 yards per run (good) but 7.0 yards per pass (bad). Facing such a solid run defense, a good offensive coordinator is forced to pass more…which would be a far more effective thing to do in the first place, especially against a relatively weak pass defense. A team like the 2007 Vikings would essentially be forcing their opponent to unwittingly play a more efficient and effective strategy, all the while exploiting their own weakness.

Should Rookie QBs Start?

2009-05-18T08:00:00.007-04:00

Wages of Wins author and loyal Detroit Lions fan Dave Berri recently asked me about research on whether rookie quarterbacks are better off standing on the sideline all season. One of the big questions in Detroit this year will certainly be whether overall number one pick Matthew Stafford should start at QB. The central issue is whether starting a rookie QB somehow harms his long-term development. Does a year holding a clipboard allow rookies to adapt to the NFL and boost their prospects for a successful career?

Names like Boller, Harrington, Couch, Shuler, and Carr underscore the danger of starting rookie passers. But there are also names such as Manning, Roethlisberger, Marino, Elway, and Aikman that indicate that starting as a rookie may not be so damaging to a QB's development.

This is a question I get often but haven’t looked into it because of complications associated with the issue. First, there aren’t that many top QBs to analyze—the sample size is fairly small. Any inference we make needs to keep in mind the small sample. Second, there is a problem of bias in the data. The better QBs would be the ones to earn starting jobs their rookie year, and would also likely tend to be the ones to enjoy successful careers.

The trick would be to properly account for underlying QB potential, which would be quite a trick. If we knew that, that’s all we’d ever really need to know about a QB. There's no perfect way to measure that, but in the end, I think the most reasonable variable to indicate potential is overall draft pick number. It’s something that is established prior to any decision to start or not as a rookie. Also, although it is often an unreliable predictor of career performance for individual QBs, it correlates very well for QBs as a group. In other words, the number one QB might not pan out or even be better than the second QB taken in any particular draft year. But as a whole, the top picks reliably tend to outperform subsequent picks.

Data

I looked at first and second round QBs drafted between 1980 and 2004. I chose 1980 because it is roughly the dawn of the modern NFL passing rules. I chose 2004 to allow at least four years to asses each QB’s performance. Data are from Pro-Football-Reference.com. To measure career success, I used Adjusted Yards per Attempt (Adj YPA). This is passing yards minus 45 yards for every interception, per pass attempt. For QBs with very low pass attempts and spurious Adj YPA stats, I replaced their actual Adj YPA with 3.0 YPA, generally the floor for QBs with a reasonable sample size of attempts. (There is no reason to expect 1989 Chief’s 2nd-round pick Mike Elkins to lose 20 yards for every pass based on only 2 attempts.)

Methodology

The first step is to account for potential using overall draft pick number. The graph below plots career Adj YPA by pick number. The scatterplot is pretty random for any individual QB, but as a whole there is a predictable trend. Top draft picks tend to end up as NFL top passers. I estimated the expected Adj YPA based on pick number. A QB from the top of the first round would be expected to average 5.0 Adj YPA, while a QB from the bottom of the second round would be expected to average 4.1 Adj YPA.

We can see a clear trend. Unsurprisingly, top picks would be expected to end up with better career passing stats than later picks. A linear regression estimates what the baseline expectation should be for each slot in the draft.

Next I calculated the 'Adj YPA above expected' for each QB. Now we can compare QBs who started their first year to those that didn’t, while holding “potential” equal.

But how do we define “started their first year?” How many starts qualifies--4, 5, 11? I don’t know, so let’s start by looking at the whole picture.

Results

We can use that baseline to compare each QB's career Adj YPA. Some QBs did better than expected given their draft slots and some did worse. We can now test if there is a connection between better than expected performance and the number of rookie year starts.

As it turns out, it appears that QBs with more rookie starts tend to enjoy greater career success, even accounting for draft order.

Some of you might have noticed that what I've really done here is a crude multivariate regression. Holding for draft order, I estimated the effect of games started. What if we just do the regression directly?

As expected we get a small negative effect with draft order. (The higher the pick number, the worse the expected stats.) The Games Started variable is positive and significant at p=0.03. The model as a whole has an r-squared of 0.15--small in absolute terms, but considerable given the highly random individual variance in QB careers.

But this is only one way of looking at whether a QB was a starter or not. What if we draw a line at say, 5 rookies starts--below 5 starts he's not a rookie starter and above it he is. The group of QBs with 5 or less starts averages -0.01 Adj YPA above expected, and the group with 6 or more starts averages +0.3 Adj YPA above expected. If we define it at zero starts, those with no rookie starts averaged -0.03 Adj YPA above expected, while those with at least one start averaged +0.02 starts above expected. In fact, no matter where I chose the endpoints of the groups, from 3 to 13 starts, the group with more starts outperforms the group with fewer starts by about 0.4 Adj YPA.

Conclusion

Does this mean teams should rush their rookies out to face the onslaught of NFL defenses to somehow make them better? I really doubt it. If I had to bet, I'd say that we simply haven't fully accounted for QB "potential" using draft order alone. I think the better QBs, those with the best chances of career success, often gain and maintain starting positions earlier.

But at the very least, we can say this: Given this analysis, there is no reason for a coach to arbitrarily keep a rookie QB on the bench. He should start his best QB, rookie or not, and not worry about incubating him under a ballcap on the sidelines. In the end, it should be the coach's qualitative judgment on the readiness of the player.

Here are the QBs and their stats I used for this article. (It's interesting just to see who the QBs are who exceeded expectations. There are some surprising names--Pennington and Batch at the top, for example. And Is Eli Manning really worse than David Carr? Wow.) You can sort the table by clicking on the column headers.

Year	Rnd	Pick	Name	Team	Adj YPA	Exp Adj YPA	Yr1 GS	AdjYPA Abv Exp
2001	2	32	Drew Brees		6.0	4.6	0	1.4
1981	2	33	Neil Lomax		5.9	4.6	7	1.4
1998	1	1	Peyton Manning		6.4	5.0	16	1.4
1998	2	60	Charlie Batch		5.5	4.1	12	1.4
2004	1	4	Philip Rivers		6.4	5.0	0	1.4
2004	1	11	Ben Roethlisberger		6.2	4.9	13	1.3
1983	1	27	Dan Marino		6.0	4.6	9	1.3
2000	1	18	Chad Pennington		6.1	4.8	0	1.3
1999	1	11	Daunte Culpepper		6.1	4.9	0	1.3
1984	2	38	Boomer Esiason		5.7	4.5	4	1.2
1985	2	37	Randall Cunningham		5.6	4.5	4	1.1
1983	1	24	Ken O'Brien		5.7	4.7	0	1.1
1995	2	45	Todd Collins		5.3	4.4	1	1.0
1991	2	33	Brett Favre		5.5	4.6	0	1.0
1983	1	14	Jim Kelly		5.8	4.8	0	0.9
1999	1	2	Donovan McNabb		5.9	5.0	6	0.9
1983	1	15	Tony Eason		5.7	4.8	4	0.8
2003	1	1	Carson Palmer		5.8	5.0	0	0.8
1987	1	26	Jim Harbaugh		5.4	4.7	0	0.7
1995	1	3	Steve McNair		5.7	5.0	2	0.7
1996	2	42	Tony Banks		5.1	4.4	13	0.7
1983	1	1	John Elway		5.7	5.0	10	0.7
1990	1	1	Jeff George		5.7	5.0	12	0.6
1997	2	42	Jake Plummer		5.1	4.4	9	0.6
2001	2	53	Quincy Carter		4.9	4.3	8	0.6
1989	1	1	Troy Aikman		5.6	5.0	11	0.6
2003	1	7	Byron Leftwich		5.5	4.9	13	0.6
1982	1	5	Jim McMahon		5.5	5.0	7	0.5
1995	2	60	Kordell Stewart		4.7	4.1	2	0.5
1986	1	3	Jim Everett		5.5	5.0	5	0.5
2002	1	32	Patrick Ramsey		5.0	4.6	5	0.4
1999	2	50	Shaun King		4.7	4.3	5	0.4
1989	2	51	Billy Joe Tolliver		4.6	4.3	5	0.3
2001	1	1	Michael Vick		5.3	5.0	2	0.3
2004	1	22	J.P. Losman		5.0	4.7	0	0.3
1987	1	13	Chris Miller		5.1	4.9	2	0.2
1993	1	1	Drew Bledsoe		5.3	5.0	12	0.2
1995	1	5	Kerry Collins		5.2	5.0	13	0.2
1987	1	1	Vinny Testaverde		5.1	5.0	4	0.1
2003	1	22	Rex Grossman		4.8	4.7	3	0.1
1992	1	25	Tommy Maddox		4.7	4.7	4	0.0
1986	2	47	Jack Trudeau		4.3	4.3	11	0.0
2002	1	1	David Carr		5.0	5.0	16	-0.1
2004	1	1	Eli Manning		4.9	5.0	7	-0.1
1991	1	24	Todd Marinovich		4.6	4.7	1	-0.1
1980	1	15	Marc Wilson		4.7	4.8	0	-0.1
1990	1	7	Andre Ware		4.7	4.9	1	-0.3
2003	1	19	Kyle Boller		4.5	4.8	9	-0.3
1999	1	1	Tim Couch		4.7	5.0	14	-0.3
1994	1	6	Trent Dilfer		4.6	5.0	2	-0.3
1999	1	12	Cade McNown		4.4	4.9	6	-0.5
1982	2	44	Oliver Luck		3.9	4.4	0	-0.5
1980	1	28	Mark Malone		4.0	4.6	0	-0.7
2002	1	3	Joey Harrington		4.3	5.0	12	-0.7
1986	1	12	Chuck Long		4.1	4.9	2	-0.8
1992	1	6	David Klingler		4.2	5.0	4	-0.8
1993	1	2	Rick Mirer		4.2	5.0	16	-0.8
1983	1	7	Todd Blackledge		4.1	4.9	0	-0.9
1991	2	34	Browning Nagle		3.6	4.5	0	-0.9
1982	2	48	Matt Kofler		3.3	4.3	0	-1.1
1994	1	3	Heath Shuler		3.7	5.0	8	-1.3
1992	2	46	Tony Sacca		3.0	4.4	0	-1.4
1992	2	40	Matt Blundin		3.0	4.4	0	-1.4
1999	1	3	Akili Smith		3.5	5.0	4	-1.5
1980	2	37	Gene Bradley		3.0	4.5		-1.5
2001	2	59	Marques Tuiasosopo		2.7	4.2	0	-1.5
1989	2	32	Mike Elkins		3.0	4.6	0	-1.6
1987	1	6	Kelly Stouffer		3.4	5.0	0	-1.6
1991	1	16	Dan McGwire		3.2	4.8	1	-1.6
1997	1	26	Jim Druckenmiller		3.0	4.7	1	-1.7
1998	1	2	Ryan Leaf		3.1	5.0	9	-1.9
1981	1	6	Rich Campbell		3.0	5.0	0	-2.0
1982	1	4	Art Schlichter		3.0	5.0	0	-2.0

Live NHL Win Probability

2009-05-17T10:45:00.005-04:00

If you've been checking out the NBA win probability site, you've probably noticed a link for the NHL too. The probabilities are calculated based on the method I outlined in this post. The model now includes power plays, a unique and important factor in hockey.

Latest play descriptions are also included at each point in the graph, which is similar to how the football graphs will work in the fall. The most challenging part of whole project has been automatically adding notes to the graph where the goals were scored. They frequently overlap or overflow off the graph, and the logic of how to arrange them is far more challenging than you'd think.

I don't know much about hockey despite my frequent attendance at Capitals games in high school, and this has been a great way to look at the sport from a different perspective. I realize there aren't nearly as many hockey fans as football fans, but hey, a ten thousand Canadians can't be wrong...or however many Canadians there are. I missed that day at school.

The conference finals start today, so check it out. And if you're not interested in hockey, pass the word to your friends who are.

NHL Win Probability

Passing Predictability Part 2

2009-05-13T08:00:00.005-04:00

Here's a tip: If you author a sports research website, never start an article as "Part 1" unless you already have a Part 2 ready to go. Way back at the end of March I began looking at how predictability affects passing success. I looked at "10 yards to go" situations--1st and 10, 2nd and 10, and 3rd and 10 plays. Passes are least predictable on 1st down and most predictable on 3rd and long. With 10 yards to go, passes are called roughly half the time on 1st and 2nd down, but on 3rd and 10 passes are called 91% of the time.

Since then, I've been tinkering with my NBA and NHL in-game win probability sites. This post will officially tie the loose end that's been dangling since the NCAA Tournament.

In Part 1 of this article, we saw the average gain in each of these situations:

Yds Per Attempt by Down, 10 Yds To Go

Type	1st	2nd	3rd	Total
Pass	7.0	6.3	6.5	6.9
Run	4.2	4.4	6.9	4.3
Total	5.5	5.4	6.5	5.6

But there were several wrinkles in this analysis. First, there are interceptions to consider. Interceptions become much more probable on 3rd and 10.

Interception % by Down, 10 Yds To Go

	1st	2nd	3rd	Total
Int Rate	2.6	2.9	3.5	2.7

Combining Yards per Attempt and Interception rate yields Adjusted YPA, which is YPA with a -45 yd adjustment for every interception thrown.

Adj Yds Per Attempt by Down, 10 Yds To Go

Type	1st	2nd	3rd	Total
Pass	5.9	5.0	4.9	5.6
Run	4.2	4.4	6.9	4.3

But there is still a problem. Adj YPA underestimates the drop off from 1st to 3rd down in passing effectiveness because defenses will allow gains, as long as they're not more than 9 yards. So we have to limit our observation to say the reduction in passing effectiveness due to predictability is likely at least 1 full adjusted yard per attempt.

Except that there's still another problem. There's a bias in the data because poor passing teams will face more 2nd & 10 and 3rd & 10 situations. So the 2nd and 3rd down numbers are lower than would be representative of the league as a whole. In other words, poor passing teams 'get more votes' in the analysis.

The solution is to give each team (or each team-year, actually) equal votes. Instead of averaging the gains for all 2nd and 10s for the entire league as a whole, I first averaged them by team-year, then averaged them by team.

Broken Out by Team & Year, Re-Averaged

Stat	1st	2nd	3rd	Total
YPA	7.0	6.3	6.5	6.8
Int Rate	2.6	2.9	3.5	2.7
Adj YPA	5.8	5.0	4.9	5.6

As it turns out, the numbers are nearly (but not completely) identical using this method. The effect of the bias isn't very pronounced and we're left with the original estimate. Going from about 50% predictability to 90% predictability costs at least 1.0 Adj YPA.

Another notable observation is that although 1st &10 and 2nd & 10 have about the same run/pass balance of about 50%, passes on 1st and 10 are considerably more effective--5.8 vs 5.0 Adj YPA. I'd have to think that the difference may be largely due to play action.

Epic Bulls-Celtics Series

2009-05-02T11:39:00.009-04:00

If you haven’t been paying attention to this series you’re missing one of the most exciting 7-game series ever, even if it’s just the first round.

Going into Thursday night’s game, there were already 3 OT games and 1 last second buzzer-beater game. Then this happened: A 3OT potential elimination game that featured multiple furious comebacks for both teams and ultimately tied the series 3-3 with a 1-point win by Chicago. These two teams are as evenly matched as it gets. They square off for Game 7 tonight at 8.

Here are all 6 games of the series so far.

Compared to basketball or football, the hockey graphs aren't as compelling at first glance, at least during the game. But a quick look at a graph after the game tells a dramatic story you just won’t get with a box score. Below is Thursday night’s Game 1 between Chicago and Vancouver.

5-3? Well, that doesn’t sound terribly exciting. But check out what happened: 3-0 lead held until the third period. With 10 minutes left in the game, another goal, and with 5 min left another to tie it 3-3. Then Vancouver gets the game-winning goal with less than 2 min to go. Then in the final seconds, a garbage goal with the goalie pulled makes it 5-3.

You can check out the beta version of the hockey graphs for today's Caps-Penguins and Blackhawks-Canucks games. Power plays have not been factored in yet, but that's in progress, and it should be ready early next week.

NHL In-Game Win Probability

2009-04-30T12:30:00.002-04:00

I was at an NHL game the other night, and with the score 2-0 someone asked me, “So Mr. Win Probability, what’s the chance the Capitals win?” I was caught off guard, and after I choked out, “I…don’t…know…,” I experienced the horror that is not knowing the exact up-to-the-second win probability of a sporting contest. Don’t let this happen to you.

The anxiety and shame lasted for two days straight. I kept blaming myself and replaying the incident over and over in my head. The only way to cure my depression was to build a win probability model for NHL hockey.

Unlike my previous models for basketball and football which were empirically based, my hockey model is theoretical. In other words, instead of being based on a massive database of actual previous games, the probabilities are calculated based on a Poisson scoring distribution. The distribution is calculated using the average goals scored per minute in the 2008-9 NHL season. It’s an extension of the model I developed in this post.

Teams score an average of 2.79 goals per 60 minutes of regulation time, which is equal to 0.0465 goals per minute. A Poisson distribution based on that per-minute scoring rate and the time remaining in the game yields the probabilities of each team scoring each number of possible goals by the end of the game. Summing up all the probabilities of all the possible combinations of final scores gives the game’s win probability.

Here’s the graph:

There are a couple wrinkles to address. First, there are power plays. When a team as a man advantage on the ice, it’s much more likely to score. About one in five power plays results in a goal for the team with the advantage. Only about 2% of the time the short-handed team will score. So at the start of a power play, a rough approximation would put the win probability a little less than one fifth of the way toward the next best curve.

For example, if the score is 2-0 with 30 minutes remaining in the game, the win probability would normally be about 13% for the trailing team (the red line). But at the beginning of a power play, the trailing team’s win probability would jump about a fifth of the way up to the ‘down by 1’ line (blue). A rough approximation puts the new win probability at 16%. Then as the power play expires and there’s no score, the win probability would gradually return to the ‘down by 2’ line.

Second, there is the ‘end-game,’ when teams down by a goal will pull their goalie in favor of an additional skater. That would increase the win probability of the trailing team slightly, but only half as much as you might expect. They’d still only be buying an opportunity in overtime. But it could still be factored in. Before I do, I’d need some data on end-game goals.

One advantage of a theoretical approach over an empirical model is that team strength can be factored in far more easily. In an empirical model, when you divide up the data by various classes of team strength, the data is sliced into tiny fragments, usually with very small and unreliable sample sizes. Theoretical formula-based models don’t suffer from that problem. I can simply adjust the mean goals scored and goals allowed for any particular opponent, then rerun the model. The resulting model would be tailored to the specific match-up instead of a generic model for the league as a whole. Home ice advantage can be factored in with a similar approach.

Remember, WPD (Win Probability Dysfunction) can happen at any time, and it’s nothing to be ashamed of. Don't analyze win probability graphs if you take nitrates, often prescribed for chest pain, as this may cause a sudden, unsafe drop in blood pressure. Discuss your health with your doctor to ensure that you are healthy enough to view win probability graphs. If you experience chest pain, nausea, or any other discomforts during a sporting contest, seek immediate medical help. In the rare event of viewing win probability graphs more than 4 hours, seek immediate medical help to avoid long-term injury.

Live NHL win probability graphs now online.

Are Rookies Overpaid?

2009-04-28T08:00:00.003-04:00

I recently looked at what might explain why the top draft picks are paid disproportionately to their expected performance compared to later picks. But that doesn't address the larger issue--are rookies overpaid compared to their veteran counterparts?

A 2005 research paper called The Loser's Curse by economists Cade Massey and Richard Thaler tackled that question. In a nutshell, the paper compares rookie pay to the pay of a 6th-year veteran who could be expected to deliver the same performance as a rookie from each slot in the draft. (Performance is defined by a mix of measures including: being on a team roster, starts, and Pro Bowls.)

The conclusion of the paper is that team executives and scouts overpay for the top picks in the draft relative to the later picks, likely due to overconfidence in their ability to identify the best players. But what might surprise some readers is that rookies at every level of the draft are bargains compared to equivalently performing veterans.

This graph from the paper is the study's bottom line. The red 'compensation' line is the average annual pay for each draft pick. The blue 'performance' line is the salary a team would have to pay a 6-year veteran free agent for the same expected performance. The green 'surplus' line is the difference between the two pay levels.

The surplus performance peaks shallowly at the bottom of the first round and through the second round. That's where teams get the biggest bang for the buck. But still, the surplus is strongly positive throughout the entire draft. According to Massey and Thaler, rookies are a bargain compared to veterans.

There's a good explanation why rookies would be underpaid. Veterans are known quantities while there is a tremendous amount of uncertainty with draft picks. Think of it this way--Peyton Manning has been to nine Pro Bowls and Ryan Leaf to zero, for an average of 4.5 between the two players. Four Pro Bowls--that's not bad. But would a GM pay more for a guaranteed 4.5 Pro-Bowl-type player or for a 50/50 shot between a total bust and Hall of Famer? Just about every modern economic and psychological theory tells us that people will pay a premium for the sure average.

Unfortunately, that's not an option in the draft. Peyton Leaf just doesn't exist. But 6-year veterans do, and GMs will be willing to pay a premium for the reduced uncertainty in performance.

One note of caution on the paper. The draft years studied were 2000-2002, and rookie salaries have increased substantially since then. But veteran salaries have too. The question is whether rookie pay increases have outpaced veteran pay increases since then. However, rookie pay would needed to have increased over 15-20% faster than veteran pay to change the conclusions of the paper.

Draft Success by Team

2009-04-25T11:14:00.001-04:00

Some teams and GMs have built reputations as good drafting teams, while others have earned the stigma of, well, as Mel Kiper says of the Jets, "not understanding what the draft is all about." What about your team? How have they fared in recent years?

I crunched some numbers for picks from the 1996 through 2008 picks. To be honest, I chose 1996 because that was the first year of the "Baltimore NFL Franchise," the team yet to become the Ravens later that year. But '96 also makes sense because it's soon after the salary cap system was put into place, and players from that draft are still enjoying success in the NFL today. For the purposes of this article, I'm defining draft success as player years as a team's primary starter, and total number of Pro Bowl selections. Neither measure is perfect, but together they'll give us a pretty good idea of which teams have recently enjoyed the most successful drafts.

The first table lists the average number of years as a starter for each team's picks, broken out by round. For example, Indianapolis's first round picks have averaged 6.5 years as a starter. The average column is the average starting years for all a team's draft picks.

(Click on the table headers to sort.)

Rank	Team	1	2	3	4	5	6	7	Avg
1	IND	6.5	3.5	0.9	2.2	0.7	0.7	0.9	1.9
2	ARI	3.9	3.3	2.1	2.3	0.7	0.7	0.7	1.9
3	BAL	6.0	2.7	0.6	2.1	1.5	0.8	0.0	1.9
4	STL	4.8	3.7	1.1	1.5	1.3	0.3	0.5	1.8
5	NYG	3.9	3.3	2.5	1.0	2.0	0.5	0.4	1.8
6	CIN	4.1	3.4	1.8	1.3	0.2	1.4	0.4	1.8
7	PIT	4.2	1.6	2.4	2.3	0.4	0.5	1.4	1.7
8	GB	3.3	2.5	1.8	1.5	1.1	1.8	1.0	1.7
9	OAK	4.2	2.2	1.6	0.2	3.0	0.6	0.3	1.7
10	NYJ	5.0	1.9	1.9	1.8	0.8	0.1	0.8	1.7
11	SEA	4.9	2.8	1.8	1.3	0.4	0.2	0.8	1.7
12	JAX	4.9	2.8	1.8	1.1	0.9	0.6	0.7	1.7
13	CHI	3.1	3.0	2.4	1.7	1.2	0.9	0.3	1.6
14	DAL	3.5	3.3	1.8	0.9	1.1	0.6	0.7	1.6
15	PHI	4.2	2.8	1.9	1.2	0.9	0.5	0.6	1.6
16	SF	2.7	3.5	1.9	0.7	0.2	1.4	1.0	1.6
17	BUF	3.9	3.2	1.5	1.3	1.3	0.5	0.4	1.5
18	WAS	4.3	2.5	2.8	1.6	0.9	0.2	0.2	1.5
19	TEN	3.6	3.0	1.5	1.2	0.9	0.9	0.6	1.5
20	MIN	4.3	1.9	1.3	1.0	0.6	1.0	0.1	1.5
21	DEN	3.7	2.5	1.2	1.3	0.3	1.0	0.6	1.5
22	CAR	2.8	4.2	1.2	1.1	0.5	0.7	0.4	1.5
23	TB	3.6	3.0	2.9	1.1	0.8	0.6	0.0	1.4
24	ATL	3.3	2.8	0.8	2.1	0.4	0.2	1.3	1.4
25	KC	4.1	1.8	1.3	1.7	1.2	0.1	0.6	1.4
26	MIA	2.0	3.5	2.2	1.0	1.4	0.4	0.3	1.4
27	NO	3.8	2.3	1.2	1.5	0.5	0.6	0.2	1.4
28	SD	2.9	2.8	2.7	1.2	0.6	0.1	0.3	1.4
29	NE	3.7	2.9	1.9	0.9	0.6	0.4	0.3	1.4
30	HOU	3.6	2.5	0.7	2.4	0.1	0.8	0.1	1.3
31	DET	3.7	2.0	1.4	0.0	0.6	0.2	0.4	1.3
32	CLE	3.2	2.5	2.0	0.9	0.8	0.1	0.2	1.2
Avg		4.0	2.8	1.7	1.4	0.9	0.6	0.5	1.6

The next table lists the total number of Pro Bowl selections for each team, broken out by round. Baltimore has had a total of 39 Pro Bowl selections by players they have drafted in the first round. (Note that some of the total numbers for the Browns and Texans will be low simply due their entry into the league in '99 and '02 respectively. The Lions are still worse, despite being in the league for the full time span.)

Rank	Team	1	2	3	4	5	6	7	Total
1	BAL	39	0	0	1	2	3	0	45
2	IND	32	3	1	0	1	2	0	39
3	PHI	14	8	9	0	2	0	0	33
4	PIT	18	2	10	1	0	0	0	31
5	SEA	21	4	4	0	1	0	0	30
6	DAL	10	8	8	1	0	1	1	29
7	CHI	10	3	11	1	1	0	0	26
8	NE	9	6	1	3	1	4	1	25
9	WAS	19	1	2	3	0	0	0	25
10	GB	4	6	1	1	3	6	3	24
11	KC	13	0	1	3	6	0	0	23
12	MIA	1	8	6	0	7	0	0	22
13	MIN	16	0	0	0	0	6	0	22
14	CIN	8	9	0	2	0	1	1	21
15	ARI	10	7	3	0	0	0	0	20
16	SF	7	2	7	1	0	2	1	20
17	DEN	12	4	2	1	0	0	0	19
18	OAK	8	0	0	0	11	0	0	19
19	SD	10	5	2	0	1	1	0	19
20	TB	4	6	7	0	0	2	0	19
21	CAR	7	7	4	0	0	0	0	18
22	STL	15	2	1	0	0	0	0	18
23	ATL	13	4	0	0	0	0	0	17
24	NYJ	13	1	1	1	0	0	0	16
25	NYG	5	6	1	1	0	1	0	14
26	BUF	7	5	0	1	0	0	0	13
27	TEN	8	2	0	2	0	0	1	13
28	HOU	8	1	0	3	0	0	0	12
29	NO	6	2	0	0	0	2	0	10
30	JAX	7	2	0	0	0	0	0	9
31	CLE	4	0	0	0	2	0	0	6
32	DET	3	3	0	0	0	0	0	6
Total		361	117	82	26	38	31	8	663

But the tables above don't really tell us how well teams draft as much as it tells how high in the draft each team has picked. A team that consistently picks in the top third of each round will tend to end up with players with more potential, and therefore have better individual careers. So we need to account for each team's draft positions over the time period studied.

To do this, I calculated the expected number of starting years and expected number of Pro Bowls for each slot in the draft. After smoothing the data, I compared each team's expected draft success to their actual draft success. For example, the Detroit Lions' 1st round picks averaged 3.7 years starting, but they should have averaged a lot more given their typically high pick in each round. If we sum up the differences between expected and average for all the players, we'll see how well teams really drafted.

This table lists the 'Starting Years Above Expected' and 'Pro Bowl Selections Above Expected' for each team, given the picks they had during each draft.

Rank	Team	St Yrs Abv Exp	PBs Abv Exp
1	IND	23.6	17.4
2	GB	17.3	1.7
3	STL	14.3	-2.5
4	OAK	13.6	-2.1
5	JAX	13.3	-1.8
6	CHI	11.8	0.2
7	BAL	11.2	8.3
8	SEA	10.4	6.3
9	NYG	7.4	-2.9
10	SF	6.4	4.4
11	NYJ	5.0	-3.7
12	CIN	4.8	-2.9
13	PHI	4.5	6.8
14	ARI	4.1	-3.6
15	KC	2.1	1.7
16	PIT	2.1	1.4
17	WAS	1.9	0.1
18	TEN	-0.8	-3.9
19	BUF	-1.3	-4.9
20	TB	-3.2	-0.8
21	CAR	-4.2	-1.1
22	DAL	-5.2	3.8
23	ATL	-5.5	-1.4
24	MIA	-6.7	-2.5
25	DEN	-6.7	1.4
26	MIN	-10.8	0.4
27	NE	-10.8	-4.7
28	NO	-15.7	1.1
29	SD	-17.1	0.5
30	HOU	-21.0	-1.2
31	DET	-21.9	-7.5
32	CLE	-23.0	-8.0

It's important to note that none of this necessarily means certain teams or GMs are really any better than the others at identifying the best players. If the draft were completely luck, there would still be teams that look like geniuses and teams that get more than their share of busts.

In fact, that's one reason I'm building these tables. I'd like to find out how much variance there would be in draft outcomes due to luck alone, and then compare it to how much actual variance there is. The difference would the true drafting "skill" of teams, executives, and scouts. Is Ozzie really that good, or is he just lucky?

Career Success by Draft Order

2009-04-24T11:26:00.005-04:00

In response to some requests from commenters I’ve put together a few more graphs of how draft picks tend to pan out according to when they were picked. These graphs are for all players and not broken out by position. Mostly it’s just some food for thought heading into the big day tomorrow.

As always the data are from Pro-Football-Reference.com. All draft picks from 1996-2008 are considered.

First is a graph of the likelihood of Pro Bowl selection according to draft order (overall pick number). The graph is grouped into sets of five picks, with the first data point as picks #1 to #5, the second as #6 to #10, etc. Without the grouping the graphs are too noisy to be helpful.

Notice the small spike for picks #11 - 15. I'm not sure if we can read anything into that or not, but it might be worth investigating.

Next is a graph of the average years as a starter by draft order. The picks are grouped into sets of 5.

Phil B. raised an important consideration. Player success has a lot to do with opportunity, and that needs to be factored into the discussion. He suggested that top picks will get the opportunities to start, (ostensibly because teams have the most invested in them). So regardless of differences in ability, top picks would naturally be expected to become Pro Bowlers more often simply due to opportunity. Phil suggested a graph plotting success divided by opportunity.

I think this graph is what he was suggesting. Below is a plot of number of Pro Bowl selections divided by years as starter (#PBs/St Yrs), by draft order.

There is a distinct downward slope. The top picks are more likely to become successful even accounting for opportunity (at least in terms of Pro Bowls, an admittedly imperfect measure). In fact, my hunch is that this would over-account for opportunity because Pro Bowl selection and being a starter are both directly proportional to player talent. So we’re really dividing talent by talent + opportunity. The “excess” Pro Bowl selections of the top picks suggests their success has to do with more than just opportunity. But it might not be all due to talent--top picks certainly get their share of notoriety, which can be a factor in Pro Bowl selection.

Notice the plateau from about pick 51 to pick 80. I'm not sure if it means anything, but perhaps this suggests 3rd round picks are better in terms of talent than their opportunities allow. Or on the other side of the coin, maybe 2nd round picks are given more opportunities than their talent merits. But it might be just noise in the data.

A quick note regarding the use of Pro Bowl selections as a measure of success. I've pointed to the flaws in using Pro Bowl selections several times, so perhaps I should explain why I do think it can be useful. Even though Pro Bowls aren't purely performance-based, the best players do rise to the top in the aggregate. Plus, being named to at least a single Pro Bowl at some point in a player's career, at the very least, confirms a pick is not a bust.

Draft Picks: Bricklayers or Gladiators?

2009-04-23T17:00:00.006-04:00

With the draft upon us, there is a lot of chatter about ballooning rookie salaries for top picks. The consensus seems to be that top picks are not worth the cost, and salaries should be capped. But there’s a good reason why the top players’ salaries are so high, and the explanation can be found in economic ‘tournament theory.’ A short example by economics professor Robert Schenk explains it nicely:

Say you’re a contractor and your company builds brick walls. Most of your bricklayers lay about 3 bricks per minute and make about $8 per hour. (You can think of this as the replacement level.) But along comes a guy who lays bricks twice as fast--6 bricks per minute. How much would you be willing to pay him? Simple fairness suggests $16 per hour. Certainly no more than that because you could just replace him by hiring two replacement-level guys and get the same production. So in this example rewards are based on absolute differences in productivity. Large differences in productivity result in large differences in rewards. Similarly, small differences in bricklaying ability would result in small differences in hourly pay.

Now consider two ancient gladiators entertaining the emperor in combat. Even if one gladiator is only slightly better than the other, he’ll very likely win, and the differences in rewards could be extreme. Here, in a winner-take-all system, absolute differences in ability do not matter, only relative differences.

What about sports like football? First, in many ways the NFL is a winner-take-all system. Whoever wins the game earns 100% of the win while the loser eats all of the loss, and there is only one champion left standing at the end of the season.

Second, football players are not like bricklayers. You cannot replace an All-Pro QB by sending two average QBs out on the field and expect the same productivity. When there is a constraint on the number of people that can be employed at one time, the value of the most productive people rapidly increases.

And when there is a constraint on the number of contributors combined with a winner-take-all reward structure, the value of the top performers will skyrocket. This is why the top NFL draft picks make so much more than the lower picks. Even if the abilities of the top picks are only marginally better than those of the picks in later rounds, there will be very large differences in pay.

It’s not much different than CEO compensation. If a company is in competition with other companies for market share, the shareholders should want the best CEO they can get--especially because a competitor with a slightly more visionary CEO will likely steal market share, even if your guy is still top-notch. And since you can’t replace a single chief executive with two average guys or a whole mob of slackers, the CEO’s pay is going to end up being wildly disproportionate to his actual ability. There can be only one guy at the top, only one winner of the tournament.

Note that I’m not claming that rookie salaries should be this high, just trying to understand why they’re so high. And I’m not comparing rookie pay to veteran pay. That’s another topic for another day.

"Must-Win" Games

2009-04-23T13:34:00.004-04:00

“This is a must-win game…” Well, unless one team has a 3 wins already no game in a 7-game MLB, NBA, or NHL series is technically ‘must win.’ But certainly some games are more crucial than others in terms of a team’s chances of winning the series.

For example, the Rangers currently have a 2-0 advantage over the Capitals in the first round of the NHL playoffs. The difference in Series Win Probability (SWP) between being down 0-3 and down 1-2 tells us exactly how critical this Game 3 is. Based on a symmetric binomial distribution (that is, a coin flip--each team has an equal chance of winning each game), the SWP of being down 0-3 is 0.0625 and the SWP of being down 1-2 is 0.3125. The potential change in SWP (∆SWP) for Game 3 is 0.25.

The difference in SWP for Game 2 however was larger. The SWP(1-1) is 0.5. And the SWP(0-2) is 0.1875. The ∆SWP for Game 2 was 0.3125. Game 2 therefore had more leverage than Game 3 will. It was about 20% more crucial (.31 vs .25).

Continuing on this path, the ∆SWP for all Game 1s is SWP(1-0) vs SWP (0-1) which is also 0.3125. So both Game 1 and Game 2 were both more critical than Game 3. But this is only because the situation of being 0-2 is already fairly dire.

Ironically, the must win situation of 0-3 yields a ∆SWP of only 0.125. Again, this is because 0-3 is very dire. A team is pretty close to elimination anyway. Winning Game 4 when down 0-3 still only buys a team a .125 chance of winning the series while losing the game would make it zero.

The most critical situation is Game 7 of a 3-3 series . The 3-3 situation features a ∆SWP of 1.0—the winner goes from 0.5 probability to 1.0 (certainty) in a single game, while the loser goes to zero.

The next most critical games are Game 6 of a 3-2 series and Game 5 of a 2-2 series. All Game 6s are 3-2 and yield either a 4-2 (1.0) or 3-3 (0.5) result for a difference of 0.5. Game 5 of a 2-2 series is just as critical. Being up 3-2 yields a SWP of 0.75, while (symmetrically) being down 2-3 yields a SWP of 0.25, the difference being 0.5.

Here is the full table:

Game	Situation	Leverage
Game 1	0-0	0.3125
Game 2	1-0	0.3125
Game 3	2-0	0.25
	1-1	0.375
Game 4	3-0	0.125
	2-1	0.375
Game 5	3-1	0.25
	2-2	0.5
Game 6	3-2	0.5
Game 7	3-3	1

Notes:

1. This method ignores home ice/court/field.
2. It also assumes teams are evenly matched. Empirical observations of teams that comeback from 0-3 deficits will be less frequent than predicted by the theoretical average because teams down 0-3 tend not to be evenly matched. I think a symmetric binomial model (coin flip) is sufficient because we're looking at the 'typical' leverage for the various series situations and not necessarily for particular match-ups.
3. You can also use this table for 5-game series. A 5-game series is no different from a 7-game series that starts tied at 1-1. To find the leverage of a game in a 5-game series, take the current record and add 1 win for each team. For example, the leverage for a 5-game series that's at 2-1 is identical to that of a 7-game series that's 3-2 (0.5).

Drafting Linebackers

2009-04-22T08:00:00.001-04:00

In some circles, the conventional wisdom is that great linebackers can be found anywhere in the draft, and that teams should think twice before taking a LB in the first round. This post will take a look at whether this is true by looking at LB performance according to draft round and pick order.

We've seen how performance varies by draft position in QBs, RBs, DEs, WRs, and DBs. How do LBs compare? Based on the careers of all LBs taken in the 1980 through 2001 NFL drafts, we'll see how scarce top LBs typically are, and what kind of performance teams can expect from their picks.

I'll start by looking at Pro Bowl selections, and I'll repeat my standard disclaimer. Pro Bowl selection is a very imperfect measure of a player's value for a lot of different reasons, but it does identify the top players at each position which is what much of the draft is about. One other advantage it offers is that player value can be compared across positions. For example, we can compare LB draft picks to QB draft picks using Pro Bowl selections, but using passing yards or tackles wouldn't work to well.

The first graph looks at the rate of Pro Bowl selection by draft round. The blue line illustrates the likelihood a pick from each round will be selected to at least one Pro Bowl at some point in his career. The red line is for two or more Pro Bowls, and the green line is for three or more.

The next graph looks at Pro Bowl selections by draft order within position. The scouts must be doing their job because first linebacker taken really outshines the second, third, etc, at least in terms of going to at least one Pro Bowl.

The third graph is the average number of years a player is a starter for his team, broken out by draft order. The careers of the first LBs taken appear to have more longevity.

The graphs remind me a lot of the ones for wide receivers. There is a relatively large drop off after the very top players taken.

The continuing theme in this series is that the best players really do come from the top of the draft. No surprise there. But the top players have more than just an incrementally higher chance of great success, but double or triple the chance. The scouts and GMs do have an ability to recognize the players with the most potential at every position we've looked at so far. It's interesting to see just how steep the drop off really is after the first few players.

Drafting Defensive Backs

2009-04-16T07:00:00.002-04:00

Continuing the series of analyzing the NFL draft by position from last year, this post will look at defensive backs. How likely do the top picks outperform the later ones? How often do later picks turn out to be solid contributors? How do they compare to other positions? Using data from all defensive backs taken from 1980 through 2001 I'll answer those questions.

Unfortunately, the draft data at Pro-Football-Reference.com does not distinguish between cornerbacks and safeties (for understandable reasons). They are all considered defensive backs, so this analysis will have to do the same.

First, let's look at Pro Bowl selections. As I've mentioned before, Pro Bowl selection is a very limited measure of a player's value. In fact, no single measure can be perfect. But Pro Bowl selection does tend to signify the top players at their positions, and that's really what much of the draft about.

The graph below illustrates the likelihood of a defensive back taken in each round to be named to 1 or more, 2 or more, and 3 or more Pro Bowls in his career. As you'd expect, there is a steep drop off after the first round.

The next graph illustrates the same thing, but by draft order--in other words, regardless of round or overall pick number, was the player the 1st, 2nd, 3rd and so on defensive back taken? After the top couple DBs taken each year, there's a steep drop off in the chances the player will turn out to be a star.

Notice anything about the shape of the distributions? They're power law probability distributions, the signatures of many natural complex systems with all the implications that come with them. The next graph isolates just the '2 or more' Pro Bowls likelihoods and fits a power law curve.

Next is a graph of years as a starter by draft position. No, this time it's not a power law distribution but an exponential one, which is a little easier to interpret. Every subsequent DB taken will have an average career as a starter 7.9% shorter than the earlier DB taken. There may not be many stars in the later picks, but there are plenty of durable starter-quality players to be had.

Although it's not the only measure of a DB by a long shot, we still want them to intercept passes as often as possible. The next graph looks at the total number of interceptions by draft order. Just like the years as starter graph, the curve is exponential with an average 7.4% difference in total interceptions from any given DB pick to the subsequent pick. I'm not sure this tells us anything more than the last graph of years as starter. The two graphs closely mimic each other for obvious reasons.

The last graph we'll look at is the likelihood the a DB draft pick will turn out better than the subsequent DB chosen. To define 'better' I used years as a starter rather than any of the other measures. All the measures are flawed, but I think years as a primary starter is the safest measure because it captures so much other information, both quantitative and qualitative, about a player's performance.

This graph will tell us how well scouts and personnel executives identify the better prospects. But looking at DBs in isolation doesn't tell us much, so I added RBs and QBs to the graph for comparison.

All three positions are fairly noisy, but it looks like scouts can identify the superior DBs slightly better than RBs and QBs, particularly deeper in the draft. What this means is that GMs can have slightly better confidence when picking a DB than the other selected positions. We should expect an inverse relationship between the ability of scouts to identify the superior prospects and how deep into the draft teams can expect to find solid starter-quality players.

And that's really the point of all this analysis. In order to eventually build a sound comprehensive model of draft strategy, we'd need to know all of the likelihoods of success for the various positions in each round and at each pick order. In the meantime, it's useful to know how deep into the draft a team can find viable contributors at the various positions.

Earthquakes, Kevin Bacon, The Financial Crisis, and Pro Bowl Selections

2009-04-13T08:00:00.010-04:00

Most of the analysis I do at this site is based on the normal distribution (aka Gaussian aka bell curve). Team records, yards per attempt, sack rate, turnovers, and just about everything else follow a bell curve distribution where most teams or players are bunched around the average and a rapidly diminishing number are found at the extremes. Most of the statistical tools used here such as regression, correlation, or even simple averages are based on the assumption of a normal or quasi-normal distribution.

Normal distributions are ubiquitous in sports for mainly two reasons. First, the rules provide level playing fields, fixed boundaries, and predictable cause-effect relationships. Football games always last 60 minutes, the field is always 100 yds long, a touchdown is always 6+ points, and a win is always a win no matter how close the score. Second, there is a significant amount of random luck involved in sports, which by definition is always distributed normally.

Other distributions with different shapes appear in sports. Recently I looked at how sports like soccer, lacrosse, and particularly hockey are better modeled with Poisson distributions. But the Poisson distribution is really just a variation of the normal distribution. (Technically, I think the normal is a specific case of the Poisson, or they are both specific versions of a more general family of curves.)

There are other distributions that often appear in nature and in sports that are completely unlike the bell curve most of us are familiar with. The power law distribution is a prime example.

The Power Law

Have you ever noticed how most of the productivity around your office seems to be accomplished by a minority of your co-workers? It’s no different in the NFL, or most anywhere else.

The power law is all around us, and is a fundamental property of natural organizations of all types. City sizes, for example, are distributed according to the power law. There are a few extremely large cities, more average sized cities, and very many smaller towns. Earthquake sizes, the structure of the Internet, stock market gains and losses, body mass indexes, gravity, social network connections, wealth distributions, and even Kevin Bacon movies all follow power law distributions. If you've ever heard people refer to the "fat tail" or the "long tail," this is what they're referring to.

The power law distribution follows this equation:

y = ax^b

where x and y are variables and a and b are constants. The constant b is known as the scaling exponent.
The Financial Crisis

Our current financial crisis was in part caused by a fundamentally wrong assumption about risk distributions in the debt markets. An oversimplified explanation is that investment companies made lucrative but risky investments, and then hedged against their failure by buying insurance in the form of complex derivatives in case they went bust. These companies thought that they had cracked the code and solved the problem of risk once and for all. (One of the reasons the company AIG is central to the problem is that it's the company that led the selling of all that insurance.)

The problem was that the insurance was priced based on an assumption of bell curve distributions of market risk. A model known as the Correlated Gaussian Copula was developed by a Chinese mathematician named Li, and it was widely used throughout the financial industry for measuring and pricing risk. Unfortunately, financial markets act more like earthquakes than normally distributed phenomena like rainfall or human height. There are lots of minor fluctuations but occasionally the bottom drops out. The power law distribution has a ‘fatter tail’ at the extremes than the normal distribution, meaning extreme outcomes are considerably more likely.

Network Organization

One reason we see power law distributions so often is because they are a signature of networks. The picture below could represent a computer network, a social network, highways between cities, or airline routes. But let’s say it represents business connections among individuals. If you’re an entrant into that business market and had the resources to afford to establish a single link, who would you prefer to hitch your wagon to?

I’d want to be associated with someone who is already well-connected. I’d want to connect to #4 or #5. Each already has 3 connections and is no more than 2 degrees removed from any other member of the community. I’d avoid #1 and especially #6. They have fewer connections and are further removed from the rest of the group.

This process tends to enrich nodes that already have a large number of links. Once the decision is made to link to either #4 or #5, that node would now be even more attractive to subsequent entrants. In organizations like this, the number of links for each node follows the power law distribution.

Scale Invariance

The fundamental feature of power law distributions is ‘scale invariance.’ For example, if you count cities of a certain size, cities half has large might be four times more common, and cities twice as large might be four times less common. If this pattern holds throughout the full range of cities, then you have scale invariance. This relationship means there is no typical city size. There will still be an arithmetic mean, but it won’t actually be the ‘average’ the way we understand it. There really is no average.

Success in College Football

What does any of this have to do with football? First, compare the NFL with college football. Think of the teams as strongly-linked clusters of individual players and coaches in the network of the overall league. The teams themselves are in turn linked and clustered by division or conference.

In college ball, elite players choose their team largely on their own, and it’s no surprise that they select their team based on the team’s current strength and the prominence of the coach. Players who aren’t recruited by the USCs and LSUs of the world will still prefer PAC 10 or SEC teams. And failing that, they’ll prefer any Division IA (or “Bowl Series”) school to the lower divisions and conferences.

The NFL is constructed differently. With the salary cap and the draft, the better players are distributed more evenly throughout the league. Its distribution of championship appearances is decidedly not a power law distribution. But BCS appearances by college teams certainly is:

Pro Bowl Selections

What does follow a power law distribution in the NFL is Pro Bowl appearances. Just like in your office where a minority of employees can account for most of the productivity, the talent in the NFL is distributed in a similar way. In doing my analysis for drafting defensive backs I noticed just how much Pro Bowl selections were concentrated among the top players.

Among all the defensive backs drafted from 1980 through 2001 who have had at least one year as their team’s primary starter, the distribution looks like this:

There are plenty of players with no appearances, a smaller group with 1 selection, and then a steadily decreasing number of players with 2, 3, 4, etc. selections. As you can see, the distribution approximates a typical power law distribution.

Here is the distribution for QB Pro Bowl selections. It approximates a power law distribution even better.

What does this tell us about Pro Bowl selections? Does this mean that being chosen for the Pro Bowl is based on how connected a player is? Partly--because the votes of other players and coaches weigh heavily in the selections, but that’s not what I’m getting at. Besides raw performance, it also depends on how popular the player is, how good the rest of his team is, how often the team plays on national TV, and how good he was in previous years. And all those things are correlated with each other--it's a complex self-organizing system of factors and influences. That’s one reason why we see the power law at work here.

Coaching Tenure

Another example of the power law in football is the tenure of coaches. This paper from the UK found that coaching tenure in the Premier League follows a power law very closely. They even looked at NFL coaching tenure and found the same pattern. I’ve done my own analysis and confirmed that coaching tenures in NFL obey the power law distribution. What the researchers conclude is that talent and ability has relatively little to do with how long a coach hangs on to his job. It mostly has to do with being ‘sacked’ or ‘poached,’ and with the random luck of his team. (For instance, Jon Gruden was poached from Oakland and sacked at Tampa Bay). Interestingly, the tenure of leaders of many kinds including Popes, British Prime Ministers, and Roman Emperors follow power law distributions.

Player Tenure

Although career length does not follow a power law distribution, years as a starter does. For example, of all the RBs drafted between 1980 and 2000, the majority will never be a starter, and the rest of the players have steadily decreasing chances of lasting long as a starter. Here is the distribution:

Why Any of This Matters

Power law distributions are noteworthy because they are the signatures of mature self-organizing complex systems. It’s also a feature of ‘rich-get-richer’ systems. So when we see power law distributions, we can make some qualitative inferences about the system we’re observing. For example, the BCS system is certainly a rich-get-richer organization. We can even quantify just how hierarchical it is and how difficult it is for second-tier teams to break into the elite.

The problem with the BCS isn’t just that it’s a rich-get-richer system. That’s just the natural way of the world. Even in supposedly ‘egalitarian’ systems like socialism, the rich still get richer. The difference is that initial outcomes in socialist systems are based primarily on one’s political connections, where in a free market they tend to be based on how productive or innovative one is. The problem is that the elite ‘nodes’ of the BCS have colluded to preserve their status on top, preventing a natural churn in who the elite are.

Understanding the implications of power law distributions also helps make more accurate models. For example, there really isn’t an average coaching tenure, and the standard deviation of tenure is not a meaningful statistic. Instead of applying the normal distribution and its associated analytical tools to everything we see, we should be more cautious.

If anyone is interested further in network theory and power law distributions, I recommend the book Nexus: Small Worlds and the Groundbreaking Theory of Networks. Regarding the current financial crisis and the misapplication of risk models, I recommend this prophetic 2005 WSJ article.

NBA Playoff Win Probabilities

2009-04-12T18:39:00.000-04:00

Live win probabilities for NBA playoff games are available at wp.advancednflstats.com/nba. Previous final games are here. The WP graphs include new features and stats. Some of the recent additions include:

Possession Value (PV): The value in WP of simply having the ball. Defined as the ‘next expected’ outcome of a possession, usually 20 sec off the clock, 10% chance of a 3-point gain, 35% chance of a 2-point gain, and a 10% of a 1-point gain.

Leverage Index (LI): A ratio of the current Poss Val to the an NBA game’s typical possession value, which is 0.04. So when the Poss Val might be 0.10 toward the end of a close game, the LI is 2.5.

Comeback Factor (CF): A measure of how big a comeback the game comprises. Based on the current winning team’s lowest WP at any point in the game. Adjusted to a scale of 1-100. Technically defined as 1/(lowest WP). For example, a team that has come back from a 0.10 WP to lead or win a game has a CF of 1/0.10 = 10. An epic comeback from a 0.01 WP would be 1/0.01 = 100.

Excitement Index (EI): (I need a better name for this.) Based on the net average deviation of the game’s WP from 0.50. A blow-out where one team takes the WP to 0.99 or 0.01 for most of the game will have a large deviation (boring). A game that teeters around 0.50 WP will have a small deviation (exciting). A game where the advantage swings wildly back and forth between teams (also exciting) will also have a small net deviation of 0.50 because the large swings cancel out.

One of the reasons I'm tinkering with the NBA win probability now is as a dress rehearsal for the upcoming NFL season. I'm working on analogous features and stats for NFL games. By the start of the NFL season I should have a pretty comprehensive WP site. Toward the end of last season, there was a healthy discussion on how to measure the 'excitement' value of a game. I think I've got a good foothold now on how to do that.

Here's a sample of what the NBA graph looks like right now. This is from a final game, a live game will have some added stuff.

If anyone is interested in the inner workings of the basketball WP model, you can check my post at the Wages of Wins blog. As always, comments and suggestions are appreciated.

Draft Analysis by Position

2009-04-09T11:37:00.006-04:00

It’s draft season again, and one of the most important aspects of draft strategy is assessing the value of players taken at the various spots throughout the draft. How well to 1st round QBs pan out compared to 2nd or 3rd rounders? How sure can a team be that the 1st QB taken will in fact turn out better than the 2nd QB taken? Can a team really find good RBs in the 3rd round compared to the 1st? Do some positions tend to be gambles compared to others which may tend to be sure things?

The purpose of this analysis is to quantify the scarcity of quality players in the various positions and the ability of scouts to actually identify who the better players are going to be. Knowing these things, teams can construct better draft strategies. For example, if the better RBs actually come from the first few taken each year and generally can’t be found in the later rounds, then teams looking for a RB need to plan accordingly. Additionally, if it’s found that top RB draft picks tend to be ‘sure things’ compared to other positions, then teams may prefer to fill other needs with ‘known-quantity’ veterans rather than relatively uncertain draft picks.

Last year I looked at most of the skill positions because they tend to be the focus of the most attention and they provide easier measures of performance with their stats. My analysis of each position usually follows a similar pattern.

I looked at how likely players taken in the various rounds would attend one, two, or three or more Pro Bowls. Although Pro Bowls are an imperfect measure of a player for several reasons, they do signal a player’s overall achievement in ways that individual performance stats cannot capture. Pro Bowl appearances also indicate whether a player was a ‘home run,’ something GMs are certainly looking for in the early rounds.

I also looked at other indications of a successful draft pick, such as numbers of years in the league and number of years as a team’s primary starter. Although a player may never make a Pro Bowl, he may still be a solid above average player, and that’s certainly of great value to the team who drafts him.

For positions like QB or RB that offer obvious performance measures such as yards per attempt (or even DE with sacks per game), it’s very helpful to look at those distributions too. Plus, a good measure of how well scouts can predict the better prospect is the likelihood that an earlier pick will actually turn out better than the subsequent pick at the same position.

Each one of these measures is imperfect for measuring the value of a draft pick in some way, but together they give us a very good idea of how scarce the various positions really are and how well scouts identify the better players.

One more thing I’ll point out is that draft round doesn’t tell the whole story. It’s also important to look at the overall order a player was taken within his position. In other words, not all first round picks are the same. If 3 QBs were taken in the first round, they’re going to have very different likelihoods of becoming a top passer.

I’ll recycle the positions I analyzed last year, partly because there are so many new readers this off-season compared to last:

QBs Part 1
QBs Part 2
RBs
WRs
DEs

Next, I’ll be looking at defensive backs, but I’ll also tie in an interesting observation about how career success is distributed among NFL players. And if anyone is waiting for part 2 of the Passing Predictability article, I’m going to put that on hold until after the draft.

Lastly, I’ll mention that most data comes from Pro-Football-Reference.com’s draft database, which is complete with great career data and features a great query tool.

SPECIAL ANNOUNCEMENT: New Feature

2009-04-01T03:00:00.004-04:00

It's with great pride that I announce the addition of another Advanced NFL Stats adjunct site. It all started here with the main site AdvancedNFLstats.com, then came the Win Probability site, followed by the Community site, and most recently the NCAA basketball win probabilities. But we're not done.

Based on requests from many, many readers, I am pleased to present Advanced NFL Stats -Love, a dating site where stat-heads from around the country can meet and find someone special to share their passion for mathematical football analysis.

Everyone is welcome, and for a limited time while the site is in beta, registration is free. Check it out! You've got nothing to lose...except maybe lonely nights hunkered over a spreadsheet of football data.

Advanced NFL Stats -Love

Win Probability Site Upgrade

2009-03-27T10:55:00.004-04:00

I'm currently working on major improvements in the function and feel of the win probability graphs. For those who have been checking in on the NCAA tournament win probs, you may have noticed a "2.0 Beta" link last night.

The new graphs not only look a lot better, but also have added features such as hovering transparent tooltip boxes for game scores and time remaining, and crosshairs for precise win probs at each point in the game. The scoreboard for each game is now integrated with the graph, but it's still a work in progress. The color scheme is still in flux as well. (I'm going for the wood of the court and the dark orange of a basketball. My banner will need to change to match.) I'd appreciate any suggestions you might have for the layout or colors, etc.

This will also give you an idea of what the football site will look like this fall. But the football version will be even better, with play-by-play available at each point on the graph, plus added stats such as 1st down probability, expected points, and scoring probabilities for the current drive. But you might not have to wait until fall. Part of my plan this year is to build WP graphs for every NFL game since 2000.

The upgrade is thanks to a suggestion by Ken Roberts of the great site "Sports Club Stats." He pointed me toward a very handy web graphing tool. His site probably deserves its own post, but I'll plug it now anyway. Sports Club Stats does playoff projections for most pro leagues and graphs them from the start to end of the season. Just as a win probability graph tells the story of a game, Ken's graphs tell the story of a season. For example, check out the heartbreaking Bucs' or Redskins' graphs for last season.