Introduction to WAR: Wilmot's Advanced Rating

By Nate Wilmot on March 31, 2020 at 8:00 am
Dec 27, 2019; San Diego, California, USA; A general view of a military flyover during the playing of the national anthem prior to the Holiday Bowl between the Southern California Trojans and the Iowa Hawkeyes at SDCCU Stadium. Mandatory Credit: Kirby Lee-USA TODAY Sports
Kirby Lee-USA TODAY Sports
2 Comments

Background

The development of statistical analytics in college and professional football has been rapidly expanding over the last decade. No longer is a team solely assessed on simple statistics like total offense or points per game. Now, more advanced measures that include efficiency and explosiveness are analyzed on situational play-by-play bases that have dramatically increased the depth of understanding of a team’s schematic goals and execution. Oftentimes, these new statistics are used to predict future performance (i.e. the outcome of upcoming games) as much as they are used to explain past performance. One of the current standards in this regard is Bill Connelly’s SP+ (formerly S&P+) metrics. In SP+, Connelly combines the five factors of efficiency, explosiveness, field position, finishing drives, and turnovers for both offenses and defenses to generate a rating for both the offense and the defense (Off. SP+ and Def SP+) and an overall rating (SP+) which is the metric used to predict a team’s performance.

Other widely cited rating systems include ESPN’s Football Power Index (FPI), Fremeau Efficiency Index (FEI), The Sagarin Ratings, Massey-Peabody, The Power Rank, Simple Rating System (SRS), and others. Each utilizes different criteria and calculations to understand past performance and predict future outcomes of games. They all vary in their specifics but generally have the same goal as SP+: to use advanced statistical analysis to rank teams on past performance and predict the future outcome of upcoming contests.

Other common tools that are used to determine performance are simple calculations like yard-per-play or yard-per point. These have been used as the first level of “advanced” analytics and are a step beyond a simple exploration of a box-score.

Introduction

There is a high variability of the level of granularity needed to execute the systems mentioned above. Some require situational play-by-play data. Some call for the exclusion of garbage time and other specific situations. Others can be estimated with game-by-game data. My goal in this effort is to gain as much information as possible about how well a team matches up against another team using the highest-level, easiest to access inputs as possible. For the retrospective study, I am utilizing a variety of full-year data to set baselines of performance and correlate the rating metric to winning percentage, point production, and defensive points allowed. It should be noted that systems like SRS and others utilize opponent-adjusted point differential as the main evaluation tool.

In WAR we will consider two main factors: scoring explosiveness and ball control. In other methods, scoring explosiveness or efficiency has been explained by points-per-game, yard-per-point, points-per-play, points-per-drive, and several other statistics. For WAR, I am choosing to instead use points-per-minute of time-of-possession (abbreviated PPM) as my explosiveness metric. PPM is chosen because, except for overtime, every game lasts for the same amount of time (60 minutes) and this helps to normalize performance independent of the scheme. It’s an easier predictive tool than a per-play or per-yard metric (i.e. it is easier to estimate how much possession time a team may have versus their opponent's scheme than it is how many plays a team might run).

To highlight an extreme example of PPM, let’s assume two teams who each hold the ball for about 30 minutes per game and score about 30 points per game (1.0 PPM). We will specifically use 2012 Troy (30.6 PPG, 30.2 TOP, 1.01 PPM) and 2010 Georgia (30.2 PPG, 29.8 TOP, 1.01 PPM). From the standpoint of PPM, these teams are virtually equal and they both played 12 games against FBS competition. It took Troy 966 total plays (0.38 point-per-play) versus UGA’s 733 plays (0.49 points-per-play) and Troy had a yard-per-point of 15.9 while UGA had 12.6. Both other metrics make it appear that Troy and UGA had significantly different performances. Yet they generated nearly the same exact total points and PPM. We will discuss in later posts how this might be used in predictability.

For ball control, I am choosing to consider a yardage metric. Chewing up yards and clock helps to keep a team’s defense fresh, wears down the opposing defenses, and keeps potentially explosive opponent offenses off the field. For ball control, the metric that I’ve chosen to use is:

Ball Control (BC) = sqrt(Yard/Possession * min-TOP).

Yards-per-Play is considered one of the best metrics to evaluate a team’s efficiency. This alone does not indicate how well a team is controlling the ball, though. Let’s consider a couple of 10 play samples.

Team 1: 5 runs for 0 yards and 4 incomplete passes followed by a 90-yard pass play. Let’s assume that this total sample took 3 minutes of possession. This gives a yard/play of 9 but this scenario requires at least 4 possessions.

Team 2: 9 runs each of 5 yards (including 4 first downs) followed by a 45-yard touchdown run. This sample took 5 minutes off the clock and happens in one possession.

Both Team 1 and Team 2 had the same YPP average of nine but the BC for Team 1 would be 8.21 and for Team 2 it is 21.2.

So, in this scenario, Team 2 has a far better ball control rate than Team 1. Let’s assume that Team 2 had three other possessions, each with 6 plays, 30 yards, 3 min-TOP, and one more touchdown. We can then calculate the Offensive Effectiveness (OE) of each team:

  • Team 1 OE = 2.33 PPM * 8.21 sqrt(yard/possession*min) = 19.15
  • Team 2 OE = 1 PPM * 25.1 sqrt(yard/possession*min) = 25.1

So, while Team 1 is more explosive (more than twice as many points-per-minute) they have somewhat lower effectiveness than Team 2 given how much Team 2 controls the ball and generates yards.

The same principles can be applied to defenses where we calculate PPMD as points allowed per minute time on the field (mTOF) where mTOF is opponent TOP or 60 – team-TOP. Defensive Ball Control (BCD) is calculated as sqrt(yards-allowed/possession * mTOF). For both of these values, lower is better. From the perspective of the defense, minimizing PPMD is obviously the more critical factor since it’s tied to point allowance but if a team is susceptible to allowing yards (a high BCD) and they are coming up a team with a high offensive BC, you might predict that they will eventually break and yield points.

Application of WAR – A 2019 Retrospective

Evaluation of Offensive and Defensive PPM in 2019

Let’s first look at explosiveness, in terms of PPM in 2019. The top 5 teams were: UCF (1.63), LSU (1.53), Alabama (1.51), Ohio State (1.48), and SMU (1.45) against a national average of 0.92 PPM. The bottom five of ODU, Vanderbilt, Bowling Green, Rutgers, and Akron each averaged 0.51 PPM or less. The UCF number is notable considering their average TOP was 25.7 minutes per game whereas the next three teams averaged 30.7 to 31.8 min per game. UCF absolutely made the most of their opportunities. The graph below shows a good correlation between PPM-O and PPG. A higher x-axis value indicates a more explosive team.

PPM-O versus PPG\

Defensively, the national average was 0.93 PPM against and the top-five were Georgia (0.44), Clemson (0.46), Ohio State (0.49), Iowa (0.50), and Penn State (0.52). This indicates that these teams were the best at not allowing opposing offenses to capitalize on their opportunities.

Evaluation of Offensive and Defensive Ball Control (OBC and DBC) in 2019

In terms of OBC, the 2019 values range from 22.8 to 39.2 with a national average of 31.2. The unit of (Yard*Min/Possession)^½ is not particularly intuitive but it should be noted that higher is better. The national leaders in 2019 were: LSU (39.2), Minnesota (38.4), Oklahoma (37.9), Navy (37.3), and Utah (37.3). PSU had a below average 30.0 value, which conjures memories of offensive stalls and a lack of consistency throughout the year. The summary of 2019 national performance is shown in the graph below (a larger number on the x-axis is better).

OBV versus PPG

For DBC, lower is better, and the range was 24.6 to 40.0 with an average of 31.4. The top teams were Wisconsin (24.6), TCU (25.7), Ohio State (25.7), Iowa (25.8), and Minnesota (25.9). Penn State sat at a below average 33.3 (ranked 97th), which when combined with the 5th best PPMD nationally, highlights PSU’s bend-don’t-break defense that we saw in 2019. In the graph below, note the reversed x-axis. The "better" teams are within the green circle.

DBC versus PPG allowed

Note the slight differences in average PPMO versus PPMD and OBC and DBC. These are minor artifacts of the calculation that I don’t believe have a meaningful impact on the overall calculation and intent.

Putting it Together – Offensive Efficiency (OE) and Defensive Efficiency (DE)

OE and DE are calculated by multiplying a team’s PPM-O and OBC for OE and PPM-D and DBC for DE.

OE is the overall offensive performance metric that makes up WAR and we find that the average is 29.3 with a high of 60.3 (LSU) and a low of 9.02 (Akron). The 60.3 for LSU is substantially better than the #2 Ohio State (52.7) and is the 6th best value since at least 2009. The 2018 Oklahoma team set the mark for the past decade at 64.3.

On the defensive side of the ball: Georgia (12.1), Ohio State (13.0), Iowa (13.5), Clemson (13.3), and San Diego State (13.9) led the nation against a national average of 29.7. Penn State finished 9th at a value of 17.5. This highlights the utility of the calculation to blend the PPM and BC calculation. Penn State had a strong PPM-D and a poor DBC. When they’re combined, more weight is given to the PPM-D, which is more important, and Penn State moves back near the top.

In the graph below the best teams are in the upper right with strong OE (high number on the y-axis) and strong DE (lower value on the reversed x-axis).

Offensive Effectiveness versus Defensive Effectiveness - 2019

Overall Effectiveness

To calculate a team’s overall performance, we calculate OE and DE and then take the difference. Large positive numbers are better, and I have applied a strength of schedule metric at a conference level. Let’s look at the final 2019 top 25 by WARS. The data include all games against FBS competition for the year. As you can see, LSU finished #1 followed by OSU, Alabama, and Clemson.

Top 15 by OE in 2019

UCF, coming in at #5 is an obvious outlier and I will not suggest that I believe they were the 5th-best team in the country last year. But, they finished the season at 9-3 against FBS teams, lost their three games by a total of 7 points (2.3 points/game), and had a margin of victory of 23.3 points per game in their wins. They may not have had the buzz of being the unofficial national champions, but they were a very good football team.

When we compare the final Top-10’s of WAR versus SP+, FEI, SRS, and pure point differential, we find some consistency but also some quick between the systems. Each of the top-10 lists for these systems are below. We find that only WAR picks LSU as the top team and each system has the same four teams in the top 4. From there though things diverge and there is little agreement between the methods. Penn State came in at #12 in WAR (behind Minnesota at 11).

Ratings comparison - 2019 top 10's

 

Summary

To recap, WAR is a new rating system that can give a retrospective view of performance against both offensive and defensive performance through a combination of points-per-minute (PPM) and ball control (BC) metrics. These, like many of the other rating systems available, can be combined to provide an overall effectiveness (OE) rating that does very well at rating the performance of a team. In future postings, we will explore the following: Penn State and Kirk Ciarrocca’s performance over the last decade, a look at the power of WAR to retroactively pick games, and look forward to the possible 2020 season.

2 Comments
View 2 Comments