PASE and SOAR: two key tourney performance metrics

If you’re new to Bracketscience.com, you may have come across these two acronyms and wondered what the heck I was talking about. PASE and SOAR are two measures of tourney performance—the latter measures the frequency that a team overperforms in the tourney and the former measures the degree of overachievement. I’ve had a couple of emails recently from people asking about these stats, so I figured I’d explain them early in the season to get everyone on the same page.

PASE is the primary metric I use to assess the degree to which a coach, team, conference or factor exceeds or falls short of expectations. PASE stands for Performance Against Seed Expectations. Over the 28 years of the 64-team era, each of the seeds has recorded an average number of wins. The 112 top seeds, for instance, have won 378 games since 1985—or 3.375 per team. When a team wins four games as a top seed, therefore, they beat expectations by 0.625 games. Using the averages of each seed, you can tally the positive or negative differences between actual and expected wins for any attribute you want to study—whether it’s an individual coach, teams with more than five straight bids, teams with all-Americans, et cetera. The total of these differences is then divided by the number of appearances to arrive at an average number of games the attribute either over-performed or under-performed per tournament.

An example will make this clear. Let’s look at Butler coach Brad Stevens. He’s gone to the tourney four times. In 2008, he won a single game as a seven seed. The average seven seed has won just 0.830 games per dance, so Stevens was .170 games ahead of expectations that year. In 2009, however, Stevens’ ninth-seeded squad bowed out in the first round of the dance, falling -.563 games short of expectations (that’s the average number of wins for a nine seed). So in his first two trips to the dance, Stevens was actually an underperformer, falling .393 games short of the number of wins that teams with his two seed positions typically amassed, or -.181 games per dance. That was his PASE before the 2010 tourney.

Then Butler went on two magical runs to the finals. In 2010, five wins that Stevens coaxed out of his Bulldogs were 3.839 games more than the typical number of wins a five seed gets in the dance. In 2011, Stevens was 4.304 games over projected eight seed wins Add the 2010-11 overages to Stevens’ previous .414-game deficit and he’s a whopping 7.752 games above seed expectations in four tourney trips, for a PASE of +1.938—nearly two games per dance better than his seed project win total. That’s really, really good. To put it in perspective, Tom Izzo is the biggest overachiever among coaches with at least five tourney appearances—and his PASE is just +.761.

Of course, Izzo has built his seed-defying record of overperformance in 15 tourney trips. So Stevens’ has a ways to go before he can claim to be as strong a tourney performer as Tom. And let’s remember: Stevens’ achievement was really built on two deep runs to the finals. Before that, he was an underachiever.  And that brings us to the concept of SOAR, or Seed Overachievement Rate. This is a much easier concept to explain. SOAR is a simple percentage measure of overachievement frequency. All you do is divide the number of overperforming tourneys by overall appearances.

Let’s return to our two overachieving coaches. Stevens’ SOAR is 75.0%, because he beat expectations in three out of four tries. Izzo’s SOAR is a notch lower. He beat expectations in 10 of 15 trips for a 66.7% SOAR. Both of those overachievement rates are strong…but they’re not the best among coaches with at least five trips to the dance. Sean Miller has exceeded seed expectations four times in five trips, for an 80.0% SOAR. However, the degree of Miller’s overachievement can’t match Stevens’ or Izzo’s; his PASE is lower at +.754.

There are coaches with big SOAR and small PASE values, like Frank Martin, and coaches with small SOAR and big PASE values, like Jim Larranaga. Martin is a consistent, if unspectular overachiever. He’s beaten expectations in three or four tries, but owns just a +.295 PASE. On the other hand, Larranaga has exceeded seed-projected wins just twice in five tries, but owns a top-ten PASE of +.584. His degree of overachievement is pretty much based on a single amazing tourney run. Larranaga, after all, was the coach of 11 seed George Mason in 2006, which reached the Final Four, exceeding expectations by 3.455 games. Take away that one dance and Larranaga is a -.134 PASE underperformer.

The moral of the story: when you’re assessing the performance of a coach, team, conference or attribute, remember to look at both their degree (PASE) and frequency (SOAR) of overachievement. Is the overperformance bankable—or just a flash in the pan?

This entry was posted in Basic Concepts. Bookmark the permalink.

2 Responses to PASE and SOAR: two key tourney performance metrics

  1. Blake Levine says:

    What about SOAR and PASE for conferences? And what would that say? If a conference had a negative PASE and low SOAR would that mean they are usually overrated in the tournament?

    • ptiernan says:

      I do an analysis of the conferences using PASE and SOAR. In fact, if you look under the Ratings menu, you’ll get a sortable list of all the conferences. The biggest overachieving tourney, not surprisingly, is the Horizon, home of Brad Steven’s amazing Butler Bulldogs.

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>