52-team Quality Curve says: “There will be blood!”

For the last couple months, I’ve been promising that I’d do a full Quality Curve of the top 52 teams in the tourney (the presumptive 1-13 seeds) once the brackets were set. I finally got around to it this morning—and what I see scares me. We could be in for one wacky dance. Feast your eyes on how the efficiency of the 2013 field compares to average fields (thick light blue), the chalky 2007 tourney (orange) and the crazy 2011 dance (dark blue):


First, a few words about 2007 and 2011. Look at the orange line. Is it any wonder that tournament was historically free of upsets? You had high quality teams from the top seed all the way down to what would be the ten seed—and then historically bad low seeds, where the upsets tend to come from. On the other hand, look at the thin blue line: the top ten seeds were all well below average…then the quality came up to average from about position 38 to 46 before tailing off dramatically.

Now, look what we have this year. The four most efficient teams—Florida, Louisville, Indiana and Gonzaga—are about average compared to their historical counterparts. Then the fifth (Ohio State) through 13th (Syracuse) best teams start drifting toward the 2011 line of all-time weakness. And from the the 14th best team (Miami) all the way to the 41st best team (Belmont), you’re looking at squads whose efficiency is worse or only slightly better than the quality of teams in the craziest dance of the modern era.

But it gets worse. Instead of the presumptive upset seeds (positions 42 to 52) tailing off, they actually get better than average—much better. In fact, they’re solidly better than even the 2007 tourney. Uh-oh. Setting aside the very best teams, we have historically weak second-echelon squads and historically strong Cinderella teams.

I had made the case a few weeks back that there wasn’t parity in college basketball. But this curve suggests that there most definitely is. It also suggests that we are in four a lot of upsets. Could we break the record of 13 shockers set in four different years—1986, 1987, 2002 and 2011? Will the Madometer soar past its record reading of 19.8% unpredictability? I don’t know…but suddenly I’m thinking a safe, chalky bracket isn’t going to get it done.

This entry was posted in Measuring Madness, Upset Spotting. Bookmark the permalink.

31 Responses to 52-team Quality Curve says: “There will be blood!”

  1. Gary Diny says:


    Great work as always. just wondering if you have looked into the upsets during the efficiency data years to see if there is a minimum difference in the seeds that leads to upsets, possibly at different seed levels. Example a 6 seed with .9000 efficiency and an 11 seed at .8750 with the difference being .0250 being more likely to spring the upset than with a difference of .0300.


    • ptiernan says:

      I haven’t done an exhaustive study, Gary. But that’s exactly the approach I used in my Outcome Matching bracket. I only have one year of results, but it did get in the 99th percentile of tourney brackets last year.

      • BobbyBuckets says:

        I noticed you have St. Mary’s going far in that Outcome Matching model. Would you have the same if Middle Tennessee wins tonight? How would the Outcome Matching model be effected if St. Mary’s loses the First Four game this evening?

    • JD says:

      The way I calculated percentage of victory was as follows:

      Step 1: Calculate Difference in KenPom*(1+COACH).
      Step 2: Look at mean (~.05) and variance (~.025) difference in KenPom*(1+COACH) per upset over last 5 years.
      Step 3: Use Standard Distribution to identify probably of victory given a specified difference.
      Step 4: Apply expected value of victory given the type of bracket you’re playing. For example, one bracket I’m in gives points based on the following:
      (Round # – First Four is NOT Round 1)*(Seed of Correct Pick)
      So if VCU(5 Seed) won in round 2 it would be worth (5)*(2) or 10 points.

      I multiply the probably of a team winning times their points for victory to get an expected value. I do the same with the other team that they are playing. Whomever has a higher expected value moves on.

      I ran a maximization linear programming model to maximize expected outcome.

      Amazingly close to the Outcome Matching results. But, then again, my seeds are weighted. Straight points would probably be much more chalky.

      • ptiernan says:

        Very impressive, JD. Let me know how this approach works in the ESPN tourney challenge. You’ve thought through a couple things (like pool scoring) that I haven’t contemplated in my models. Thanks.

      • John says:

        What were the results..who was your F4?

  2. ptiernan says:

    Great question Bobby. And I’ve alluded to this a couple times in the flood of comments. If St. Mary’s goes down, I’m going to redo the Outcome Matching bracket, using the same “Efficiency proximity” rules as before. But with St. Mary’s out and Middle Tennessee in, the Cinderellas might alter slightly. I haven’t gotten that far yet. Thanks for giving me a forum to clarify that.

    • Tom says:

      For what it’s worth, MTSU ranks higher than Memphis in terms of efficiency as well. But they might be less likely to beat Michigan State.

  3. DrMiller says:

    I Love this graph and breakdown, great work. My question is can this be translated into a percentage. Meaning can we break this down and come to a percentage of how much more madness we should see from 2011. Thanks!

  4. John R. says:

    I picked 8 “upsets” in this bracket. I did not count seeds like 10 over 7 as upsets (can’t remember if they count or not). I had ten but narrowed it to eight eventually. I figured that was a safe number, while picking upsets. My crazy half of the bracket is the bottom half of the East. I have Davidson in the Sweet Sixteen, and Illinois in the Elite Eight. Just don’t believe in Miami at all. I can see any four coming out of the Davidson, Marquette, Bucknell, Butler pod. Can’t wait to get this madness started….

  5. Dave K says:

    I think we all need to start reserving ourselves to the fact that our brackets will probably be running red with ink by the end of the weekend, especially since the seed by seed guide doesn’t necessarily point to a lot of upsets (low seeds meeting upset parameters but not their opponents for victims and vice versa).

    I’m very interested to see if the Final Four qualifications for the top 3 seeds you posted a little while back hold up because that would be very helpful in “risk managing” future tourney that have a quality curve similar to this year.

    Thanks for all the info Pete.

  6. John R. says:

    I’ve got one more question, Pete. The last few years, you had it available to all of us to play a Great 8 game. Are you doing that this year? I enjoyed that so much in years past….

  7. ptiernan says:

    I’m sorry to say, John, that we’re not doing the Great Eight this year. With the site switchover…and my added commitments to CBS…we just couldn’t swing it. Sadly, I have a day off in the non-tourney season!

    • John R. says:

      That’s sad, but I understand. It was one of my favorite bracket games to play on-line because of the relatively small number of participants (compared to the numerous Bracket Challenges) and the fun rules. I should run one myself next year, maybe.

  8. Jonathan says:

    So my championship match-up keeps coming down to Indiana and Louisville. The toss-up “stats that matter” chart suggests Indiana. The advanced toss-up rules suggest Louisville. The Seed Guide 2013 suggests Indiana. And the Final Four/Champ rules suggest Indiana. Then outcome matching aka curve-fitting goes back to Louisville! Everything contradicts. So… who wins?

  9. BobbyBuckets says:

    Are we going to see what teams the statistical rules say are Final Four contenders and the ultimate champ? I’m very interested in that.

  10. dee qualls says:

    Pete, are we going to get to see your bracket before Thursday morning?

    Thanks. Enjoy the dance.

  11. Tom says:

    Not that it’s anything out of the ordinary, but I’m noticing a lot of paid college hoops analysts with very chalk-heavy brackets.

    Almost makes me wonder if they’ve been paying attention for the last two months. (Also, why these guys are paid college hoops analysts.)

  12. Michal says:

    I’ve watched Gonzaga quite a bit this year and think they are definitely good enough to win it all but have a hard time picking them because of the “snake-bitten coach” aspect (plus the mid-major thing). My question is, should there be different rules on that for coaches who have only been at mid-major programs? Mark Few’s SOAR is 61.5% and his PASE is basically 0. So his teams on average perform how they should though more often than not they perform better than their seed. While Pitino, Self, and Donovan have SOAR’s of only 50% but higher PASE (much higher for Pitino and Donovan). This tells me their teams either greatly outperform or completely flame out. Is there any data for mid-major coaches like Few that maybe they shouldn’t be lumped in with the likes of John Thompson III (I realize he has made one Elite 8)? Maybe Few has just had too few teams actually good enough to make the Elite 8?

    • ptiernan says:

      That may be true, Michael. But 14 trips without an Elite Eight is a lot. I can forgive a coach like Fran Dunphy, who’s had a low average seed. Let’s just say this: Few hasn’t exactly distinguished himself in the tourney. This is his year to get at least to the E8. If that doesn’t happen with this quality, then his legacy of underachievement will come much more to the fore. Look: he’s not Steve Alford, Jamie Dixon or Mike Brey bad (hey! they’re all in the same region!), but he’s not Donovan/Izzo good either.

      • Michal says:

        I agree that this has to be this year and didn’t mean to imply that he’s as good as the likes of Donovan. I’m a Texas fan and can’t stand Barnes. Use to always hope Few would go to Texas but I’ve begun to question that over the last couple of years.

  13. Jason Rosko says:

    Am I crazy or is Montana not on the graph?

    • ptiernan says:

      They may not be if they weren’t one of KenPom’s top 52 teams. It doesn’t go by seeding.

    • ptiernan says:

      This is the KenPom top 52…Montana mustn’t be in it.

      • Scott says:

        I don’t get how comparing the top 52 in KenPom relates to upsets. Seems like ordering by seed would make more sense, as it would show the difference in strength of the lower seeds.. Another way to get at it would be to show the Pythag difference for each matchup. In this view, narrowing margins would indicate more parity.
        Keep up the great work, Pete!

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>