Model performance: nothing spectacular, but most still in running

Heading into the second day of the round of 32, most of the bracket models posted under the Tips+ section still have a fighting chance of success. All but two of the models are above the 60th percentile–and even Final Four/Champ has its Final Four intact. The only one that’s dead in the water is “Upset/Toss-up.” That’s the one that crazily liked Belmont in the Final Four. Hey–the West is certainly the wildcard region. The model just picked the wrong wildcard. Here are the results:

  • Model #5 Pulse Check, 340 – 95.3
  • Model #6 Factor PASE, 320 – 84.1
  • Model #7 Baseline, 320 – 84.1
  • Model #12 Nate Silver, 320 – 84.1
  • Model #13 Coach + Six, 320 – 84.1
  • Model #14 Costanza, 320 – 84.1
  • Model #15 Keeper, 320 – 84.1
  • Model #1 Kenpom, 310 – 73.0
  • Model #4 Eff+Coach, 310 – 73.0
  • Model #8 Seed Match-ups, 310 – 73.0
  • Model #11 Contrarian, 310 – 73.0
  • Model #3 Outcome, 300 – 61.3
  • Model #10 ESPN BPI, 300 – 61.3
  • Model #2 F4Champ, 260 – 18.4
  • Model #9 Upset/Toss-up, 260 – 18.4

I find it interesting to track the performance of Nate Silver, Ken Pomeroy and Dean Oliver (architect of ESPN’s new BPI system) against the no-brainer higher-seed strategy (model #7). So far, Silver and the no-brainer approach have a one-pick lead on Pomeroy…and Oliver is 20 points off. The higher-seed approach doesn’t have a chance of winning though: it liked Gonzaga going all the way. Nobody told it that Mark Few is snakebit.

This entry was posted in General News. Bookmark the permalink.

9 Responses to Model performance: nothing spectacular, but most still in running

  1. HoopAndAPrayer says:

    Very interesting to track this. I’m not sure how long ratings approaches have been in existence, such as BPI… But assuming we can use the algorithms to calculate historical ratings, would be intriguing to look at previous tournament results to see which preform well over a significant # of samples. Also, any thought on whether some of these different models can be “averaged” together to produce and aggregate tournament result that is superior to the individual models? The example I recall was a study (sorry, no link) that looked at movie box office predictions and found that the average of a group of 5-10 experts was superior to any individual expert’s predictions over time.

    • ptiernan says:

      That approach tends to yield a very chalky braket. But it might work. The ESPN BPI results are new, since this is the first year of its existence.

  2. Jonathan says:

    Very interesting info. I think it would also make sense to show each brackets max number of points they can still get. That would show very different results I think.

    • cderrick77584 says:

      Good point.

      The Upset/Toss-up actually has 3 FF teams left and the championship game in tact. I would think there are quite a few PPR.

      It will lose some ground with nothing coming out of the West but if Ohio State goes down it will even out some.

      I don’t ever use this model and I have been coming here for a long time, but it is probably my favorite to track.

  3. jbessa says:

    I like kenpom efficiency ratings but not the pythagorean calculation. I multiplied adj tempo by the difference in adj off and adj def to create an adj margin of victory. I then ranked people on this adj MOV and advanced the better team. It is currently 97% percetile with a chalky bracket but it has Gonzaga in the F4 so it will start to perform worse now. I hated Gonzaga but I wanted to check the strict performance of this methodology. There were a few differences in early rounds versus straight kenpom (Iowa St advanced over Notre Dame); Arizona into Sweet 16. It also has Indiana beating Lousville in championship (not Florida).

    • jbessa says:

      I also created a bracket using the consensus of 11 of the 15 models on bracket science. I can’t remember which 4 I left off but one was baseline and a few others I thought were not as theoretically sound or were redundant. This bracket science concensus pick is also in 97% but I like it better than kenpom “adj MOV” since it moved Ohio St into Final 4 over Gonzaga. Otherwise it is very similar to kenpom “adj MOV”.

      • ptiernan says:

        Send me the specifics on that formula, Jbessa. I may want to track it. Also, can you keep me updated on the “Bracket Science Consensus” model going forward? I should’ve thought of doing that.

        • jbessa says:

          Formula…
          Using KenPom rating headings
          AdjMOV = (AdjO – AdjD) * AdjT

          If you think there is any flaw in the theory of this formula let me know. It doesn’t move things that drastically but there are some subtle changes.

          Some big movers are Iowa St who “was” 37th on kenPom pythag moves to 26th on adj MOV. They currently are 30th on kenPom pythag after going 1-1 in tourney. Another big mover was Cincy which moved from 40th on pythag to 49th using adj MOV.

          I will keep you posted on the bracket science consensus as well.

  4. BillM says:

    Here’s a simple model that’s working well and with a little luck could have really made some hay.
    Big Six Chalk – Take the highest seed from the six Power Conferences. This provided 22 winners in the round of 64, 12 in the round of 32. Seven of the elite 8 are still alive and all of the Final Four. The only miss for a potential Elite 8 sweep is Kansas State who would be favored tonight if they hadn’t laid an egg in the first half against La Salle or could have held onto their late lead after a 19 point turn-around in the second half of that game. Looking forward to seeing how this plays out. How often have we seen a Butler or VCU run in the last 20+ years?

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>