The Open Source Rankings Project

The Open Source Rankings Project

The Open Source Rankings Project

The Open Source Rankings Project

In an effort to create the most reliable ranking system for college lacrosse, has just introduced code that grades each team by strength of schedule, ratings percentage index, the USILA Coach's Poll, and a “power rank” that is computed by weighting game results against several variables (inclusive of the prior three components above) and coefficients.

You can see the ratings for the three divisions here: D1  D2   D3

The above links are also now permanently on each division's scores page.

We know there are other opinion polls, rankings, and algorithms out there already. But what makes's system unique is that we're going to tell you exactly how we calculate it, and we want your feedback and suggestions on how to make it even more accurate. In short, we're open-sourcing our rankings.

We'll be taking your suggestions (please leave in the comments of this story, which will have a permanent link from the ratings pages) and refining the code over the off-season with any of the ideas that will likely improve it. We'll then be posting various versions of the differing algorithms and be asking for your input as to which methods and weightings make for the most accurate solution. So, below, is how it works for now:

RPI (25%) - shorthand for Ratings Percentage Index, this is a pretty standard metric used by the BCS and NCAA basketball. It's comprised of a team's winning percentage (25%), the team's opponents' winning percentage (50%), and the team's opponents' opponents' winning percentage (25%). Note that only games inside of the team's division are counted, and games against the team for which the percentage is being calculated are excluded (ie, in calculating a Syracuse opponent's winning percentage the game against Syracuse itself is not counted). The max value for the RPI is 1.000

SOS (25%) - Strength of schedule. Another standard metric which is 66.6% a team's opponents' winning percentage and 33.3% a team's opponents' opponents' winning percentage. The max value for the SOS is 1.000

USILA Coaches Poll (25%) - We take the total points each team has received in that week's poll and normalize the numbers so that 1.000 is the max. For example, in the most recent poll Syracuse came in #1 with 213 points, while Villanova came in at #10 with 118 points. So we just divide every vote total by 213, giving Cuse a score of 1.000 and Nova 0.554. Doing it based on total votes, as opposed to the poll rank that USILA generates from those votes, makes the rank much more accurate. If there were 5 teams bunched up at the top that only differed by maybe 10 total votes, their rank should reflect that proximity. Otherwise we'd have to do it on discrete stepwise points system where #5 had a score 20% lower than #1, when that #5 team could very well only be a smattering of total votes behind the #1 team. PowerRank (25%) - This is calculated in two stages as a recursion is needed to generate the score. As with the USILA poll above, the end result is normalized so that 1.000 is the max score.

Part 1: We cycle through a team's schedule, excluding any games that were outside of their division, and create an initial (temporary) PowerRank to be used as the basis for the calculation in Part 2, each points result below being summed for the team in question as we go through all their games. If you lost the game you're awarded 0 points. If you won then the following points are awarded (as a sum of each line below):

- the opponent's RPI score * 0.75 (our RPI coefficient)
- the opponent's normalized Poll score * 1.0 (our Poll coefficient)
- 1.5 (the fixed var for winning a game) * the opponent's SOS score
- the margin of victory * 0.04 (our goal differential coefficient)
- if win was home we add -0.1, if away we add 0.1 (our fixed var for home advantage)

After summing the above for every game on a team's schedule, we then divide by the number of games played, and normalize this result so the top score is again 1.000. This result represents the initial PowerRank score for that team.

Part 2: We now cycle through every game on a team's schedule again, this time with the initial (temporary) PowerRank number available for each opponent. If you won the game then you get 4 * the opponent's PowerRank. If you lost the game, and your opponent's PowerRank is greater than 0.333, then you get 0 points. But if you lost to a team with a PowerRank <= 0.333 then we subtract the amount of the difference between 0.500 and PowerRank of the team to which you lost.

We sum these points for every game then, as we do in Part 1, divide by the number of games played and normalize that result so 1.000 is the max. The resulting number represents the real, final PowerRank.

We look forward to your feedback.

Post A Comment To This Story >

Close defeats
    by (#233158) on 5/06/11 @12:50AM
The one possible issue I have at first glance with this system is that other than in the coaches' poll, it doesn't take into account margin of victories/defeats. In my mind, there is a huge difference to losing to Syracuse in OT (like Georgetown or Hopkins) and getting blown out by the third quarter.

Reply to this

    by (#2) on 5/07/11 @12:16AM
there are teams that can literally be .500 but have every loss come by one goal from a top 10 team.

i'll pop a minor upwards adjustment in next week. there'd have to be a hurdle to clear though, cause a loss is still a loss. likely, in order to get awarded at least some points, you have to lose by no more than 1, and it has to be to a team with a PowerRank over .600

i'll be updating this story to include a changelog of any ideas that get implemented.
Reply to this

(no subject)
    by (#252624) on 3/21/12 @11:26PM
I think some consideration might be made to scale out the lax power ranking at the beginning of the season.

Right now I think the 15-20 on this poll might not be the most solid rankings, because Adrian for example has a great record with solid winning margins over middling competition. They definitely are not a top 20 team.

But at the end of the season the laxpower rankings are rock solid statistically. If you were to increase their weighting from the beginning of the season (date X to the middle of April) to have it arrive at 25% when the laxpower rankings are in their prime, this might be a better measure. I look at these rankings and see some deficiencies on the low end of the top 20'ish.
Reply to this

(no subject)
    by (#252624) on 3/21/12 @11:27PM
My dad works at Goldman Sach's and I can have him put one of his algo nerds on this for you if you want?
Reply to this

This is hysterical and very flawed
    by (#217112) on 3/03/13 @3:30PM
So you have a Duke team that is getting the snot kicked out of them #1? And Maryland, Hopkins, Notre Dame and Loyola are barely in Top 20? This is a complete joke as all 4 of those will be in the Final 4, likely. Maybe the guy from Goldman Sachs can figure out a better formula not using the Dookie math! Oh yeah that #20 beat Duke at Duke in the worst beat down since 1984 yesterday - oops!

Just for the record I cut & Pasted exactly what you said below:

How Are These Calculated?
Rank Name Poll Rank RPI SOS Power Rating Total Points
1 Duke (2-3)
2 Albany (1-1)
3 Syracuse (1-1)
4 Lehigh (2-2)
5 Ohio State (3-0)
6 Drexel (1-1)
7 Pennsylvania (1-0)
8 Fairfield (2-1)
9 Princeton (1-0)
10 Colgate (3-0)
11 Denver (3-1)
12 North Carolina (2-1)
13 Penn State (2-1)
14 Cornell (1-0)
15 Virginia (3-0)
16 Umass (3-0)
17 Loyola (2-1)
18 Johns Hopkins (3-0)
19 Notre Dame (2-0)
20 Maryland (3-0)
Reply to this