Breaking – Netflix Prize, we’ve got a winner, and it’s Greek! (updated)

It is to our great pleasure to share with you and the world that the Netflix Prize is finally over, with Nicholas Ampazis and George Tsagas being the core members of the winning team.

picture-2

The contest, featuring one of the biggest datasets ever published, opened new frontiers in the areas of Data Mining & Machine Learning and could easily be characterized as’ The Olympiad of Data Mining’. Netflix, the biggest online video rental service in the US, literally crowdsourced the optimization of its recommendation system, giving away -next to the data- a prize of $1M. (You may read more in the NYT or Wired, or you’d better wait for more updated articles to come).

To put this success in context, let me also mention that over the 2 years that the contest was running, more than 40,000 teams from over 160 countries participated. The final winner in a very tight race with consecutive submissions till the very last moment is The Ensemble team, the full member’s list is available here.

Nicholas Ampazis presented a few months ago in Open Coffee Athens XXII his own start-up, Feeds 2.0, and gave a fascinating speech on the Netflix Prize a couple of months ago in the greek Foss Conference, but rest assured that we’ll have him again to speak on this tremendous success in a Open Coffee event in the very near future. :)

update: Netflix will officially announce the winners in a couple of weeks, based on a separate test set to validate the robustness of submissions to overfitting.

update 2: It seems that, Bellkor’s submission was better in the test set, at least this is what a member of the team states at the netflix forum. And we really have no reason to argue against that, it looks like a proper time to congratulate the team. The official announcement is still pending.

5 comments ↓

#1 An Insider on 07.27.09 at 12:52 am

Sorry to ruin a party, but “BellKor’s Pragmatic Chaos” is the winner.
Even though the leaderboard shows “The Ensemble” at the 1st place, it is only on the Quiz set, which approximates the important one – the Test set. However, on the Test set BellKor’s won.

#2 George Tziralis on 07.27.09 at 8:44 am

Thank you insider, this is what I’m hearing, too. I’ve updated the post, still waiting for the official confirmation. Let me also state my personal view that Netflix’s approach of not instantly publicizing the actual test set results is kind of poor, even not so high end competitions like mill.ucsd.edu seem to have get this right.

#3 Kostas Stathakos on 07.28.09 at 12:19 am

Dissapointing, on the one hand, but keeping things in perspective… 40,000 teams? Wow..

This is still a big success, nonetheless.

A huge congratulations to Nicholas and George

#4 Nicholas Ampazis on 07.28.09 at 12:24 am

Thank you very much for the coverage.

From the official Netflix announcement, at present, we only know that:

“There are submissions from two teams that meet the minimum requirements for the Grand Prize.”

This means that both teams achieved at least 10% on the test set. At this level of accuracy, assuming that our test results will be slightly worse than quiz, the difference between the two teams is probably going to be meaningless. For instance, if an algorithm is restarted with different seed values for randomization, then the 5th decimal place of performance might be affected. From a machine learning perspective this is statistically insignificant and the outcome is much like a coin-flip.

At the moment the only comment we (“The Ensemble”) can make is that:

* We’re excited to be vying for the $1 million prize but we won’t know who the winner will be for several weeks
* We submitted our best work and now the Netflix Prize judges are validating the submissions
* Netflix tells us they will announce the winner in a few weeks

Thanks

#5 Netflixer on 07.28.09 at 10:15 pm

Nicholas,

I think your response here is particularly important, as the recent message of your team is unclear. Please clarify:
Am I understanding correctly from your post that you are also submtting your methods to Netflix to be judged?

In any case, you deserve much congratulations on your fantastic achievment!

Leave a Comment