The Camelyon16 ISBI challenge took place on Wednesday, 13 April. The presentations from the organizing team are now available. Camelyons16 was closed in November 2016 following the launch of Camelyon17.

b'

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Title of presentationPresenterDownload
How computers shape the future of pathologyJeroen van der Laak
Camelyon16: Aim, dataset, and evaluationBabak Ehteshami Bejnordi
Statistics, Leaderboards, Results and Comparison to PathologistBabak Ehteshami Bejnordi
'

 

Public Leaderboard 1 - Whole-slide-image classification

  • The results are computed on the independent test set.
  • Evaluation 1: Teams are ranked based on area under ROC curve (AUC).

Top-five ranked teams until the challenge event deadline (Apr 1, 2016):

b'

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
RankTeamAUCSubmission dateDescription
01Harvard Medical School and MIT, Method 10.923401 Apr 2016
02EXB Research and Development co., Germany0.915601 Apr 2016
03Independent participant, Germany0.865401 Apr 2016
04Middle East Technical University, Departments of EEE, NSNT and HS, Turkey0.864201 Apr 2016
05NLP LOGIX co., USA0.829801 Apr 2016
\n'

Leaderboard including all submissions (updated after each new entry):

* Indicates that the team has achieved an AUC value that surpasses the AUC of the pathologist in our study.

b'

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
RankTeamAUCSubmission dateDescription
01 *Harvard Medical School and MIT, Method 2 (updated)0.993506 Nov 2016
02 *Harvard Medical School, Gordon Center for Medical Imaging, MGH, Method 30.976324 Oct 2016
03 Harvard Medical School, Gordon Center for Medical Imaging, MGH, Method 10.965007 Sep 2016
04The Chinese University of Hong Kong (CU lab, Hong Kong), Method 30.941529 Aug 2016
05Harvard Medical School and MIT, Method 10.923401 Apr 2016
06EXB Research and Development co., Germany0.915601 Apr 2016
07The Chinese University of Hong Kong (CU lab), Hong Kong, Method 10.908608 June 2016
08Harvard Medical School, Gordon Center for Medical Imaging, MGH, Method 20.908224 Oct 2016
09The Chinese University of Hong Kong (CU lab), Hong Kong, Method 20.905620 July 2016
10DeepCare Inc, China0.883305 Nov 2016
11Independent participant, Germany0.865401 Apr 2016
12Middle East Technical University, Departments of EEE, NSNT and HS, Turkey0.864201 Apr 2016
13NLP LOGIX co., USA0.829801 Apr 2016
14Smart Imaging Technologies co., USA0.820714 May 2016
15University of Toronto, Electrical and Computer Engineering, Canada0.814901 Apr 2016
16The Warwick-QU Team, United Kingdom0.795801 Apr 2016
17Radboud University Medical Center (DIAG), Netherlands0.778601 Apr 2016 \n
18HTW-BERLIN, Germany0.767601 Apr 2016
19University of Toronto, Electrical and Computer Engineering, Canada0.762101 Apr 2016
20BioMediTech, University of Tampere, Finland0.761201 Apr 2016
21Smart Imaging Technologies co., USA0.757401 Apr 2016
22Technical University of Munich (CAMP), Germany - Method 20.736730 Aug 2016
23Osaka University, Department of Bioinformatic Engineering, Japan0.731901 Apr 2016
24University of South Florida, Computer Science and Engineering, USA0.727001 Apr 2016
25NSS college of Engineering, India0.726901 Apr 2016
26BioMediTech, University of Tampere, Finland0.713201 Apr 2016
27Technical University of Munich (CAMP), Germany0.691001 Apr 2016
28United Institute of Informatics Problems, Belarus0.689001 Apr 2016
29VISILAB, University of Castilla-La Mancha, Spain0.653101 Apr 2016
30VISILAB, University of Castilla-La Mancha, Spain0.651301 Apr 2016
31Mines Paris Tec, France0.627701 Apr 2016
32Sorbonne Universites, Laboratoire d\xe2\x80\x99Imagerie Biomdicale, France0.556101 Apr 2016\n
'

 

Public Leaderboard 2 - Tumor localization

  • The results are computed on the independent test set.
  • Evaluation 2: The detection/localization performance is summarized using Free Response Operating Characteristic (FROC) curves. The final score is defined as the average sensitivity at 6 predefined false positive rates: 1/4, 1/2, 1, 2, 4, and 8 FPs per whole slide image.

Top-five ranked teams until the challenge event deadline (Apr 1, 2016):

b'

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
RankTeamscoreSubmission dateDescription
01Harvard Medical School and MIT, Method 10.693301 Apr 2016
02Radboud University Medical Center (DIAG), Netherlands0.574801 Apr 2016
03EXB Research and Development co., Germany0.511101 Apr 2016
04Middle East Technical University, Departments of EEE, NSNT and HS, Turkey0.388901 Apr 2016
05NLP LOGIX co., USA0.385901 Apr 2016
'

Leaderboard including all submissions (updated after each new entry):

b'

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
RankTeamscoreSubmission dateDescription
01Harvard Medical School and MIT, Method 2 (updated)0.807406 Nov 2016
02Harvard Medical School, Gordon Center for Medical Imaging, MGH, Method 30.760024 Oct 2016
03Harvard Medical School, Gordon Center for Medical Imaging, MGH, Method 20.728924 Oct 2016
04The Chinese University of Hong Kong (CU lab), Method 30.703029 Aug 2016
05Harvard Medical School and MIT, Method 10.693301 Apr 2016
06Harvard Medical School, Gordon Center for Medical Imaging, MGH, Method 10.596307 Sep 2016
07Radboud University Medical Center (DIAG), Netherlands0.574801 Apr 2016
08The Chinese University of Hong Kong (CU lab) - Method 10.544408 June 2016
09The Chinese University of Hong Kong (CU lab) - Method 20.527420 July 2016
10EXB Research and Development co., Germany0.511101 Apr 2016
11Middle East Technical University, Departments of EEE, NSNT and HS, Turkey0.388901 Apr 2016
12NLP LOGIX co., USA0.385901 Apr 2016
13University of Toronto, Electrical and Computer Engineering, Canada0.382201 Apr 2016
14Independent participant, Germany0.366701 Apr 2016
15University of Toronto, Electrical and Computer Engineering, Canada0.351901 Apr 2016
16Osaka University, Department of Bioinformatic Engineering, Japan0.346701 Apr 2016
17Smart Imaging Technologies, USA0.338514 May 2016
18The Warwick-QU Team, United Kingdom0.305201 Apr 2016
19Technical University of Munich (CAMP), Germany, Method 20.273330 Aug 2016
20BioMediTech, University of Tampere, Finland0.257001 Apr 2016
21BioMediTech, University of Tampere, Finland0.251901 Apr 2016
22DeepCare Inc, China0.243005 Nov 2016
23United Institute of Informatics Problems, Belarus0.226701 Apr 2016
24Smart Imaging Technologies, USA0.208101 Apr 2016
25HTW-BERLIN, Germany0.186701 Apr 2016
26Technical University of Munich (CAMP), Germany0.183701 Apr 2016
27University of South Florida, Computer Science and Engineering, USA0.179301 Apr 2016
28NSS college of Engineering, India0.165201 Apr 2016
29VISILAB, University of Castilla-La Mancha, Spain0.142201 Apr 2016
30Sorbonne Universites, Laboratoire d\xe2\x80\x99Imagerie Biomdicale, France0.120001 Apr 2016
31VISILAB, University of Castilla-La Mancha, Spain0.116301 Apr 2016
32Mines Paris Tec, France0.097001 Apr 2016
'