Develop an algorithm which can most accurately determine skeletal age on a validation set of pediatric hand radiographs.
The Pediatric Bone Age Challenge will utilize three skeletal age datasets acquired from Stanford Children’s Hospital, Colorado Children’s Hospital and the University of California Los Angeles. A training set of hand radiographs and corresponding skeletal ages will be provided to the participants.
Artificial intelligence (AI) algorithms have existed for decades, and have recently been propelled to the forefront of medical imaging research. Two factors are primarily responsible for this revolution: very powerful computer hardware available at relatively low cost, and the recognition that a certain types of algorithms are particularly well suited to image analysis. The latter discovery was made possible through the IMAGENET competition and represents the power of a fundamental transformation in research mechanics.
Currently, most research studies collect data, perform analysis, and publish results. The same researchers may continue to augment and expand the data set and perform subsequent analysis, with resulting publications. The data for each study is held quite closely, and outside of multi-center trials is rarely shared amongst institutions. Competitions are fundamentally different model of research: research data is made available to the public, usually with a baseline performance metric. Groups around the world are invited to analyze the data, and create algorithms to beat performance of the prior generation.
This model is so compelling in the rapidly growing data science community that a startup called Kaggle, which hosts ML datasets, competitions and algorithms was recently purchased by Google.
RSNA Mission Statement: The RSNA promotes excellence in patient care and health care delivery through education, research and technologic innovation.
ML competitions align with the research and technologic innovation aspects of the RSNA mission.
The availability of training and test data sets will adhere to the following schedule:
Training Data Available: Aug. 5
Training Phase: Aug. 5 - Sept. 1
Leaderboard Data Available: Sept. 1
Leaderboard Phase: Sept. 1 - Oct. 7
Test Data Available: Oct. 7
Submission of Test Results: Oct. 7 - Oct. 15
Review and Confirmation of Results and Notification of Awardees: Oct. 15
Acknowledgment of Awardees at RSNA Annual Meeting: Monday, Nov. 27
Participants should submit a csv file with the case ID and the predicted age in months. A prediction should be provided for all cases in each phase. A zip file containing this csv file as well as a text file describing the algorithm is uploaded as the participant's submission. An example test submission is available here
The primary evaluation measure is Mean Absolute Distance (MAD), calculated as the mean of the absolute values of the difference between the model estimates and those of the reference standard bone age.
In case of ties, the concordance correlation coefficient (CCC) will be used to break ties.
We ask that you personally abide by codes of honor and ethical behavior in your participation. Only one registration is allowed per participant and compliance will be monitored. During the Test phase, starting on October 7th, 2017, registration to the site will be closed and only existing participants/teams will be able to submit results (maximum of 3 submissions). Team creation will also be disabled during the Test phase. We view this competition as a unique opportunity to further contribute to the field of machine learning in medical imaging and we appreciate your most legitimate efforts to make this activity a great success.
Organizers of the challenge and groups that provided the data sets used are not allowed to participate as entrants in the challenge.
Questions about the terms and conditions of the challenge should be directed to the RSNA Informatics Department: firstname.lastname@example.org.
The Radiological Society of North America (RSNA) Radiology Informatics Committee (RIC) Pediatric Bone Age Machine Learning Challenge Organizing Committee:
Kathy Andriole, Massachusetts General Hospital
Brad Erickson, Mayo Clinic
Adam Flanders, Thomas Jefferson University
Safwan Halabi, Stanford University
Jayashree Kalpathy-Cramer, Massachusetts General Hospital
Marc Kohli, University of California - San Francisco
Luciano Prevedello, The Ohio State University
Data sets used in the Pediatric Bone Age Challenge have been contributed by Stanford University, the University of Colorado and the University of California - Los Angeles.
The MedICI platform (built CodaLab) used for the challenge is provided by Jayashree Kalpathy-Cramer, supported through NIH grants (U24CA180927) and a contract from Leidos.
The challenge is coordinated by the RSNA. Questions about the challenge can be directed to the RSNA Informatics Department, email@example.com.
Start: Aug. 5, 2017, midnight
Start: Sept. 1, 2017, midnight
Start: Oct. 7, 2017, midnight
Oct. 15, 2017, midnight
You must be logged in to participate in competitions.Sign In