YORKTOWN HEIGHTS, N.Y.—In the case of amyotrophic lateralsclerosis (ALS), why do some patients—such as renowned baseball player LouGehrig—die quickly, while others—Stephen Hawking comes to mind—survive for manyyears? This is one of the questions that Dialogue for Reverse EngineeringAssessment and Methods (DREAM) Project leader and founder Gustavo Stolovitzkyhopes can be answered in the coming weeks by "the wisdom of crowds."
Established in 2006 by the IBM Computational Biology Centerand the MAGNet National Center for Biomedical Computing at Columbia University,DREAM's main objective is "to catalyze the interaction between experiment andtheory in the area of cellular network inference and quantitative modelbuilding in systems biology." The initiative's mission, according to itswebsite, is "to assess how well we are describing the networks of interactingmolecules that underlie biological systems … and how can we know how well weare predicting the outcome of previously unseen experiments from our models?"
Three of the four challenges for this year's DREAM 7challenge call for the development of informatics tools and methods that willsupport ongoing efforts to treat cancer and ALS. Stolovitzky, a Yale UniversityPh.D. and manager of functional genomics and systems biology at IBM, notes thatfor this year's challenges, DREAM organizers are partnering with the NationalCancer Institute, Sage Bionetworks and Prize4Life, a not-for-profitorganization that supports efforts to discover treatments for ALS.
The NCI-DREAM Drug Sensitivity Prediction Challenge willfocus on using genomic information to build models that can estimate thesensitivity of cancer cell lines to a set of small-molecule compounds—bothalone and in combination. The goal is to understand how well computationalanalysis of 'omics data—including proteomics data, SNP data, gene expressiondata and drug dose response data—can be used to predict drug activity in celllines, and ultimately to select the best treatments for patients based ongenetic profiling of tumors, Stolovitzky explains. The best-performing teamwill have the opportunity to publish its results in Nature Biotechnology.
The second challenge for the year, organized withPrize4Life, will attempt to develop methods that can predict the futureprogression of ALS. Prize4Life is collecting data from ALS patients and, usingthis data, the DREAM team will use combinations of mathematical and statisticalmodels to find a combination that defines fast and slow progression of thedisease. Emerging from this data may be chemical differences that are at thecausative root of ALS. Finally, the team will try to determine what drugs wouldwork best in each patient's case, Stolovitzky says. Prize4Life is offering a$25,000 prize for the winning submission in this category.
The third challenge, created in collaboration with SageBionetworks, aims to develop algorithms for predicting breast cancer survival.The best performer in this category will have a paper published in Science Translational Medicine.
The final challenge is "pure informatics" and focuses onnetwork topology and parameter inference. Participants are expected to developoptimization methods that accurately estimate parameters, predict outcomes ofperturbations and rewire networks in systems biology network models. This challengeis meant to help researchers "understand how to do experimental design when itcomes to reconstruction of gene regulatory networks, and understand how tocreate mathematical models, including what parameters to use," Stolovitzkysays.
Stolovitzky notes that in DREAM 1, he "created thescaffolding and did all the scoring." The contest has grown since, and nowincludes a number of collaborators to help him with the various evaluations."In 7," he observes, "results will be scrutinized as part of a peer-reviewprocess." For example, the journals that have agreed to publish results fromtwo of the challenges—Nature Biotechand Science Translational Medicine—willprovide what Stolovitzky refers to as "challenge-assisted peer review." Henotes that DREAM 7 has reached out to participants involved in previouschallenges as well as individuals who attend conferences on related subjects.
"We are as inclusive as possible," he says. "The more themerrier."
As its centerpiece, the DREAM process relies upon the wisdomof crowds, a process based on thetitle of a book published in 2004 and written by JamesSurowiecki aboutthe aggregation of information in groups and resulting in decisions that areoften better than could have been made by any single member of the group. Stolovitzkyis a believer, arguing that an aggregate solution is better than any singlesolution.
"Everyone knows alittle bit about something, but knowledge is diffused among many people. Wedistribute questions in such a way that participants compete to provide theirown important piece of the truth," he states.
He claims thataggregating the best results from teams in each of the categories from previouschallenges has resulted in improved methods and better results.
"We have verified this in challenge after challenge," hesays.
While the timelines haven't yet been finalized, it's likelythat most submissions will be due by mid-October, although entries for somechallenges could be due as early as Oct. 1, Stolovitzky says. The winningentries from each challenge will be presented at the DREAM 7 conference to beheld in San Francisco Nov. 12 to 14.
Prize4Life and ALS
Prize4Life was founded in 2006 by a group of HarvardBusiness School students when one of them, Avichai Kremer, then 29, wasdiagnosed with ALS. After his diagnosis, Kremer and his colleagues decided topilot an innovative new way to accelerate ALS research.
Prize4Life is a results-oriented nonprofit organization founded to accelerateALS research by offering substantial prizes to scientists who solve the mostcritical scientific problems preventing the discovery of an effective ALStreatment. In a statement to ddn, Dr.Neta Zach, the organization's scientific director, said: "Prize4Life wants togenerate new solutions and breakthroughs in ALS. One way to do this is tointroduce new minds to the research of ALS. For this challenge, we areappealing to computational savants, many of whom don't often have theopportunity to use their skills to generate meaningful change in human health,let alone a terrible disease like ALS. While the potential for Big Data toimpact health is much discussed, ALS patients, with their short lifespan, needto see these benefits as soon as possible. We are proud to be one of the firstinitiatives to actually develop and launch such a challenge and hope thatothers will follow."
The new Prize4Life Challenge is based on the PRO-ACT database,which upon completion will contain clinical data for more than 7,500 ALSpatients from clinical trials by companies including Sanofi, Teva and Novartis.The entire PRO-ACT database will be available for research purposes by year'send, and the ALS Prediction Prize will use a subset of this data.
Ultimately, it is expected that the solution resulting fromthis challenge will improve disease prediction and lead to more accuratemethods of forecasting progression earlier on in the course of the disease.
DREAM 7 and breastcancer survival
Based on initial promising clinical results, computationalapproaches to infer molecular predictors of cancer clinical phenotypes are oneof the most active areas of research in both industrial and academicinstitutions, leading to a flood of published reports of signatures predictiveof cancer phenotypes, notes Dr. Adam Margolin, director of computationalbiology at Sage Bionetworks. To evaluate such predictors, the Sage/DREAM BreastCancer Prognosis Challenge will create a community-based effort to provide anunbiased assessment of models and methodologies for the prediction of breastcancer survival. A common dataset containing gene expression, copy number, andclinical information for 2,000 breast cancer samples will be provided forparticipants to build models to predict survival time (median 10 years, alsoprovided for all samples). A novel dataset of 350 samples will be generated atthe end of the challenge and used to provide a final, unbiased score for eachmodel. More than 200 participants from 20 different countries have alreadysigned up for the challenge.
The journal ScienceTranslational Medicine (STM) has agreed for the best performing individualor team in the final evaluation (using the newly generated data) to publishtheir results as a "prize" for best performance provided its score is betterthan the score of a pre-defined baseline set of models. STM representativesagreed that having an evaluation committee re-run and compare all models in atransparent environment can serve the role of challenge-assisted peer review,allowing the results from the winning individual or team to be publishedwithout additional review.
