The organizers of sbv IMPROVER (Industrial Methodology for PROcess VErification in Research) are currently accepting submissions for the Species Translation Challenge, an innovative crowd-sourcing initiative designed to help define the limits of in vitro rodent models as predictors of human biology. This second sbv IMPROVER Challenge presents another valuable opportunity for scientists around the world to work with high-quality data sets, improve their research methods, compete for research funding, and gain recognition among the scientific community.
The sbv IMPROVER project is a collaborative effort that includes scientists from IBM Research (Yorktown Heights, NY) and Philip Morris International (PMI), Research & Development (Neuchâtel, Switzerland). The project was initiated to develop a more transparent and robust process for assessing complex scientific data in systems biology (the study of biological organisms, viewed holistically as integrated and interacting networks of genes, proteins, and biochemical reactions). This approach has implications for a wide variety of industries including pharmaceuticals, biotechnology, nutrition, and environmental safety–essentially any area that requires a more meaningful scientific analysis of Big Data. For PMI, the verification of data derived from systems biology is planned to form part of the assessment of an innovative range of products, currently in development, which may have the potential to reduce the risk of smoking-related diseases.
Species translation is getting a lot of attention at the moment, with the FDA looking carefully at how it can be applied to advance medical development. Donna Mendrick, Ph.D., Director of the FDA’s Division of Systems Biology, presented at the sbv IMPROVER Symposium in Boston, MA, in October 2012, highlighting the need for new species-translational biomarkers to empower personalized medicine. The Species Translation Challenge asks participants to verify that translational mapping can enable prediction of the biological effects of “stimulus-induced” genetic alterations in one species, given information about the same alterations in another species. It is divided into four subchallenges:
- An intraspecies protein phosphorylation challenge that asks: Can rodent phosphorylation status be inferred from rodent gene expression data?
- An interspecies protein phosphorylation challenge that asks: Can human phosphorylation status be inferred from rodent gene expression and phosphorylation data, as well as human gene expression data?
- An interspecies pathway perturbation challenge that asks: Can rodent gene expression data be used to infer responsive gene sets and pathways in humans?
- A species-specific network inference that asks participants to consider whether a consensus network can be constructed from gene expression, phosphorylation, and cytokine data.
The first sbv IMPROVER Challenge asked participants to identify diagnostic signatures in four disease areas: chronic obstructive pulmonary disease (COPD), lung cancer, multiple sclerosis, and psoriasis. The Diagnostic Signature Challenge ran in 2012 and attracted the participation of 54 teams from America, Asia, and Europe. Once the Species Translation Challenge is complete, other challenges are planned to look specifically at COPD, the third leading cause of death in the U.S.A.1 and a tobacco-related disease that is a focus for the scientists at PMI Research & Development.
PMI is funding this project to develop and use systems biology verification techniques, like those being explored in sbv IMPROVER, to complement traditional peer review and ensure the quality of the methods with which they intend to analyze the nonclinical assessments data of their novel range of products. The FDA is developing guidance for the assessment of what they term “modified risk tobacco products” (MRTPs), but since there is currently no regulatory framework in place, initiatives like sbv IMPROVER could provide a valuable contribution that enables independent and unbiased verification of research processes and methods. It could also support scientific substantiation of reduced risk.
Beyond PMI’s interests, the sbv IMPROVER methodology is expected to also be applied to a number of different fields that require the complex analysis of Big Data. To date, systems biology has no standard methodology in place to verify results, claims, and processes, and traditional peer review of systems biology research has struggled with the volume and complexity of the data that are generated. The U.S. National Academy of Sciences has highlighted that verification of systems biology could make major contributions to food production, improvement of human health, optimized biofuels, and ecosystem restoration.2 Using sbv IMPROVER to accurately analyze and verify systems biology data thus has the potential to accelerate product design and development and facilitate policy decisions.
sbv IMPROVER combines and incorporates high-performance computing technology, algorithmic models, and crowd-sourcing techniques to provide a methodology that can evaluate the robustness of a research work flow. The methodology involves dividing the work flow into building blocks, small functional units of a research pipeline that are based on input–output relationships. When precise knowledge of the input–output relationship is not possible, sbv IMPROVER helps verify which building blocks work synergistically in a research work flow. If a building block is verified, then its outcomes can be trusted with a higher degree of confidence.
Approaches like sbv IMPROVER may also help to relieve the intense pressure that the traditional peer-review process faces. While peer review remains the cornerstone of data analysis and is the gold standard for assessing and verifying scientific data and results, the process faces a number of challenges related to time, accuracy, reliability, and cost. In recent months and years, there have been an increasing number of high-profile retractions of scientific manuscripts after publication. The sbv IMPROVER project is designed not to replace, but to complement, peer review by addressing many of these challenges, enabling more rigorous evaluation of large and complex sets and verifying the conclusions reached by peer review.
The Species Translation Challenge is planned to stay open for submissions until early August this year. Participation is particularly encouraged from scientists in commercial entities and also from academic or research institutions. Submissions will be scored by a team of experts from the IBM Computational Biology Centre and independently reviewed by the sbv IMPROVER Scoring Review Panel.
A number of incentives are available; for example, the best performing teams may be eligible to receive financial support for further research. The best performers will also be invited to present their work at an international scientific symposium that is planned to be held in Athens, Greece, in October. The results of the challenge and the outcomes from the symposium are also planned to be published in peer-reviewed journals. For further information and to join a pioneering community working together to improve how scientific research is verified, visit the sbv IMPROVER web site at www.sbvimprover.com.
- Qaseem, A.; Wilt, T.J. et al. Diagnosis and management of stable chronic obstructive pulmonary disease: a clinical practice guideline update from the American College of Physicians, American College of Chest Physicians, American Thoracic Society and European Respiratory Society. Ann. Intern. Med. Aug 2011, 155(3), 179–91; http://annals.org/article.aspx?articleid=479627.
- The U.S. National Academy of Sciences. A New Biology for the 21st Century. National Academies Press: Washington, DC, 2009.
Manuel C. Peitsch, Ph.D., is Vice President, Biological Systems Research, Philip Morris International (PMI), Research & Development, Quai Jeanrenaud 5, 2000 Neuchâtel, Switzerland; tel.: +41 (0) 58 242 2111; e-mail: SBVINFO@ch.ibm.com.