Software is the Achilles’ Heel of Mass Spectrometry

report in the Journal of Proteome Research describes a significant and widespread deficiency of data analysis software for mass spectrometry. Rob Smith of the University of Montana interviewed 100 users, the majority of whom were working in proteomics. The cohort also included scientists using MS in metabolomics, lipidomics, and extractables; 66% were affiliated with for-profit firms such as biopharma.

Perceptions of the users were dramatically different from those of the developers, including vendors. The problems seem serious, since there is general concern in science about the lack of reproducibility between reports in the literature and the experience of people who try to repeat or use the reports. This difficulty is not limited to mass spectrometry.

Some of the comments are:

  • There are so many software packages available that the user community is not able to focus on one or a small set. One exception: An industrial user reports that, “We evaluate every single software that comes out. They are all bad.”
  • The programs are so complex that users do not take the time to make thorough comparisons.
  • When programs were compared, the scientists found that the software gave poor results, lacked critical functionality, or excessive manual control. Manual control such as parameter setting accounts for 10–50% of the operators’ time.
  • The programs lack adequate technical support.
  • Customer responsiveness is not part of the culture of MS software vendors

Commercial software developers talked about competing with open-source software. Further, they say the open-source software provides misidentifications and incorrect quantitative results and are not transparent. Free software suppliers often charge for support and maintenance.

Papers that do not use the latest edition of software are not typically reviewed by journal editors. This could be a real problem with legacy data that were obtained with software that is not backwards-compatible.

Users in commercial biopharma labs report difficulty and time delays of calendar quarters to years to get intellectual property protection so the service provider can access the software installation. Workflow in academic labs differs significantly with industrial labs.

Perhaps the most disturbing result is that a “supermajority (86%) of developers felt that unaddressed problems were minor or nonexistent.”

Smith concludes that there needs to be more dialog between developers and users. I’ve suggested this to CASSS as a topic for the MS meeting scheduled for September 2018 in San Francisco.

Robert L. Stevenson, Ph.D., is Editor Emeritus, American Laboratory/Labcompare; e-mail: [email protected]

Related Products

Comments