Artificial intelligence tools designed to speed up the research process by helping to extract data have led to a flood of “low-quality” research papers that threaten to damage the “foundations of scientific rigour”, according to a new study.
A research team at the University of Surrey looked at papers that used the US-based National Health and Nutrition Examination Survey (NHANES) to propose links between health conditions, lifestyle and clinical outcomes.
Such studies have ballooned in recent years, rising from just four published between 2014 and 2021 to 190 in 2024 alone.
Many of these papers were found to be low quality and used a superficial and oversimplified approach to analysis – often focusing on single variables.
ADVERTISEMENT
Narrow datasets were cherry-picked with seemingly little justification, which the researchers said raised concerns about poor research practice, including data dredging or changing research questions after seeing the results.
The paper, published in PLOS Biology, warns that as AI makes data mining more accessible, and large language models can help write papers, “manufactured publications [could] outnumber legitimate publications in certain data-driven fields”.
ADVERTISEMENT
This could “increase the risks of misleading findings being introduced to the body of scientific literature”, it adds.
“Our analysis is not intended to criticise individual manuscripts or attribute them to paper mills”, the paper says, noting that many were “still statistically significant and of interest to the scientific community”.
But it noted that this did “not detract from the wider point that holistic research is a better way to deal with the challenges posed by big data than the manufacture of very large numbers of single-factor manuscripts”.
Matt Spick, a lecturer in health and biomedical data analytics at the University of Surrey and co-author of the study, said that while AI has the “clear potential to help the scientific community make breakthroughs that benefit society, our study has found that it is also part of a perfect storm that could be damaging the foundations of scientific rigour”.
ADVERTISEMENT
“We’ve seen a surge in papers that look scientific but don’t hold up under scrutiny – this is ‘science fiction’ using national health datasets to masquerade as science fact,” Spick added.
The trend was threatening to overwhelm some journals and peer reviewers, Spick warned, which could reduce “their ability to assess more meaningful research – ultimately weakening the quality of science overall”.
The paper calls for the issues to be addressed by researchers, journals and peer reviewers.
It recommends researchers use the full datasets available to them unless there’s a clear and well-explained reason to do otherwise, and that they are transparent about which parts of the data were used, over what time periods, and for which groups.
ADVERTISEMENT
Journals should strengthen peer review by involving more researchers with statistical expertise and make greater use of early desk rejection to reduce the number of formulaic or low-value papers entering the system.
“We believe that in the AI era scientific publishing needs better guard rails”, Anietie Aliu, co-author of the study and a postgraduate student at Surrey, said.
ADVERTISEMENT
“Our suggestions are simple things that could help stop weak or misleading studies from slipping through, without blocking the benefits of AI and open data. These tools are here to stay, so we need to act now to protect trust in research.”
juliette.rowsell
