Artificial intelligence has become a double-edged sword in the world of scientific research; while it opens vast horizons for researchers to develop knowledge and accelerate discoveries, other parties exploit it to produce fake research and fabricate data, references, and scientific images that are difficult to distinguish from original materials, a phenomenon that threatens the credibility of scientific output and public trust in science.
With the widespread proliferation of large language models and tools for generating text, images, and data, preparing a study that appears scientific and comprehensive has become cheaper and faster than ever before. Fraud operations are no longer limited to writing poor texts or copying previous research, but have extended to fabricating scientific references, experimental data, and laboratory images that look realistic.
Dr. Heba Fahmy, a professor of medical biophysics at the Faculty of Science, Cairo University, confirms that one of the most prominent manifestations of this crisis is the emergence of what is known as "hallucinated references," as AI tools sometimes produce citations for studies and sources that do not exist, but appear to the reader as if they were real scientific references. Reports related to this type of misconduct have increased in tandem with the spread of AI tools.
The problem does not stop at references and texts, but includes scientific images used to prove experimental results, especially in medical and biological fields. Some images generated or modified by AI have become highly similar to real images, making detection of manipulation more difficult, to the point that some prestigious scientific journals have had to retract published materials after the use of AI tools in editing their images was uncovered.
Research mills exploit technology
Dr. Nils Meid, an assistant professor of scientific communication at Wageningen University, points to indicators that AI tools contribute to an increase in the production of fake or low-quality research, while noting the difficulty of accurately determining the true scale of this phenomenon.
The phenomenon of "research mills" stands out here, which are commercial entities that sell ready-made studies or fabricated data, and may even offer researchers the ability to buy authorship rights and add their names to research they did not participate in preparing. Their activity often targets researchers facing academic pressures related to promotion, graduation, or obtaining funding.
Artificial intelligence has helped these entities develop their working methods through three main paths:
Increasing production and reducing costs: Whereas writing fabricated research used to rely on human labor and took a long time, chatbots are now capable of helping prepare dozens of articles in short periods, which has expanded these mills' capacity for production.
Evasion of detection tools: Fake research was sometimes detected due to poor drafting or repetitive images and data, but the use of automatically generated texts and images has made the detection of traditional patterns of fraud more difficult.
Exploitation of open databases: Some entities resort to publicly available health and government data to produce similar studies that link a single variable, such as vitamin D levels or sleep quality, to complex diseases like depression and heart disease, without sufficient consideration of the multiplicity of factors influencing these diseases.
The activity of research mills is not limited to fabricating studies, but also includes selling authorship rights and citations, submitting the same research to more than one journal, and manipulating peer review processes—practices that undermine the foundations of scientific integrity.
Figures reveal the widening of the crisis
Some studies reveal a notable growth in the volume of suspicious research. Researchers have tracked a significant increase in studies that rely on open US health databases; the number of research papers following similar patterns in using this data rose from four in 2014 to 190 during the first nine months of 2024.
Research estimates also indicate that the number of new research papers suspected of being linked to research mills doubles approximately every year and a half, while the total scientific output doubles at a much slower pace. As for research retracted after the discovery of manipulation, its number doubles every 3.3 years according to one study.
In the field of cancer research, a study published in the British Medical Journal in 2026 covered more than 2.6 million research papers and estimated that the percentage of research suspected of being linked to research mills increased over two decades from about 1% to nearly 16%.
However, experts warn against blaming artificial intelligence alone for the crisis; the phenomenon of research manipulation preceded the emergence of generative AI tools, but these tools helped accelerate it, reduce its costs, and increase the difficulty of detecting it.
Who is responsible for protecting science?
The problem is largely linked to the academic evaluation system that places excessive importance on the number of published research papers at the expense of their scientific value and quality. As long as promotions, funding, and professional opportunities are linked to the quantity of publication, the demand for entities that sell ready-made research will increase.
Dr. Heba Fahmy believes that addressing the crisis requires a radical change in researcher evaluation criteria, so that the quality, originality, and impact of research become more important than the number of publications.
Solutions also include strengthening auditing procedures within scientific journals, verifying the validity of references and authors' identities, and examining images and data using specialized tools, in addition to fostering cooperation between publishers to counter networks that move between different journals.
Also among the proposed solutions is mandating that researchers make the raw data they relied on in their studies available, allowing specialists to re-analyze results and verify their validity, while developing mechanisms to monitor the use of open databases without restricting scientific access to them.
Conversely, artificial intelligence itself can be employed to counter this phenomenon by identifying non-existent references, detecting image manipulation, and analyzing abnormal patterns in authorship, publication, and scientific citations.
Science between fraud and self-correction
The danger of fabricated research is not limited to distorting the scientific record, but extends to influencing medical and health decisions and public policies if entities or institutions rely on incorrect results. These practices can also lead to weakening public trust in scientific institutions and researchers.
Nils Meid emphasizes that addressing the crisis does not require only the development of detection tools, but also entails training researchers, editors, and reviewers on the principles of scientific integrity, imposing clear consequences for buying authorship rights and manipulating study results, and supporting whistleblowers who uncover these violations.
At the same time, announcing and retracting fake research should not be viewed as evidence of the collapse of science, but rather as part of the self-correction mechanisms that protect knowledge from errors and manipulation.
In the end, the problem does not lie in using artificial intelligence within laboratories and universities, but in employing it to falsify knowledge and turn scientific research into a commodity. Unless the "publish or perish" culture changes and quality and integrity become the basis of academic evaluation, science will continue to face an open race between those who produce knowledge and those who trade in its illusion.
