AI has made research faster than ever before – the challenge is knowing which answers deserve your confidence.
Why AI outputs that look credible can lead research down the wrong path.
What if the biggest risk in AI-assisted research isn't an obviously bad response, but a convincing one?
Somewhere in a research pipeline right now, an AI system is generating output that's almost entirely correct. Not invented. Not nonsense. Almost right. A citation from a real-sounding journal, or from a paper that doesn't quite exist. A summary that misses the one detail that would have flipped the conclusion. Nobody catches it, because there's nothing obvious to catch. It doesn't look wrong. It looks finished.
That's the harder problem, and it's not the one most people picture when they imagine AI going wrong.
Not wrong. Worse.
“When people talk about AI making things up, they picture something absurd, like a nonsense answer or a citation to a paper that obviously doesn't exist,” said Armughan Rafat, 
Senior Vice President, Chief AI & Data Analytics Officer. “That's not the dangerous case. The dangerous case is the one that's 90 percent right but actually misinforms. Many won’t stop to check something that already sounds correct. Our work with AI-assisted research is tackling this specific challenge head on.”
The risk of almost right answers is growing because researchers increasingly turn to general AI platforms to find and make sense of scientific content. Meanwhile, Reddit accounts for 40 percent of LLM citations across the major AI engines. Which means many people are trusting platforms with no editorial process standing between a plausible answer and the person relying on it.
Adoption is outrunning trust
In 2024, just over half of researchers said they used AI in their work.
By 2025 it was 84 percent, according to a global survey of more than 2,400 researchers we published in our second annual ExplanAItions study.
The tools made work faster, and most said it made their work better too.
But the more researchers use these tools,
the less they trust them. Concerns about AI models climbed to 87 percent, up from 81 percent the year before. And within that, worry about AI inventing sources specifically jumped 13 points in a single year. Usage kept climbing the whole time, too — this wasn't early adopters cooling off. The researchers relying on AI most heavily were also the ones growing most wary of it.
The importance of evidence
Ask any of the major AI tools a question and there's a real chance the answer traces back not to a peer-reviewed study, but to a Reddit thread. Independent citation audits have found that Reddit is now among the single most-cited sources across ChatGPT, Claude, Perplexity, and Google AI Overviews. When a research tool can't reliably tell the difference between a well-cited meta-analysis and a confidently-worded Reddit comment, the burden shifts to the researcher to interrogate every source AI hands them, rather than trusting the citation because it's there.
The lack of proper evidence is rampant. An analysis of accepted papers to a major AI research conference found more than 100 fabricated citations embedded in work that had already been assessed by expert reviewers trained to catch issues. “That's exactly the problem we're focused on solving,” said Rafat. “Rigor can't just mean checking whether an argument holds up anymore. It has to mean checking whether the information that informs it is real in the first place.”
Where it hides
The errors cluster at the frontier — the niche, the recent, the highly specific — exactly where a general AI platform has the least real data to draw from. It's also where breakthroughs tend to happen, and where a researcher has the thinnest basis for spotting a made-up detail on sight.
In the scientific pipeline, which builds on itself, a fabricated detail doesn't stay contained. It gets cited. It gets built on. Months later, it shapes a decision made by people with no way of knowing if the foundation is flawed.
Where this gets expensive
Nowhere does “almost right” carry more weight than in healthcare and life sciences 
R&D, where a summary of the evidence is the basis of a multi-year, multi-million-dollar bet. Incomplete evidence, dressed up as confident evidence, can steer a program in ways that don't show up for months or years, and by the time they do, the cost is real. Gartner estimates poor data quality costs organizations an average of $12.9 million every year in wasted resources and lost opportunities.
A program built on an almost-right guess can add years of misguided development. Trials can end up designed around assumptions nobody double-checked: who to enroll, what to measure, what's driving the disease. Contradictory findings get waved off, and a hypothesis with nothing solid behind it is impossible for anyone outside the room to verify. Follow that far enough and weak evidence becomes a weak trial. When a regulator asks for the scientific rationale behind a target population, a mechanism, or a biomarker, an unverified AI summary is not evidence. And the cost is certainly not only in the numbers. The truest cost is the opportunity to deliver better health outcomes for patients.
None of this requires the AI to be malicious, or even wrong. It just has to be convincing enough, at the wrong moment, to stop someone from asking one more question,
one that would expose the mistake.
This isn't an argument for setting the tools aside. Nearly every researcher who's adopted AI says it's made them faster and better at their work. The problem is that “is this right or wrong” has quietly become a harder question: how much of this can I actually trust, and how would I know if I couldn't?
What we're doing about it
When it comes to AI, the source matters. When working with our partners, we aim to ensure that proper citations and sources are tracked for use in AI systems, as well as leveraging full text access instead of just abstracts. Wiley has robust detection tools built into our publishing processes and platforms to mitigate the risks. Peer review means real experts check a claim before it's published. Citation traceability across our 2,000+ journals means a claim can always be traced back to the real study behind it (exactly what's missing when a general AI platform hands someone a summary that lacks proper citation). Wiley's content, platforms, and expertise provide a trustworthy knowledge layer for AI tool development.
What comes next
With a deeper understanding of the impact of general AI use in scientific research, we continue to explore the risks of “almost right.” Our research has already surfaced new data points, from a study we ran earlier this year, measuring
exactly how much of the picture goes missing when an AI system works from summaries instead of full research text. The results, as relate to Alzheimer's research specifically, are coming in the next installment of this series.
AI is transforming scientific research. But confidence depends on knowing where every answer comes from.
For more than 200 years, Wiley has helped researchers access trusted knowledge. Today, we're bringing that same rigor to AI by grounding every response in peer-reviewed research, expert authorship, and transparent evidence.
Because better AI starts with better sources.
Follow the expert
Armughan Rafat, Senior Vice President, Chief AI & Data Analytics Officer
Armughan Rafat leads Wiley's AI and data services initiatives, focusing on developing and commercializing AI-ready content and data products for AI developers and corporate R&D teams. Armughan has a proven track record as an innovator responsible for building high-margin businesses and capabilities that convert content assets into predictive data and AI services. Most recently, Rafat served as chief analytics officer at Norstella, where he drove AI-powered innovation in pharma and healthcare. Prior to that, as chief data officer at Clarivate, he led the company's data strategy and operations, delivering intelligence products for life sciences, pharma, financial and legal markets.