Hallucinations

[danielxmed]

Hello, everyone!

Im using the agents for medical literature research, but Ive having a lot of trouble with hallucinated links mostly. How are you guys handling those?

[danielxmed]

Done! Prompt engeneering and changing the models works.

[ElishaKay]

Welcome @danielxmed

Sounds good

I was gonna ask which LLM model are you using and which report type?

We're doing most of our testing with the OpenAI LLM

[danielxmed]

Hello!

I’ve noticed that using o3-mini in more stages of the pipeline significantly reduces hallucinations. Providing clear instructions about this in the prompt is also important, as well as guiding the agents to avoid using sources that can’t be clearly referenced — even if that means losing some content. The current prompts are a bit too forceful about leveraging the retrieved content heavily, but for actual scientific writing — especially in medicine — that approach isn’t ideal.

I also observed that the deep research mode clearly hallucinates much less. The faster the report, the more hallucinations appear — at least based on my testing so far.

I’ve also instructed the agents not to search lay websites, only platforms that host scientific articles.

One suggestion: in medicine we often use Vancouver-style referencing instead of APA. I tried switching it through the prompt, but it doesn’t work well. I think it would be very valuable to be able to choose between Vancouver and APA. However, it would probably require a dedicated module within the architecture. I’m working on it and will try to commit if I manage to build it.