To start, not all RAGs are the same caliber. The accuracy of content in a custom database is important for consistent output, but it’s not the only variable. Joel Hron, global head of AI at Thomson Reuters. “It’s the quality of searching and retrieving relevant content based on the question.” Mastering each step in the process is important because one wrong step can cause the model to completely fail.
“Any lawyer who has ever tried to use natural language search in one of the research tools will find that there are often cases where semantic similarity leads you to completely unrelated documents.” Human-centered AI Institute. Ho’s research AI legal tools relying on RAG found higher error rates in outputs than those found by modeling companies.
This brings us to the most difficult question in the discussion: how do you identify hallucinations in a RAG implementation? Is it only when the chatbot produces quote-free output that makes up the information? Is that also where the tool might miss relevant data or misunderstand aspects of the citation?
According to Lewis, the illusion in the RAG system depends on whether the output matches what the model found during data retrieval. However, Stanford’s research on AI tools for lawyers expands this definition a bit by examining whether the output is grounded in the data provided as well as whether it whether it is factually correct or not. High standards for legal professionals who often analyze complex cases and navigate complex hierarchies of precedent.
While the RAG system tailored to legal issues is clearly better at answering questions about case law than OpenAI’s ChatGPT or Google’s Gemini, it can still miss finer details. and make random errors. All the AI experts I spoke with emphasized the ongoing need for thoughtful human interaction throughout the process to double-check quotes and verify accuracy overall results.
Law is an area where there is a lot of activity around RAG-based AI tools, but the potential of this process is not limited to a single white-collar job. “Choose any career or business. You need to get the answers documented on actual documents,” Arredondo said. “So I think the RAG is going to be the predominant device used in basically every professional application, at least in the near to medium term.” Risk-averse executives seem excited about the prospect of using AI tools to better understand their proprietary data without having to upload sensitive information to a standard, public chatbot .
However, it is important that users understand the limitations of these tools and that AI-focused companies should not over-promise the accuracy of their answers. Anyone using an AI tool should still avoid trusting the output completely, and they should approach its answers with healthy skepticism even if the answers are improved through RAG.
“The hallucinations are still here,” Ho said. “We still don’t have a way to actually eliminate the illusion.” Even if RAG reduces error rates, human judgment is still the most important factor. And that’s not a lie.
