"FACTOOL: A Versatile Framework for Accurate Text Error Detection in Large Language Models"

Researchers have developed FACTOOL, a task- and domain-agnostic framework for detecting factual errors in texts generated by large language models (LLMs) like GPT-4. LLMs often produce text with errors or departures from the truth, limiting their use in industries with high risks. FACTOOL uses various resources, including Google Search, Google Scholar, code interpreters, Python, and LLMs, to verify the factuality of generated information. The framework integrates "tool use" and "factuality detection" to provide a unified and adaptable approach. Experimental results show that GPT-4 has the highest factuality, while carefully honed chatbots struggle with more complex tasks like writing scientific literature reviews and solving math problems.