## FacTool: Factuality Detection in Generative AI
Credit : [I-Chun Chern](https://arxiv.org/search/cs?searchtype=author&query=Chern%2C+I), [Steffi Chern](https://arxiv.org/search/cs?searchtype=author&query=Chern%2C+S), [Shiqi Chen](https://arxiv.org/search/cs?searchtype=author&query=Chen%2C+S), [Weizhe Yuan](https://arxiv.org/search/cs?searchtype=author&query=Yuan%2C+W), [Kehua Feng](https://arxiv.org/search/cs?searchtype=author&query=Feng%2C+K), [Chunting Zhou](https://arxiv.org/search/cs?searchtype=author&query=Zhou%2C+C), [Junxian He](https://arxiv.org/search/cs?searchtype=author&query=He%2C+J), [Graham Neubig](https://arxiv.org/search/cs?searchtype=author&query=Neubig%2C+G), [Pengfei Liu](https://arxiv.org/search/cs?searchtype=author&query=Liu%2C+P)
Research Paper: https://arxiv.org/abs/2307.13528

## Abstract

The emergence of generative pre-trained models has facilitated the synthesis of high-quality text, but it has also posed challenges in identifying factual errors in the generated text.

In particular:  
(1) A wider range of tasks now face an increasing risk of containing factual errors when handled by generative models.  
(2) Generated texts tend to be lengthy and lack a clearly defined granularity for individual facts.  
(3) There is a scarcity of explicit evidence available during the process of fact checking.

With the above challenges in mind, we propose FacTool, a task and domain agnostic framework for detecting factual errors of texts generated by large language models (e.g., ChatGPT).

FacTool now supports factual error detection of four different tasks: knowledge-based QA, code generation, math problem solving, and scientific literature review.

![Figure 1](https://ethanc111.github.io/factool_website/static/images/factool-framework.png)

![Figure 2](https://ethanc111.github.io/factool_website/static/images/factool-examples.png)
Code and ChatGPT plugin configs can be found  [here](https://github.com/GAIR-NLP/factool).

## BibTeX

```
@article{chern2023factool,
  title={FacTool: Factuality Detection in Generative AI--A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios},
  author={Chern, I-Chun and Chern, Steffi and Chen, Shiqi and Yuan, Weizhe and Feng, Kehua and Zhou, Chunting and He, Junxian and Neubig, Graham and Liu, Pengfei},
  journal={arXiv preprint arXiv:2307.13528},
  year={2023}
}
```
