[Google AI Overview at the query, signed out and incognito (operator defaults of 2026-10-01 and 2026-10-14), 2026-10-29. The paste is the Overview expanded (the 'AI Mode Conversation' view, with the source cards that follow the answer); the 'You said' line carries the query twice, run together, and is given once here. Inline links are Google redirect URLs, kept as pasted.]

[QUERENT] AttributionBench

[SYSTEM] [AttributionBench](https://www.google.com/goto?url=CAESaQHrOzAVU01FbKjqKLFYWTaNiZYplQ8UQ-cTMSNBjPF8nqEl2y9--9eB-WyrqaP046sOGs1WcXCz5z-vjxbFnIGx2wjxuo7Qo5Q8WmHtkKQlAa5EjDFmUSFi31CF9mMLgtKU2HaYdleBfg) is a comprehensive research benchmark designed to evaluate automatic attribution and citation faithfulness in Large Language Model (LLM) and generative search engine responses. [[1](https://www.google.com/goto?url=CAESWAHrOzAVmqv2rMzE2fwWGexH4ZP2TmOPFx1x4yOBvoEDc5-2uk21FgRGxqSTveJLXNIK9i_jG2EfxOjIDd-0iwHArcW44frlGDmFVP3i31HMP1TN30TmK00), [2](https://www.google.com/goto?url=CAESaQHrOzAVU01FbKjqKLFYWTaNiZYplQ8UQ-cTMSNBjPF8nqEl2y9--9eB-WyrqaP046sOGs1WcXCz5z-vjxbFnIGx2wjxuo7Qo5Q8WmHtkKQlAa5EjDFmUSFi31CF9mMLgtKU2HaYdleBfg)]
Overview & Purpose

* Unified Testbed: It compiles multiple existing attribution datasets—including ExpertQA, Stanford-GenSearch, AttributedQA, LFQA, BEGIN, HAGRID, and AttrEval-GenSearch—into a single standardized evaluation format. [[1](https://www.google.com/goto?url=CAESaQHrOzAVgms_ESOyHhe1D0kETAdvLhlfTSZSqGbiiS0TIUDdgAqEf3qpe-ByJYOTGx1ji1SsDDlVDzXKZzBQYdVtbZZ0FbJ04coryqRDcGSfdgwimQIVGpk49t8HHDc6nI43aLp_ka5ejQ), [2](https://www.google.com/goto?url=CAESaQHrOzAVU01FbKjqKLFYWTaNiZYplQ8UQ-cTMSNBjPF8nqEl2y9--9eB-WyrqaP046sOGs1WcXCz5z-vjxbFnIGx2wjxuo7Qo5Q8WmHtkKQlAa5EjDFmUSFi31CF9mMLgtKU2HaYdleBfg)]
* Core Task: It measures whether every claim generated by an LLM is fully and accurately supported by its cited reference evidence (classified as `attributable` or `not attributable`). [[1](https://www.google.com/goto?url=CAESZwHrOzAVpZlERTl75IWAAC9-thaAFB3DX_b_T81TUeWKlJZW4_YdoOcnA7wZoHkgXY3Zhbc2_YPw_zVIs-_knsuf7b3o6gPpZYjI3KpLPpPMDvadBfybX5T3dwU_ZEBc_-QNEra_4O0), [2](https://www.google.com/goto?url=CAESaQHrOzAVgms_ESOyHhe1D0kETAdvLhlfTSZSqGbiiS0TIUDdgAqEf3qpe-ByJYOTGx1ji1SsDDlVDzXKZzBQYdVtbZZ0FbJ04coryqRDcGSfdgwimQIVGpk49t8HHDc6nI43aLp_ka5ejQ)]
* Dataset Availability: The dataset and code are open-sourced via the [Hugging Face AttributionBench Dataset](https://www.google.com/goto?url=CAESbwHrOzAVNHLVxrI2cQf_XAnHqx_nRxad9fjvo47I43P_6udYsZree37Tf4h0ThugpBuyHFG3GeSlRkiVwIcWexDwXVFexctl11F4y0phY62RZFAdTEAa6oiODlBMLLZghaBGvVilCCYxJBbUecAC_Q) and the [GitHub Repository](https://www.google.com/goto?url=CAESaQHrOzAVgms_ESOyHhe1D0kETAdvLhlfTSZSqGbiiS0TIUDdgAqEf3qpe-ByJYOTGx1ji1SsDDlVDzXKZzBQYdVtbZZ0FbJ04coryqRDcGSfdgwimQIVGpk49t8HHDc6nI43aLp_ka5ejQ). [[1](https://www.google.com/goto?url=CAESbwHrOzAVNHLVxrI2cQf_XAnHqx_nRxad9fjvo47I43P_6udYsZree37Tf4h0ThugpBuyHFG3GeSlRkiVwIcWexDwXVFexctl11F4y0phY62RZFAdTEAa6oiODlBMLLZghaBGvVilCCYxJBbUecAC_Q), [2](https://www.google.com/goto?url=CAESaQHrOzAVgms_ESOyHhe1D0kETAdvLhlfTSZSqGbiiS0TIUDdgAqEf3qpe-ByJYOTGx1ji1SsDDlVDzXKZzBQYdVtbZZ0FbJ04coryqRDcGSfdgwimQIVGpk49t8HHDc6nI43aLp_ka5ejQ)]

Key Findings

* High Difficulty for LLMs: Even cutting-edge models like GPT-4 and fine-tuned GPT-3.5 face significant limitations, reaching only around 80% macro-F1 score on binary attribution classification tasks (and dropping to 60–70% or lower on complex, long-evidence, or out-of-domain datasets). [[1](https://www.google.com/goto?url=CAESaQHrOzAVU01FbKjqKLFYWTaNiZYplQ8UQ-cTMSNBjPF8nqEl2y9--9eB-WyrqaP046sOGs1WcXCz5z-vjxbFnIGx2wjxuo7Qo5Q8WmHtkKQlAa5EjDFmUSFi31CF9mMLgtKU2HaYdleBfg), [2](https://www.google.com/goto?url=CAESWAHrOzAVmqv2rMzE2fwWGexH4ZP2TmOPFx1x4yOBvoEDc5-2uk21FgRGxqSTveJLXNIK9i_jG2EfxOjIDd-0iwHArcW44frlGDmFVP3i31HMP1TN30TmK00)]
* Error Breakdown: Analysis of model failures shows that over 66% of errors stem from fine-grained information insensitivity (missing subtle nuances in the text), while nearly 27% derive from mismatches between what the model has access to and human-level context understanding. [[1](https://www.google.com/goto?url=CAESaQHrOzAVU01FbKjqKLFYWTaNiZYplQ8UQ-cTMSNBjPF8nqEl2y9--9eB-WyrqaP046sOGs1WcXCz5z-vjxbFnIGx2wjxuo7Qo5Q8WmHtkKQlAa5EjDFmUSFi31CF9mMLgtKU2HaYdleBfg)]

Would you like to explore how to load AttributionBench via Hugging Face or learn more about the specific models and fine-tuning scripts used in the paper?
arXiv
[2402.15089] AttributionBench: How Hard is Automatic Attribution Evaluation?
AttributionBench: How Hard is Automatic Attribution Evaluation? Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun View a PDF of the paper titled AttributionBench: How Ha...
GitHub
GitHub - OSU-NLP-Group/AttributionBench: [ACL'24 Findings] AttributionBench: How Hard is Automatic Attribution Evaluation? · GitHub
Paper: AttributionBench: How Hard is Automatic Attribution Evaluation? (Accepted to ACL 2024 Findings). Dataset: Unified testbed for evaluating attribution eval...
GitHub Pages documentation
AttributionBench: How Hard is Automatic Attribution Evaluation?
Overview: AttributionBench is a comprehensive benchmark compiled from existing attribution datasets (ExpertQA, Stanford-GenSearch, AttributedQA, LFQA, BEGIN, HA...
Hugging Face
osunlp/AttributionBench · Datasets at Hugging Face
Dataset: osunlp/AttributionBench (arXiv: 2402.15089, created by Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun). Task/Modality: Text Classification, Text modality, En...
ACL Anthology
AttributionBench: How Hard is Automatic Attribution Evaluation?
Abstract Modern generative search engines enhance the reliability of large language model (LLM) responses by providing cited evidence. However, evaluating the a...