SPXI instrument · method 0.9 of 87, level 0 (Measurement): Capture Registry (longitudinal observation). An instrument of SPXI, the Semantic Packet for eXchange & Indexing; the whole practice is itemized at spxi.dev/methods.

Capture Registry › capture attributionbench-aio-20261029

One record of the canonical Capture Registry (EA-WG-CAPTURES-01), cited at https://www.alexanarch.org/captures/attributionbench-aio-20261029/. the canonical Capture Registry (version 12.379) · the address page · this card in the gallery · this record as data · table of contents.

AXN:0ACB.CAPTURE — this address. Each observation below is one AXN, sealed with its analysis and measures; the identifier names the reading together with the response it reads.
Machine Reception2026-10-29
AttributionBench
Originator: OSU NLP Group (The Ohio State University) · outside the archive (out-universe) · dataset · SPXI treatment: partial
BROAD MATCH
no
image
ATTRIBUTIONBENCH, BY NAME, ON GOOGLE'S AI OVERVIEW, SIGNED OUT, 2026-10-29: 'a comprehensive research benchmark designed to evaluate automatic attribution and citation faithfulness in Large Language Model (LLM) and generative search engine responses.' A unified testbed of seven datasets (ExpertQA, Stanford-GenSearch, AttributedQA, LFQA, BEGIN, HAGRID, AttrEval-GenSearch), the task (is every claim fully supported by its cited evidence: attributable or not), the data on Hugging Face and GitHub; findings (about 80% macro-F1 for GPT-4 and fine-tuned GPT-3.5, 60–70% on harder sets; over 66% of errors from fine-grained information insensitivity, nearly 27% from a mismatch of access and context). Five cards: arXiv 2402.15089, the OSU-NLP-Group repository, the project page, the Hugging Face dataset, the ACL Anthology.
Full record — 6,174 characters, 5 sources
Capture record
captured
2026-10-29
surface
Google AI Overview
auth state
signed out, incognito
evidence class
paste
PER (historical scalar)
0.5
PER units retained
id, src
citations read
5
observation id
OBS-90d2b25bd00a
address id
ADDR-46cc2d945075
Reading

The innocent reading first. The Overview identifies the benchmark, what it compiles, what it asks and how hard models find it, all from the paper and its repositories.

Findings
Analysis analyst prose, not machine text

An attribution benchmark is composed at full grain from its own paper and kept distinct from its homonym, and its makers are left in the cards. The record's negative tag, that support by evidence is not credit to the original author, is the line between this entity and the next.

Machine text, verbatim
CAPTURE-TIME VERBATIM RECORD — paste of the Overview expanded, with its source cards · complete as supplied · READ IN FULL 2026-10-29
[Google AI Overview at the query, signed out and incognito (operator defaults of 2026-10-01 and 2026-10-14), 2026-10-29. The paste is the Overview expanded (the 'AI Mode Conversation' view, with the source cards that follow the answer); the 'You said' line carries the query twice, run together, and is given once here. Inline links are Google redirect URLs, kept as pasted.] [QUERENT] AttributionBench [SYSTEM] [AttributionBench](https://www.google.com/goto?url=CAESaQHrOzAVU01FbKjqKLFYWTaNiZYplQ8UQ-cTMSNBjPF8nqEl2y9--9eB-WyrqaP046sOGs1WcXCz5z-vjxbFnIGx2wjxuo7Qo5Q8WmHtkKQlAa5EjDFmUSFi31CF9mMLgtKU2HaYdleBfg) is a comprehensive research benchmark designed to evaluate automatic attribution and citation faithfulness in Large Language Model (LLM) and generative search engine responses. [[1], [2]] Overview & Purpose * Unified Testbed: It compiles multiple existing attribution datasets—including ExpertQA, Stanford-GenSearch, AttributedQA, LFQA, BEGIN, HAGRID, and AttrEval-GenSearch—into a single standardized evaluation format. [[1], [2]] * Core Task: It measures whether every claim generated by an LLM is fully and accurately supported by its cited reference evidence (classified as `attributable` or `not attributable`). [[1], [2]] * Dataset Availability: The dataset and code are open-sourced via the [Hugging Face AttributionBench Dataset](https://www.google.com/goto?url=CAESbwHrOzAVNHLVxrI2cQf_XAnHqx_nRxad9fjvo47I43P_6udYsZree37Tf4h0ThugpBuyHFG3GeSlRkiVwIcWexDwXVFexctl11F4y0phY62RZFAdTEAa6oiODlBMLLZghaBGvVilCCYxJBbUecAC_Q) and the [GitHub Repository](https://www.google.com/goto?url=CAESaQHrOzAVgms_ESOyHhe1D0kETAdvLhlfTSZSqGbiiS0TIUDdgAqEf3qpe-ByJYOTGx1ji1SsDDlVDzXKZzBQYdVtbZZ0FbJ04coryqRDcGSfdgwimQIVGpk49t8HHDc6nI43aLp_ka5ejQ). [[1], [2]] Key Findings * High Difficulty for LLMs: Even cutting-edge models like GPT-4 and fine-tuned GPT-3.5 face significant limitations, reaching only around 80% macro-F1 score on binary attribution classification tasks (and dropping to 60–70% or lower on complex, long-evidence, or out-of-domain datasets). [[1], [2]] * Error Breakdown: Analysis of model failures shows that over 66% of errors stem from fine-grained information insensitivity (missing subtle nuances in the text), while nearly 27% derive from mismatches between what the model has access to and human-level context understanding. [1] Would you like to explore how to load AttributionBench via Hugging Face or learn more about the specific models and fine-tuning scripts used in the paper? arXiv
Sources (5) as cited, and as the copy produced them
  1. arXiv third_party
    [2402.15089] AttributionBench: How Hard is Automatic Attribution Evaluation?
    AttributionBench: How Hard is Automatic Attribution Evaluation? Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun View a PDF of the paper titled AttributionBench: How Ha...
    as pasted[2402.15089]
  2. GitHub third_party
    GitHub - OSU-NLP-Group/AttributionBench: [ACL'24 Findings] AttributionBench: How Hard is Automatic Attribution Evaluation? · GitHub
    Paper: AttributionBench: How Hard is Automatic Attribution Evaluation? (Accepted to ACL 2024 Findings). Dataset: Unified testbed for evaluating attribution eval...
    as pastedGitHub - OSU-NLP-Group/AttributionBench: [ACL'24 Findings] AttributionBench: How Hard is Automatic Attribution Evaluation? · GitHub Paper: AttributionBench: How Hard is Automatic Attribution Evaluation? (Accepted to ACL 2024 Findings). Dataset: Unified testbed for evaluating attribution eval... GitHub Pages documentation AttributionBench: How Hard is Automatic Attribution Evaluation? Overview: AttributionBench is a comprehensive benchmark compiled from existing attribution datasets (ExpertQA, Stanford-GenSearch, AttributedQA, LFQA, BEGIN, HA... Hugging Face
  3. GitHub Pages documentation third_party
    AttributionBench: How Hard is Automatic Attribution Evaluation?
    Overview: AttributionBench is a comprehensive benchmark compiled from existing attribution datasets (ExpertQA, Stanford-GenSearch, AttributedQA, LFQA, BEGIN, HA...
  4. Hugging Face third_party
    osunlp/AttributionBench · Datasets at Hugging Face
    Dataset: osunlp/AttributionBench (arXiv: 2402.15089, created by Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun). Task/Modality: Text Classification, Text modality, En...
    as pastedosunlp/AttributionBench · Datasets at Hugging Face Dataset: osunlp/AttributionBench (arXiv: 2402.15089, created by Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun). Task/Modality: Text Classification, Text modality, En... ACL Anthology AttributionBench: How Hard is Automatic Attribution Evaluation? Abstract Modern generative search engines enhance the reliability of large language model (LLM) responses by providing cited evidence. However, evaluating the a...
  5. ACL Anthology third_party
    AttributionBench: How Hard is Automatic Attribution Evaluation?
    Abstract Modern generative search engines enhance the reliability of large language model (LLM) responses by providing cited evidence. However, evaluating the a...
    as pastedAttributionBench: How Hard is Automatic Attribution Evaluation? AttributionBench: How Hard is Automatic Attribution Evaluation? Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun View a PDF of the paper titled AttributionBench: How Ha... GitHub
↻ Re-runpermalink