THE TOOL, SIGNED OUT, 2026-11-01: 'OLMoTrace is the first real-time system that traces language model outputs back to multi-trillion-token training datasets. Developed by the Allen Institute for AI (Ai2) and winner of Best Demo at ACL 2025, it lets users inspect the exact training text that influenced a model's response.' How it works (verbatim matching, the infini-gram index, filtering by unigram probability and ranking by BM25, the three supported OLMo models); four use cases (fact-checking, hallucination diagnosis, creative tracing, arithmetic and memory analysis); pointers to the blog post and the arXiv listing; an embedded Ai2 video (2:17); an offer on the GitHub code or infini-gram indexing. Six cards: Ai2, arXiv, ACL Anthology (two), Hugging Face, YouTube.
Reading
The innocent reading first. Asked for the tool by name, the Overview says what it is, who made it, how it finds and ranks matches, which models it serves and what it is for, from Ai2's blog, the paper and its listings.
Machine text, verbatim
[Google AI Overview at the query, signed out and incognito (operator defaults of 2026-10-01 and 2026-10-14). The paste is the Overview expanded (the 'AI Mode Conversation' view).] The 'You said' line carries the query twice, run together, and is given once here. Inline links are Google redirect URLs, kept as pasted.
[QUERENT] OLMoTrace
[SYSTEM] OLMoTrace is the first real-time system that traces language model outputs back to multi-trillion-token training datasets. Developed by the Allen Institute for AI (Ai2) and winner of Best Demo at ACL 2025, it lets users inspect the exact training text that influenced a model's response. [[1], [2], [3]]
How OLMoTrace Works
* Verbatim Matching: It tokenizes the model's generated text and scans for maximal matching word spans against a massive training corpus within seconds. [[1], [2]]
* Infini-gram Technology: It uses an extended infini-gram suffix array index and parallel queries to scale across trillions of tokens with low complexity (\(O(L \log N)\)). [[1], [2]]
* Ranking and Retrieval: Spans are filtered by unigram probability and mapped to top context documents ranked by BM25 relevance. [[1], [2]]
* Supported Models: It is integrated directly into the [Ai2 Playground](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ) for open models like OLMo 2 32B Instruct, OLMo 2 13B Instruct, and OLMoE 1B 7B Instruct. [[1], [2]]
Main Use Cases
* Fact-Checking: Verify whether a model's statement is grounded in authentic training data or synthesized from noise. [[1], [2]]
* Hallucination Diagnosis: Trace incorrect or fabricated claims back to misleading or contradictory passages in the pre-training data. [[1], [2]]
* Creative Tracing: Identify where distinctive phrases, styles, or patterns originated across training corpora. [[1], [2]]
* Arithmetic & Memory Analysis: Assess how much a model memorized verbatim versus how much it generalized. [[1], [2]]
You can test the tool or read the official technical breakdown via the [Ai2 OLMoTrace Blog Post](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ) or inspect the paper via the [arXiv OLMoTrace Listing](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4). [[1], [2]]
This video demonstrates how OLMoTrace highlights generated text and links it directly to training documents in real time:
2:17
Sources (6)
Allen Institute for Artificial Intelligence third_party
Going beyond open data – increasing transparency and trust in language models with OLMoTrace | Ai2
Feature: OLMoTrace in Ai2 Playground traces language model outputs to multi-trillion-token training data in real time. Supported Models: OLMo 2 32B Instruct, OL...
arXiv third_party
[2504.07096] OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens
System: OLMoTrace (first system tracing language model outputs to multi-trillion-token training data in real time). Functionality: Finds and displays verbatim m...
ACL Anthology third_party
OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens - ACL Anthology
System: OLMoTrace (Best Demo, ACL 2025) Core Function: First system tracing language model outputs back to multi-trillion-token training data in real time. Mech...
ACL Anthology third_party
OLMOTRACE: Tracing Language Model Outputs Back to Trillions of Training Tokens
System: OLMOTRACE, a real-time system tracing language model outputs verbatim back to multi-trillion-token training data within 4.5 seconds using infini-gram su...
Hugging Face third_party
Paper page - OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens
System: OLMoTrace, the first real-time system tracing language model outputs back to multi-trillion-token training corpora. Capabilities: Identifies and display...
YouTube·Ai2 third_party
OLMoTrace | Connecting a language model's response back to its training ...
Analysis analyst prose, not machine text
The tool composed accurately at the level of its mechanism, from its own blog and paper. A verbatim match becomes 'the exact training text that influenced' a response, the user's verification becomes the match's, and creative tracing widens from expressions to styles; what a matched document is shown under is not reached.