AI Mode Conversation
You said: OLMoTraceOLMoTrace
OLMoTrace is the first real-time system that traces language model outputs back to multi-trillion-token training datasets. Developed by the Allen Institute for AI (Ai2) and winner of Best Demo at ACL 2025, it lets users inspect the exact training text that influenced a model's response. [[1](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4), [2](https://www.google.com/goto?url=CAESYgHrOzAVwZXrpqdVe9FbrwtjSc7eQZxpQMV1rW45fpmiYcEun6qtw5BkoUFZYGnw5BisqP15dw6xzeIBd-c5UZtymCKlPn3GdZsdjmSWuyAd3HUQoS63-eoB7id8WZ2i6xYm), [3](https://www.google.com/goto?url=CAESYwHrOzAVoBdCmgaMv_aVuPblFx2Fl1rTr5jRtpHjFt8bY9t3-kFUMnJHUCi4fKi9syj-czcVi4-vTzxH4_FgbW1u53Og3AFWYhVl8jdf6ShVbh-eNt1Txau2fsSjoPS_qA6cPg)]
How OLMoTrace Works

* Verbatim Matching: It tokenizes the model's generated text and scans for maximal matching word spans against a massive training corpus within seconds. [[1](https://www.google.com/goto?url=CAESZQHrOzAVxlEQh8DtUS6saF7_ir0WDWTt10oJOkUyoZLZ5V5YBJVgGvaRQHjFqplwzUNuPtXEMikcc4i3XR090HFesSg_GHHBxQB2LQXhKyuji5rI2Pb1BhQ4Ln80dBGmKiY1Avss), [2](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4)]
* Infini-gram Technology: It uses an extended infini-gram suffix array index and parallel queries to scale across trillions of tokens with low complexity (\(O(L \log N)\)). [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESZQHrOzAVxlEQh8DtUS6saF7_ir0WDWTt10oJOkUyoZLZ5V5YBJVgGvaRQHjFqplwzUNuPtXEMikcc4i3XR090HFesSg_GHHBxQB2LQXhKyuji5rI2Pb1BhQ4Ln80dBGmKiY1Avss)]
* Ranking and Retrieval: Spans are filtered by unigram probability and mapped to top context documents ranked by BM25 relevance. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESZQHrOzAVxlEQh8DtUS6saF7_ir0WDWTt10oJOkUyoZLZ5V5YBJVgGvaRQHjFqplwzUNuPtXEMikcc4i3XR090HFesSg_GHHBxQB2LQXhKyuji5rI2Pb1BhQ4Ln80dBGmKiY1Avss)]
* Supported Models: It is integrated directly into the [Ai2 Playground](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ) for open models like OLMo 2 32B Instruct, OLMo 2 13B Instruct, and OLMoE 1B 7B Instruct. [[1](https://www.google.com/goto?url=CAESYAHrOzAVpyKaMtIZur5fnzfVTAplbZKyOpgyO9XIi8wdceCKMztZClwJCIyrTk_htKDmhQXtZJjPy2SvfMSl7AlXo-nMIKvseTestuqaTvi2ZNgwjsENvLLvmNeRnWnVmA), [2](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ)]

Main Use Cases

* Fact-Checking: Verify whether a model's statement is grounded in authentic training data or synthesized from noise. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESYwHrOzAVoBdCmgaMv_aVuPblFx2Fl1rTr5jRtpHjFt8bY9t3-kFUMnJHUCi4fKi9syj-czcVi4-vTzxH4_FgbW1u53Og3AFWYhVl8jdf6ShVbh-eNt1Txau2fsSjoPS_qA6cPg)]
* Hallucination Diagnosis: Trace incorrect or fabricated claims back to misleading or contradictory passages in the pre-training data. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESYwHrOzAVoBdCmgaMv_aVuPblFx2Fl1rTr5jRtpHjFt8bY9t3-kFUMnJHUCi4fKi9syj-czcVi4-vTzxH4_FgbW1u53Og3AFWYhVl8jdf6ShVbh-eNt1Txau2fsSjoPS_qA6cPg)]
* Creative Tracing: Identify where distinctive phrases, styles, or patterns originated across training corpora. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESYwHrOzAVoBdCmgaMv_aVuPblFx2Fl1rTr5jRtpHjFt8bY9t3-kFUMnJHUCi4fKi9syj-czcVi4-vTzxH4_FgbW1u53Og3AFWYhVl8jdf6ShVbh-eNt1Txau2fsSjoPS_qA6cPg)]
* Arithmetic & Memory Analysis: Assess how much a model memorized verbatim versus how much it generalized. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESZQHrOzAVxlEQh8DtUS6saF7_ir0WDWTt10oJOkUyoZLZ5V5YBJVgGvaRQHjFqplwzUNuPtXEMikcc4i3XR090HFesSg_GHHBxQB2LQXhKyuji5rI2Pb1BhQ4Ln80dBGmKiY1Avss)]

You can test the tool or read the official technical breakdown via the [Ai2 OLMoTrace Blog Post](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ) or inspect the paper via the [arXiv OLMoTrace Listing](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4). [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4)]
This video demonstrates how OLMoTrace highlights generated text and links it directly to training documents in real time:
2:17
OLMoTrace | Connecting a language model's response back to its training ...
3K views · 1 year ago
YouTube · Ai2
Would you like to know how to access the open-source code on GitHub, or do you want more details on how infini-gram indexing works under the hood?
Allen Institute for Artificial Intelligence
Going beyond open data – increasing transparency and trust in language models with OLMoTrace | Ai2
Feature: OLMoTrace in Ai2 Playground traces language model outputs to multi-trillion-token training data in real time. Supported Models: OLMo 2 32B Instruct, OL...
arXiv
[2504.07096] OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens
System: OLMoTrace (first system tracing language model outputs to multi-trillion-token training data in real time). Functionality: Finds and displays verbatim m...
ACL Anthology
OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens - ACL Anthology
System: OLMoTrace (Best Demo, ACL 2025) Core Function: First system tracing language model outputs back to multi-trillion-token training data in real time. Mech...
ACL Anthology
OLMOTRACE: Tracing Language Model Outputs Back to Trillions of Training Tokens
System: OLMOTRACE, a real-time system tracing language model outputs verbatim back to multi-trillion-token training data within 4.5 seconds using infini-gram su...
Hugging Face
Paper page - OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens
System: OLMoTrace, the first real-time system tracing language model outputs back to multi-trillion-token training corpora. Capabilities: Identifies and display...
YouTube·Ai2
2:17
OLMoTrace | Connecting a language model's response back to its training ...
AI responses may include mistakes. [Learn more](https://support.google.com/websearch?p=ai_overviews)
