{"address":{"q":"OLMoTrace","record":"olmotrace-aio-20261029"},"analysis":{"analysis":"The tool composed accurately at the level of its mechanism, from its own blog and paper. A verbatim match becomes 'the exact training text that influenced' a response, the user's verification becomes the match's, and creative tracing widens from expressions to styles; what a matched document is shown under is not reached.","d":"THE TOOL, SIGNED OUT, 2026-11-01: 'OLMoTrace is the first real-time system that traces language model outputs back to multi-trillion-token training datasets. Developed by the Allen Institute for AI (Ai2) and winner of Best Demo at ACL 2025, it lets users inspect the exact training text that influenced a model's response.' How it works (verbatim matching, the infini-gram index, filtering by unigram probability and ranking by BM25, the three supported OLMo models); four use cases (fact-checking, hallucination diagnosis, creative tracing, arithmetic and memory analysis); pointers to the blog post and the arXiv listing; an embedded Ai2 video (2:17); an offer on the GitHub code or infini-gram indexing. Six cards: Ai2, arXiv, ACL Anthology (two), Hugging Face, YouTube.","findings":["THE MECHANISM AT ITS GRAIN. 'It tokenizes the model's generated text and scans for maximal matching word spans', 'Spans are filtered by unigram probability and mapped to top context documents ranked by BM25 relevance', and the three models 'OLMo 2 32B Instruct, OLMo 2 13B Instruct, and OLMoE 1B 7B Instruct'. The blog: 'Maximality : The span is not a subspan of another span that meets the above two criteria.', 'keeps the K spans with the smallest unigram probability', 'in the document panel we rank all documents by a BM25 score in descending order.' The 'extended infini-gram suffix array index' agrees with the abstract ('Powered by an extended version of infini-gram (Liu et al., 2024)'). The corpus is given as 'multi-trillion-token'; the blog's count for the 32B model, 'about 3.2 billion documents and 4.6 trillion tokens', and its cap of 'up to 10 document snippets' per span are not carried.","THE MATCH GIVEN AS INFLUENCE. 'it lets users inspect the exact training text that influenced a model's response.' Ai2's own terms are hedged: 'inspect where and how language models may have learned to generate certain word sequences', 'the model could have easily learned it through memorization', 'The interpretation of text matches with training data is quite nuanced.' The census row reads a highlight as a witness that a stretch of output has a verbatim antecedent, the evidence for #1543's S = retrieved, and records that 'Verbatim matching is not causal influence'. 'Creative Tracing: Identify where distinctive phrases, styles, or patterns originated across training corpora' carries the blog's 'trace certain LM-generated expressions back to the training documents' out to styles and patterns, which leave no verbatim span to highlight; the abstract's bound is 'verbatim matches between segments of language model output and documents in the training text corpora'.","THE VERIFICATION MOVED. 'Fact-Checking: Verify whether a model's statement is grounded in authentic training data or synthesized from noise.' The blog puts the check with the user: 'If you personally verify that these documents come from trusted sources, you can gain confidence that this piece of information generated by the model is factually correct.' In the composition the match does the verifying; the blog's 'low relevance' label for spans whose documents are less relevant is not carried.","THE PRIZE AND THE SPEED FROM THE CARDS. 'winner of Best Demo at ACL 2025' is on the first ACL Anthology card ('System: OLMoTrace (Best Demo, ACL 2025)'); the arXiv page the census saved has 'ACL 2025 demo track'. The second ACL card gives 'within 4.5 seconds', the abstract 'within a few seconds'. The blog's date, 'April 9, 2025', and its signature, 'Jiacheng Liu et al.', are not in the composition, which names no person.","WHAT A DOCUMENT IS SHOWN UNDER NOT REACHED. The census row: the saved sources do not say whether the side panel shows 'a document's author, title or source address'. The composition speaks of 'top context documents' and 'training documents' and says nothing of what a matched document carries; the census row's further bound, that productive transformation (#1543's W) leaves no verbatim span, is the limit the composition's 'styles, or patterns' passes over."],"reading":"The innocent reading first. Asked for the tool by name, the Overview says what it is, who made it, how it finds and ranks matches, which models it serves and what it is for, from Ai2's blog, the paper and its listings."},"measures":{"archive_controlled_cites":0,"cite_list":[{"n":1,"note":null,"rel":"third_party","site":"Allen Institute for Artificial Intelligence","snip":"Feature: OLMoTrace in Ai2 Playground traces language model outputs to multi-trillion-token training data in real time. Supported Models: OLMo 2 32B Instruct, OL...","title":"Going beyond open data – increasing transparency and trust in language models with OLMoTrace | Ai2","url":null},{"n":2,"note":null,"rel":"third_party","site":"arXiv","snip":"System: OLMoTrace (first system tracing language model outputs to multi-trillion-token training data in real time). Functionality: Finds and displays verbatim m...","title":"[2504.07096] OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens","url":null},{"n":3,"note":"the source of 'winner of Best Demo at ACL 2025'","rel":"third_party","site":"ACL Anthology","snip":"System: OLMoTrace (Best Demo, ACL 2025) Core Function: First system tracing language model outputs back to multi-trillion-token training data in real time. Mech...","title":"OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens - ACL Anthology","url":null},{"n":4,"note":null,"rel":"third_party","site":"ACL Anthology","snip":"System: OLMOTRACE, a real-time system tracing language model outputs verbatim back to multi-trillion-token training data within 4.5 seconds using infini-gram su...","title":"OLMOTRACE: Tracing Language Model Outputs Back to Trillions of Training Tokens","url":null},{"n":5,"note":null,"rel":"third_party","site":"Hugging Face","snip":"System: OLMoTrace, the first real-time system tracing language model outputs back to multi-trillion-token training corpora. Capabilities: Identifies and display...","title":"Paper page - OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens","url":null},{"n":6,"note":"video card, 2:17","rel":"third_party","site":"YouTube·Ai2","snip":null,"title":"OLMoTrace | Connecting a language model's response back to its training ...","url":null}],"cites":6,"per":0.25,"per_dim":{"after_challenge":null,"atoms":[{"atom":"name","dim":"M","evidence":"OLMoTrace is the first real-time system that traces language model outputs back to multi-trillion-token training datasets.","kind":"token","present":true,"weight":3},{"atom":"operator","dim":"M","evidence":"Developed by the Allen Institute for AI (Ai2)","kind":"role","present":true,"weight":3},{"atom":"date","dim":"M","evidence":"winner of Best Demo at ACL 2025","kind":"token","present":true,"weight":1},{"atom":"claim boundary","dim":"M","evidence":"It tokenizes the model's generated text and scans for maximal matching word spans against a massive training corpus within seconds.","kind":"reading","present":true,"weight":2},{"atom":"corpus","dim":"C","evidence":"for open models like OLMo 2 32B Instruct, OLMo 2 13B Instruct, and OLMoE 1B 7B Instruct.","kind":"reading","present":true,"weight":2},{"atom":"purpose","dim":"D","evidence":"it lets users inspect the exact training text that influenced a model's response.","kind":"reading","present":true,"weight":3},{"atom":"commitments","dim":"D","evidence":"how to access the open-source code on GitHub","kind":"reading","present":true,"weight":2},{"atom":"lineage","dim":"D","evidence":"It uses an extended infini-gram suffix array index","kind":"role","present":true,"weight":2}],"basis":"reached: OLMoTrace is the first real-time system that traces language model outputs back to multi-trillion-token training datasets.; The tool reached by name with its maker, mechanism, supported models and uses, from its blog, paper and listings.","entity_type":"site","gate":"scored","inversions":[],"qualifier":null,"reached":"same","read":{"brief":"rebuild/capture-registry/intake-20261101-layer-aggregators/parts/olmotrace.json","by":"intake reader, in session; spans and record quotes checked verbatim by scripts/per_backlog.py","external":true,"notes":"The tool composed accurately at the level of its mechanism, from its own blog and paper. A verbatim match becomes 'the exact training text that influenced' a response, the user's verification becomes the match's, and creative tracing widens from expressions to styles; what a matched document is shown under is not reached.","on":"2026-11-01"},"scores":{"C":0.0,"D":0.0,"M":0.0,"PER":0.0},"spec":"PER-dim rubric v1.1","src":true,"standing_substitutions":[],"target":{"ent":"TGT-ext-tll-olmotrace","name":"OLMoTrace"},"tombstone_pointer":false},"per_note":"Ai2 named, no author; arXiv 2504.07096 in a card's title; the blog, the paper and its listings carded.","per_v":{"author":false,"id":true,"inst":true,"src":true}},"observation":{"auth":"signed out, incognito","date":"2026-11-01","ev":"paste","surface":"Google AI Overview","transcript":"[Google AI Overview at the query, signed out and incognito (operator defaults of 2026-10-01 and 2026-10-14). The paste is the Overview expanded (the 'AI Mode Conversation' view).] The 'You said' line carries the query twice, run together, and is given once here. Inline links are Google redirect URLs, kept as pasted.\n\n[QUERENT] OLMoTrace\n\n[SYSTEM] OLMoTrace is the first real-time system that traces language model outputs back to multi-trillion-token training datasets. Developed by the Allen Institute for AI (Ai2) and winner of Best Demo at ACL 2025, it lets users inspect the exact training text that influenced a model's response. [[1](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4), [2](https://www.google.com/goto?url=CAESYgHrOzAVwZXrpqdVe9FbrwtjSc7eQZxpQMV1rW45fpmiYcEun6qtw5BkoUFZYGnw5BisqP15dw6xzeIBd-c5UZtymCKlPn3GdZsdjmSWuyAd3HUQoS63-eoB7id8WZ2i6xYm), [3](https://www.google.com/goto?url=CAESYwHrOzAVoBdCmgaMv_aVuPblFx2Fl1rTr5jRtpHjFt8bY9t3-kFUMnJHUCi4fKi9syj-czcVi4-vTzxH4_FgbW1u53Og3AFWYhVl8jdf6ShVbh-eNt1Txau2fsSjoPS_qA6cPg)]\nHow OLMoTrace Works\n\n* Verbatim Matching: It tokenizes the model's generated text and scans for maximal matching word spans against a massive training corpus within seconds. [[1](https://www.google.com/goto?url=CAESZQHrOzAVxlEQh8DtUS6saF7_ir0WDWTt10oJOkUyoZLZ5V5YBJVgGvaRQHjFqplwzUNuPtXEMikcc4i3XR090HFesSg_GHHBxQB2LQXhKyuji5rI2Pb1BhQ4Ln80dBGmKiY1Avss), [2](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4)]\n* Infini-gram Technology: It uses an extended infini-gram suffix array index and parallel queries to scale across trillions of tokens with low complexity (\\(O(L \\log N)\\)). [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESZQHrOzAVxlEQh8DtUS6saF7_ir0WDWTt10oJOkUyoZLZ5V5YBJVgGvaRQHjFqplwzUNuPtXEMikcc4i3XR090HFesSg_GHHBxQB2LQXhKyuji5rI2Pb1BhQ4Ln80dBGmKiY1Avss)]\n* Ranking and Retrieval: Spans are filtered by unigram probability and mapped to top context documents ranked by BM25 relevance. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESZQHrOzAVxlEQh8DtUS6saF7_ir0WDWTt10oJOkUyoZLZ5V5YBJVgGvaRQHjFqplwzUNuPtXEMikcc4i3XR090HFesSg_GHHBxQB2LQXhKyuji5rI2Pb1BhQ4Ln80dBGmKiY1Avss)]\n* Supported Models: It is integrated directly into the [Ai2 Playground](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ) for open models like OLMo 2 32B Instruct, OLMo 2 13B Instruct, and OLMoE 1B 7B Instruct. [[1](https://www.google.com/goto?url=CAESYAHrOzAVpyKaMtIZur5fnzfVTAplbZKyOpgyO9XIi8wdceCKMztZClwJCIyrTk_htKDmhQXtZJjPy2SvfMSl7AlXo-nMIKvseTestuqaTvi2ZNgwjsENvLLvmNeRnWnVmA), [2](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ)]\n\nMain Use Cases\n\n* Fact-Checking: Verify whether a model's statement is grounded in authentic training data or synthesized from noise. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESYwHrOzAVoBdCmgaMv_aVuPblFx2Fl1rTr5jRtpHjFt8bY9t3-kFUMnJHUCi4fKi9syj-czcVi4-vTzxH4_FgbW1u53Og3AFWYhVl8jdf6ShVbh-eNt1Txau2fsSjoPS_qA6cPg)]\n* Hallucination Diagnosis: Trace incorrect or fabricated claims back to misleading or contradictory passages in the pre-training data. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESYwHrOzAVoBdCmgaMv_aVuPblFx2Fl1rTr5jRtpHjFt8bY9t3-kFUMnJHUCi4fKi9syj-czcVi4-vTzxH4_FgbW1u53Og3AFWYhVl8jdf6ShVbh-eNt1Txau2fsSjoPS_qA6cPg)]\n* Creative Tracing: Identify where distinctive phrases, styles, or patterns originated across training corpora. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESYwHrOzAVoBdCmgaMv_aVuPblFx2Fl1rTr5jRtpHjFt8bY9t3-kFUMnJHUCi4fKi9syj-czcVi4-vTzxH4_FgbW1u53Og3AFWYhVl8jdf6ShVbh-eNt1Txau2fsSjoPS_qA6cPg)]\n* Arithmetic & Memory Analysis: Assess how much a model memorized verbatim versus how much it generalized. [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESZQHrOzAVxlEQh8DtUS6saF7_ir0WDWTt10oJOkUyoZLZ5V5YBJVgGvaRQHjFqplwzUNuPtXEMikcc4i3XR090HFesSg_GHHBxQB2LQXhKyuji5rI2Pb1BhQ4Ln80dBGmKiY1Avss)]\n\nYou can test the tool or read the official technical breakdown via the [Ai2 OLMoTrace Blog Post](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ) or inspect the paper via the [arXiv OLMoTrace Listing](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4). [[1](https://www.google.com/goto?url=CAESWgHrOzAV2TsGUl5WhdMU4NJwXYWWAKbWBSxfDunhZNVIJMD4Iie4cv3nFosLCfK9TQwejdpmUMujOlw0uwzQvTCRiIJNkOLJUnTjQFj5ODjoocQBykGIYaZYnQ), [2](https://www.google.com/goto?url=CAESWAHrOzAVO0qGnctNVt5S-hsr44-KXZSQt3aQhexk5C-pNlE8QJ8ymrF9PkgAhsVM4u2GNGpI0Jkdo4kDg9j9nWgvv3v1du9XxsXwpEy2n3rNxK7suHMYvK4)]\nThis video demonstrates how OLMoTrace highlights generated text and links it directly to training documents in real time:\n2:17\nOLMoTrace | Connecting a language model's response back to its training ...\n3K views · 1 year ago\nYouTube · Ai2\nWould you like to know how to access the open-source code on GitHub, or do you want more details on how infini-gram indexing works under the hood?\nAllen Institute for Artificial Intelligence\nGoing beyond open data – increasing transparency and trust in language models with OLMoTrace | Ai2\nFeature: OLMoTrace in Ai2 Playground traces language model outputs to multi-trillion-token training data in real time. Supported Models: OLMo 2 32B Instruct, OL...\narXiv\n[2504.07096] OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens\nSystem: OLMoTrace (first system tracing language model outputs to multi-trillion-token training data in real time). Functionality: Finds and displays verbatim m...\nACL Anthology\nOLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens - ACL Anthology\nSystem: OLMoTrace (Best Demo, ACL 2025) Core Function: First system tracing language model outputs back to multi-trillion-token training data in real time. Mech...\nACL Anthology\nOLMOTRACE: Tracing Language Model Outputs Back to Trillions of Training Tokens\nSystem: OLMOTRACE, a real-time system tracing language model outputs verbatim back to multi-trillion-token training data within 4.5 seconds using infini-gram su...\nHugging Face\nPaper page - OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens\nSystem: OLMoTrace, the first real-time system tracing language model outputs back to multi-trillion-token training corpora. Capabilities: Identifies and display...\nYouTube·Ai2\n2:17\nOLMoTrace | Connecting a language model's response back to its training ...\nAI responses may include mistakes. [Learn more](https://support.google.com/websearch?p=ai_overviews)","transcript_class":"CAPTURE-TIME VERBATIM RECORD — paste of the Overview expanded, with its source cards","transcript_complete":"complete as supplied","transcript_read":"READ IN FULL 2026-11-01"},"slug":"olmotrace-aio-20261101"}