SPXI instrument · method 0.9 of 87, level 0 (Measurement): Capture Registry (longitudinal observation). An instrument of SPXI, the Semantic Packet for eXchange & Indexing; the whole practice is itemized at spxi.dev/methods.

Capture Registry › capture dolma-dataset-aio-20261101

One record of the canonical Capture Registry (EA-WG-CAPTURES-01), cited at https://www.alexanarch.org/captures/dolma-dataset-aio-20261101/. the canonical Capture Registry (version 12.518) · the address page · this card in the gallery · this record as data · table of contents.

AXN:0BAE.CAPTURE — this address. Each observation below is one AXN, sealed with its analysis and measures; the identifier names the reading together with the response it reads.

Related deposits: #1543 Generative Uptake: Training-Layer Literature Extended from Machine Reading to Machine Comp…

Machine Reception2026-11-01
Dolma dataset
Originator: Dolma · outside the archive (out-universe) · site · SPXI treatment: partial
BROAD MATCH
no
image
THE CORPUS, SIGNED OUT, 2026-11-01: 'Dolma is an open-source 3-trillion-token English corpus created by the Allen Institute for Artificial Intelligence (Ai2) to support language model pretraining and transparency.' Access (Hugging Face, the GitHub toolkit); key features: scale ('version 1.7 utilizes about 1.715 trillion actual training tokens across ~5.2 billion documents totaling over 5,300 GB'), data sources by kind with their named sets, the curation pipeline ('Bloom filter deduplication (by URL, paragraph, and document), PII/toxicity masking'), licensing ('ODC-BY and Ai2's ImpACT license'), primary use (OLMo); two follow-up questions. Six cards: Hugging Face, the GitHub Pages documentation, Ai2, arXiv, Deep (Learning) Focus, ACL Anthology.
Full record — 7,542 characters, 6 sources
Capture record
captured
2026-11-01
surface
Google AI Overview
auth state
signed out, incognito
evidence class
paste
PER (dimensional, rubric v1.1)
PER 0.19 · M 0.33 · C 0.00 · D 0.00
PER (historical scalar)
0.25
PER units retained
inst, id, src
citations read
6
observation id
OBS-6dd2d86d5018
address id
ADDR-5a024e0d48d0
Reading

The innocent reading first. Asked for the dataset, the Overview says what it is, who made it and why, where to get it, what it holds, how it was cleaned, under what licences it is released and which models it trains, from the dataset's own pages and its paper.

Findings
Analysis analyst prose, not machine text

The corpus composed accurately and more finely than the saved abstract, from the dataset's own pages, with Ai2 as maker where the census row names only the paper's authors. Whether a document in the corpus keeps its author's name, the census row's open question, is not reached.

Machine text, verbatim
CAPTURE-TIME VERBATIM RECORD — paste of the Overview expanded, with its source cards · complete as supplied · READ IN FULL 2026-11-01
[Google AI Overview at the query, signed out and incognito (operator defaults of 2026-10-01 and 2026-10-14). The paste is the Overview expanded (the 'AI Mode Conversation' view).] The 'You said' line carries the query twice, run together, and is given once here. Inline links are Google redirect URLs, kept as pasted. [QUERENT] Dolma dataset [SYSTEM] Dolma is an open-source 3-trillion-token English corpus created by the [Allen Institute for Artificial Intelligence (Ai2)](https://www.google.com/goto?url=CAEShQEB6zswFdYTFzcT8iARHW7j2rb03Q9fzUvuXk92kI5vJW1ys8rU0ZyNqvi9nb9SEfxVE6GDLUXfzIOJ9Mdgxo_wLR3UpEXqPziCd7TsCkT04tfI3NGm7zbcWyRpgISmYcfpVr3bLZKfJFJXV2yp769te1gUIyCeAsDgwURFI-czaqs1Tqxx) to support language model pretraining and transparency. [1] You can access and download the files directly via the [AllenAI Dolma on Hugging Face](https://www.google.com/goto?url=CAESZQHrOzAVqDGWZgFkNAiGOTr1a3I1Et4PFMQit55QdiU3kqPsTyA9okSIwbdfjlhNbKbOp0k5Zv-YXrOnL6SgaiaRrWOD1-DMaGhXR8JXgYW-jE3KNA8m5Y1C3hoIQbK8y1B0uWYy) hub or explore the curation pipeline using the Dolma GitHub Toolkit. [[1], [2]] Key Features * Scale: Contains roughly 3 trillion tokens (version 1.7 utilizes about 1.715 trillion actual training tokens across ~5.2 billion documents totaling over 5,300 GB). [[1], [2]] * Data Sources: Combines a diverse mix of web content (Common Crawl, C4, RefinedWeb), academic papers (arXiv, Semantic Scholar, peS2o), code (The Stack, StarCoder), public-domain books (Project Gutenberg), encyclopedic material (Wikipedia, Wikibooks), and social media discussions (Reddit, StackExchange). [[1], [2]] * Curation Pipeline: Involves language filtering, Gopher/C4 quality heuristics, Bloom filter deduplication (by URL, paragraph, and document), PII/toxicity masking, and evaluation decontamination. [[1], [2]] * Licensing: Released under permissive research licenses (such as ODC-BY and Ai2's ImpACT license) alongside the [Dolma ACL Research Paper](https://www.google.com/goto?url=CAESYwHrOzAVvKhxmIIzUklJEDSJyWY8Ro1YnSpFLFxZhQ0EcW4hazs3O-L1Y8BHPxI9cIxyUxxa4A8aVoomoKlEYiUEFBtlTpFdl0qVShVTplvTAOrrsSsWQ_H2MW35T8flseiMVA). [[1], [2], [3]] * Primary Use: Designed and used specifically to pre-train the [OLMo](https://www.google.com/goto?url=CAESfgHrOzAVf6rn5OZttIM0us6WvAir5Q1-WR0EyYm9ecTP7XBTIM-DinlhVME8i_KIkf8LZ3DAOz14u-sSPbfl_Pc4u7hcHYX6Q1oBjXL5hFzJgdfGrRKVE1UMINo78prcPEMu19EHqLlLGx8laR-2Dzu5RINQkc4u25jqyRpHwQ) open-source language model family. [[1], [2]] If you'd like, let me know: * Are you planning to train or fine-tune an LLM using Dolma? * Are you looking for specific subsets or data processing scripts? I can provide the right code commands or pipeline details. Hugging Face
Sources (6) as cited, and as the copy produced them
  1. Hugging Face third_party
    allenai/dolma · Datasets at Hugging Face
    Overview: AllenAI Dolma is an open corpus dataset containing 3 trillion tokens (version 1.7 utilizing 1.715 trillion actual training tokens) designed for langua...
    as pastedallenai/dolma · Datasets at Hugging Face Overview: AllenAI Dolma is an open corpus dataset containing 3 trillion tokens (version 1.7 utilizing 1.715 trillion actual training tokens) designed for langua... GitHub Pages documentation
  2. GitHub Pages documentation third_party
    dolma | Data and tools for generating and inspecting OLMo pre-training data.
    Dataset: Open 3-trillion-token corpus (web content, academic publications, code, books, encyclopedic materials) created as training data for OLMo by AI2; availa...
    as pasteddolma | Data and tools for generating and inspecting OLMo pre-training data. Dataset: Open 3-trillion-token corpus (web content, academic publications, code, books, encyclopedic materials) created as training data for OLMo by AI2; availa... Allen Institute for Artificial Intelligence
  3. Allen Institute for Artificial Intelligence third_party
    Ai2 Dolma: 3 trillion token open corpus for language model pretraining | Ai2
    Dataset: Dolma (3 trillion tokens, 5,245M documents, 5,334 GB) developed by Ai2 for OLMo pretraining. Sources: CommonCrawl, C4, peS2o, The Stack, Project Gutenb...
    as pastedAi2 Dolma: 3 trillion token open corpus for language model pretraining | Ai2 Dataset: Dolma (3 trillion tokens, 5,245M documents, 5,334 GB) developed by Ai2 for OLMo pretraining. Sources: CommonCrawl, C4, peS2o, The Stack, Project Gutenb... arXiv
  4. arXiv third_party
    [2402.00159] Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
    Overview: Dolma is an open-source, three-trillion-token English corpus designed for language model pretraining research, addressing the scarcity of publicly ava...
    as pasted[2402.00159] Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research Overview: Dolma is an open-source, three-trillion-token English corpus designed for language model pretraining research, addressing the scarcity of publicly ava... Deep (Learning) Focus·Deep (Learning) Focus
  5. Deep (Learning) Focus·Deep (Learning) Focus third_party
    Dolma, OLMo, and the Future of Open-Source LLMs
    Overview: Dolma (3T-token open pretraining corpus) and OLMo (fully open-source LLM suite) demystify language model development by openly releasing training data...
    as pastedDolma, OLMo, and the Future of Open-Source LLMs Overview: Dolma (3T-token open pretraining corpus) and OLMo (fully open-source LLM suite) demystify language model development by openly releasing training data... ACL Anthology
  6. ACL Anthology third_party
    Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research - ACL Anthology
    Resource: Dolma (ACL 2024 Best Resource Paper Award). Corpus: Open English dataset containing three trillion tokens for language model pretraining research. Con...
    as pastedDolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research - ACL Anthology Resource: Dolma (ACL 2024 Best Resource Paper Award). Corpus: Open English dataset containing three trillion tokens for language model pretraining research. Con... AI responses may include mistakes. [Learn more](https://support.google.com/websearch?p=ai_overviews)
↻ Re-runpermalink