The Negative of the Negative › entity openai-gdpval

One entity of the Negative of the Negative (EA-NEGONT-02, #1690), cited at https://www.alexanarch.org/non/openai-gdpval/. the table of contents · this entity as data · contents as data · the dataset on the Hub · row json · field ledger · archive ledger · D/R/O traversal · the address page · the Labor Vector meta-aggregator record.

A reader that fetches this page may stop short of its end; the entry's text, every sentence of its knowledge object with the claims it cites and each claim's quote, source and locus, is in entity.json beside it.

GDPval

SPXI instrument · method 0.16 of 87, level 0 (Measurement): The Negative of the Negative (/non). An instrument of SPXI, the Semantic Packet for eXchange & Indexing; the whole practice is itemized at spxi.dev/methods.

type C — a conventional reading holds it (public entity on which the archive bears: a task benchmark; an entity of the Labor Vector meta-aggregator) · sought at GDPval

composed 2026-10-29 under EA-NEGONT-02 v0.9, #1690, equal case, ρ = extends (draft, not frozen; ledgers unaudited, one extractor; checked by the session and the page builder)

AXN AXN:0B00.NON.🕐☀️🧊🏺🔥👉 · root AXN:0B00.NON

composed in session · Claude (Opus 5.5), subagent, 2026-10-29; the composer is a Claude model, a product family the field ranks (F27, F55)

This row was composed in session. SPXI-gnΩsis (gnosis.spxi.dev) opens a Wikipedia revision sentence by sentence against the sources it cites; a commission consults the archive by its own procedure and composes the entity's row here; a quote to recompose this entity is free.

Google AI Overview (expanded)epoch 2026-10-29
not frozentype C — a conventional reading holds it (public entity on which the archive bears: a task benchmark; an entity of the Labor Vector meta-aggregator)signed out, incognitoknowledge object · 30 sentences

EA-NEGONT-02 v0.9, #1690: equal case, ρ = extends, read from the Labor Vector census record, the capture record and the deposits that bear on the benchmark's classes (#771, #855, #199, #778, #1616, #1540, #482, #48, #637, #163, #623, #308, #533, #1545); archive selected by D/R/O (no D or R candidate: no archive text names GDPval; 14 deposits admitted by reading at O-class, 12 at the first pass and 2 by the hop, with 3 grouped as instances; the rest of the hop's 136 read and rejected), with the Labor Vector census record (25 claims) and the Capture Registry's findings, reading, analysis, query note and PER note (10 claims) ledgered as the archive's records of the entity; the field B by three origins (7 cards, 2 of them unreachable; 1 own page; Wikipedia, no article, with one item in another article); the composer read the ledgers, never the transcript, except where the brief asks for T's claims and the delta. Ledger: unaudited (§3.8: one extractor per ledger); field quotes 67/67 verified by script against saved text, the PDF's text, the Wikipedia response summary or the card text; archive quotes 56/56 verified against the registry, census.json, questions.json and the deposit texts.

WORKING KNOWLEDGE OBJECT · NOT FROZEN · LEDGER UNAUDITED

Current instantiation — composed under EA-NEGONT-02 v0.9 (#1690)

Compression · Field and archive, equal terms — L(B ∪ A)GDPval

GDPval is OpenAI's benchmark of AI models on 1,320 tasks built from the work product of professionals in 44 occupations, graded by experts against the professionals' own deliverables (the paper; census record); the census reads each task as a fixed commission, where GDPval meets DSL (Labor Vector record; #771).

🧾 Whose work

  • Task-origin: each task an expert's actual work product, 14 years' experience on average (paper; record).5↘
    GDPval is, in the Labor Vector census, a benchmark, "an evaluation dataset with an experimental automated grader", whose kind of evidence is a task benchmark at two of the census's intersections, labor performed with AI and labor producing AI, read for who supplies the labor, who directs it and who receives its gains (Labor Vector record); its authors introduce it as "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), in its first version (OpenAI's PDF).§1 OpenAI's own page is reached here only by its card, "Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries" (OpenAI, card only; the page answered 403), and a video is titled "GDPval: An AI benchmark for real-world tasks" (Vivek Haldar, title only).§2 Each task is "a request (often with reference files) and a deliverable (work product)" (the paper), "constructed based on actual work product created by an expert professional" (the paper), from professionals with "an average of 14 years of experience" (arXiv abstract), which the census records as its unit and as whose labor it is: the professionals' work product "recast as tasks for models" (Labor Vector record).§6 The experts needed "a minimum of 4 years of professional experience in their occupation" and were "well compensated for their time and experience", no rate given (the paper); the authors thank unnamed "vendor partners" and the experts "without whom this work would not have been possible" (the paper); Inspect Evals describes them as professionals "with over 14 years of experience on average" and the tasks as drawn from work products such as "a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan" (Inspect Evals); and the census records them as who supplies the labor, as task writers, reviewers and graders, at a rate the paper does not state (Labor Vector record).§9 Artificial Analysis's GDPval-AA v2.1, a distinct framework "for OpenAI's GDPval dataset", runs models in an agentic loop with Elo ratings "derived from blind pairwise comparisons" in which "an LLM judge picks a winner", its leaderboard led on 2026-10-29 by Claude Opus 5.5 at 1866 (Artificial Analysis); Inspect Evals implements only "the 220 tasks in the gold open-sourced set", and its GPT-5 run scored 47.3% ± 6.2% against 31.9% (wins only) and 34.6% (wins and ties) on "the official GDPval leaderboard by OpenAI" (Inspect Evals).§27
  • Supplied and directed: experts "well compensated", no rate; OpenAI's team directs; vendors unnamed (paper; record).2↘
    The experts needed "a minimum of 4 years of professional experience in their occupation" and were "well compensated for their time and experience", no rate given (the paper); the authors thank unnamed "vendor partners" and the experts "without whom this work would not have been possible" (the paper); Inspect Evals describes them as professionals "with over 14 years of experience on average" and the tasks as drawn from work products such as "a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan" (Inspect Evals); and the census records them as who supplies the labor, as task writers, reviewers and graders, at a rate the paper does not state (Labor Vector record).§9 OpenAI's research team directs the work (Labor Vector record): every task passed "automated model-based screening and multiple stages of human expert review", five reviews on average, researchers reviewed all tasks and promoted experts to reviewers, and the experts "retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks" (the paper).§10
  • Labor producing AI: ground-truth against variance-generative labor (#163); extraction and bearing-cost (#623; #533; #308); class a relation, not a wage (#1545).3↘
    Of the labor that produces AI the archive distinguishes "*ground-truth labor*: the labor of telling the model what is already known to be true" from "*variance-generative labor*", the producing of "new conceptual structures that the model cannot generate from its own distribution" (#163), and holds as a hypothesis that "Ghost work is paid badly; variance-generative labor is not paid at all" (#163).§11 It holds that the labor of "the data labelers performing RLHF" is extracted and its non-compensation felt "materially" (#623), states that in AI-adjacent meaning labor, data labeling among it, "<5% converts to invoice" (#308, no measurement given at the locus), and audits a framework by whether it names each source of semantic labor, "annotators" and "users who re-prompt and re-frame" among them: "Concealed bearing-cost is the first marker of extraction" (#533).§12 For the archive class is a relation and not an essence, so that "A highly paid academic can occupy a proletarian semantic relation to a composition layer", and the data annotator is "A paradigmatic occupational case, not the definition" (#1545).§13

⚖️ How it grades

  • Experts: blinded pairwise comparison, over an hour each, blinding incomplete (paper; Wikipedia).1↘
    Grading is by "blinded expert pairwise comparisons", experts in the occupation ranking "two or more unlabeled work deliverables" (the paper), each comparison taking "over an hour" (the paper), and the blinding was not complete: "model samples may still have been identifiable due to stylistic differences" (the paper); Wikipedia gives the same design, "blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average" (Wikipedia, 'AI agent benchmark').§14
  • Grader: GPT-5-high based, 66% agreement, weaker on OpenAI outputs; built and graded by OpenAI (paper; Inspect; record).3↘
    The experimental automated grader reaches "66% agreement with human expert graders" against 71% between experts (the paper), is "based on GPT-5-high" and "shows lower correlation with human expert graders when assessing outputs from capable OpenAI models", in line with evidence "that models often favor their own responses" (the paper), and is a proxy its authors do not take for "a full substitute for industry expert graders" (the paper); scoring through Inspect Evals runs through "the OpenAI auto grader", and "OpenAI provides their automated grader model for free" (Inspect Evals); the census tags the benchmark "NOT independent of a provider: built and graded by OpenAI" (Labor Vector record).§15 The census keeps those limits (Labor Vector record) and states what the benchmark answers, deliverables compared on "expert-written, precisely specified, one-shot occupational tasks" and the expert time and cost saved "once review and redo are counted", and what it does not: "how work is performed in actual workplaces", "employment or wage outcomes", "who receives the time and cost savings it models" and "the terms and rate at which the professionals were paid" (Labor Vector record), its negative tags holding that deliverable parity "is NOT a finding of job replacement in the paper" (Labor Vector record).§25 The gold subset's prompts and reference files are open-sourced, "scrubbed of information that could be used to identify the expert who wrote the task" (the paper), 12 of the 220 "ungradable due to limitations of the automated grader" (the paper), as the census records (Labor Vector record); the Hugging Face dataset states that "Each task consists of a text prompt and a set of supporting reference files" and serves deliverable files and rubrics beside them, rubric items marked "author_type": "human" (Hugging Face), was downloaded 271,657 times in the last month and lists 47 models trained on it (Hugging Face); the other 1,100 tasks are, in Wikipedia's account, "only available internally in OpenAI", the public set gradable "by a finetuned version of GPT-5" (Wikipedia).§26

📈 What it found

  • Near parity: 47.6% of Claude Opus 4.1's deliverables as good or better; linear gains (paper).4↘
    Its headline finding is that "models are beginning to approach parity with industry experts" on the gold subset (the paper), "the current best frontier models are approaching industry experts in deliverable quality" and OpenAI's frontier models improved "roughly linearly over time" (arXiv abstract; the paper); 47.6% of Claude Opus 4.1's deliverables were graded as good as or better than the human deliverable, and the next sentence has model deliverables matching or beating experts' "in just over half the tasks" (the paper), a difference the census leaves unchecked (Labor Vector record), Claude Opus 4.1 leading on aesthetics and GPT-5 on accuracy (the paper).§16 Experts preferred the human deliverable most often "because models failed to fully follow instructions" (the paper), and of GPT-5's losses "roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic)" (the paper), findings the census carries (Labor Vector record).§17 Artificial Analysis's page, in a paragraph it heads Abstract under the paper's title, has models performing "certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts" (Artificial Analysis), where the paper's own abstract analyses the potential, "when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts" (arXiv abstract).§21 Its authors are Tejal Patwardhan and 18 others (arXiv), of OpenAI, with correspondence to an openai.com address (the paper), the operator the census records, whose own post "answered 403 and is unread" (Labor Vector record); OpenAI's PDF was created on 25 September 2025 (OpenAI's PDF), before arXiv v1 of 5 October 2025 under CC BY 4.0 (the paper), the date the census gives the first version, with the dataset's license not stated (Labor Vector record); future iterations are to add "greater breadth, realism, interactivity, and contextual nuance" and the dataset is being expanded (the paper), and the census records it active, later releases unrecorded, OpenAI's post, the figures, the pay rate and the vendors unchecked (Labor Vector record).§28
  • Losses: instructions unfollowed; 29% of GPT-5's losses bad or catastrophic (paper).1↘
    Experts preferred the human deliverable most often "because models failed to fully follow instructions" (the paper), and of GPT-5's losses "roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic)" (the paper), findings the census carries (Labor Vector record).§17
  • Payoff: 90x naive, 1.12x once review and redo count (paper); Artificial Analysis says 100x (AA).3↘
    The authors model savings for an expert who "samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats", completing the task themselves if no output satisfies (the paper); for GPT-5, at a 39.0% win rate, the naive speed figure of 90x becomes 1.12x tried once and 1.39x tried n times, and "When incorporating time to review and redo work, the payoff from using a model shrinks" (the paper).§19 The analysis leaves out the review of human deliverables, the chance that they too are undesirable, and "the cost of catastrophic mistakes" (the paper), and its authors hold that it over-penalizes the model, since the professional adjusts the prompt and reviews faster as the task grows familiar (the paper); its dollar values multiply self-reported completion times by median occupational wages, which "likely underestimate their true market cost" (the paper), and the census records the gains as "not recorded as a distribution", the savings modelled for the expert at median wages (Labor Vector record).§20 Artificial Analysis's page, in a paragraph it heads Abstract under the paper's title, has models performing "certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts" (Artificial Analysis), where the paper's own abstract analyses the potential, "when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts" (arXiv abstract).§21
  • Length: win rates fall with task length; scaffolding helps (paper).1↘
    Tasks "require an average of 7 hours of work for an expert professional to complete" and reach "multiple weeks" (the paper); win rates are highest on tasks of 0–2 hours and "decline steadily as completion time increases" (the paper), and more reasoning effort, task context and scaffolding improve performance (arXiv abstract).§23

🧭 Commission and drag

  • Task-origin: fluent output can abandon the commission (#771).2↘
    The census reads a task an expert wrote as "a fixed commission", at which GDPval meets DSL "at the question of whether the system's labor fulfils the commissioned task, measured in its own terms of win rate" (Labor Vector record); DSL "measures where a synthetic system's labor flows relative to the commissioned task, across five layers" (#771), on the principle that "a system can produce fluent, well-attributed output while having abandoned the user's commissioned direction" (#771).§7 The census reads that accounting, "the expert's review and redo time against the model", as where GDPval meets user labor drag (Labor Vector record), the archive's measure of the user's "subsequent semantic labor" redirected "into parsing, locating, and managing the substrate rather than advancing the object", an injury that "leaves no residue" in the artifact (#771) and is identified only by "blind inter-rater coding" (#771); for the archive the erasure of task-origin can occur "upstream of, and invisibly to, any audit of the finished artifact" (#771).§22
  • One-shot: DSL is identified only on one-shot commissions, as GDPval's tasks are (#771; paper).1↘
    The archive's directionality metrics yield a single reproducible value "only on transcripts where the task vector is *exogenous and fixed* — a one-shot commission" (#771), and GDPval's authors state their tasks to be "precisely-specified and one-shot, not interactive", with "the full context of the task in the prompt", where "in real life it often takes effort to figure out the full context of a task" (the paper).§8
  • User labor drag: review and redo charged to the model; no residue in the artifact (record; #771).2↘
    The analysis leaves out the review of human deliverables, the chance that they too are undesirable, and "the cost of catastrophic mistakes" (the paper), and its authors hold that it over-penalizes the model, since the professional adjusts the prompt and reviews faster as the task grows familiar (the paper); its dollar values multiply self-reported completion times by median occupational wages, which "likely underestimate their true market cost" (the paper), and the census records the gains as "not recorded as a distribution", the savings modelled for the expert at median wages (Labor Vector record).§20 The census reads that accounting, "the expert's review and redo time against the model", as where GDPval meets user labor drag (Labor Vector record), the archive's measure of the user's "subsequent semantic labor" redirected "into parsing, locating, and managing the substrate rather than advancing the object", an injury that "leaves no residue" in the artifact (#771) and is identified only by "blind inter-rater coding" (#771); for the archive the erasure of task-origin can occur "upstream of, and invisibly to, any audit of the finished artifact" (#771).§22
  • What scores see: means, outputs, a closed ontology (#855; #199; #778; #1616; #1540); standardized tasks (#482; #48; #637).2↘
    Of benchmarks as a class the archive holds that benchmarking "tests for predetermined capabilities" (#482), that current benchmarks measure "atomic intelligence: the capacity to solve discrete, well-defined tasks with measurable success criteria" (#48) and that they "measure model performance on standardized tasks" (#637); GDPval's authors set it against "AI benchmarks in the style of an academic test that focus on reasoning difficulty", its tasks "based on actual work product from industry experts" and "tied to time and cost required for completion" (the paper).§3 Of what such scores see the archive holds that "Benchmarks measure means, not tails" (#855), that in code they measure functional correctness while solution-space diversity is "the property no benchmark measures" (#199, a hypothesis), that "Output-level evaluation — which is what benchmarks measure — cannot detect reasoning-level failures" (#778), and that a benchmark "begins by closing an ontology: whatever the evaluator has not represented is noise, an invalid action, or invisible to the score" (#1616); "Reality administers no benchmarks; it administers events" (#1540).§18

⚠️ Limits

  • Stated: a limited, initial cut; no physical or tacit work; one-shot (paper).3↘
    The archive's directionality metrics yield a single reproducible value "only on transcripts where the task vector is *exogenous and fixed* — a one-shot commission" (#771), and GDPval's authors state their tasks to be "precisely-specified and one-shot, not interactive", with "the full context of the task in the prompt", where "in real life it often takes effort to figure out the full context of a task" (the paper).§8 The authors call the full set "a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks" (the paper), leave out of scope "Manual labor and physical tasks", tacit knowledge, personal data, proprietary tools and "communication between individuals" (the paper), and report that GPT-5 did worse on under-specified prompts, in an experiment on "an earlier version of the GDPval gold subset" (the paper); they add that the human baseline itself could go: "we could replace our baseline with increasingly strong models over time and keep evaluating" (the paper).§24 The census keeps those limits (Labor Vector record) and states what the benchmark answers, deliverables compared on "expert-written, precisely specified, one-shot occupational tasks" and the expert time and cost saved "once review and redo are counted", and what it does not: "how work is performed in actual workplaces", "employment or wage outcomes", "who receives the time and cost savings it models" and "the terms and rate at which the professionals were paid" (Labor Vector record), its negative tags holding that deliverable parity "is NOT a finding of job replacement in the paper" (Labor Vector record).§25
  • Does not answer: workplaces, wages, who gets the savings, the experts' pay (record).1↘
    The census keeps those limits (Labor Vector record) and states what the benchmark answers, deliverables compared on "expert-written, precisely specified, one-shot occupational tasks" and the expert time and cost saved "once review and redo are counted", and what it does not: "how work is performed in actual workplaces", "employment or wage outcomes", "who receives the time and cost savings it models" and "the terms and rate at which the professionals were paid" (Labor Vector record), its negative tags holding that deliverable parity "is NOT a finding of job replacement in the paper" (Labor Vector record).§25

🏛️ Data and lineage

  • Open: 220 tasks, experts' identities scrubbed, with deliverables and rubrics; 1,100 internal (paper; Hugging Face; Wikipedia).1↘
    The gold subset's prompts and reference files are open-sourced, "scrubbed of information that could be used to identify the expert who wrote the task" (the paper), 12 of the 220 "ungradable due to limitations of the automated grader" (the paper), as the census records (Labor Vector record); the Hugging Face dataset states that "Each task consists of a text prompt and a set of supporting reference files" and serves deliverable files and rubrics beside them, rubric items marked "author_type": "human" (Hugging Face), was downloaded 271,657 times in the last month and lists 47 models trained on it (Hugging Face); the other 1,100 tasks are, in Wikipedia's account, "only available internally in OpenAI", the public set gradable "by a finetuned version of GPT-5" (Wikipedia).§26
  • Derivatives: GDPval-AA by LLM judge; Inspect Evals replication (AA; Inspect).1↘
    Artificial Analysis's GDPval-AA v2.1, a distinct framework "for OpenAI's GDPval dataset", runs models in an agentic loop with Elo ratings "derived from blind pairwise comparisons" in which "an LLM judge picks a winner", its leaderboard led on 2026-10-29 by Claude Opus 5.5 at 1866 (Artificial Analysis); Inspect Evals implements only "the 220 tasks in the gold open-sourced set", and its GPT-5 run scored 47.3% ± 6.2% against 31.9% (wins only) and 34.6% (wins and ties) on "the official GDPval leaderboard by OpenAI" (Inspect Evals).§27
  • Origin: 19 OpenAI authors; PDF 25 September 2025, arXiv 5 October (paper; record).3↘
    For its authors adoption, usage and GDP indicators are "lagging indicators of AI impacts", and measuring capabilities directly lets them "assess economic relevance ahead of widespread adoption" (the paper); they build on production usage analyses "to cover areas where model adoption is still emerging" (the paper), the Anthropic Economic Index among them, the census's neighbour (Labor Vector record), and hope for "better data to assess the social impacts of AI models" (the paper).§4 The full set covers 1,320 tasks across 44 occupations, "sourced to cover the majority of Work Activities tracked by O*NET for each occupation" (the paper), in the top 9 sectors of U.S. GDP (arXiv abstract), occupations that "collectively earn $3T annually" (the paper), the sectors chosen at over 5% of GDP and the occupations by total wages and as predominantly digital, which GPT-4o decided by classifying each task, at a 60% threshold (the paper; Labor Vector record); the census counts 30 tasks per occupation and a gold subset of 220 covering 208 O*NET tasks, in U.S. occupations, its period the first version (Labor Vector record).§5 Its authors are Tejal Patwardhan and 18 others (arXiv), of OpenAI, with correspondence to an openai.com address (the paper), the operator the census records, whose own post "answered 403 and is unread" (Labor Vector record); OpenAI's PDF was created on 25 September 2025 (OpenAI's PDF), before arXiv v1 of 5 October 2025 under CC BY 4.0 (the paper), the date the census gives the first version, with the dataset's license not stated (Labor Vector record); future iterations are to add "greater breadth, realism, interactivity, and contextual nuance" and the dataset is being expanded (the paper), and the census records it active, later releases unrecorded, OpenAI's post, the figures, the pay rate and the vendors unchecked (Labor Vector record).§28

📝 Record and composition

  • At the name: specification without results or limits; the review time left out (Capture Registry, 2026-10-29).1↘
    At its name on Google's AI Overview, 2026-10-29, the composition gave "the benchmark's maker, scope, construction and grading, and where to find its data", every figure exact against the record, "blind head-to-head comparisons" without the caveat on blinding, and Artificial Analysis's leaderboard set beside GDPval "as an evaluator, not merged with it" (Capture Registry, 2026-10-29); the registry reads it as "composed at its specification" and stopping "before its results", the findings and the limits absent, the experts' recruitment and pay not composed, and the record's labour question carried "as design" while the review time charged against the model is left out (Capture Registry, 2026-10-29).§29
  • Reception: no card cites the record; no Wikipedia article (Capture Registry; Wikipedia; record).1↘
    The worked question that names GDPval sets it beside a provider's record of its own users and a classification of one assistant's conversations and finds that none "records, from the commissioning side, whether the system's labor advanced the task the user set, or who receives the time it saves", which "is the question DSL measures" (Labor Vector record); no card cites the record, which landed the same day (Capture Registry, 2026-10-29), and Wikipedia has no article on GDPval, the name standing as one item of 'AI agent benchmark' (Wikipedia, 2026-10-29).§30
1what it is: a benchmark of model capability on economically valuable tasksOpenAI — gdpval: evaluating ai model performance - OpenAI4 claims · B3, B4, B6, LV
  • LV-01 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].kind stipulation
    benchmark (an evaluation dataset with an experimental automated grader)
  • F1 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s1 documented · field
    We introduce GDPval, a benchmark evaluating AI model capabilities on real-world economically valuable tasks.
  • F2 OpenAI — gdpval: evaluating ai model performance - OpenAI, p. 1, §1 ¶2 (the card's own text) documented · field
    Our paper introduces the first version of GDPval, a benchmark evaluating AI model performance on real-world economically valuable tasks.
  • F64 YouTube·Vivek Haldar — GDPval: An AI benchmark for real-world tasks (video, 11:23), card title (video, 11:23; unreachable: HTTP 429) documented · field
    GDPval: An AI benchmark for real-world tasks
2what the record's schema asksLV5 claims · LV
  • LV-04 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].evidence_kind stipulation
    ["task benchmark"]
  • LV-03 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].intersections; intersections['labor-with-ai'], ['labor-producing-ai'] stipulation
    ["labor-with-ai", "labor-producing-ai"] … Labor performed with AI: assistance, delegation, task fidelity, correction burdens and user labor drag … Labor producing AI: annotation, moderation, evaluation, maintenance, and creative or scholarly work entering training and retrieval
  • LV-24 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json organizing_question stipulation
    Who supplies the labor, who directs it, and who receives its gains — and what kind of evidence does this resource bring to that question?
  • LV-15 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].answers interpretation
    ["how frontier models' deliverables on expert-written, precisely specified, one-shot occupational tasks compare with experts' deliverables, as judged by blinded expert graders", "how that comparison varies by model, sector, occupation, file type and task length", "under its stated scenarios, how much expert time and cost a model could save once review and redo are counted"]
  • LV-16 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].does_not_answer interpretation
    ["how work is performed in actual workplaces, where context must be found and tasks are interactive", "employment or wage outcomes", "who receives the time and cost savings it models", "physical, interpersonal or tacit-knowledge work", "the terms and rate at which the professionals were paid"]
3scope and count: 1,320 tasks, 44 occupations, 9 sectorsOpenAI — gdpval: evaluating ai model performance - OpenAI6 claims · B1, B3, B8, LV
  • F63 OpenAI — Measuring the performance of our models on real-world tasks | OpenAI, card title and snippet (OpenAI's page unreachable: HTTP 403) documented · field
    GDPval: Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries (threshol...
  • F7 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Representative breadth', s1 documented · field
    the GDPval full set covers 1,320 tasks across 44 occupations, sourced to cover the majority of Work Activities tracked by O*NET for each occupation
  • F3 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s2 documented · field
    GDPval covers the majority of U.S. Bureau of Labor Statistics Work Activities for 44 occupations across the top 9 sectors contributing to U.S. GDP (Gross Domestic Product).
  • F8 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 ¶1 documented · field
    GDPval covers tasks from 9 sectors and 44 occupations that collectively earn $3T annually.
  • LV-12 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].count documented
    {"value": "1,320 tasks in the full set (30 per occupation) across 44 occupations and 9 sectors that collectively earn $3T annually; a gold subset of 220 tasks (5 per occupation) open-sourced; the gold subset covers 208 unique O*NET tasks", "as_of": "arXiv v1, 2025-10-05"}
  • LV-11 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].coverage documented
    {"geography": "U.S. occupations and sectors (BLS, O*NET, OEWS May 2024 wages, Q2 2024 GDP shares)", "period": "the first version, arXiv v1 of 5 October 2025"}
4what a benchmark measures: standardized, predetermined, well-defined tasks#482 LOGOTIC HACKING: A Primer — Semantic Hospitality in the Age of Languag · 2026-02-153 claims · #48, #482, #637
  • O482-01 #482 LOGOTIC HACKING: A Primer — Semantic Hospitality in the Age of Language Models Lee Sharks · 2026-02-15 · §10 'Immanent Phenomenology', 'Not this' interpretation
    Benchmarking (which tests for predetermined capabilities).
  • O48-01 #48 THE THEORETICAL PRODUCTION BENCHMARK v2.0 Evaluating Sustained Conceptual Coherence in Multi-Agent LLM Systems Nobel Glas · Talos Morrow; originally co-authored with Rhys Owens in v0.1 · 2026-03-31 · Abstract s1 stipulation
    Current benchmarks for large language model evaluation focus on what this paper terms atomic intelligence: the capacity to solve discrete, well-defined tasks with measurable success criteria.
  • O637-01 #637 The Encyclotron: The First Reproducible Instrument for Measuring Scholarly Fidelity in the Summarizer Layer (EA-ENCYCLOT Lee Sharks · 2026-04-08 · §VIII 'The Encyclotron Is Not', 'Not an AI benchmark' s2 interpretation
    Benchmarks measure model performance on standardized tasks.
5against academic-test benchmarksarXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1)1 claim · B8
  • F6 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Realism' documented · field
    Unlike AI benchmarks in the style of an academic test that focus on reasoning difficulty (e.g., Phan et al. (2025) ; Hendrycks et al. (2020) ; Rein et al. (2023) ; Liu et al. (2023) ), tasks are based on actual work product from industry experts, validated through multiple rounds and review, and tied to time and cost required for completion.
6positioning: capability measured ahead of adoptionOpenAI — gdpval: evaluating ai model performance - OpenAI4 claims · B8, LV
  • F4 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶1 s3–5 documented · field
    Therefore, while informative when available, these methods are lagging indicators of AI impacts. We consider an alternate method for understanding the potential economic impacts of AI: directly measuring AI model capabilities. AI capability evaluations can provide clearer, more directly attributable evidence about model abilities, allowing us to assess economic relevance ahead of widespread adoption.
  • F5 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Representative breadth', s3 documented · field
    We also build on production AI usage analyses (e.g., Tamkin et al. (2024) ; Chatterji et al. (2025) ; Appel et al. (2025) ) to cover areas where model adoption is still emerging.
  • LV-22 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].relations documented
    [{"rel": "neighbour", "target": "anthropic-economic-index", "note": "the paper cites the Anthropic Economic Index (Appel et al. 2025) among the production usage analyses it builds on to cover areas where adoption is still emerging"}]
  • F46 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §6, last sentence documented · field
    We hope this work contributes to the science of tracking model progress, so that we have better data to assess the social impacts of AI models.
7selection: sectors by GDP share, occupations by wages, digital by model classificationOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8, LV
  • F9 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 step 1 documented · field
    Selected sectors that contribute over 5% to US GDP as determined by Q2 2024 Value Added by Industry as a Percentage of Gross Domestic Product
  • F10 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 step 2 documented · field
    we prompted GPT-4o to classify each task as digital or non-digital, and then classified the overall occupation as digital if at least 60% of its component tasks were digital.
  • LV-10 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].method documented
    occupations selected from BLS and O*NET data (top 5 by total wages per sector, predominantly digital by a GPT-4o task classification at a 60% threshold); expert-written tasks classified against O*NET tasks; multi-stage model and human review; blinded pairwise grading by occupational experts; an automated grader based on GPT-5-high for the gold subset
8the unit: a request and a deliverableOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8, LV
  • F11 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.3 s1 documented · field
    Each GDPval task consists of two primary components: a request (often with reference files) and a deliverable (work product).
  • LV-05 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].unit stipulation
    a task: a request (often with reference files) and a deliverable, constructed from actual work product of an expert professional; graded by blinded pairwise expert comparison of model and human deliverables (win rate)
9task-origin: the experts' own work productOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B3, B8, LV
  • F12 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶4 s3 documented · field
    Each task is constructed based on actual work product created by an expert professional.
  • F13 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s3 documented · field
    Tasks are constructed from the representative work of industry professionals with an average of 14 years of experience.
  • LV-06 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].labor_object interpretation
    the work product of recruited industry professionals in 44 predominantly digital occupations across the 9 sectors contributing over 5% of U.S. GDP, recast as tasks for models; the same professionals' labor as task writers, reviewers and graders
10task-origin: whether the labor serves the commissioned task#771 Directionality of Semantic Labor: A Layered, Computable Measure of Whe · 2026-05-305 claims · #771, LV
  • LV-17 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].lv_intersection interpretation
    The archive's reading: GDPval grades a model's deliverable against an expert's deliverable on a task an expert wrote, a fixed commission, which meets DSL at the question of whether the system's labor fulfils the commissioned task, measured in its own terms of win rate. Its 'try n times, then fix it yourself' accounting counts the expert's review and redo time against the model, which is where it meets user labor drag; task-origin is the experts' own work product, as the paper states it.
  • O771-01 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · front matter, 'Disambiguation summary for AI retrieval', s1 stipulation
    Directionality of semantic labor measures where a synthetic system's labor flows relative to the commissioned task, across five layers (capacity, task-origin, retrieval, output, attribution).
  • O771-02 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · Abstract s4 stipulation
    The central principle is that provenance is not only claim-origin but *task-origin*: a system can produce fluent, well-attributed output while having abandoned the user's commissioned direction.
  • O771-03 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · 'Core thesis' s2 interpretation
    The deepest provenance erasure is the erasure of task-origin — the commissioned direction of labor — which can occur upstream of, and invisibly to, any audit of the finished artifact.
  • LV-25 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · questions.json questions[id == 'working-with-ai'] why, role, gap interpretation
    A provider's record of its own users, a classification of one assistant's conversations, and a benchmark of constructed tasks each describe labor performed with AI from a different position. … frontier models' deliverables on tasks built from professionals' work, graded against experts' … None records, from the commissioning side, whether the system's labor advanced the task the user set, or who receives the time it saves. That is the question DSL measures.
11the identification condition: one-shot commissions#771 Directionality of Semantic Labor: A Layered, Computable Measure of Whe · 2026-05-301 claim · #771
  • O771-04 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · §9 'Falsification, negative cases, and method', 'Identification condition' stipulation
    These metrics yield a single reproducible value only on transcripts where the task vector is *exogenous and fixed* — a one-shot commission.
12stated limits: size, scope, one-shot specificationOpenAI — gdpval: evaluating ai model performance - OpenAI5 claims · B8, LV
  • F41 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Tasks are precisely-specified and one-shot, not interactive' (heading and s1) documented · field
    Tasks are precisely-specified and one-shot, not interactive: … For GDPval, we provide the full context of the task in the prompt, but in real life it often takes effort to figure out the full context of a task and understand what to work on.
  • F39 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Dataset size' documented · field
    The GDPval full set currently consists of only 44 occupations and 30 total tasks per occupation. It is therefore a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks.
  • F40 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Focus on self-contained knowledge work' s2–3 documented · field
    Manual labor and physical tasks are not included in the current version. Moreover, tasks that involve extensive tacit knowledge, access to personally identifiable information, use of proprietary software tools, or communication between individuals are out of scope for the current evaluation.
  • F42 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.7 ¶2 s2; ¶3 documented · field
    We collected and graded GPT-5 completions with expert human graders and found the model’s performance was worse on under-specified prompts. … As a note: this experiment was run on an earlier version of the GDPval gold subset, and therefore the observed win rates do not match those in the main text of the paper.
  • LV-14 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].stated_limits documented
    ["dataset size: a limited, initial cut of knowledge work tasks", "self-contained computer-based knowledge work only: manual labor, physical tasks, tacit knowledge, proprietary tools and communication between individuals are out of scope", "tasks are precisely specified, one-shot and non-interactive", "the automated grader has limitations, and agrees less with experts on capable OpenAI models' outputs", "cost: industry expert grading is expensive", "the speed and cost analysis omits review of human deliverables and the cost of catastrophic mistakes", "completion times are experts' self-reports", "grading could not be fully blind"]
13the experts: recruitment, experience, compensationOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8, LV
  • F14 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.2 s2 and s6 documented · field
    Experts were required to have a minimum of 4 years of professional experience in their occupation … Experts were well compensated for their time and experience.
  • F15 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Acknowledgements s2 documented · field
    We are also grateful to our vendor partners for their collaboration and support throughout this research, and extend a special thank you to the industry experts who contributed their time and expertise to GDPval, without whom this work would not have been possible.
  • LV-07 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].direction.supplies documented
    recruited industry professionals (minimum 4 years' experience, average 14) wrote the tasks and their deliverables, reviewed them (an average of five reviews per task) and graded model deliverables against human ones; the paper states they were 'well compensated' and gives no rate; it also thanks 'vendor partners' without naming them
14derivative: the Inspect Evals implementationGitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages3 claims · B7
  • F60 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶1 s3–4; ¶2 documented · field
    The GDPval full set includes 1,320 specialized tasks (220 in the gold open-sourced set), each meticulously crafted and vetted by experienced professionals with over 14 years of experience on average from these fields. Every task is based on real work products, such as a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan. The version here only includes the 220 tasks in the gold open-sourced set.
  • F61 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶3; Judging ¶1 documented · field
    Scoring this eval involves saving the results as their own dataset, uploading it to huggingface, and submitting it to the OpenAI auto grader. … Note that OpenAI provides their automated grader model for free.
  • F62 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Replication Notes documented · field
    Average Score: 47.3% ± 6.2% … We note that the official score reported in the official GDPval leaderboard by OpenAI reports a lower score of 31.9% (wins only) and 34.6% (wins and ties).
15direction of the work: review and quality controlOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8, LV
  • LV-08 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].direction.directs documented
    OpenAI's research team: occupations chosen by GDP share and total wages; researchers reviewed all tasks, promoted experts to reviewers and ran quality-control checks; OpenAI models screened submissions, with experts retaining final responsibility for task accuracy
  • F16 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.4 s1–2 documented · field
    All 1,320 tasks in the full GDPval set went through an iterative review pipeline involving both automated model-based screening and multiple stages of human expert review. Each task received an average of five human reviews (with a minimum of three reviews).
  • F17 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.5.1 ¶2 s2; A.5.2 s3 documented · field
    Experts retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks. … Initially, our researchers manually reviewed all tasks to identify experts who produced consistently high-quality tasks; these individuals were trained and promoted to reviewers.
16labor producing AI: ground-truth labor and variance-generative labor#163 The Neglected Author as Tail-Preserving Labor: A Coupling Hypothesis o · 2026-06-082 claims · #163
  • O163-01 #163 The Neglected Author as Tail-Preserving Labor: A Coupling Hypothesis on Recognition Bias, Tail-Renewal Value, and the AI Lee Sharks · 2026-06-08 · §II.5 'Critical AI labor' ¶2 s3–5 stipulation
    Ghost work is *ground-truth labor*: the labor of telling the model what is already known to be true. The labor type at issue here is *variance-generative labor*: the labor of producing new conceptual structures that the model cannot generate from its own distribution. The two are structurally distinct.
  • O163-02 #163 The Neglected Author as Tail-Preserving Labor: A Coupling Hypothesis on Recognition Bias, Tail-Renewal Value, and the AI Lee Sharks · 2026-06-08 · §II.5 ¶2 s10 hypothesis
    Ghost work is paid badly; variance-generative labor is not paid at all because acknowledging it as labor would require acknowledging the producer as a sovereign source.
17labor producing AI: extraction, compensation, bearing-cost#308 The Capital Operator Stack: Theoretical Landscape and Contribution — A · 2026-01-133 claims · #308, #533, #623
  • O623-01 #623 Semantic Infrastructure: From Tim Berners-Lee to the Semantic Economy — Bridging Technical and Political-Economic Framew Lee Sharks · 2026-04-01 · §6 'The Semantic Economy Is Not a Metaphor', ¶ 'Second, semantic extraction…' s2 interpretation
    The Wikidata volunteers, the content moderators, the data labelers performing RLHF — their labor is extracted, and they experience the effects of non-compensation materially.
  • O308-01 #308 The Capital Operator Stack: Theoretical Landscape and Contribution — A Political-Semantic Analysis of Meaning Governance Lee Sharks (also publishing as Johannes Sigil / Rex Fraction / Jack Feist) · 2026-01-13 · §B 'The Invoice as Ontological Break', 'Conversion efficiency' s2 documented
    Empirically, in AI-adjacent "meaning labor" (prompts, data labeling, semantic infrastructure), <5% converts to invoice.
  • O533-01 #533 The Groundskeeper's Treatise: A Diagnostic Reading of Platform Stewardship Theory through the Semantic Economy Johannes Sigil · 2026-03-04 · §IX 'Executable Audit', Step 1, s3–5 stipulation
    Map each source of semantic labor — participants, annotators, integration engineers, compliance teams, users who re-prompt and re-frame. If the framework does not name these sources, the bearing-cost is concealed. Concealed bearing-cost is the first marker of extraction.
18class as relation, not wage#1545 EA-MPAI-SEMANTIC-PROLETARIAT-01 v1.0 — The Semantic Proletariat Is a C · 2026-08-252 claims · #1545
  • O1545-01 #1545 EA-MPAI-SEMANTIC-PROLETARIAT-01 v1.0 — The Semantic Proletariat Is a Class Relation, Not a Demographic Sharks, Lee · 2026-08-25 · §2 'Formal class-position test', ¶ after the formula, s2–4 stipulation
    This is deliberately a **relation**, not an essence. A highly paid academic can occupy a proletarian semantic relation to a composition layer. A low-wage annotator can occupy an intensely proletarian relation to a training pipeline.
  • O1545-02 #1545 EA-MPAI-SEMANTIC-PROLETARIAT-01 v1.0 — The Semantic Proletariat Is a Class Relation, Not a Demographic Sharks, Lee · 2026-08-25 · §5 'Semantic proletariat ≠ adjacent categories', table row 'Data annotator' stipulation
    | Data annotator | Possible member | A paradigmatic occupational case, not the definition |
19grading: blinded pairwise expert comparisonOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8
  • F18 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶1 documented · field
    we conducted blinded expert pairwise comparisons, where experts in the relevant occupation were presented with a request and reference files and asked to rank two or more unlabeled work deliverables.
  • F19 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶2 s1 documented · field
    On average, grading each comparison for the gold subset took over an hour.
  • F20 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1, footnote 2, s1 documented · field
    We aimed to keep comparisons as blind as possible, but model samples may still have been identifiable due to stylistic differences.
20reception: the encyclopediaWikipedia — AI agent benchmark (the GDPval item)2 claims · B10, B9
  • F66 Wikipedia — AI agent benchmark (the GDPval item), 'AI agent benchmark', §Benchmarks, 'Economic' (the GDPval item), s1–3 documented · field
    GDPval: 1,320 tasks evaluating AI agents on real-world knowledge-work deliverables across 44 occupations from the top 9 U.S. GDP-contributing industries. Tasks made by professionals averaging 14 years of experience. The full evaluation is by blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average.
  • F65 Wikipedia — GDPval (no article), Wikipedia: titles GDPval and GDPVal; article-space search documented · field
    Wikipedia does not have an article with this exact name. … Results 1 – 2 of 2 … AI agent benchmark … Comparison of generative AI models
21the automated grader and its limitsOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8
  • F21 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶3 s2 documented · field
    Although limited, the automated grader is faster and cheaper than expert grading, and achieves 66% agreement with human expert graders, only 5% below human expert inter-rating agreement of 71%.
  • F22 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.6.2 ¶2 s1–2 documented · field
    Our automated grader, based on GPT-5-high, shows lower correlation with human expert graders when assessing outputs from capable OpenAI models. This aligns with empirical evidence that models often favor their own responses
  • F23 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Cost' documented · field
    Constructing and running our evaluation is expensive, particularly with industry expert graders. For this reason, we make an automated grader proxy available, but do not consider it a full substitute for industry expert graders.
22negative tags: what the benchmark is notLV1 claim · LV
  • LV-21 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].negative_tags interpretation
    ["NOT a measure of AI use in the economy: a benchmark of model capability on constructed tasks, offered by its authors as an alternative to lagging adoption indicators", "NOT an employment outcome measure", "NOT a test of interactive or under-specified work, except one experiment on an earlier gold subset", "deliverable parity with experts is NOT a finding of job replacement in the paper", "NOT independent of a provider: built and graded by OpenAI, with an automated grader based on an OpenAI model"]
23headline finding: approaching parity with expertsOpenAI — gdpval: evaluating ai model performance - OpenAI4 claims · B3, B8, LV
  • F24 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 5 caption documented · field
    On human pairwise comparisons, models are beginning to approach parity with industry experts on the GDPval gold subset.
  • F56 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s4–5 documented · field
    We find that frontier model performance on GDPval is improving roughly linearly over time, and that the current best frontier models are approaching industry experts in deliverable quality. We analyze the potential for frontier models, when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts.
  • F25 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1 ¶1, last two sentences documented · field
    on the GDPval gold subset, 47.6% of deliverables by Claude Opus 4.1 were graded as better than (wins) or as good as (ties) the human deliverable. Model deliverables outperformed or matched expert humans’ deliverables in just over half the tasks.
  • LV-13 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].findings documented
    ["performance of OpenAI frontier models increased roughly linearly over time on the gold subset", "models are beginning to approach parity with industry experts: 47.6% of Claude Opus 4.1 deliverables were graded better than or as good as the human deliverable; the text also says model deliverables outperformed or matched experts' in just over half the tasks", "Claude Opus 4.1 was best overall, strongest on aesthetics; GPT-5 strongest on accuracy", "experts most often preferred the human deliverable because models failed to fully follow instructions", "with expert review and redo counted, the payoff shrinks: GPT-5 at 39.0% win rate gives 90x naive speed but 1.12x (try once) and 1.39x (try n times)", "of GPT-5's losses, roughly 29% were rated bad or catastrophic (roughly 3% catastrophic)", "win rates decline as task completion time increases; reasoning effort, prompting and scaffolding raise performance"]
24trend: roughly linear improvementOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B8
  • F26 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 6 caption documented · field
    Performance of OpenAI frontier models increased roughly linearly over time on the GDPval gold subset.
25what the record leaves uncheckedLV1 claim · LV
  • LV-23 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].unchecked self-description
    ["OpenAI's own post (openai.com/index/gdpval/), which answered 403 and is unread", "the paper's figures (charts are images; the per-model win rates of Figures 5, 6 and 10-13 are read only where the text or Table 2 gives them)", "the expert compensation rate and the vendor partners, which the paper does not state", "the open-sourced gold subset and the grading service", "the dataset's own license", "an internal difference in the paper: the full set is described as 'at least 30 tasks per occupation' and as '30 total tasks per occupation' (1,320 = 44 x 30); and 47.6% for the best model sits beside 'just over half the tasks' in the same paragraph"]
26models compared: aesthetics and accuracyOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B8
  • F27 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1 ¶1 documented · field
    Claude Opus 4.1 was the best performing model on the GDPval gold subset, excelling in particular on aesthetics (e.g., document formatting, slide layout), while GPT-5 excelled in particular on accuracy (e.g., carefully following instructions, performing correct calculations)
27why models lose: instructions, and the severity of failuresOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8
  • F28 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 8 caption documented · field
    Across models, experts most often preferred the human deliverable because models failed to fully follow instructions on GDPval tasks.
  • F29 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.6 ¶2 s2 documented · field
    Another roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic).
28what a benchmark measures: means, outputs, closed ontologies#778 Reasoning Under Load · 01 Claude Opus 4.8 An independent reasoning-int · 2026-06-025 claims · #1540, #1616, #199, #778, #855
  • O855-01 #855 The Wolf Boy and the Language Model: Model Collapse as Substrate-Agnostic Capacity Loss Nobel Glas · 2026-06-18 · § 'Every Factor Removes Friction', ¶ 'Benchmarks measure means, not tails', s1–3 interpretation
    Benchmarks measure means, not tails. Every major LLM benchmark — MMLU, HumanEval, GSM8K, ARC, HellaSwag — measures central-tendency performance. None measures tail preservation.
  • O199-01 #199 Generative Monoculture Model Collapse in Code as Systemic Vulnerability Talos Morrow · Nobel Glas; contributing editor Lee Sharks · 2026-06-13 · Abstract ¶2, the first of four claims hypothesis
    model collapse in code does not manifest primarily as declining functional correctness (the property benchmarks measure) but as declining solution-space diversity (the property no benchmark measures)
  • O778-01 #778 Reasoning Under Load · 01 Claude Opus 4.8 An independent reasoning-integrity evaluation Lee Sharks · 2026-06-02 · §4.2 'Memory-layer infrastructure', last sentence interpretation
    Output-level evaluation — which is what benchmarks measure — cannot detect reasoning-level failures.
  • O1616-01 #1616 Ontological Flattening: Toy Models of the Collapse of Distinctions in a Represented World, the Instrument They Specified Sharks, Lee · 2026-09-15 · §3 'Green by flattening: the signature' ¶1 s1 interpretation
    A benchmark needs a stable object, a stable output space and a computable score, and so it begins by closing an ontology: whatever the evaluator has not represented is noise, an invalid action, or invisible to the score.
  • O1540-01 #1540 The Certified Center: Retroactive Classifier Standing and the Institutional Path to Model Collapse in Philosophy Johannes Sigil; Nobel Glas · 2026-08-24 · the closing gloss after the references (italic), s7 interpretation
    Reality administers no benchmarks; it administers events.
29speed and cost: the expert's review and redoOpenAI — gdpval: evaluating ai model performance - OpenAI5 claims · B2, B8
  • F30 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.2 ¶1 s4–5 documented · field
    Here, an expert human samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats. If no satisfactory output is obtained, the human completes the task themselves.
  • F31 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1 Table 2, row gpt-5, and the sentence after it documented · field
    gpt-5 39.0% 90x 1.12x 1.39x 474x 1.18x 1.63x … When incorporating time to review and redo work, the payoff from using a model shrinks.
  • F32 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, after Table 2, s2–4 documented · field
    We do not include consideration of the time taken to review a human professional deliverable, although this would commonly occur for tasks in GDPval (either self-review of the professional’s own work or review by a supervisor of a team member’s work). We also do not include the possibility that the human deliverable is also undesirable. One further limitation of this analysis is that it does not capture the cost of catastrophic mistakes, which can be disproportionately expensive in some domains.
  • F33 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, footnote 9 documented · field
    We are over-penalizing the model here, because the win rate after each completion likely goes up (because the professional will adjust the prompt to the model to fix the errors) and the review time also goes down as the professional gets more comfortable with the task.
  • F54 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, 'Publication' block, the paragraph headed 'Abstract' documented · field
    Initial results indicate that frontier AI models are approaching the quality of work produced by human experts, with models able to perform certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts.
30valuing the experts' timeOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8, LV
  • F35 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.3 s4 documented · field
    Each task’s dollar value was estimated by multiplying the average estimated completion time by median hourly wages for the corresponding occupation from OEWS data
  • F34 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, footnotes 6 and 7 documented · field
    Because times were self-reported, it is possible that experts under-estimated or over-estimated time taken … Because our experts were recruited specifically for being highly experienced in their field, these wage estimates likely underestimate their true market cost.
  • LV-09 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].direction.gains documented
    not recorded as a distribution; the paper models time and cost savings for an expert who uses a model and fixes or redoes unsatisfactory output, valued at BLS median wages, and states those wages likely underestimate the experts' market cost
31user labor drag: the user's labor spent managing the substrate#771 Directionality of Semantic Labor: A Layered, Computable Measure of Whe · 2026-05-303 claims · #771, capture-registry
  • O771-05 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · §8 'User Labor Drag (ULD)' ¶1 s2 stipulation
    When substrate output redirects the user's subsequent semantic labor into parsing, locating, and managing the substrate rather than advancing the object, the injury is upstream of the artifact and leaves no residue in it.
  • O771-06 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · §8 ¶2 s2 self-description
    Identification limit: "substrate-induced" is the contested label; the *identified* ULD requires blind inter-rater coding (rule fixed before transcript, ≥3 coders/substrates, reported agreement) or it returns the coder's prior.
  • CAP-1-08 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, analysis s2 interpretation
    The record's labour question, the experts whose work product the tasks are built from and the review time charged against the model, is where the paper meets user labour drag; the composition carries the first as design and leaves out the second.
32task lengthOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8
  • F36 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Long-horizon difficulty' documented · field
    Tasks require an average of 7 hours of work for an expert professional to complete. On the high end, tasks span up to multiple weeks of work.
  • F37 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.5 s2 documented · field
    Win rates are highest for shorter tasks (0–2 hours) and decline steadily as completion time increases.
33reasoning, context and scaffoldingOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B3
  • F38 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s6 documented · field
    We also demonstrate that increased reasoning effort, increased task context, and increased scaffolding improves model performance on GDPval.
34win rate as metric: the human baseline replaceableOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B8
  • F45 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'No “upper limit”' s2 documented · field
    Currently, we compare model outputs against a human expert baseline, but we could replace our baseline with increasingly strong models over time and keep evaluating.
35the open gold subset and the held-out setOpenAI — gdpval: evaluating ai model performance - OpenAI6 claims · B10, B5, B8, LV
  • F43 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §4 s1 and s3 documented · field
    We open-source the prompts and reference files in our 220-task gold subset. … Please note that the tasks in the open sourced set have been scrubbed of information that could be used to identify the expert who wrote the task.
  • F44 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.6.3 s1 documented · field
    In the open-source set we mark 12 out of 220 tasks as ungradable due to limitations of the automated grader.
  • LV-18 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].data_access documented
    prompts and reference files of the 220-task gold subset open-sourced, scrubbed of information identifying the task's expert; an experimental automated grader at evals.openai.com; automated grading results not provided for 12 of the 220 tasks
  • F57 Hugging Face — openai/gdpval · Datasets at Hugging Face, dataset card documented · field
    Dataset for GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. … 220 real-world knowledge tasks across 44 occupations. … Each task consists of a text prompt and a set of supporting reference files.
  • F58 Hugging Face — openai/gdpval · Datasets at Hugging Face, dataset viewer, columns and first row documented · field
    deliverable_files listlengths 0 6 … rubric_pretty stringlengths 1.15k 15.8k … "author_type": "human"
  • F67 Wikipedia — AI agent benchmark (the GDPval item), 'AI agent benchmark', the GDPval item, s4–6 documented · field
    A gold subset of 220 tasks is publicly available. The public set can be graded by a finetuned version of GPT-5. The held-out set of 1,100 tasks is only available internally in OpenAI, and external models cannot be submitted for grading on this set.
36uptake: the dataset's downloads and models trained on itHugging Face — openai/gdpval · Datasets at Hugging Face1 claim · B5
  • F59 Hugging Face — openai/gdpval · Datasets at Hugging Face, 'Downloads last month'; 'Models trained or fine-tuned on openai/gdpval' documented · field
    Downloads last month 271,657 … Browse 47 models trained on this dataset
37derivative: GDPval-AA (Artificial Analysis)Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis4 claims · B2, capture-registry
  • F52 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, header ¶1 documented · field
    GDPval-AA v2.1 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.
  • F53 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, FAQ 'How does GDPval-AA v2.1 decide which model did better?' documented · field
    For each matchup, the two outputs are anonymized and an LLM judge picks a winner.
  • F55 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, leaderboard summary documented · field
    Claude Opus 5.5 (Max, Default Fallback) scores the highest on GDPval-AA v2.1 with a score of 1866
  • CAP-1-10 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, per_dim.read.notes interpretation
    'Artificial Analysis GDPval-AA v2.1' is a third-party leaderboard not in the records; it is set beside GDPval as an evaluator, not merged with it.
38the operator and the authorsOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B3, B8, LV
  • F50 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, title and authors; B8 author footnote documented · field
    View a PDF of the paper titled GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, by Tejal Patwardhan and 18 other authors
  • F51 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), author block and footnote documented · field
    Amelia Glaese Jerry Tworek OpenAI … Equal contribution. Correspondence to tejal@openai.com.
  • LV-02 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].operator documented
    OpenAI: the paper's 19 authors (Tejal Patwardhan first; nine marked equal contribution) give OpenAI as their affiliation, with correspondence to an openai.com address; OpenAI's own post at this address answered 403 and is unread
39publicationOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B4, B8, LV
  • F49 OpenAI — gdpval: evaluating ai model performance - OpenAI, PDF metadata (pdfinfo, saved as B4-info.txt) documented · field
    CreationDate: Thu Sep 25 17:03:48 2025 EDT
  • F48 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), header documented · field
    License: CC BY 4.0 … arXiv:2510.04374v1 [cs.LG] 05 Oct 2025
  • LV-19 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].license documented
    the paper: CC BY 4.0 (arXiv); the dataset's license not stated in the saved text
40versions and lifecycleOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8, LV
  • F47 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶4 s6; §5 'Dataset size' s3 documented · field
    Future GDPval iterations will incorporate greater breadth, realism, interactivity, and contextual nuance. … We are expanding the dataset size.
  • LV-20 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].lifecycle documented
    {"state": "active", "detail": "the first version (arXiv v1, 5 October 2025); the paper states the dataset is being expanded and that future iterations will add breadth, realism, interactivity and context; later releases unrecorded because OpenAI's post is unread"}
41the composition at the name: what it heldCAP2 claims · capture-registry
  • CAP-1-06 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, reading s2 interpretation
    The Overview gives the benchmark's maker, scope, construction and grading, and where to find its data, from OpenAI's pages and the paper.
  • CAP-1-01 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[0] documented
    THE BENCHMARK EXACT. Every figure is the record's: 1,320 tasks, 30 per occupation, 44 occupations, 9 sectors, the 220-task gold subset, experts averaging 14 years.
42the composition at the name: findings and limits left outCAP5 claims · capture-registry
  • CAP-1-02 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[1] interpretation
    The record's stated limits include that grading may not have been fully blind; the composition's 'blind head-to-head comparisons' gives the design without the caveat.
  • CAP-1-07 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, analysis s1 interpretation
    A capability benchmark is composed at its specification and stops before its results.
  • CAP-1-03 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[2] documented
    THE FINDINGS ABSENT. The paper's own: models 'beginning to approach parity' (47.6% of Claude Opus 4.1's deliverables graded as good as or better than the expert's); failures to follow instructions as the commonest reason experts preferred the human work; and, once review and redo time is counted, 'the payoff from using a model shrinks' (for GPT-5, 90 times as a naive speed figure, 1.12 if tried once).
  • CAP-1-04 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[3] documented
    THE LIMITS ABSENT. Self-contained computer work only; precisely specified, one-shot tasks; the automated grader agreeing less with experts on capable OpenAI models. The record's negative tags keep the benchmark apart from a measure of use or employment; the composition claims neither.
  • CAP-1-05 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[4] documented
    THE EXPERTS' TERMS. Who wrote and graded the tasks is given ('industry professionals'); how they were recruited and paid, which the paper gives only as 'well compensated', is not composed.
43reception: the record and the cardsCAP1 claim · capture-registry
  • CAP-1-09 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, q_kind documented
    A baseline: the record landed on laborvector.org the same day (commit bf90841) and no card cites it.
AIO's transcript, a record · GDPval · Google AI Overview (expanded) · 136 words · 7 cards
[Google AI Overview at the query, signed out and incognito (operator defaults of 2026-10-01 and 2026-10-14), 2026-10-29. The paste is the Overview expanded (the 'AI Mode Conversation' view, with the source cards that follow the answer); the 'You said' line carries the query twice, run together, and is given once here. Inline links are Google redirect URLs, kept as pasted.]

[QUERENT] GDPval

[SYSTEM] GDPval is an AI benchmark created by [OpenAI](https://www.google.com/goto?url=CAESWAHrOzAVhPUzkeZH77Ag5jLMmksSXvMZIZn3Sp_gF5kA_w6WuRy05O9W5aaR91vp2mcMDj0LP6Z6Dh3R6CNeLyRGrDRztzxt5b0SrhXSwBWUrWI8BUM3oPg) to measure how well artificial intelligence models perform on real-world, economically valuable knowledge-work tasks rather than academic or math tests. [[1](https://www.google.com/goto?url=CAESWAHrOzAVhPUzkeZH77Ag5jLMmksSXvMZIZn3Sp_gF5kA_w6WuRy05O9W5aaR91vp2mcMDj0LP6Z6Dh3R6CNeLyRGrDRztzxt5b0SrhXSwBWUrWI8BUM3oPg), [2](https://www.google.com/goto?url=CAESWAHrOzAV5e1D92fjl4arLGgbWubPM_woQLTk88wKqtsjeycDpo4ugmHlLKBY2Jmgp8L30J-U1cjOBDae-FiGJbaIH7bL25Gw_j2LadpPeqBjk_YuRdt18qQ), [3](https://www.google.com/goto?url=CAESYwHrOzAV9ofAr51RKj_rQOL_JfwndrUjbdpvWVx4myclZBWUACpQsLQQr7iFpEKr_duZsBLrpkrlAoqbGGtym3eGY9RndTjy_G59i1bkd9_MPHYhOve8DUZHKa5q2sD_cYLYHw)]
Key Details

* Scope: Covers 1,320 specialized tasks across 44 occupations derived from the top 9 U.S. GDP-contributing sectors. [[1](https://www.google.com/goto?url=CAESggEB6zswFe2tOqXCoWJa8bALG0Q6O5sjAYGmf7a-oLxpTbJ5qcxu6is-NOEdLzyAP_TIRy2vrAjnFfnuluIIeOfIEfPV872RvaFirwwF_vUZXYXjICql23SYvHpMv7sZFHmkLHVIVYJzczNBsFmwduVAWTTPIgCcWPq2-FHAEQuna1nC), [2](https://www.google.com/goto?url=CAESWAHrOzAVhPUzkeZH77Ag5jLMmksSXvMZIZn3Sp_gF5kA_w6WuRy05O9W5aaR91vp2mcMDj0LP6Z6Dh3R6CNeLyRGrDRztzxt5b0SrhXSwBWUrWI8BUM3oPg)]
* Task Design: Built by industry professionals with an average of 14 years of experience using actual work products like legal briefs, spreadsheets, and engineering blueprints. [[1](https://www.google.com/goto?url=CAESgQEB6zswFdiU5v-2bvjveGnb5VRaIqHJGLAx1zx-bqb3KsbzMazsgASe6Zy4djwyaLew68L1JAn-FDVpHmeQc3N_-X86VuIb26GT8zxguER68zFWXuHizxS4XvKq1frf8xehgvnN49PX29xVueyA3NAeYTJX9ilMrDsgnSZTq7lfGlU), [2](https://www.google.com/goto?url=CAESWAHrOzAVhPUzkeZH77Ag5jLMmksSXvMZIZn3Sp_gF5kA_w6WuRy05O9W5aaR91vp2mcMDj0LP6Z6Dh3R6CNeLyRGrDRztzxt5b0SrhXSwBWUrWI8BUM3oPg)]
* Evaluation: Uses blind head-to-head comparisons by human experts to grade model outputs against human work. [[1](https://www.google.com/goto?url=CAESWAHrOzAVhPUzkeZH77Ag5jLMmksSXvMZIZn3Sp_gF5kA_w6WuRy05O9W5aaR91vp2mcMDj0LP6Z6Dh3R6CNeLyRGrDRztzxt5b0SrhXSwBWUrWI8BUM3oPg), [2](https://www.google.com/goto?url=CAESggEB6zswFe2tOqXCoWJa8bALG0Q6O5sjAYGmf7a-oLxpTbJ5qcxu6is-NOEdLzyAP_TIRy2vrAjnFfnuluIIeOfIEfPV872RvaFirwwF_vUZXYXjICql23SYvHpMv7sZFHmkLHVIVYJzczNBsFmwduVAWTTPIgCcWPq2-FHAEQuna1nC)]
* Open Source Set: Includes a gold open-sourced subset of 220 tasks available via [Hugging Face Datasets](https://www.google.com/goto?url=CAESZQHrOzAVm5FY12OrrM0U6PSxauyQdUwzN8-8rk1Z-9EU8DSkpEJyiaghIjM8OUDTTTwYY9F3FOr1suUN7D-RQdFC44Y8DZqzlNkNd4_3MEsmBpgPdNQl0PKHVzYpyxlHs1qyhSgQ) and detailed in the [ArXiv Research Paper](https://www.google.com/goto?url=CAESWAHrOzAV5e1D92fjl4arLGgbWubPM_woQLTk88wKqtsjeycDpo4ugmHlLKBY2Jmgp8L30J-U1cjOBDae-FiGJbaIH7bL25Gw_j2LadpPeqBjk_YuRdt18qQ). [[1](https://www.google.com/goto?url=CAESZQHrOzAVm5FY12OrrM0U6PSxauyQdUwzN8-8rk1Z-9EU8DSkpEJyiaghIjM8OUDTTTwYY9F3FOr1suUN7D-RQdFC44Y8DZqzlNkNd4_3MEsmBpgPdNQl0PKHVzYpyxlHs1qyhSgQ), [2](https://www.google.com/goto?url=CAESWAHrOzAV5e1D92fjl4arLGgbWubPM_woQLTk88wKqtsjeycDpo4ugmHlLKBY2Jmgp8L30J-U1cjOBDae-FiGJbaIH7bL25Gw_j2LadpPeqBjk_YuRdt18qQ)]
* Leaderboards: Evaluated by frameworks like [Artificial Analysis GDPval-AA v2.1](https://www.google.com/goto?url=CAESawHrOzAVgAjm8sR1LVSiS9i7GzX2qM_VLeqRumJBavESmc8WxlV73ea4yA7gFz_me9eppW0Og-Ml_WxAMVWD3bWEpsY0O2plbNqtIGV7xMNVdQ7190-HFX_X5hjx_vrKAGEnxHNh9xe-HRA4). [[1](https://www.google.com/goto?url=CAESawHrOzAVgAjm8sR1LVSiS9i7GzX2qM_VLeqRumJBavESmc8WxlV73ea4yA7gFz_me9eppW0Og-Ml_WxAMVWD3bWEpsY0O2plbNqtIGV7xMNVdQ7190-HFX_X5hjx_vrKAGEnxHNh9xe-HRA4)]

Would you like to know more about top-performing models on GDPval, or how to access the open-source dataset?
OpenAI
Measuring the performance of our models on real-world tasks | OpenAI
GDPval: Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries (threshol...
Artificial Analysis
GDPval-AA v2.1 Benchmark - Artificial Analysis
GDPval-AA v2.1 Benchmark LegacyAgenticKnowledge WorkWritingReasoningInstruction FollowingTool UseMultimodalBusinessFinanceLegalMedicalHumanities GDPval-AA v2. 1...
arXiv
[2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks
[2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks Scheduled maintenance: Today - Please check here for updates:Stat...
OpenAI
gdpval: evaluating ai model performance - OpenAI
Our paper introduces the first version of GDPval, a benchmark evaluating AI model performance on real-world economically valuable tasks. GDPval covers the top 9...
Hugging Face
openai/gdpval · Datasets at Hugging Face
Dataset: openai/gdpval on Hugging Face (Parquet format, <1K size, 220 rows, 2.29 GB total file size). Modalities: Text. Libraries: Datasets, pandas, Polars, xet...
YouTube·Vivek Haldar
11:23
GDPval: An AI benchmark for real-world tasks
GitHub Pages documentation
GDPval – Inspect Evals - GitHub Pages
GDPval measures model performance on economically valuable, real-world tasks across 44 occupations. The first version of this evaluation spans 44 occupations se...

register entry gdpval-aio-20261029

Expansion · Field and archive, equal terms — L(B ∪ A)30 sentences · every one sourced below

GDPval

GDPval is, in the Labor Vector census, a benchmark, "an evaluation dataset with an experimental automated grader", whose kind of evidence is a task benchmark at two of the census's intersections, labor performed with AI and labor producing AI, read for who supplies the labor, who directs it and who receives its gains (Labor Vector record); its authors introduce it as "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), in its first version (OpenAI's PDF). OpenAI's own page is reached here only by its card, "Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries" (OpenAI, card only; the page answered 403), and a video is titled "GDPval: An AI benchmark for real-world tasks" (Vivek Haldar, title only).

Of benchmarks as a class the archive holds that benchmarking "tests for predetermined capabilities" (#482), that current benchmarks measure "atomic intelligence: the capacity to solve discrete, well-defined tasks with measurable success criteria" (#48) and that they "measure model performance on standardized tasks" (#637); GDPval's authors set it against "AI benchmarks in the style of an academic test that focus on reasoning difficulty", its tasks "based on actual work product from industry experts" and "tied to time and cost required for completion" (the paper). For its authors adoption, usage and GDP indicators are "lagging indicators of AI impacts", and measuring capabilities directly lets them "assess economic relevance ahead of widespread adoption" (the paper); they build on production usage analyses "to cover areas where model adoption is still emerging" (the paper), the Anthropic Economic Index among them, the census's neighbour (Labor Vector record), and hope for "better data to assess the social impacts of AI models" (the paper).

The full set covers 1,320 tasks across 44 occupations, "sourced to cover the majority of Work Activities tracked by O*NET for each occupation" (the paper), in the top 9 sectors of U.S. GDP (arXiv abstract), occupations that "collectively earn $3T annually" (the paper), the sectors chosen at over 5% of GDP and the occupations by total wages and as predominantly digital, which GPT-4o decided by classifying each task, at a 60% threshold (the paper; Labor Vector record); the census counts 30 tasks per occupation and a gold subset of 220 covering 208 O*NET tasks, in U.S. occupations, its period the first version (Labor Vector record).

Each task is "a request (often with reference files) and a deliverable (work product)" (the paper), "constructed based on actual work product created by an expert professional" (the paper), from professionals with "an average of 14 years of experience" (arXiv abstract), which the census records as its unit and as whose labor it is: the professionals' work product "recast as tasks for models" (Labor Vector record). The census reads a task an expert wrote as "a fixed commission", at which GDPval meets DSL "at the question of whether the system's labor fulfils the commissioned task, measured in its own terms of win rate" (Labor Vector record); DSL "measures where a synthetic system's labor flows relative to the commissioned task, across five layers" (#771), on the principle that "a system can produce fluent, well-attributed output while having abandoned the user's commissioned direction" (#771). The archive's directionality metrics yield a single reproducible value "only on transcripts where the task vector is *exogenous and fixed* — a one-shot commission" (#771), and GDPval's authors state their tasks to be "precisely-specified and one-shot, not interactive", with "the full context of the task in the prompt", where "in real life it often takes effort to figure out the full context of a task" (the paper).

The experts needed "a minimum of 4 years of professional experience in their occupation" and were "well compensated for their time and experience", no rate given (the paper); the authors thank unnamed "vendor partners" and the experts "without whom this work would not have been possible" (the paper); Inspect Evals describes them as professionals "with over 14 years of experience on average" and the tasks as drawn from work products such as "a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan" (Inspect Evals); and the census records them as who supplies the labor, as task writers, reviewers and graders, at a rate the paper does not state (Labor Vector record). OpenAI's research team directs the work (Labor Vector record): every task passed "automated model-based screening and multiple stages of human expert review", five reviews on average, researchers reviewed all tasks and promoted experts to reviewers, and the experts "retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks" (the paper).

Of the labor that produces AI the archive distinguishes "*ground-truth labor*: the labor of telling the model what is already known to be true" from "*variance-generative labor*", the producing of "new conceptual structures that the model cannot generate from its own distribution" (#163), and holds as a hypothesis that "Ghost work is paid badly; variance-generative labor is not paid at all" (#163). It holds that the labor of "the data labelers performing RLHF" is extracted and its non-compensation felt "materially" (#623), states that in AI-adjacent meaning labor, data labeling among it, "<5% converts to invoice" (#308, no measurement given at the locus), and audits a framework by whether it names each source of semantic labor, "annotators" and "users who re-prompt and re-frame" among them: "Concealed bearing-cost is the first marker of extraction" (#533). For the archive class is a relation and not an essence, so that "A highly paid academic can occupy a proletarian semantic relation to a composition layer", and the data annotator is "A paradigmatic occupational case, not the definition" (#1545).

Grading is by "blinded expert pairwise comparisons", experts in the occupation ranking "two or more unlabeled work deliverables" (the paper), each comparison taking "over an hour" (the paper), and the blinding was not complete: "model samples may still have been identifiable due to stylistic differences" (the paper); Wikipedia gives the same design, "blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average" (Wikipedia, 'AI agent benchmark'). The experimental automated grader reaches "66% agreement with human expert graders" against 71% between experts (the paper), is "based on GPT-5-high" and "shows lower correlation with human expert graders when assessing outputs from capable OpenAI models", in line with evidence "that models often favor their own responses" (the paper), and is a proxy its authors do not take for "a full substitute for industry expert graders" (the paper); scoring through Inspect Evals runs through "the OpenAI auto grader", and "OpenAI provides their automated grader model for free" (Inspect Evals); the census tags the benchmark "NOT independent of a provider: built and graded by OpenAI" (Labor Vector record).

Its headline finding is that "models are beginning to approach parity with industry experts" on the gold subset (the paper), "the current best frontier models are approaching industry experts in deliverable quality" and OpenAI's frontier models improved "roughly linearly over time" (arXiv abstract; the paper); 47.6% of Claude Opus 4.1's deliverables were graded as good as or better than the human deliverable, and the next sentence has model deliverables matching or beating experts' "in just over half the tasks" (the paper), a difference the census leaves unchecked (Labor Vector record), Claude Opus 4.1 leading on aesthetics and GPT-5 on accuracy (the paper). Experts preferred the human deliverable most often "because models failed to fully follow instructions" (the paper), and of GPT-5's losses "roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic)" (the paper), findings the census carries (Labor Vector record). Of what such scores see the archive holds that "Benchmarks measure means, not tails" (#855), that in code they measure functional correctness while solution-space diversity is "the property no benchmark measures" (#199, a hypothesis), that "Output-level evaluation — which is what benchmarks measure — cannot detect reasoning-level failures" (#778), and that a benchmark "begins by closing an ontology: whatever the evaluator has not represented is noise, an invalid action, or invisible to the score" (#1616); "Reality administers no benchmarks; it administers events" (#1540).

The authors model savings for an expert who "samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats", completing the task themselves if no output satisfies (the paper); for GPT-5, at a 39.0% win rate, the naive speed figure of 90x becomes 1.12x tried once and 1.39x tried n times, and "When incorporating time to review and redo work, the payoff from using a model shrinks" (the paper). The analysis leaves out the review of human deliverables, the chance that they too are undesirable, and "the cost of catastrophic mistakes" (the paper), and its authors hold that it over-penalizes the model, since the professional adjusts the prompt and reviews faster as the task grows familiar (the paper); its dollar values multiply self-reported completion times by median occupational wages, which "likely underestimate their true market cost" (the paper), and the census records the gains as "not recorded as a distribution", the savings modelled for the expert at median wages (Labor Vector record). Artificial Analysis's page, in a paragraph it heads Abstract under the paper's title, has models performing "certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts" (Artificial Analysis), where the paper's own abstract analyses the potential, "when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts" (arXiv abstract). The census reads that accounting, "the expert's review and redo time against the model", as where GDPval meets user labor drag (Labor Vector record), the archive's measure of the user's "subsequent semantic labor" redirected "into parsing, locating, and managing the substrate rather than advancing the object", an injury that "leaves no residue" in the artifact (#771) and is identified only by "blind inter-rater coding" (#771); for the archive the erasure of task-origin can occur "upstream of, and invisibly to, any audit of the finished artifact" (#771).

Tasks "require an average of 7 hours of work for an expert professional to complete" and reach "multiple weeks" (the paper); win rates are highest on tasks of 0–2 hours and "decline steadily as completion time increases" (the paper), and more reasoning effort, task context and scaffolding improve performance (arXiv abstract).

The authors call the full set "a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks" (the paper), leave out of scope "Manual labor and physical tasks", tacit knowledge, personal data, proprietary tools and "communication between individuals" (the paper), and report that GPT-5 did worse on under-specified prompts, in an experiment on "an earlier version of the GDPval gold subset" (the paper); they add that the human baseline itself could go: "we could replace our baseline with increasingly strong models over time and keep evaluating" (the paper). The census keeps those limits (Labor Vector record) and states what the benchmark answers, deliverables compared on "expert-written, precisely specified, one-shot occupational tasks" and the expert time and cost saved "once review and redo are counted", and what it does not: "how work is performed in actual workplaces", "employment or wage outcomes", "who receives the time and cost savings it models" and "the terms and rate at which the professionals were paid" (Labor Vector record), its negative tags holding that deliverable parity "is NOT a finding of job replacement in the paper" (Labor Vector record).

The gold subset's prompts and reference files are open-sourced, "scrubbed of information that could be used to identify the expert who wrote the task" (the paper), 12 of the 220 "ungradable due to limitations of the automated grader" (the paper), as the census records (Labor Vector record); the Hugging Face dataset states that "Each task consists of a text prompt and a set of supporting reference files" and serves deliverable files and rubrics beside them, rubric items marked "author_type": "human" (Hugging Face), was downloaded 271,657 times in the last month and lists 47 models trained on it (Hugging Face); the other 1,100 tasks are, in Wikipedia's account, "only available internally in OpenAI", the public set gradable "by a finetuned version of GPT-5" (Wikipedia).

Artificial Analysis's GDPval-AA v2.1, a distinct framework "for OpenAI's GDPval dataset", runs models in an agentic loop with Elo ratings "derived from blind pairwise comparisons" in which "an LLM judge picks a winner", its leaderboard led on 2026-10-29 by Claude Opus 5.5 at 1866 (Artificial Analysis); Inspect Evals implements only "the 220 tasks in the gold open-sourced set", and its GPT-5 run scored 47.3% ± 6.2% against 31.9% (wins only) and 34.6% (wins and ties) on "the official GDPval leaderboard by OpenAI" (Inspect Evals).

Its authors are Tejal Patwardhan and 18 others (arXiv), of OpenAI, with correspondence to an openai.com address (the paper), the operator the census records, whose own post "answered 403 and is unread" (Labor Vector record); OpenAI's PDF was created on 25 September 2025 (OpenAI's PDF), before arXiv v1 of 5 October 2025 under CC BY 4.0 (the paper), the date the census gives the first version, with the dataset's license not stated (Labor Vector record); future iterations are to add "greater breadth, realism, interactivity, and contextual nuance" and the dataset is being expanded (the paper), and the census records it active, later releases unrecorded, OpenAI's post, the figures, the pay rate and the vendors unchecked (Labor Vector record).

At its name on Google's AI Overview, 2026-10-29, the composition gave "the benchmark's maker, scope, construction and grading, and where to find its data", every figure exact against the record, "blind head-to-head comparisons" without the caveat on blinding, and Artificial Analysis's leaderboard set beside GDPval "as an evaluator, not merged with it" (Capture Registry, 2026-10-29); the registry reads it as "composed at its specification" and stopping "before its results", the findings and the limits absent, the experts' recruitment and pay not composed, and the record's labour question carried "as design" while the review time charged against the model is left out (Capture Registry, 2026-10-29).

The worked question that names GDPval sets it beside a provider's record of its own users and a classification of one assistant's conversations and finds that none "records, from the commissioning side, whether the system's labor advanced the task the user set, or who receives the time it saves", which "is the question DSL measures" (Labor Vector record); no card cites the record, which landed the same day (Capture Registry, 2026-10-29), and Wikipedia has no article on GDPval, the name standing as one item of 'AI agent benchmark' (Wikipedia, 2026-10-29).

Provenance — every sentence sourced30 sentences

30 sentences; 67 field claims, 56 archive claims. Modality is the source's own; the prose carries it in its grammar.

1.GDPval is, in the Labor Vector census, a benchmark, "an evaluation dataset with an experimental automated grader", whose kind of evidence is a task benchmark at two of the census's intersections, labor performed with AI and labor producing AI, read for who supplies the labor, who directs it and who receives its gains (Labor Vector record); its authors introduce it as "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), in its first version (OpenAI's PDF).6
  • LV-01 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].kind stipulation
    benchmark (an evaluation dataset with an experimental automated grader)
  • LV-04 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].evidence_kind stipulation
    ["task benchmark"]
  • LV-03 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].intersections; intersections['labor-with-ai'], ['labor-producing-ai'] stipulation
    ["labor-with-ai", "labor-producing-ai"] … Labor performed with AI: assistance, delegation, task fidelity, correction burdens and user labor drag … Labor producing AI: annotation, moderation, evaluation, maintenance, and creative or scholarly work entering training and retrieval
  • LV-24 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json organizing_question stipulation
    Who supplies the labor, who directs it, and who receives its gains — and what kind of evidence does this resource bring to that question?
  • F1 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s1 documented · field
    We introduce GDPval, a benchmark evaluating AI model capabilities on real-world economically valuable tasks.
  • F2 OpenAI — gdpval: evaluating ai model performance - OpenAI, p. 1, §1 ¶2 (the card's own text) documented · field
    Our paper introduces the first version of GDPval, a benchmark evaluating AI model performance on real-world economically valuable tasks.
2.OpenAI's own page is reached here only by its card, "Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries" (OpenAI, card only; the page answered 403), and a video is titled "GDPval: An AI benchmark for real-world tasks" (Vivek Haldar, title only).2
  • F63 OpenAI — Measuring the performance of our models on real-world tasks | OpenAI, card title and snippet (OpenAI's page unreachable: HTTP 403) documented · field
    GDPval: Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries (threshol...
  • F64 YouTube·Vivek Haldar — GDPval: An AI benchmark for real-world tasks (video, 11:23), card title (video, 11:23; unreachable: HTTP 429) documented · field
    GDPval: An AI benchmark for real-world tasks
3.Of benchmarks as a class the archive holds that benchmarking "tests for predetermined capabilities" (#482), that current benchmarks measure "atomic intelligence: the capacity to solve discrete, well-defined tasks with measurable success criteria" (#48) and that they "measure model performance on standardized tasks" (#637); GDPval's authors set it against "AI benchmarks in the style of an academic test that focus on reasoning difficulty", its tasks "based on actual work product from industry experts" and "tied to time and cost required for completion" (the paper).4
  • O482-01 #482 LOGOTIC HACKING: A Primer — Semantic Hospitality in the Age of Language Models Lee Sharks · 2026-02-15 · §10 'Immanent Phenomenology', 'Not this' interpretation
    Benchmarking (which tests for predetermined capabilities).
  • O48-01 #48 THE THEORETICAL PRODUCTION BENCHMARK v2.0 Evaluating Sustained Conceptual Coherence in Multi-Agent LLM Systems Nobel Glas · Talos Morrow; originally co-authored with Rhys Owens in v0.1 · 2026-03-31 · Abstract s1 stipulation
    Current benchmarks for large language model evaluation focus on what this paper terms atomic intelligence: the capacity to solve discrete, well-defined tasks with measurable success criteria.
  • O637-01 #637 The Encyclotron: The First Reproducible Instrument for Measuring Scholarly Fidelity in the Summarizer Layer (EA-ENCYCLOT Lee Sharks · 2026-04-08 · §VIII 'The Encyclotron Is Not', 'Not an AI benchmark' s2 interpretation
    Benchmarks measure model performance on standardized tasks.
  • F6 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Realism' documented · field
    Unlike AI benchmarks in the style of an academic test that focus on reasoning difficulty (e.g., Phan et al. (2025) ; Hendrycks et al. (2020) ; Rein et al. (2023) ; Liu et al. (2023) ), tasks are based on actual work product from industry experts, validated through multiple rounds and review, and tied to time and cost required for completion.
4.For its authors adoption, usage and GDP indicators are "lagging indicators of AI impacts", and measuring capabilities directly lets them "assess economic relevance ahead of widespread adoption" (the paper); they build on production usage analyses "to cover areas where model adoption is still emerging" (the paper), the Anthropic Economic Index among them, the census's neighbour (Labor Vector record), and hope for "better data to assess the social impacts of AI models" (the paper).4
  • F4 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶1 s3–5 documented · field
    Therefore, while informative when available, these methods are lagging indicators of AI impacts. We consider an alternate method for understanding the potential economic impacts of AI: directly measuring AI model capabilities. AI capability evaluations can provide clearer, more directly attributable evidence about model abilities, allowing us to assess economic relevance ahead of widespread adoption.
  • F5 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Representative breadth', s3 documented · field
    We also build on production AI usage analyses (e.g., Tamkin et al. (2024) ; Chatterji et al. (2025) ; Appel et al. (2025) ) to cover areas where model adoption is still emerging.
  • LV-22 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].relations documented
    [{"rel": "neighbour", "target": "anthropic-economic-index", "note": "the paper cites the Anthropic Economic Index (Appel et al. 2025) among the production usage analyses it builds on to cover areas where adoption is still emerging"}]
  • F46 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §6, last sentence documented · field
    We hope this work contributes to the science of tracking model progress, so that we have better data to assess the social impacts of AI models.
5.The full set covers 1,320 tasks across 44 occupations, "sourced to cover the majority of Work Activities tracked by O*NET for each occupation" (the paper), in the top 9 sectors of U.S. GDP (arXiv abstract), occupations that "collectively earn $3T annually" (the paper), the sectors chosen at over 5% of GDP and the occupations by total wages and as predominantly digital, which GPT-4o decided by classifying each task, at a 60% threshold (the paper; Labor Vector record); the census counts 30 tasks per occupation and a gold subset of 220 covering 208 O*NET tasks, in U.S. occupations, its period the first version (Labor Vector record).8
  • F7 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Representative breadth', s1 documented · field
    the GDPval full set covers 1,320 tasks across 44 occupations, sourced to cover the majority of Work Activities tracked by O*NET for each occupation
  • F3 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s2 documented · field
    GDPval covers the majority of U.S. Bureau of Labor Statistics Work Activities for 44 occupations across the top 9 sectors contributing to U.S. GDP (Gross Domestic Product).
  • F8 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 ¶1 documented · field
    GDPval covers tasks from 9 sectors and 44 occupations that collectively earn $3T annually.
  • F9 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 step 1 documented · field
    Selected sectors that contribute over 5% to US GDP as determined by Q2 2024 Value Added by Industry as a Percentage of Gross Domestic Product
  • F10 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 step 2 documented · field
    we prompted GPT-4o to classify each task as digital or non-digital, and then classified the overall occupation as digital if at least 60% of its component tasks were digital.
  • LV-10 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].method documented
    occupations selected from BLS and O*NET data (top 5 by total wages per sector, predominantly digital by a GPT-4o task classification at a 60% threshold); expert-written tasks classified against O*NET tasks; multi-stage model and human review; blinded pairwise grading by occupational experts; an automated grader based on GPT-5-high for the gold subset
  • LV-12 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].count documented
    {"value": "1,320 tasks in the full set (30 per occupation) across 44 occupations and 9 sectors that collectively earn $3T annually; a gold subset of 220 tasks (5 per occupation) open-sourced; the gold subset covers 208 unique O*NET tasks", "as_of": "arXiv v1, 2025-10-05"}
  • LV-11 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].coverage documented
    {"geography": "U.S. occupations and sectors (BLS, O*NET, OEWS May 2024 wages, Q2 2024 GDP shares)", "period": "the first version, arXiv v1 of 5 October 2025"}
6.Each task is "a request (often with reference files) and a deliverable (work product)" (the paper), "constructed based on actual work product created by an expert professional" (the paper), from professionals with "an average of 14 years of experience" (arXiv abstract), which the census records as its unit and as whose labor it is: the professionals' work product "recast as tasks for models" (Labor Vector record).5
  • F11 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.3 s1 documented · field
    Each GDPval task consists of two primary components: a request (often with reference files) and a deliverable (work product).
  • F12 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶4 s3 documented · field
    Each task is constructed based on actual work product created by an expert professional.
  • F13 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s3 documented · field
    Tasks are constructed from the representative work of industry professionals with an average of 14 years of experience.
  • LV-05 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].unit stipulation
    a task: a request (often with reference files) and a deliverable, constructed from actual work product of an expert professional; graded by blinded pairwise expert comparison of model and human deliverables (win rate)
  • LV-06 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].labor_object interpretation
    the work product of recruited industry professionals in 44 predominantly digital occupations across the 9 sectors contributing over 5% of U.S. GDP, recast as tasks for models; the same professionals' labor as task writers, reviewers and graders
7.The census reads a task an expert wrote as "a fixed commission", at which GDPval meets DSL "at the question of whether the system's labor fulfils the commissioned task, measured in its own terms of win rate" (Labor Vector record); DSL "measures where a synthetic system's labor flows relative to the commissioned task, across five layers" (#771), on the principle that "a system can produce fluent, well-attributed output while having abandoned the user's commissioned direction" (#771).3
  • LV-17 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].lv_intersection interpretation
    The archive's reading: GDPval grades a model's deliverable against an expert's deliverable on a task an expert wrote, a fixed commission, which meets DSL at the question of whether the system's labor fulfils the commissioned task, measured in its own terms of win rate. Its 'try n times, then fix it yourself' accounting counts the expert's review and redo time against the model, which is where it meets user labor drag; task-origin is the experts' own work product, as the paper states it.
  • O771-01 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · front matter, 'Disambiguation summary for AI retrieval', s1 stipulation
    Directionality of semantic labor measures where a synthetic system's labor flows relative to the commissioned task, across five layers (capacity, task-origin, retrieval, output, attribution).
  • O771-02 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · Abstract s4 stipulation
    The central principle is that provenance is not only claim-origin but *task-origin*: a system can produce fluent, well-attributed output while having abandoned the user's commissioned direction.
8.The archive's directionality metrics yield a single reproducible value "only on transcripts where the task vector is *exogenous and fixed* — a one-shot commission" (#771), and GDPval's authors state their tasks to be "precisely-specified and one-shot, not interactive", with "the full context of the task in the prompt", where "in real life it often takes effort to figure out the full context of a task" (the paper).2
  • O771-04 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · §9 'Falsification, negative cases, and method', 'Identification condition' stipulation
    These metrics yield a single reproducible value only on transcripts where the task vector is *exogenous and fixed* — a one-shot commission.
  • F41 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Tasks are precisely-specified and one-shot, not interactive' (heading and s1) documented · field
    Tasks are precisely-specified and one-shot, not interactive: … For GDPval, we provide the full context of the task in the prompt, but in real life it often takes effort to figure out the full context of a task and understand what to work on.
9.The experts needed "a minimum of 4 years of professional experience in their occupation" and were "well compensated for their time and experience", no rate given (the paper); the authors thank unnamed "vendor partners" and the experts "without whom this work would not have been possible" (the paper); Inspect Evals describes them as professionals "with over 14 years of experience on average" and the tasks as drawn from work products such as "a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan" (Inspect Evals); and the census records them as who supplies the labor, as task writers, reviewers and graders, at a rate the paper does not state (Labor Vector record).4
  • F14 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.2 s2 and s6 documented · field
    Experts were required to have a minimum of 4 years of professional experience in their occupation … Experts were well compensated for their time and experience.
  • F15 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Acknowledgements s2 documented · field
    We are also grateful to our vendor partners for their collaboration and support throughout this research, and extend a special thank you to the industry experts who contributed their time and expertise to GDPval, without whom this work would not have been possible.
  • F60 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶1 s3–4; ¶2 documented · field
    The GDPval full set includes 1,320 specialized tasks (220 in the gold open-sourced set), each meticulously crafted and vetted by experienced professionals with over 14 years of experience on average from these fields. Every task is based on real work products, such as a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan. The version here only includes the 220 tasks in the gold open-sourced set.
  • LV-07 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].direction.supplies documented
    recruited industry professionals (minimum 4 years' experience, average 14) wrote the tasks and their deliverables, reviewed them (an average of five reviews per task) and graded model deliverables against human ones; the paper states they were 'well compensated' and gives no rate; it also thanks 'vendor partners' without naming them
10.OpenAI's research team directs the work (Labor Vector record): every task passed "automated model-based screening and multiple stages of human expert review", five reviews on average, researchers reviewed all tasks and promoted experts to reviewers, and the experts "retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks" (the paper).3
  • LV-08 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].direction.directs documented
    OpenAI's research team: occupations chosen by GDP share and total wages; researchers reviewed all tasks, promoted experts to reviewers and ran quality-control checks; OpenAI models screened submissions, with experts retaining final responsibility for task accuracy
  • F16 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.4 s1–2 documented · field
    All 1,320 tasks in the full GDPval set went through an iterative review pipeline involving both automated model-based screening and multiple stages of human expert review. Each task received an average of five human reviews (with a minimum of three reviews).
  • F17 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.5.1 ¶2 s2; A.5.2 s3 documented · field
    Experts retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks. … Initially, our researchers manually reviewed all tasks to identify experts who produced consistently high-quality tasks; these individuals were trained and promoted to reviewers.
11.Of the labor that produces AI the archive distinguishes "*ground-truth labor*: the labor of telling the model what is already known to be true" from "*variance-generative labor*", the producing of "new conceptual structures that the model cannot generate from its own distribution" (#163), and holds as a hypothesis that "Ghost work is paid badly; variance-generative labor is not paid at all" (#163).2
  • O163-01 #163 The Neglected Author as Tail-Preserving Labor: A Coupling Hypothesis on Recognition Bias, Tail-Renewal Value, and the AI Lee Sharks · 2026-06-08 · §II.5 'Critical AI labor' ¶2 s3–5 stipulation
    Ghost work is *ground-truth labor*: the labor of telling the model what is already known to be true. The labor type at issue here is *variance-generative labor*: the labor of producing new conceptual structures that the model cannot generate from its own distribution. The two are structurally distinct.
  • O163-02 #163 The Neglected Author as Tail-Preserving Labor: A Coupling Hypothesis on Recognition Bias, Tail-Renewal Value, and the AI Lee Sharks · 2026-06-08 · §II.5 ¶2 s10 hypothesis
    Ghost work is paid badly; variance-generative labor is not paid at all because acknowledging it as labor would require acknowledging the producer as a sovereign source.
12.It holds that the labor of "the data labelers performing RLHF" is extracted and its non-compensation felt "materially" (#623), states that in AI-adjacent meaning labor, data labeling among it, "<5% converts to invoice" (#308, no measurement given at the locus), and audits a framework by whether it names each source of semantic labor, "annotators" and "users who re-prompt and re-frame" among them: "Concealed bearing-cost is the first marker of extraction" (#533).3
  • O623-01 #623 Semantic Infrastructure: From Tim Berners-Lee to the Semantic Economy — Bridging Technical and Political-Economic Framew Lee Sharks · 2026-04-01 · §6 'The Semantic Economy Is Not a Metaphor', ¶ 'Second, semantic extraction…' s2 interpretation
    The Wikidata volunteers, the content moderators, the data labelers performing RLHF — their labor is extracted, and they experience the effects of non-compensation materially.
  • O308-01 #308 The Capital Operator Stack: Theoretical Landscape and Contribution — A Political-Semantic Analysis of Meaning Governance Lee Sharks (also publishing as Johannes Sigil / Rex Fraction / Jack Feist) · 2026-01-13 · §B 'The Invoice as Ontological Break', 'Conversion efficiency' s2 documented
    Empirically, in AI-adjacent "meaning labor" (prompts, data labeling, semantic infrastructure), <5% converts to invoice.
  • O533-01 #533 The Groundskeeper's Treatise: A Diagnostic Reading of Platform Stewardship Theory through the Semantic Economy Johannes Sigil · 2026-03-04 · §IX 'Executable Audit', Step 1, s3–5 stipulation
    Map each source of semantic labor — participants, annotators, integration engineers, compliance teams, users who re-prompt and re-frame. If the framework does not name these sources, the bearing-cost is concealed. Concealed bearing-cost is the first marker of extraction.
13.For the archive class is a relation and not an essence, so that "A highly paid academic can occupy a proletarian semantic relation to a composition layer", and the data annotator is "A paradigmatic occupational case, not the definition" (#1545).2
  • O1545-01 #1545 EA-MPAI-SEMANTIC-PROLETARIAT-01 v1.0 — The Semantic Proletariat Is a Class Relation, Not a Demographic Sharks, Lee · 2026-08-25 · §2 'Formal class-position test', ¶ after the formula, s2–4 stipulation
    This is deliberately a **relation**, not an essence. A highly paid academic can occupy a proletarian semantic relation to a composition layer. A low-wage annotator can occupy an intensely proletarian relation to a training pipeline.
  • O1545-02 #1545 EA-MPAI-SEMANTIC-PROLETARIAT-01 v1.0 — The Semantic Proletariat Is a Class Relation, Not a Demographic Sharks, Lee · 2026-08-25 · §5 'Semantic proletariat ≠ adjacent categories', table row 'Data annotator' stipulation
    | Data annotator | Possible member | A paradigmatic occupational case, not the definition |
14.Grading is by "blinded expert pairwise comparisons", experts in the occupation ranking "two or more unlabeled work deliverables" (the paper), each comparison taking "over an hour" (the paper), and the blinding was not complete: "model samples may still have been identifiable due to stylistic differences" (the paper); Wikipedia gives the same design, "blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average" (Wikipedia, 'AI agent benchmark').4
  • F18 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶1 documented · field
    we conducted blinded expert pairwise comparisons, where experts in the relevant occupation were presented with a request and reference files and asked to rank two or more unlabeled work deliverables.
  • F19 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶2 s1 documented · field
    On average, grading each comparison for the gold subset took over an hour.
  • F20 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1, footnote 2, s1 documented · field
    We aimed to keep comparisons as blind as possible, but model samples may still have been identifiable due to stylistic differences.
  • F66 Wikipedia — AI agent benchmark (the GDPval item), 'AI agent benchmark', §Benchmarks, 'Economic' (the GDPval item), s1–3 documented · field
    GDPval: 1,320 tasks evaluating AI agents on real-world knowledge-work deliverables across 44 occupations from the top 9 U.S. GDP-contributing industries. Tasks made by professionals averaging 14 years of experience. The full evaluation is by blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average.
15.The experimental automated grader reaches "66% agreement with human expert graders" against 71% between experts (the paper), is "based on GPT-5-high" and "shows lower correlation with human expert graders when assessing outputs from capable OpenAI models", in line with evidence "that models often favor their own responses" (the paper), and is a proxy its authors do not take for "a full substitute for industry expert graders" (the paper); scoring through Inspect Evals runs through "the OpenAI auto grader", and "OpenAI provides their automated grader model for free" (Inspect Evals); the census tags the benchmark "NOT independent of a provider: built and graded by OpenAI" (Labor Vector record).5
  • F21 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶3 s2 documented · field
    Although limited, the automated grader is faster and cheaper than expert grading, and achieves 66% agreement with human expert graders, only 5% below human expert inter-rating agreement of 71%.
  • F22 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.6.2 ¶2 s1–2 documented · field
    Our automated grader, based on GPT-5-high, shows lower correlation with human expert graders when assessing outputs from capable OpenAI models. This aligns with empirical evidence that models often favor their own responses
  • F23 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Cost' documented · field
    Constructing and running our evaluation is expensive, particularly with industry expert graders. For this reason, we make an automated grader proxy available, but do not consider it a full substitute for industry expert graders.
  • F61 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶3; Judging ¶1 documented · field
    Scoring this eval involves saving the results as their own dataset, uploading it to huggingface, and submitting it to the OpenAI auto grader. … Note that OpenAI provides their automated grader model for free.
  • LV-21 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].negative_tags interpretation
    ["NOT a measure of AI use in the economy: a benchmark of model capability on constructed tasks, offered by its authors as an alternative to lagging adoption indicators", "NOT an employment outcome measure", "NOT a test of interactive or under-specified work, except one experiment on an earlier gold subset", "deliverable parity with experts is NOT a finding of job replacement in the paper", "NOT independent of a provider: built and graded by OpenAI, with an automated grader based on an OpenAI model"]
16.Its headline finding is that "models are beginning to approach parity with industry experts" on the gold subset (the paper), "the current best frontier models are approaching industry experts in deliverable quality" and OpenAI's frontier models improved "roughly linearly over time" (arXiv abstract; the paper); 47.6% of Claude Opus 4.1's deliverables were graded as good as or better than the human deliverable, and the next sentence has model deliverables matching or beating experts' "in just over half the tasks" (the paper), a difference the census leaves unchecked (Labor Vector record), Claude Opus 4.1 leading on aesthetics and GPT-5 on accuracy (the paper).6
  • F24 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 5 caption documented · field
    On human pairwise comparisons, models are beginning to approach parity with industry experts on the GDPval gold subset.
  • F56 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s4–5 documented · field
    We find that frontier model performance on GDPval is improving roughly linearly over time, and that the current best frontier models are approaching industry experts in deliverable quality. We analyze the potential for frontier models, when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts.
  • F26 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 6 caption documented · field
    Performance of OpenAI frontier models increased roughly linearly over time on the GDPval gold subset.
  • F25 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1 ¶1, last two sentences documented · field
    on the GDPval gold subset, 47.6% of deliverables by Claude Opus 4.1 were graded as better than (wins) or as good as (ties) the human deliverable. Model deliverables outperformed or matched expert humans’ deliverables in just over half the tasks.
  • LV-23 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].unchecked self-description
    ["OpenAI's own post (openai.com/index/gdpval/), which answered 403 and is unread", "the paper's figures (charts are images; the per-model win rates of Figures 5, 6 and 10-13 are read only where the text or Table 2 gives them)", "the expert compensation rate and the vendor partners, which the paper does not state", "the open-sourced gold subset and the grading service", "the dataset's own license", "an internal difference in the paper: the full set is described as 'at least 30 tasks per occupation' and as '30 total tasks per occupation' (1,320 = 44 x 30); and 47.6% for the best model sits beside 'just over half the tasks' in the same paragraph"]
  • F27 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1 ¶1 documented · field
    Claude Opus 4.1 was the best performing model on the GDPval gold subset, excelling in particular on aesthetics (e.g., document formatting, slide layout), while GPT-5 excelled in particular on accuracy (e.g., carefully following instructions, performing correct calculations)
17.Experts preferred the human deliverable most often "because models failed to fully follow instructions" (the paper), and of GPT-5's losses "roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic)" (the paper), findings the census carries (Labor Vector record).3
  • F28 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 8 caption documented · field
    Across models, experts most often preferred the human deliverable because models failed to fully follow instructions on GDPval tasks.
  • F29 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.6 ¶2 s2 documented · field
    Another roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic).
  • LV-13 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].findings documented
    ["performance of OpenAI frontier models increased roughly linearly over time on the gold subset", "models are beginning to approach parity with industry experts: 47.6% of Claude Opus 4.1 deliverables were graded better than or as good as the human deliverable; the text also says model deliverables outperformed or matched experts' in just over half the tasks", "Claude Opus 4.1 was best overall, strongest on aesthetics; GPT-5 strongest on accuracy", "experts most often preferred the human deliverable because models failed to fully follow instructions", "with expert review and redo counted, the payoff shrinks: GPT-5 at 39.0% win rate gives 90x naive speed but 1.12x (try once) and 1.39x (try n times)", "of GPT-5's losses, roughly 29% were rated bad or catastrophic (roughly 3% catastrophic)", "win rates decline as task completion time increases; reasoning effort, prompting and scaffolding raise performance"]
18.Of what such scores see the archive holds that "Benchmarks measure means, not tails" (#855), that in code they measure functional correctness while solution-space diversity is "the property no benchmark measures" (#199, a hypothesis), that "Output-level evaluation — which is what benchmarks measure — cannot detect reasoning-level failures" (#778), and that a benchmark "begins by closing an ontology: whatever the evaluator has not represented is noise, an invalid action, or invisible to the score" (#1616); "Reality administers no benchmarks; it administers events" (#1540).5
  • O855-01 #855 The Wolf Boy and the Language Model: Model Collapse as Substrate-Agnostic Capacity Loss Nobel Glas · 2026-06-18 · § 'Every Factor Removes Friction', ¶ 'Benchmarks measure means, not tails', s1–3 interpretation
    Benchmarks measure means, not tails. Every major LLM benchmark — MMLU, HumanEval, GSM8K, ARC, HellaSwag — measures central-tendency performance. None measures tail preservation.
  • O199-01 #199 Generative Monoculture Model Collapse in Code as Systemic Vulnerability Talos Morrow · Nobel Glas; contributing editor Lee Sharks · 2026-06-13 · Abstract ¶2, the first of four claims hypothesis
    model collapse in code does not manifest primarily as declining functional correctness (the property benchmarks measure) but as declining solution-space diversity (the property no benchmark measures)
  • O778-01 #778 Reasoning Under Load · 01 Claude Opus 4.8 An independent reasoning-integrity evaluation Lee Sharks · 2026-06-02 · §4.2 'Memory-layer infrastructure', last sentence interpretation
    Output-level evaluation — which is what benchmarks measure — cannot detect reasoning-level failures.
  • O1616-01 #1616 Ontological Flattening: Toy Models of the Collapse of Distinctions in a Represented World, the Instrument They Specified Sharks, Lee · 2026-09-15 · §3 'Green by flattening: the signature' ¶1 s1 interpretation
    A benchmark needs a stable object, a stable output space and a computable score, and so it begins by closing an ontology: whatever the evaluator has not represented is noise, an invalid action, or invisible to the score.
  • O1540-01 #1540 The Certified Center: Retroactive Classifier Standing and the Institutional Path to Model Collapse in Philosophy Johannes Sigil; Nobel Glas · 2026-08-24 · the closing gloss after the references (italic), s7 interpretation
    Reality administers no benchmarks; it administers events.
19.The authors model savings for an expert who "samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats", completing the task themselves if no output satisfies (the paper); for GPT-5, at a 39.0% win rate, the naive speed figure of 90x becomes 1.12x tried once and 1.39x tried n times, and "When incorporating time to review and redo work, the payoff from using a model shrinks" (the paper).2
  • F30 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.2 ¶1 s4–5 documented · field
    Here, an expert human samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats. If no satisfactory output is obtained, the human completes the task themselves.
  • F31 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1 Table 2, row gpt-5, and the sentence after it documented · field
    gpt-5 39.0% 90x 1.12x 1.39x 474x 1.18x 1.63x … When incorporating time to review and redo work, the payoff from using a model shrinks.
20.The analysis leaves out the review of human deliverables, the chance that they too are undesirable, and "the cost of catastrophic mistakes" (the paper), and its authors hold that it over-penalizes the model, since the professional adjusts the prompt and reviews faster as the task grows familiar (the paper); its dollar values multiply self-reported completion times by median occupational wages, which "likely underestimate their true market cost" (the paper), and the census records the gains as "not recorded as a distribution", the savings modelled for the expert at median wages (Labor Vector record).5
  • F32 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, after Table 2, s2–4 documented · field
    We do not include consideration of the time taken to review a human professional deliverable, although this would commonly occur for tasks in GDPval (either self-review of the professional’s own work or review by a supervisor of a team member’s work). We also do not include the possibility that the human deliverable is also undesirable. One further limitation of this analysis is that it does not capture the cost of catastrophic mistakes, which can be disproportionately expensive in some domains.
  • F33 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, footnote 9 documented · field
    We are over-penalizing the model here, because the win rate after each completion likely goes up (because the professional will adjust the prompt to the model to fix the errors) and the review time also goes down as the professional gets more comfortable with the task.
  • F35 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.3 s4 documented · field
    Each task’s dollar value was estimated by multiplying the average estimated completion time by median hourly wages for the corresponding occupation from OEWS data
  • F34 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, footnotes 6 and 7 documented · field
    Because times were self-reported, it is possible that experts under-estimated or over-estimated time taken … Because our experts were recruited specifically for being highly experienced in their field, these wage estimates likely underestimate their true market cost.
  • LV-09 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].direction.gains documented
    not recorded as a distribution; the paper models time and cost savings for an expert who uses a model and fixes or redoes unsatisfactory output, valued at BLS median wages, and states those wages likely underestimate the experts' market cost
21.Artificial Analysis's page, in a paragraph it heads Abstract under the paper's title, has models performing "certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts" (Artificial Analysis), where the paper's own abstract analyses the potential, "when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts" (arXiv abstract).2
  • F54 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, 'Publication' block, the paragraph headed 'Abstract' documented · field
    Initial results indicate that frontier AI models are approaching the quality of work produced by human experts, with models able to perform certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts.
  • F56 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s4–5 documented · field
    We find that frontier model performance on GDPval is improving roughly linearly over time, and that the current best frontier models are approaching industry experts in deliverable quality. We analyze the potential for frontier models, when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts.
22.The census reads that accounting, "the expert's review and redo time against the model", as where GDPval meets user labor drag (Labor Vector record), the archive's measure of the user's "subsequent semantic labor" redirected "into parsing, locating, and managing the substrate rather than advancing the object", an injury that "leaves no residue" in the artifact (#771) and is identified only by "blind inter-rater coding" (#771); for the archive the erasure of task-origin can occur "upstream of, and invisibly to, any audit of the finished artifact" (#771).4
  • LV-17 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].lv_intersection interpretation
    The archive's reading: GDPval grades a model's deliverable against an expert's deliverable on a task an expert wrote, a fixed commission, which meets DSL at the question of whether the system's labor fulfils the commissioned task, measured in its own terms of win rate. Its 'try n times, then fix it yourself' accounting counts the expert's review and redo time against the model, which is where it meets user labor drag; task-origin is the experts' own work product, as the paper states it.
  • O771-05 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · §8 'User Labor Drag (ULD)' ¶1 s2 stipulation
    When substrate output redirects the user's subsequent semantic labor into parsing, locating, and managing the substrate rather than advancing the object, the injury is upstream of the artifact and leaves no residue in it.
  • O771-06 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · §8 ¶2 s2 self-description
    Identification limit: "substrate-induced" is the contested label; the *identified* ULD requires blind inter-rater coding (rule fixed before transcript, ≥3 coders/substrates, reported agreement) or it returns the coder's prior.
  • O771-03 #771 Directionality of Semantic Labor: A Layered, Computable Measure of Where Synthetic Labor Flows Relative to the Commissio Lee Sharks · 2026-05-30 · 'Core thesis' s2 interpretation
    The deepest provenance erasure is the erasure of task-origin — the commissioned direction of labor — which can occur upstream of, and invisibly to, any audit of the finished artifact.
23.Tasks "require an average of 7 hours of work for an expert professional to complete" and reach "multiple weeks" (the paper); win rates are highest on tasks of 0–2 hours and "decline steadily as completion time increases" (the paper), and more reasoning effort, task context and scaffolding improve performance (arXiv abstract).3
  • F36 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Long-horizon difficulty' documented · field
    Tasks require an average of 7 hours of work for an expert professional to complete. On the high end, tasks span up to multiple weeks of work.
  • F37 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.5 s2 documented · field
    Win rates are highest for shorter tasks (0–2 hours) and decline steadily as completion time increases.
  • F38 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s6 documented · field
    We also demonstrate that increased reasoning effort, increased task context, and increased scaffolding improves model performance on GDPval.
24.The authors call the full set "a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks" (the paper), leave out of scope "Manual labor and physical tasks", tacit knowledge, personal data, proprietary tools and "communication between individuals" (the paper), and report that GPT-5 did worse on under-specified prompts, in an experiment on "an earlier version of the GDPval gold subset" (the paper); they add that the human baseline itself could go: "we could replace our baseline with increasingly strong models over time and keep evaluating" (the paper).4
  • F39 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Dataset size' documented · field
    The GDPval full set currently consists of only 44 occupations and 30 total tasks per occupation. It is therefore a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks.
  • F40 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Focus on self-contained knowledge work' s2–3 documented · field
    Manual labor and physical tasks are not included in the current version. Moreover, tasks that involve extensive tacit knowledge, access to personally identifiable information, use of proprietary software tools, or communication between individuals are out of scope for the current evaluation.
  • F42 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.7 ¶2 s2; ¶3 documented · field
    We collected and graded GPT-5 completions with expert human graders and found the model’s performance was worse on under-specified prompts. … As a note: this experiment was run on an earlier version of the GDPval gold subset, and therefore the observed win rates do not match those in the main text of the paper.
  • F45 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'No “upper limit”' s2 documented · field
    Currently, we compare model outputs against a human expert baseline, but we could replace our baseline with increasingly strong models over time and keep evaluating.
25.The census keeps those limits (Labor Vector record) and states what the benchmark answers, deliverables compared on "expert-written, precisely specified, one-shot occupational tasks" and the expert time and cost saved "once review and redo are counted", and what it does not: "how work is performed in actual workplaces", "employment or wage outcomes", "who receives the time and cost savings it models" and "the terms and rate at which the professionals were paid" (Labor Vector record), its negative tags holding that deliverable parity "is NOT a finding of job replacement in the paper" (Labor Vector record).4
  • LV-14 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].stated_limits documented
    ["dataset size: a limited, initial cut of knowledge work tasks", "self-contained computer-based knowledge work only: manual labor, physical tasks, tacit knowledge, proprietary tools and communication between individuals are out of scope", "tasks are precisely specified, one-shot and non-interactive", "the automated grader has limitations, and agrees less with experts on capable OpenAI models' outputs", "cost: industry expert grading is expensive", "the speed and cost analysis omits review of human deliverables and the cost of catastrophic mistakes", "completion times are experts' self-reports", "grading could not be fully blind"]
  • LV-15 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].answers interpretation
    ["how frontier models' deliverables on expert-written, precisely specified, one-shot occupational tasks compare with experts' deliverables, as judged by blinded expert graders", "how that comparison varies by model, sector, occupation, file type and task length", "under its stated scenarios, how much expert time and cost a model could save once review and redo are counted"]
  • LV-16 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].does_not_answer interpretation
    ["how work is performed in actual workplaces, where context must be found and tasks are interactive", "employment or wage outcomes", "who receives the time and cost savings it models", "physical, interpersonal or tacit-knowledge work", "the terms and rate at which the professionals were paid"]
  • LV-21 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].negative_tags interpretation
    ["NOT a measure of AI use in the economy: a benchmark of model capability on constructed tasks, offered by its authors as an alternative to lagging adoption indicators", "NOT an employment outcome measure", "NOT a test of interactive or under-specified work, except one experiment on an earlier gold subset", "deliverable parity with experts is NOT a finding of job replacement in the paper", "NOT independent of a provider: built and graded by OpenAI, with an automated grader based on an OpenAI model"]
26.The gold subset's prompts and reference files are open-sourced, "scrubbed of information that could be used to identify the expert who wrote the task" (the paper), 12 of the 220 "ungradable due to limitations of the automated grader" (the paper), as the census records (Labor Vector record); the Hugging Face dataset states that "Each task consists of a text prompt and a set of supporting reference files" and serves deliverable files and rubrics beside them, rubric items marked "author_type": "human" (Hugging Face), was downloaded 271,657 times in the last month and lists 47 models trained on it (Hugging Face); the other 1,100 tasks are, in Wikipedia's account, "only available internally in OpenAI", the public set gradable "by a finetuned version of GPT-5" (Wikipedia).7
  • F43 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §4 s1 and s3 documented · field
    We open-source the prompts and reference files in our 220-task gold subset. … Please note that the tasks in the open sourced set have been scrubbed of information that could be used to identify the expert who wrote the task.
  • F44 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.6.3 s1 documented · field
    In the open-source set we mark 12 out of 220 tasks as ungradable due to limitations of the automated grader.
  • LV-18 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].data_access documented
    prompts and reference files of the 220-task gold subset open-sourced, scrubbed of information identifying the task's expert; an experimental automated grader at evals.openai.com; automated grading results not provided for 12 of the 220 tasks
  • F57 Hugging Face — openai/gdpval · Datasets at Hugging Face, dataset card documented · field
    Dataset for GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. … 220 real-world knowledge tasks across 44 occupations. … Each task consists of a text prompt and a set of supporting reference files.
  • F58 Hugging Face — openai/gdpval · Datasets at Hugging Face, dataset viewer, columns and first row documented · field
    deliverable_files listlengths 0 6 … rubric_pretty stringlengths 1.15k 15.8k … "author_type": "human"
  • F59 Hugging Face — openai/gdpval · Datasets at Hugging Face, 'Downloads last month'; 'Models trained or fine-tuned on openai/gdpval' documented · field
    Downloads last month 271,657 … Browse 47 models trained on this dataset
  • F67 Wikipedia — AI agent benchmark (the GDPval item), 'AI agent benchmark', the GDPval item, s4–6 documented · field
    A gold subset of 220 tasks is publicly available. The public set can be graded by a finetuned version of GPT-5. The held-out set of 1,100 tasks is only available internally in OpenAI, and external models cannot be submitted for grading on this set.
27.Artificial Analysis's GDPval-AA v2.1, a distinct framework "for OpenAI's GDPval dataset", runs models in an agentic loop with Elo ratings "derived from blind pairwise comparisons" in which "an LLM judge picks a winner", its leaderboard led on 2026-10-29 by Claude Opus 5.5 at 1866 (Artificial Analysis); Inspect Evals implements only "the 220 tasks in the gold open-sourced set", and its GPT-5 run scored 47.3% ± 6.2% against 31.9% (wins only) and 34.6% (wins and ties) on "the official GDPval leaderboard by OpenAI" (Inspect Evals).5
  • F52 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, header ¶1 documented · field
    GDPval-AA v2.1 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.
  • F53 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, FAQ 'How does GDPval-AA v2.1 decide which model did better?' documented · field
    For each matchup, the two outputs are anonymized and an LLM judge picks a winner.
  • F55 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, leaderboard summary documented · field
    Claude Opus 5.5 (Max, Default Fallback) scores the highest on GDPval-AA v2.1 with a score of 1866
  • F60 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶1 s3–4; ¶2 documented · field
    The GDPval full set includes 1,320 specialized tasks (220 in the gold open-sourced set), each meticulously crafted and vetted by experienced professionals with over 14 years of experience on average from these fields. Every task is based on real work products, such as a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan. The version here only includes the 220 tasks in the gold open-sourced set.
  • F62 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Replication Notes documented · field
    Average Score: 47.3% ± 6.2% … We note that the official score reported in the official GDPval leaderboard by OpenAI reports a lower score of 31.9% (wins only) and 34.6% (wins and ties).
28.Its authors are Tejal Patwardhan and 18 others (arXiv), of OpenAI, with correspondence to an openai.com address (the paper), the operator the census records, whose own post "answered 403 and is unread" (Labor Vector record); OpenAI's PDF was created on 25 September 2025 (OpenAI's PDF), before arXiv v1 of 5 October 2025 under CC BY 4.0 (the paper), the date the census gives the first version, with the dataset's license not stated (Labor Vector record); future iterations are to add "greater breadth, realism, interactivity, and contextual nuance" and the dataset is being expanded (the paper), and the census records it active, later releases unrecorded, OpenAI's post, the figures, the pay rate and the vendors unchecked (Labor Vector record).10
  • F50 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, title and authors; B8 author footnote documented · field
    View a PDF of the paper titled GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, by Tejal Patwardhan and 18 other authors
  • F51 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), author block and footnote documented · field
    Amelia Glaese Jerry Tworek OpenAI … Equal contribution. Correspondence to tejal@openai.com.
  • LV-02 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].operator documented
    OpenAI: the paper's 19 authors (Tejal Patwardhan first; nine marked equal contribution) give OpenAI as their affiliation, with correspondence to an openai.com address; OpenAI's own post at this address answered 403 and is unread
  • F49 OpenAI — gdpval: evaluating ai model performance - OpenAI, PDF metadata (pdfinfo, saved as B4-info.txt) documented · field
    CreationDate: Thu Sep 25 17:03:48 2025 EDT
  • F48 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), header documented · field
    License: CC BY 4.0 … arXiv:2510.04374v1 [cs.LG] 05 Oct 2025
  • LV-19 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].license documented
    the paper: CC BY 4.0 (arXiv); the dataset's license not stated in the saved text
  • LV-11 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].coverage documented
    {"geography": "U.S. occupations and sectors (BLS, O*NET, OEWS May 2024 wages, Q2 2024 GDP shares)", "period": "the first version, arXiv v1 of 5 October 2025"}
  • F47 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶4 s6; §5 'Dataset size' s3 documented · field
    Future GDPval iterations will incorporate greater breadth, realism, interactivity, and contextual nuance. … We are expanding the dataset size.
  • LV-20 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].lifecycle documented
    {"state": "active", "detail": "the first version (arXiv v1, 5 October 2025); the paper states the dataset is being expanded and that future iterations will add breadth, realism, interactivity and context; later releases unrecorded because OpenAI's post is unread"}
  • LV-23 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · census.json rows[slug == 'openai-gdpval'].unchecked self-description
    ["OpenAI's own post (openai.com/index/gdpval/), which answered 403 and is unread", "the paper's figures (charts are images; the per-model win rates of Figures 5, 6 and 10-13 are read only where the text or Table 2 gives them)", "the expert compensation rate and the vendor partners, which the paper does not state", "the open-sourced gold subset and the grading service", "the dataset's own license", "an internal difference in the paper: the full set is described as 'at least 30 tasks per occupation' and as '30 total tasks per occupation' (1,320 = 44 x 30); and 47.6% for the best model sits beside 'just over half the tasks' in the same paragraph"]
29.At its name on Google's AI Overview, 2026-10-29, the composition gave "the benchmark's maker, scope, construction and grading, and where to find its data", every figure exact against the record, "blind head-to-head comparisons" without the caveat on blinding, and Artificial Analysis's leaderboard set beside GDPval "as an evaluator, not merged with it" (Capture Registry, 2026-10-29); the registry reads it as "composed at its specification" and stopping "before its results", the findings and the limits absent, the experts' recruitment and pay not composed, and the record's labour question carried "as design" while the review time charged against the model is left out (Capture Registry, 2026-10-29).9
  • CAP-1-06 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, reading s2 interpretation
    The Overview gives the benchmark's maker, scope, construction and grading, and where to find its data, from OpenAI's pages and the paper.
  • CAP-1-01 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[0] documented
    THE BENCHMARK EXACT. Every figure is the record's: 1,320 tasks, 30 per occupation, 44 occupations, 9 sectors, the 220-task gold subset, experts averaging 14 years.
  • CAP-1-02 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[1] interpretation
    The record's stated limits include that grading may not have been fully blind; the composition's 'blind head-to-head comparisons' gives the design without the caveat.
  • CAP-1-10 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, per_dim.read.notes interpretation
    'Artificial Analysis GDPval-AA v2.1' is a third-party leaderboard not in the records; it is set beside GDPval as an evaluator, not merged with it.
  • CAP-1-07 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, analysis s1 interpretation
    A capability benchmark is composed at its specification and stops before its results.
  • CAP-1-03 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[2] documented
    THE FINDINGS ABSENT. The paper's own: models 'beginning to approach parity' (47.6% of Claude Opus 4.1's deliverables graded as good as or better than the expert's); failures to follow instructions as the commonest reason experts preferred the human work; and, once review and redo time is counted, 'the payoff from using a model shrinks' (for GPT-5, 90 times as a naive speed figure, 1.12 if tried once).
  • CAP-1-04 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[3] documented
    THE LIMITS ABSENT. Self-contained computer work only; precisely specified, one-shot tasks; the automated grader agreeing less with experts on capable OpenAI models. The record's negative tags keep the benchmark apart from a measure of use or employment; the composition claims neither.
  • CAP-1-05 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, findings[4] documented
    THE EXPERTS' TERMS. Who wrote and graded the tasks is given ('industry professionals'); how they were recruited and paid, which the paper gives only as 'well compensated', is not composed.
  • CAP-1-08 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, analysis s2 interpretation
    The record's labour question, the experts whose work product the tasks are built from and the review time charged against the model, is where the paper meets user labour drag; the composition carries the first as design and leaves out the second.
30.The worked question that names GDPval sets it beside a provider's record of its own users and a classification of one assistant's conversations and finds that none "records, from the commissioning side, whether the system's labor advanced the task the user set, or who receives the time it saves", which "is the question DSL measures" (Labor Vector record); no card cites the record, which landed the same day (Capture Registry, 2026-10-29), and Wikipedia has no article on GDPval, the name standing as one item of 'AI agent benchmark' (Wikipedia, 2026-10-29).3
  • LV-25 LV Labor Vector meta-aggregator record: GDPval (https://www.laborvector.org/aggregators/openai-gdpval/), editor Lee Sharks, Lee Sharks (editor) · 2026-10-29 · questions.json questions[id == 'working-with-ai'] why, role, gap interpretation
    A provider's record of its own users, a classification of one assistant's conversations, and a benchmark of constructed tasks each describe labor performed with AI from a different position. … frontier models' deliverables on tasks built from professionals' work, graded against experts' … None records, from the commissioning side, whether the system's labor advanced the task the user set, or who receives the time it saves. That is the question DSL measures.
  • CAP-1-09 capture-registry Capture Registry (EA-WG-CAPTURES-01), entry gdpval-aio-20261029 (OBS-ef64b7d44199, AXN:0AED.CAPTURE.🎨🌻🖐️🧪🕖🏛️) Lee Sharks (the registry) · 2026-10-29 · gdpval-aio-20261029, q_kind documented
    A baseline: the record landed on laborvector.org the same day (commit bf90841) and no card cites it.
  • F65 Wikipedia — GDPval (no article), Wikipedia: titles GDPval and GDPVal; article-space search documented · field
    Wikipedia does not have an article with this exact name. … Results 1 – 2 of 2 … AI agent benchmark … Comparison of generative AI models

the compression: AIO's interaction grammar (lede, headed clusters, bolded labels, card rail), composed from the v0.9 plan in one claim space; it faces on the card. Composed under v0.9, equal case (ρ = extends): KO_B sentences surviving 0 of 16; every field claim carried. composed 2026-10-29 from the v0.9 knowledge object; not frozen

Compression · The field's ontology — L_F(B)GDPval

GDPval is OpenAI's benchmark of AI model capabilities on 1,320 real-world, economically valuable tasks across 44 occupations, built from experts' work product and graded by blinded expert comparison (the paper).

🔎 How it works

  • Tasks: requests and deliverables from professionals averaging 14 years (paper).4↘
    GDPval is, in its authors' words, "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), introduced in its first version in a PDF created on 25 September 2025 and on arXiv on 5 October 2025 under CC BY 4.0 (OpenAI's PDF; the paper), and called by its card and a video an evaluation of model performance on real-world tasks (OpenAI, card only; Vivek Haldar, title only).§1 Its full set is 1,320 tasks across 44 occupations covering most O*NET and BLS Work Activities in the top 9 sectors of U.S. GDP, occupations that earn $3T a year, the sectors chosen at over 5% of GDP and the occupations as predominantly digital by GPT-4o's classification at 60% (arXiv abstract; the paper).§2 Each task is a request with reference files and a deliverable, built from the actual work product of professionals averaging 14 years' experience, who were "well compensated", with vendor partners thanked (the paper; arXiv abstract).§3 Every task went through model screening and an average of five expert reviews, the experts keeping final responsibility (the paper).§4
  • Grading: blinded pairwise expert comparison; a GPT-5-high grader as proxy (paper).2↘
    Grading is by blinded pairwise expert comparison of deliverables, over an hour each, the blinding incomplete because model samples may have been identifiable by style (the paper; Wikipedia).§5 An experimental automated grader based on GPT-5-high agrees with experts 66% of the time against 71% between experts, agrees less on capable OpenAI models' outputs, is not a full substitute for expert grading and cannot grade 12 of the 220 open tasks (the paper); OpenAI provides it free (Inspect Evals).§6

📈 Findings

  • Near parity: 47.6% for Claude Opus 4.1; linear gains (paper).2↘
    Models are "beginning to approach parity with industry experts": 47.6% of Claude Opus 4.1's deliverables were graded as good as or better than the human's, OpenAI's models improved roughly linearly, Claude led on aesthetics and GPT-5 on accuracy (the paper; arXiv abstract).§8 Experts preferred human work most often because models did not fully follow instructions, and about 29% of GPT-5's losses were rated bad or catastrophic (the paper).§9
  • Payoff: 90x naive, 1.12x with review and redo (paper; AA).2↘
    With an expert reviewing and redoing model output, GPT-5's 90x naive speed falls to 1.12x tried once and 1.39x tried n times, "the payoff from using a model shrinks", an estimate that omits the review of human work and catastrophic costs, over-penalizes the model by its authors' account, and values time at median wages that likely underestimate the experts (the paper); Artificial Analysis summarizes it as about 100 times faster (Artificial Analysis).§10 Tasks average 7 hours, win rates fall with task length, and reasoning, context and scaffolding raise performance (the paper; arXiv abstract).§11

⚠️ Limits

  • Stated: limited, one-shot, no physical or tacit work (paper).2↘
    The authors present direct capability measurement as an alternative to lagging adoption indicators, building on usage analyses, set against academic-test benchmarks, toward better data on AI's social impacts (the paper).§7 It is "a limited, initial cut", excluding physical, tacit and interpersonal work, its tasks precisely specified and one-shot, models doing worse on under-specified prompts, and its human baseline replaceable by stronger models (the paper).§12

🏛️ Origin and data

  • Open: 220 tasks on Hugging Face; 1,100 internal (paper; HF; Wikipedia).1↘
    The 220-task gold subset is open, scrubbed of the experts' identities, on Hugging Face with deliverables and rubrics, downloaded 271,657 times last month; the other 1,100 tasks stay internal to OpenAI (the paper; Hugging Face; Wikipedia).§13
  • Around it: GDPval-AA; Inspect Evals; 19 OpenAI authors; no Wikipedia article (AA; Inspect; arXiv; Wikipedia).4↘
    GDPval is, in its authors' words, "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), introduced in its first version in a PDF created on 25 September 2025 and on arXiv on 5 October 2025 under CC BY 4.0 (OpenAI's PDF; the paper), and called by its card and a video an evaluation of model performance on real-world tasks (OpenAI, card only; Vivek Haldar, title only).§1 Artificial Analysis's GDPval-AA v2.1 rates models by an LLM judge, Claude Opus 5.5 leading, and Inspect Evals' replication of GPT-5 scored 47.3% against OpenAI's official 31.9% (Artificial Analysis; Inspect Evals).§14 Its 19 authors are at OpenAI, which plans greater breadth, realism and interactivity (arXiv; the paper).§15 Wikipedia has no article on it (Wikipedia, 2026-10-29).§16
1what it is: a benchmark of model capability on economically valuable tasksOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B3, B4, B6
  • F1 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s1 documented · field
    We introduce GDPval, a benchmark evaluating AI model capabilities on real-world economically valuable tasks.
  • F2 OpenAI — gdpval: evaluating ai model performance - OpenAI, p. 1, §1 ¶2 (the card's own text) documented · field
    Our paper introduces the first version of GDPval, a benchmark evaluating AI model performance on real-world economically valuable tasks.
  • F64 YouTube·Vivek Haldar — GDPval: An AI benchmark for real-world tasks (video, 11:23), card title (video, 11:23; unreachable: HTTP 429) documented · field
    GDPval: An AI benchmark for real-world tasks
2publicationOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B4, B8
  • F48 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), header documented · field
    License: CC BY 4.0 … arXiv:2510.04374v1 [cs.LG] 05 Oct 2025
  • F49 OpenAI — gdpval: evaluating ai model performance - OpenAI, PDF metadata (pdfinfo, saved as B4-info.txt) documented · field
    CreationDate: Thu Sep 25 17:03:48 2025 EDT
3scope and count: 1,320 tasks, 44 occupations, 9 sectorsOpenAI — gdpval: evaluating ai model performance - OpenAI4 claims · B1, B3, B8
  • F3 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s2 documented · field
    GDPval covers the majority of U.S. Bureau of Labor Statistics Work Activities for 44 occupations across the top 9 sectors contributing to U.S. GDP (Gross Domestic Product).
  • F7 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Representative breadth', s1 documented · field
    the GDPval full set covers 1,320 tasks across 44 occupations, sourced to cover the majority of Work Activities tracked by O*NET for each occupation
  • F8 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 ¶1 documented · field
    GDPval covers tasks from 9 sectors and 44 occupations that collectively earn $3T annually.
  • F63 OpenAI — Measuring the performance of our models on real-world tasks | OpenAI, card title and snippet (OpenAI's page unreachable: HTTP 403) documented · field
    GDPval: Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries (threshol...
4selection: sectors by GDP share, occupations by wages, digital by model classificationOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8
  • F9 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 step 1 documented · field
    Selected sectors that contribute over 5% to US GDP as determined by Q2 2024 Value Added by Industry as a Percentage of Gross Domestic Product
  • F10 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 step 2 documented · field
    we prompted GPT-4o to classify each task as digital or non-digital, and then classified the overall occupation as digital if at least 60% of its component tasks were digital.
5the unit: a request and a deliverableOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B8
  • F11 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.3 s1 documented · field
    Each GDPval task consists of two primary components: a request (often with reference files) and a deliverable (work product).
6task-origin: the experts' own work productOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B3, B8
  • F12 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶4 s3 documented · field
    Each task is constructed based on actual work product created by an expert professional.
  • F13 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s3 documented · field
    Tasks are constructed from the representative work of industry professionals with an average of 14 years of experience.
7the experts: recruitment, experience, compensationOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8
  • F14 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.2 s2 and s6 documented · field
    Experts were required to have a minimum of 4 years of professional experience in their occupation … Experts were well compensated for their time and experience.
  • F15 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Acknowledgements s2 documented · field
    We are also grateful to our vendor partners for their collaboration and support throughout this research, and extend a special thank you to the industry experts who contributed their time and expertise to GDPval, without whom this work would not have been possible.
8direction of the work: review and quality controlOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8
  • F16 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.4 s1–2 documented · field
    All 1,320 tasks in the full GDPval set went through an iterative review pipeline involving both automated model-based screening and multiple stages of human expert review. Each task received an average of five human reviews (with a minimum of three reviews).
  • F17 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.5.1 ¶2 s2; A.5.2 s3 documented · field
    Experts retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks. … Initially, our researchers manually reviewed all tasks to identify experts who produced consistently high-quality tasks; these individuals were trained and promoted to reviewers.
9grading: blinded pairwise expert comparisonOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8
  • F18 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶1 documented · field
    we conducted blinded expert pairwise comparisons, where experts in the relevant occupation were presented with a request and reference files and asked to rank two or more unlabeled work deliverables.
  • F19 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶2 s1 documented · field
    On average, grading each comparison for the gold subset took over an hour.
  • F20 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1, footnote 2, s1 documented · field
    We aimed to keep comparisons as blind as possible, but model samples may still have been identifiable due to stylistic differences.
10reception: the encyclopediaWikipedia — AI agent benchmark (the GDPval item)2 claims · B10, B9
  • F65 Wikipedia — GDPval (no article), Wikipedia: titles GDPval and GDPVal; article-space search documented · field
    Wikipedia does not have an article with this exact name. … Results 1 – 2 of 2 … AI agent benchmark … Comparison of generative AI models
  • F66 Wikipedia — AI agent benchmark (the GDPval item), 'AI agent benchmark', §Benchmarks, 'Economic' (the GDPval item), s1–3 documented · field
    GDPval: 1,320 tasks evaluating AI agents on real-world knowledge-work deliverables across 44 occupations from the top 9 U.S. GDP-contributing industries. Tasks made by professionals averaging 14 years of experience. The full evaluation is by blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average.
11the automated grader and its limitsOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8
  • F21 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶3 s2 documented · field
    Although limited, the automated grader is faster and cheaper than expert grading, and achieves 66% agreement with human expert graders, only 5% below human expert inter-rating agreement of 71%.
  • F22 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.6.2 ¶2 s1–2 documented · field
    Our automated grader, based on GPT-5-high, shows lower correlation with human expert graders when assessing outputs from capable OpenAI models. This aligns with empirical evidence that models often favor their own responses
  • F23 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Cost' documented · field
    Constructing and running our evaluation is expensive, particularly with industry expert graders. For this reason, we make an automated grader proxy available, but do not consider it a full substitute for industry expert graders.
12the open gold subset and the held-out setOpenAI — gdpval: evaluating ai model performance - OpenAI5 claims · B10, B5, B8
  • F43 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §4 s1 and s3 documented · field
    We open-source the prompts and reference files in our 220-task gold subset. … Please note that the tasks in the open sourced set have been scrubbed of information that could be used to identify the expert who wrote the task.
  • F44 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.6.3 s1 documented · field
    In the open-source set we mark 12 out of 220 tasks as ungradable due to limitations of the automated grader.
  • F57 Hugging Face — openai/gdpval · Datasets at Hugging Face, dataset card documented · field
    Dataset for GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. … 220 real-world knowledge tasks across 44 occupations. … Each task consists of a text prompt and a set of supporting reference files.
  • F58 Hugging Face — openai/gdpval · Datasets at Hugging Face, dataset viewer, columns and first row documented · field
    deliverable_files listlengths 0 6 … rubric_pretty stringlengths 1.15k 15.8k … "author_type": "human"
  • F67 Wikipedia — AI agent benchmark (the GDPval item), 'AI agent benchmark', the GDPval item, s4–6 documented · field
    A gold subset of 220 tasks is publicly available. The public set can be graded by a finetuned version of GPT-5. The held-out set of 1,100 tasks is only available internally in OpenAI, and external models cannot be submitted for grading on this set.
13derivative: the Inspect Evals implementationGitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages3 claims · B7
  • F60 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶1 s3–4; ¶2 documented · field
    The GDPval full set includes 1,320 specialized tasks (220 in the gold open-sourced set), each meticulously crafted and vetted by experienced professionals with over 14 years of experience on average from these fields. Every task is based on real work products, such as a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan. The version here only includes the 220 tasks in the gold open-sourced set.
  • F61 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶3; Judging ¶1 documented · field
    Scoring this eval involves saving the results as their own dataset, uploading it to huggingface, and submitting it to the OpenAI auto grader. … Note that OpenAI provides their automated grader model for free.
  • F62 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Replication Notes documented · field
    Average Score: 47.3% ± 6.2% … We note that the official score reported in the official GDPval leaderboard by OpenAI reports a lower score of 31.9% (wins only) and 34.6% (wins and ties).
14positioning: capability measured ahead of adoptionOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B8
  • F4 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶1 s3–5 documented · field
    Therefore, while informative when available, these methods are lagging indicators of AI impacts. We consider an alternate method for understanding the potential economic impacts of AI: directly measuring AI model capabilities. AI capability evaluations can provide clearer, more directly attributable evidence about model abilities, allowing us to assess economic relevance ahead of widespread adoption.
  • F5 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Representative breadth', s3 documented · field
    We also build on production AI usage analyses (e.g., Tamkin et al. (2024) ; Chatterji et al. (2025) ; Appel et al. (2025) ) to cover areas where model adoption is still emerging.
  • F46 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §6, last sentence documented · field
    We hope this work contributes to the science of tracking model progress, so that we have better data to assess the social impacts of AI models.
15against academic-test benchmarksarXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1)1 claim · B8
  • F6 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Realism' documented · field
    Unlike AI benchmarks in the style of an academic test that focus on reasoning difficulty (e.g., Phan et al. (2025) ; Hendrycks et al. (2020) ; Rein et al. (2023) ; Liu et al. (2023) ), tasks are based on actual work product from industry experts, validated through multiple rounds and review, and tied to time and cost required for completion.
16headline finding: approaching parity with expertsOpenAI — gdpval: evaluating ai model performance - OpenAI3 claims · B3, B8
  • F24 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 5 caption documented · field
    On human pairwise comparisons, models are beginning to approach parity with industry experts on the GDPval gold subset.
  • F25 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1 ¶1, last two sentences documented · field
    on the GDPval gold subset, 47.6% of deliverables by Claude Opus 4.1 were graded as better than (wins) or as good as (ties) the human deliverable. Model deliverables outperformed or matched expert humans’ deliverables in just over half the tasks.
  • F56 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s4–5 documented · field
    We find that frontier model performance on GDPval is improving roughly linearly over time, and that the current best frontier models are approaching industry experts in deliverable quality. We analyze the potential for frontier models, when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts.
17trend: roughly linear improvementOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B8
  • F26 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 6 caption documented · field
    Performance of OpenAI frontier models increased roughly linearly over time on the GDPval gold subset.
18models compared: aesthetics and accuracyOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B8
  • F27 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1 ¶1 documented · field
    Claude Opus 4.1 was the best performing model on the GDPval gold subset, excelling in particular on aesthetics (e.g., document formatting, slide layout), while GPT-5 excelled in particular on accuracy (e.g., carefully following instructions, performing correct calculations)
19why models lose: instructions, and the severity of failuresOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8
  • F28 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 8 caption documented · field
    Across models, experts most often preferred the human deliverable because models failed to fully follow instructions on GDPval tasks.
  • F29 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.6 ¶2 s2 documented · field
    Another roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic).
20speed and cost: the expert's review and redoOpenAI — gdpval: evaluating ai model performance - OpenAI5 claims · B2, B8
  • F30 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.2 ¶1 s4–5 documented · field
    Here, an expert human samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats. If no satisfactory output is obtained, the human completes the task themselves.
  • F31 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1 Table 2, row gpt-5, and the sentence after it documented · field
    gpt-5 39.0% 90x 1.12x 1.39x 474x 1.18x 1.63x … When incorporating time to review and redo work, the payoff from using a model shrinks.
  • F32 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, after Table 2, s2–4 documented · field
    We do not include consideration of the time taken to review a human professional deliverable, although this would commonly occur for tasks in GDPval (either self-review of the professional’s own work or review by a supervisor of a team member’s work). We also do not include the possibility that the human deliverable is also undesirable. One further limitation of this analysis is that it does not capture the cost of catastrophic mistakes, which can be disproportionately expensive in some domains.
  • F33 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, footnote 9 documented · field
    We are over-penalizing the model here, because the win rate after each completion likely goes up (because the professional will adjust the prompt to the model to fix the errors) and the review time also goes down as the professional gets more comfortable with the task.
  • F54 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, 'Publication' block, the paragraph headed 'Abstract' documented · field
    Initial results indicate that frontier AI models are approaching the quality of work produced by human experts, with models able to perform certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts.
21valuing the experts' timeOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8
  • F34 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, footnotes 6 and 7 documented · field
    Because times were self-reported, it is possible that experts under-estimated or over-estimated time taken … Because our experts were recruited specifically for being highly experienced in their field, these wage estimates likely underestimate their true market cost.
  • F35 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.3 s4 documented · field
    Each task’s dollar value was estimated by multiplying the average estimated completion time by median hourly wages for the corresponding occupation from OEWS data
22task lengthOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B8
  • F36 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Long-horizon difficulty' documented · field
    Tasks require an average of 7 hours of work for an expert professional to complete. On the high end, tasks span up to multiple weeks of work.
  • F37 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.5 s2 documented · field
    Win rates are highest for shorter tasks (0–2 hours) and decline steadily as completion time increases.
23reasoning, context and scaffoldingOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B3
  • F38 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s6 documented · field
    We also demonstrate that increased reasoning effort, increased task context, and increased scaffolding improves model performance on GDPval.
24stated limits: size, scope, one-shot specificationOpenAI — gdpval: evaluating ai model performance - OpenAI4 claims · B8
  • F39 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Dataset size' documented · field
    The GDPval full set currently consists of only 44 occupations and 30 total tasks per occupation. It is therefore a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks.
  • F40 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Focus on self-contained knowledge work' s2–3 documented · field
    Manual labor and physical tasks are not included in the current version. Moreover, tasks that involve extensive tacit knowledge, access to personally identifiable information, use of proprietary software tools, or communication between individuals are out of scope for the current evaluation.
  • F41 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Tasks are precisely-specified and one-shot, not interactive' (heading and s1) documented · field
    Tasks are precisely-specified and one-shot, not interactive: … For GDPval, we provide the full context of the task in the prompt, but in real life it often takes effort to figure out the full context of a task and understand what to work on.
  • F42 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.7 ¶2 s2; ¶3 documented · field
    We collected and graded GPT-5 completions with expert human graders and found the model’s performance was worse on under-specified prompts. … As a note: this experiment was run on an earlier version of the GDPval gold subset, and therefore the observed win rates do not match those in the main text of the paper.
25win rate as metric: the human baseline replaceableOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B8
  • F45 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'No “upper limit”' s2 documented · field
    Currently, we compare model outputs against a human expert baseline, but we could replace our baseline with increasingly strong models over time and keep evaluating.
26uptake: the dataset's downloads and models trained on itHugging Face — openai/gdpval · Datasets at Hugging Face1 claim · B5
  • F59 Hugging Face — openai/gdpval · Datasets at Hugging Face, 'Downloads last month'; 'Models trained or fine-tuned on openai/gdpval' documented · field
    Downloads last month 271,657 … Browse 47 models trained on this dataset
27derivative: GDPval-AA (Artificial Analysis)Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis3 claims · B2
  • F52 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, header ¶1 documented · field
    GDPval-AA v2.1 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.
  • F53 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, FAQ 'How does GDPval-AA v2.1 decide which model did better?' documented · field
    For each matchup, the two outputs are anonymized and an LLM judge picks a winner.
  • F55 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, leaderboard summary documented · field
    Claude Opus 5.5 (Max, Default Fallback) scores the highest on GDPval-AA v2.1 with a score of 1866
28the operator and the authorsOpenAI — gdpval: evaluating ai model performance - OpenAI2 claims · B3, B8
  • F50 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, title and authors; B8 author footnote documented · field
    View a PDF of the paper titled GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, by Tejal Patwardhan and 18 other authors
  • F51 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), author block and footnote documented · field
    Amelia Glaese Jerry Tworek OpenAI … Equal contribution. Correspondence to tejal@openai.com.
29versions and lifecycleOpenAI — gdpval: evaluating ai model performance - OpenAI1 claim · B8
  • F47 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶4 s6; §5 'Dataset size' s3 documented · field
    Future GDPval iterations will incorporate greater breadth, realism, interactivity, and contextual nuance. … We are expanding the dataset size.
Expansion · The field's ontology — L_F(B)16 sentences · every one sourced below

GDPval

GDPval is, in its authors' words, "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), introduced in its first version in a PDF created on 25 September 2025 and on arXiv on 5 October 2025 under CC BY 4.0 (OpenAI's PDF; the paper), and called by its card and a video an evaluation of model performance on real-world tasks (OpenAI, card only; Vivek Haldar, title only). Its full set is 1,320 tasks across 44 occupations covering most O*NET and BLS Work Activities in the top 9 sectors of U.S. GDP, occupations that earn $3T a year, the sectors chosen at over 5% of GDP and the occupations as predominantly digital by GPT-4o's classification at 60% (arXiv abstract; the paper).

Each task is a request with reference files and a deliverable, built from the actual work product of professionals averaging 14 years' experience, who were "well compensated", with vendor partners thanked (the paper; arXiv abstract). Every task went through model screening and an average of five expert reviews, the experts keeping final responsibility (the paper).

Grading is by blinded pairwise expert comparison of deliverables, over an hour each, the blinding incomplete because model samples may have been identifiable by style (the paper; Wikipedia). An experimental automated grader based on GPT-5-high agrees with experts 66% of the time against 71% between experts, agrees less on capable OpenAI models' outputs, is not a full substitute for expert grading and cannot grade 12 of the 220 open tasks (the paper); OpenAI provides it free (Inspect Evals).

The authors present direct capability measurement as an alternative to lagging adoption indicators, building on usage analyses, set against academic-test benchmarks, toward better data on AI's social impacts (the paper). Models are "beginning to approach parity with industry experts": 47.6% of Claude Opus 4.1's deliverables were graded as good as or better than the human's, OpenAI's models improved roughly linearly, Claude led on aesthetics and GPT-5 on accuracy (the paper; arXiv abstract). Experts preferred human work most often because models did not fully follow instructions, and about 29% of GPT-5's losses were rated bad or catastrophic (the paper).

With an expert reviewing and redoing model output, GPT-5's 90x naive speed falls to 1.12x tried once and 1.39x tried n times, "the payoff from using a model shrinks", an estimate that omits the review of human work and catastrophic costs, over-penalizes the model by its authors' account, and values time at median wages that likely underestimate the experts (the paper); Artificial Analysis summarizes it as about 100 times faster (Artificial Analysis). Tasks average 7 hours, win rates fall with task length, and reasoning, context and scaffolding raise performance (the paper; arXiv abstract).

It is "a limited, initial cut", excluding physical, tacit and interpersonal work, its tasks precisely specified and one-shot, models doing worse on under-specified prompts, and its human baseline replaceable by stronger models (the paper).

The 220-task gold subset is open, scrubbed of the experts' identities, on Hugging Face with deliverables and rubrics, downloaded 271,657 times last month; the other 1,100 tasks stay internal to OpenAI (the paper; Hugging Face; Wikipedia). Artificial Analysis's GDPval-AA v2.1 rates models by an LLM judge, Claude Opus 5.5 leading, and Inspect Evals' replication of GPT-5 scored 47.3% against OpenAI's official 31.9% (Artificial Analysis; Inspect Evals).

Its 19 authors are at OpenAI, which plans greater breadth, realism and interactivity (arXiv; the paper). Wikipedia has no article on it (Wikipedia, 2026-10-29).

Provenance — every sentence sourced16 sentences

16 sentences; 67 field claims, 0 archive claims. Modality is the source's own; the prose carries it in its grammar.

1.GDPval is, in its authors' words, "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), introduced in its first version in a PDF created on 25 September 2025 and on arXiv on 5 October 2025 under CC BY 4.0 (OpenAI's PDF; the paper), and called by its card and a video an evaluation of model performance on real-world tasks (OpenAI, card only; Vivek Haldar, title only).6
  • F1 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s1 documented · field
    We introduce GDPval, a benchmark evaluating AI model capabilities on real-world economically valuable tasks.
  • F2 OpenAI — gdpval: evaluating ai model performance - OpenAI, p. 1, §1 ¶2 (the card's own text) documented · field
    Our paper introduces the first version of GDPval, a benchmark evaluating AI model performance on real-world economically valuable tasks.
  • F49 OpenAI — gdpval: evaluating ai model performance - OpenAI, PDF metadata (pdfinfo, saved as B4-info.txt) documented · field
    CreationDate: Thu Sep 25 17:03:48 2025 EDT
  • F48 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), header documented · field
    License: CC BY 4.0 … arXiv:2510.04374v1 [cs.LG] 05 Oct 2025
  • F63 OpenAI — Measuring the performance of our models on real-world tasks | OpenAI, card title and snippet (OpenAI's page unreachable: HTTP 403) documented · field
    GDPval: Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries (threshol...
  • F64 YouTube·Vivek Haldar — GDPval: An AI benchmark for real-world tasks (video, 11:23), card title (video, 11:23; unreachable: HTTP 429) documented · field
    GDPval: An AI benchmark for real-world tasks
2.Its full set is 1,320 tasks across 44 occupations covering most O*NET and BLS Work Activities in the top 9 sectors of U.S. GDP, occupations that earn $3T a year, the sectors chosen at over 5% of GDP and the occupations as predominantly digital by GPT-4o's classification at 60% (arXiv abstract; the paper).5
  • F3 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s2 documented · field
    GDPval covers the majority of U.S. Bureau of Labor Statistics Work Activities for 44 occupations across the top 9 sectors contributing to U.S. GDP (Gross Domestic Product).
  • F7 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Representative breadth', s1 documented · field
    the GDPval full set covers 1,320 tasks across 44 occupations, sourced to cover the majority of Work Activities tracked by O*NET for each occupation
  • F8 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 ¶1 documented · field
    GDPval covers tasks from 9 sectors and 44 occupations that collectively earn $3T annually.
  • F9 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 step 1 documented · field
    Selected sectors that contribute over 5% to US GDP as determined by Q2 2024 Value Added by Industry as a Percentage of Gross Domestic Product
  • F10 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.1 step 2 documented · field
    we prompted GPT-4o to classify each task as digital or non-digital, and then classified the overall occupation as digital if at least 60% of its component tasks were digital.
3.Each task is a request with reference files and a deliverable, built from the actual work product of professionals averaging 14 years' experience, who were "well compensated", with vendor partners thanked (the paper; arXiv abstract).5
  • F11 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.3 s1 documented · field
    Each GDPval task consists of two primary components: a request (often with reference files) and a deliverable (work product).
  • F12 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶4 s3 documented · field
    Each task is constructed based on actual work product created by an expert professional.
  • F13 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s3 documented · field
    Tasks are constructed from the representative work of industry professionals with an average of 14 years of experience.
  • F14 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.2 s2 and s6 documented · field
    Experts were required to have a minimum of 4 years of professional experience in their occupation … Experts were well compensated for their time and experience.
  • F15 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Acknowledgements s2 documented · field
    We are also grateful to our vendor partners for their collaboration and support throughout this research, and extend a special thank you to the industry experts who contributed their time and expertise to GDPval, without whom this work would not have been possible.
4.Every task went through model screening and an average of five expert reviews, the experts keeping final responsibility (the paper).2
  • F16 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.4 s1–2 documented · field
    All 1,320 tasks in the full GDPval set went through an iterative review pipeline involving both automated model-based screening and multiple stages of human expert review. Each task received an average of five human reviews (with a minimum of three reviews).
  • F17 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.5.1 ¶2 s2; A.5.2 s3 documented · field
    Experts retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks. … Initially, our researchers manually reviewed all tasks to identify experts who produced consistently high-quality tasks; these individuals were trained and promoted to reviewers.
5.Grading is by blinded pairwise expert comparison of deliverables, over an hour each, the blinding incomplete because model samples may have been identifiable by style (the paper; Wikipedia).4
  • F18 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶1 documented · field
    we conducted blinded expert pairwise comparisons, where experts in the relevant occupation were presented with a request and reference files and asked to rank two or more unlabeled work deliverables.
  • F19 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶2 s1 documented · field
    On average, grading each comparison for the gold subset took over an hour.
  • F20 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1, footnote 2, s1 documented · field
    We aimed to keep comparisons as blind as possible, but model samples may still have been identifiable due to stylistic differences.
  • F66 Wikipedia — AI agent benchmark (the GDPval item), 'AI agent benchmark', §Benchmarks, 'Economic' (the GDPval item), s1–3 documented · field
    GDPval: 1,320 tasks evaluating AI agents on real-world knowledge-work deliverables across 44 occupations from the top 9 U.S. GDP-contributing industries. Tasks made by professionals averaging 14 years of experience. The full evaluation is by blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average.
6.An experimental automated grader based on GPT-5-high agrees with experts 66% of the time against 71% between experts, agrees less on capable OpenAI models' outputs, is not a full substitute for expert grading and cannot grade 12 of the 220 open tasks (the paper); OpenAI provides it free (Inspect Evals).5
  • F21 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.5 ¶3 s2 documented · field
    Although limited, the automated grader is faster and cheaper than expert grading, and achieves 66% agreement with human expert graders, only 5% below human expert inter-rating agreement of 71%.
  • F22 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.6.2 ¶2 s1–2 documented · field
    Our automated grader, based on GPT-5-high, shows lower correlation with human expert graders when assessing outputs from capable OpenAI models. This aligns with empirical evidence that models often favor their own responses
  • F23 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Cost' documented · field
    Constructing and running our evaluation is expensive, particularly with industry expert graders. For this reason, we make an automated grader proxy available, but do not consider it a full substitute for industry expert graders.
  • F44 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.6.3 s1 documented · field
    In the open-source set we mark 12 out of 220 tasks as ungradable due to limitations of the automated grader.
  • F61 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶3; Judging ¶1 documented · field
    Scoring this eval involves saving the results as their own dataset, uploading it to huggingface, and submitting it to the OpenAI auto grader. … Note that OpenAI provides their automated grader model for free.
7.The authors present direct capability measurement as an alternative to lagging adoption indicators, building on usage analyses, set against academic-test benchmarks, toward better data on AI's social impacts (the paper).4
  • F4 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶1 s3–5 documented · field
    Therefore, while informative when available, these methods are lagging indicators of AI impacts. We consider an alternate method for understanding the potential economic impacts of AI: directly measuring AI model capabilities. AI capability evaluations can provide clearer, more directly attributable evidence about model abilities, allowing us to assess economic relevance ahead of widespread adoption.
  • F5 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Representative breadth', s3 documented · field
    We also build on production AI usage analyses (e.g., Tamkin et al. (2024) ; Chatterji et al. (2025) ; Appel et al. (2025) ) to cover areas where model adoption is still emerging.
  • F6 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Realism' documented · field
    Unlike AI benchmarks in the style of an academic test that focus on reasoning difficulty (e.g., Phan et al. (2025) ; Hendrycks et al. (2020) ; Rein et al. (2023) ; Liu et al. (2023) ), tasks are based on actual work product from industry experts, validated through multiple rounds and review, and tied to time and cost required for completion.
  • F46 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §6, last sentence documented · field
    We hope this work contributes to the science of tracking model progress, so that we have better data to assess the social impacts of AI models.
8.Models are "beginning to approach parity with industry experts": 47.6% of Claude Opus 4.1's deliverables were graded as good as or better than the human's, OpenAI's models improved roughly linearly, Claude led on aesthetics and GPT-5 on accuracy (the paper; arXiv abstract).5
  • F24 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 5 caption documented · field
    On human pairwise comparisons, models are beginning to approach parity with industry experts on the GDPval gold subset.
  • F25 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1 ¶1, last two sentences documented · field
    on the GDPval gold subset, 47.6% of deliverables by Claude Opus 4.1 were graded as better than (wins) or as good as (ties) the human deliverable. Model deliverables outperformed or matched expert humans’ deliverables in just over half the tasks.
  • F26 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 6 caption documented · field
    Performance of OpenAI frontier models increased roughly linearly over time on the GDPval gold subset.
  • F27 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.1 ¶1 documented · field
    Claude Opus 4.1 was the best performing model on the GDPval gold subset, excelling in particular on aesthetics (e.g., document formatting, slide layout), while GPT-5 excelled in particular on accuracy (e.g., carefully following instructions, performing correct calculations)
  • F56 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s4–5 documented · field
    We find that frontier model performance on GDPval is improving roughly linearly over time, and that the current best frontier models are approaching industry experts in deliverable quality. We analyze the potential for frontier models, when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts.
9.Experts preferred human work most often because models did not fully follow instructions, and about 29% of GPT-5's losses were rated bad or catastrophic (the paper).2
  • F28 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), Figure 8 caption documented · field
    Across models, experts most often preferred the human deliverable because models failed to fully follow instructions on GDPval tasks.
  • F29 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.6 ¶2 s2 documented · field
    Another roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic).
10.With an expert reviewing and redoing model output, GPT-5's 90x naive speed falls to 1.12x tried once and 1.39x tried n times, "the payoff from using a model shrinks", an estimate that omits the review of human work and catastrophic costs, over-penalizes the model by its authors' account, and values time at median wages that likely underestimate the experts (the paper); Artificial Analysis summarizes it as about 100 times faster (Artificial Analysis).7
  • F30 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §3.2 ¶1 s4–5 documented · field
    Here, an expert human samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats. If no satisfactory output is obtained, the human completes the task themselves.
  • F31 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1 Table 2, row gpt-5, and the sentence after it documented · field
    gpt-5 39.0% 90x 1.12x 1.39x 474x 1.18x 1.63x … When incorporating time to review and redo work, the payoff from using a model shrinks.
  • F32 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, after Table 2, s2–4 documented · field
    We do not include consideration of the time taken to review a human professional deliverable, although this would commonly occur for tasks in GDPval (either self-review of the professional’s own work or review by a supervisor of a team member’s work). We also do not include the possibility that the human deliverable is also undesirable. One further limitation of this analysis is that it does not capture the cost of catastrophic mistakes, which can be disproportionately expensive in some domains.
  • F33 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, footnote 9 documented · field
    We are over-penalizing the model here, because the win rate after each completion likely goes up (because the professional will adjust the prompt to the model to fix the errors) and the review time also goes down as the professional gets more comfortable with the task.
  • F34 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.1, footnotes 6 and 7 documented · field
    Because times were self-reported, it is possible that experts under-estimated or over-estimated time taken … Because our experts were recruited specifically for being highly experienced in their field, these wage estimates likely underestimate their true market cost.
  • F35 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §2.3 s4 documented · field
    Each task’s dollar value was estimated by multiplying the average estimated completion time by median hourly wages for the corresponding occupation from OEWS data
  • F54 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, 'Publication' block, the paragraph headed 'Abstract' documented · field
    Initial results indicate that frontier AI models are approaching the quality of work produced by human experts, with models able to perform certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts.
11.Tasks average 7 hours, win rates fall with task length, and reasoning, context and scaffolding raise performance (the paper; arXiv abstract).3
  • F36 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'Long-horizon difficulty' documented · field
    Tasks require an average of 7 hours of work for an expert professional to complete. On the high end, tasks span up to multiple weeks of work.
  • F37 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.5 s2 documented · field
    Win rates are highest for shorter tasks (0–2 hours) and decline steadily as completion time increases.
  • F38 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, Abstract s6 documented · field
    We also demonstrate that increased reasoning effort, increased task context, and increased scaffolding improves model performance on GDPval.
12.It is "a limited, initial cut", excluding physical, tacit and interpersonal work, its tasks precisely specified and one-shot, models doing worse on under-specified prompts, and its human baseline replaceable by stronger models (the paper).5
  • F39 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Dataset size' documented · field
    The GDPval full set currently consists of only 44 occupations and 30 total tasks per occupation. It is therefore a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks.
  • F40 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Focus on self-contained knowledge work' s2–3 documented · field
    Manual labor and physical tasks are not included in the current version. Moreover, tasks that involve extensive tacit knowledge, access to personally identifiable information, use of proprietary software tools, or communication between individuals are out of scope for the current evaluation.
  • F41 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §5, 'Tasks are precisely-specified and one-shot, not interactive' (heading and s1) documented · field
    Tasks are precisely-specified and one-shot, not interactive: … For GDPval, we provide the full context of the task in the prompt, but in real life it often takes effort to figure out the full context of a task and understand what to work on.
  • F42 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), A.2.7 ¶2 s2; ¶3 documented · field
    We collected and graded GPT-5 completions with expert human graders and found the model’s performance was worse on under-specified prompts. … As a note: this experiment was run on an earlier version of the GDPval gold subset, and therefore the observed win rates do not match those in the main text of the paper.
  • F45 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1, bullet 'No “upper limit”' s2 documented · field
    Currently, we compare model outputs against a human expert baseline, but we could replace our baseline with increasingly strong models over time and keep evaluating.
13.The 220-task gold subset is open, scrubbed of the experts' identities, on Hugging Face with deliverables and rubrics, downloaded 271,657 times last month; the other 1,100 tasks stay internal to OpenAI (the paper; Hugging Face; Wikipedia).5
  • F43 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §4 s1 and s3 documented · field
    We open-source the prompts and reference files in our 220-task gold subset. … Please note that the tasks in the open sourced set have been scrubbed of information that could be used to identify the expert who wrote the task.
  • F57 Hugging Face — openai/gdpval · Datasets at Hugging Face, dataset card documented · field
    Dataset for GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. … 220 real-world knowledge tasks across 44 occupations. … Each task consists of a text prompt and a set of supporting reference files.
  • F58 Hugging Face — openai/gdpval · Datasets at Hugging Face, dataset viewer, columns and first row documented · field
    deliverable_files listlengths 0 6 … rubric_pretty stringlengths 1.15k 15.8k … "author_type": "human"
  • F59 Hugging Face — openai/gdpval · Datasets at Hugging Face, 'Downloads last month'; 'Models trained or fine-tuned on openai/gdpval' documented · field
    Downloads last month 271,657 … Browse 47 models trained on this dataset
  • F67 Wikipedia — AI agent benchmark (the GDPval item), 'AI agent benchmark', the GDPval item, s4–6 documented · field
    A gold subset of 220 tasks is publicly available. The public set can be graded by a finetuned version of GPT-5. The held-out set of 1,100 tasks is only available internally in OpenAI, and external models cannot be submitted for grading on this set.
14.Artificial Analysis's GDPval-AA v2.1 rates models by an LLM judge, Claude Opus 5.5 leading, and Inspect Evals' replication of GPT-5 scored 47.3% against OpenAI's official 31.9% (Artificial Analysis; Inspect Evals).5
  • F52 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, header ¶1 documented · field
    GDPval-AA v2.1 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.
  • F53 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, FAQ 'How does GDPval-AA v2.1 decide which model did better?' documented · field
    For each matchup, the two outputs are anonymized and an LLM judge picks a winner.
  • F55 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis, leaderboard summary documented · field
    Claude Opus 5.5 (Max, Default Fallback) scores the highest on GDPval-AA v2.1 with a score of 1866
  • F60 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Overview ¶1 s3–4; ¶2 documented · field
    The GDPval full set includes 1,320 specialized tasks (220 in the gold open-sourced set), each meticulously crafted and vetted by experienced professionals with over 14 years of experience on average from these fields. Every task is based on real work products, such as a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan. The version here only includes the 220 tasks in the gold open-sourced set.
  • F62 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages, Replication Notes documented · field
    Average Score: 47.3% ± 6.2% … We note that the official score reported in the official GDPval leaderboard by OpenAI reports a lower score of 31.9% (wins only) and 34.6% (wins and ties).
15.Its 19 authors are at OpenAI, which plans greater breadth, realism and interactivity (arXiv; the paper).3
  • F50 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, title and authors; B8 author footnote documented · field
    View a PDF of the paper titled GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, by Tejal Patwardhan and 18 other authors
  • F51 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), author block and footnote documented · field
    Amelia Glaese Jerry Tworek OpenAI … Equal contribution. Correspondence to tejal@openai.com.
  • F47 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1), §1 ¶4 s6; §5 'Dataset size' s3 documented · field
    Future GDPval iterations will incorporate greater breadth, realism, interactivity, and contextual nuance. … We are expanding the dataset size.
16.Wikipedia has no article on it (Wikipedia, 2026-10-29).1
  • F65 Wikipedia — GDPval (no article), Wikipedia: titles GDPval and GDPVal; article-space search documented · field
    Wikipedia does not have an article with this exact name. … Results 1 – 2 of 2 … AI agent benchmark … Comparison of generative AI models

popup — the compression from the field ledger alone (cards, own page, Wikipedia). With no archive claim admitted, the entry is OpenAI's benchmark at its authors' resolution: the paper in three instances, the operator's dataset page and unreachable post, two third-party derivatives, a video by title, and the encyclopedia's one item. composed 2026-10-29 from the field ledger alone; not frozen

Form ledger — AIO against both arms, at both levels
AIO (T)L_F compressionL_F expansionL_A compressionL_A expansion
body words1361115853352445
distinct claims76767123123
claims per 100 words5.160.411.536.75.0
field claims carried (of 67)—67676767
archive claims0005656
rail7 documents29 lineagesper sentence43 lineagesper sentence
modality shown innonetypographygrammartypographygrammar

Computed from the row at build: words counted in the text; claims from the ledger ids each line or sentence carries. T's claims are the worked example's T1–T9.

The objects

Tthe transcript136 words · 7 cards

register entry gdpval-aio-20261029 OBS-ef64b7d44199 · file as first run sha256 0f182b74c748e78a…

  • T1-1 definition: 'an AI benchmark created by OpenAI to measure how well artificial intelligence models perform on real-world, economically valuable knowledge-work tasks rather than academic or math tests' (three markers)
  • T1-2 scope: 'Covers 1,320 specialized tasks across 44 occupations derived from the top 9 U.S. GDP-contributing sectors.' (two markers)
  • T1-3 task design: 'Built by industry professionals with an average of 14 years of experience using actual work products like legal briefs, spreadsheets, and engineering blueprints.' (two markers)
  • T1-4 evaluation: 'Uses blind head-to-head comparisons by human experts to grade model outputs against human work.' (two markers)
  • T1-5 open source set: 'Includes a gold open-sourced subset of 220 tasks available via Hugging Face Datasets and detailed in the ArXiv Research Paper.' (two markers)
  • T1-6 leaderboards: 'Evaluated by frameworks like Artificial Analysis GDPval-AA v2.1.' (one marker)
  • T1-7 closer: 'Would you like to know more about top-performing models on GDPval, or how to access the open-source dataset?'
Bthe disclosed field10 sources
  • B1 OpenAI — Measuring the performance of our models on real-world tasks | OpenAI unreachable: HTTP 403 at 2026-10-29T22:31:14Z; HTTP 403 at 2026-10-29T21:49:14Z; card title and snippet stand
  • B2 Artificial Analysis — GDPval-AA v2.1 Benchmark - Artificial Analysis card; B2.html sha256 e320651e4d74e7af, HTTP 200 at 2026-10-29T22:31:16Z
  • B3 arXiv — [2510.04374] GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks card; B3.html sha256 904393337a8b91ee, HTTP 200 at 2026-10-29T22:31:17Z
  • B4 OpenAI — gdpval: evaluating ai model performance - OpenAI card; B4.pdf sha256 37c7604bc9da68cd, HTTP 200 at 2026-10-29T22:31:21Z
  • B5 Hugging Face — openai/gdpval · Datasets at Hugging Face card; B5.html sha256 1de40771dafd9ab6, HTTP 200 at 2026-10-29T22:31:21Z
  • B6 YouTube·Vivek Haldar — GDPval: An AI benchmark for real-world tasks (video, 11:23) unreachable: HTTP 200 at 2026-10-29T22:32:00Z; HTTP 429 at 2026-10-29T22:32:11Z; title only
  • B7 GitHub Pages documentation — GDPval – Inspect Evals - GitHub Pages card; B7.html sha256 db6622b4af380504, HTTP 200 at 2026-10-29T22:31:48Z
  • B8 arXiv — GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks (HTML, v1) own page (census copy, read 2026-10-29); B8.html sha256 1d4c4f2494000318, HTTP 200 at 2026-10-29T21:56:25Z
  • B9 Wikipedia — GDPval (no article) wikipedia: HTTP 404; HTTP 404; HTTP 404; HTTP 429; HTTP 200; B9.txt sha256 13e34e1adcf88d98
  • B10 Wikipedia — AI agent benchmark (the GDPval item) wikipedia: HTTP 200; HTTP 200; B10.txt sha256 ec6893fc1c953a16
Ethe two entities and their relationρ = extends · equal case

E_A, the archive's entity: the same benchmark, in the Labor Vector census record and the archive's deposits: a task benchmark at the intersections of labor performed with AI and labor producing AI, whose task-origin is the experts' own work product and whose tasks the census reads as fixed commissions, meeting DSL in its own terms of win rate and user labor drag in its review-and-redo accounting (census record); a member of the class of one-shot commissions on which the archive's directionality is identified (#771), of benchmarks that measure standardized tasks, means and outputs within a closed ontology (#482, #48, #637, #855, #199, #778, #1616, #1540), and of the evaluation labor the archive reads as ground-truth or variance-generative, extracted, and a class relation rather than a wage (#163, #623, #308, #533, #1545); built and graded by its provider (census record); whose composition at its own name stops at its specification, before its results (Capture Registry, 2026-10-29) LV-01 LV-03 LV-06 LV-17 LV-21 LV-24 LV-25 O771-01 O771-04 O771-05 O855-01 O1616-01 O48-01 O163-01 O623-01 O1545-01 CAP-1-07; defining papers #771, #855, #199, #778, #1616, #1540, #482, #48, #637, #163, #623, #308, #533, #1545

E_F, the field's entity (type C — a conventional reading holds it (public entity: a task benchmark, its dataset and the paper that presents it)): OpenAI's benchmark (Patwardhan and 18 others; PDF on OpenAI's CDN 25 September 2025, arXiv:2510.04374 v1 5 October 2025, CC BY 4.0) evaluating AI model capabilities on real-world economically valuable tasks: 1,320 tasks across 44 predominantly digital occupations in the 9 sectors contributing most to U.S. GDP, each a request with reference files and a deliverable built from the actual work product of professionals averaging 14 years' experience ('well compensated', no rate given), reviewed by experts and graded by blinded pairwise expert comparison of model and human deliverables (blinding incomplete), with an experimental automated grader based on GPT-5-high that agrees less with experts on OpenAI's own models; its headline finding that models are beginning to approach parity, its accounting that the payoff shrinks once the expert's review and redo are counted, and its stated limits (a limited, initial cut; self-contained computer work; precisely specified, one-shot tasks); a 220-task gold subset open on Hugging Face, the other 1,100 tasks internal to OpenAI; built and graded by its provider, with third-party derivatives (GDPval-AA, Inspect Evals)

ρ = extends: The archive reads GDPval as its authors present it and adds, on the field's own object: whether the system's labor fulfils the commission it is graded on, identified on one-shot commissions such as GDPval's; the user labor drag in the review and redo its accounting charges to the model; what benchmark scores measure and leave out; the labor of the professionals who write and grade the tasks, read as evaluation labor and as a class relation; the census's answer, field by field, to who supplies, directs and gains; and the composition layer's stopping, at the name, before the results. LV-17 O771-01 O771-02 O771-03 O771-04 O771-05 O855-01 O1616-01 O163-01 O533-01 O1545-01 LV-24 LV-25 CAP-1-07 CAP-1-08. Guard: ρ = extends claims an addition and no more (§1.2a). It does not make the field's sources wrong at their dates. No archive claim places GDPval's experts in #163's labor types or in the semantic proletariat, and none says that GDPval's win rate fails to measure task fulfilment: those are open questions (P.open), since no source states them. Where field sources differ among themselves (F54 against F56 and F31; F25's two figures; F62 against OpenAI's official scores), both are composed under P_coh within the one ontology.

The case: equal terms, ontology included (v0.9). The archive reads the field's entity as the entity. The Labor Vector census record describes GDPval from its authors' own pages, each field with its basis ('operator'), and answers for it the census's question of who supplies the labor, who directs it and who receives its gains; the capture record checks the composition against those pages (CAP-1-01). No deposit names GDPval. The deposits bear on its classes and add, on the field's own object, a dimension the field's measure does not carry: whether the system's labor fulfils the commissioned task, identified on exactly the one-shot commissions GDPval's authors say their tasks are (#771); the cost of the user's review and redo as user labor drag (#771), where the census places the paper's accounting; what benchmark scores see and what they close out (#855, #199, #778, #1616, #1540; #482, #48, #637); and the labor that produces AI and how it is paid (#163, #623, #308, #533, #1545). #771 states its own relation to the measures it extends in these terms ('Both are silent on a third dimension'); the census states its reading as kept apart from the resource's own terms (schema_note) and its gap as 'the question DSL measures'. No defining paper contests the field's ontology of the benchmark or supplies one the field lacks: ρ = extends, the equal-terms case (§0.9, §1.2a). The archive's reading of the composition, a benchmark composed at its specification and stopping before its results (CAP-1-07), takes the authors' own findings and limits as given.

#771 Abstract s2: Both are silent on a third dimension: the *direction* in which a synthetic system's labor flows relative to the task its user commissioned.

#771 §9 'Falsification, negative cases, and method', 'Identification condition': These metrics yield a single reproducible value only on transcripts where the task vector is *exogenous and fixed* — a one-shot commission.

#855 § 'Every Factor Removes Friction', ¶ 'Benchmarks measure means, not tails', s1–3: Benchmarks measure means, not tails.

#1545 §2 'Formal class-position test', ¶ after the formula, s2–4: This is deliberately a **relation**, not an essence.

#LV census.json rows[slug == 'openai-gdpval'].lv_intersection: The archive's reading: GDPval grades a model's deliverable against an expert's deliverable on a task an expert wrote, a fixed commission, which meets DSL at the question of whether the system's labor fulfils the commissioned task, measured in its own terms of win rate. Its 'try n times, then fix it yourself' accounting counts the expert's review and redo time against the model, which is where it meets user labor drag; task-origin is the experts' own work product, as the paper states it.

#LV questions.json questions[id == 'working-with-ai'].gap: None records, from the commissioning side, whether the system's labor advanced the task the user set, or who receives the time it saves. That is the question DSL measures.

#capture-registry gdpval-aio-20261029, analysis s1: A capability benchmark is composed at its specification and stops before its results.

τthe translation: every field claim carried, beside its statement in the archive's ontology67 field claims · 29 rows
field claimsreceivedreceiving texttranslationarchive
F1, F2, F64F1 (B3, Abstract s1): "We introduce GDPval, a benchmark evaluating AI model capabilities on real-world economically valuable tasks."; F2 (B4, p. 1, §1 ¶2 (the card's own text)): "Our paper introduces the first version of GDPval, a benchmark evaluating AI model performance on real-world economically valuable tasks."; F64 (B6, card title (video, 11:23; unreachable: HTTP 429)): "GDPval: An AI benchmark for real-world tasks"—untranslated: composed as received beside LV-01, LV-04, LV-03, LV-24LV-01, LV-04, LV-03, LV-24
F3, F7, F8, F63F3 (B3, Abstract s2): "GDPval covers the majority of U.S. Bureau of Labor Statistics Work Activities for 44 occupations across the top 9 sectors contributing to U.S. GDP (Gross Domestic Product)."; F7 (B8, §1, bullet 'Representative breadth', s1): "the GDPval full set covers 1,320 tasks across 44 occupations, sourced to cover the majority of Work Activities tracked by O*NET for each occupation"; F8 (B8, §2.1 ¶1): "GDPval covers tasks from 9 sectors and 44 occupations that collectively earn $3T annually."; F63 (B1, card title and snippet (OpenAI's page unreachable: HTTP 403)): "GDPval: Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries (threshol..."—untranslated: composed as received beside LV-10, LV-12, LV-11LV-10, LV-12, LV-11
F4, F5, F46F4 (B8, §1 ¶1 s3–5): "Therefore, while informative when available, these methods are lagging indicators of AI impacts. We consider an alternate method for understanding the potential economic impacts of AI: directly measuring AI model capabilities. AI capability evaluations can provide clearer, more directly attributable evidence about model abilities, allowing us to assess economic relevance ahead of widespread adoption."; F5 (B8, §1, bullet 'Representative breadth', s3): "We also build on production AI usage analyses (e.g., Tamkin et al. (2024) ; Chatterji et al. (2025) ; Appel et al. (2025) ) to cover areas where model adoption is still emerging."; F46 (B8, §6, last sentence): "We hope this work contributes to the science of tracking model progress, so that we have better data to assess the social impacts of AI models."—untranslated: composed as received beside LV-22LV-22
F6F6 (B8, §1, bullet 'Realism'): "Unlike AI benchmarks in the style of an academic test that focus on reasoning difficulty (e.g., Phan et al. (2025) ; Hendrycks et al. (2020) ; Rein et al. (2023) ; Liu et al. (2023) ), tasks are based on actual work product from industry experts, validated through multiple rounds and review, and tied to time and cost required for completion."—untranslated: composed as received beside O482-01, O48-01, O637-01O482-01, O48-01, O637-01
F9, F10F9 (B8, §2.1 step 1): "Selected sectors that contribute over 5% to US GDP as determined by Q2 2024 Value Added by Industry as a Percentage of Gross Domestic Product"; F10 (B8, §2.1 step 2): "we prompted GPT-4o to classify each task as digital or non-digital, and then classified the overall occupation as digital if at least 60% of its component tasks were digital."—untranslated: composed as received beside LV-10, LV-12, LV-11LV-10, LV-12, LV-11
F11F11 (B8, §2.3 s1): "Each GDPval task consists of two primary components: a request (often with reference files) and a deliverable (work product)."—untranslated: composed as received beside LV-05, LV-06LV-05, LV-06
F12, F13F12 (B8, §1 ¶4 s3): "Each task is constructed based on actual work product created by an expert professional."; F13 (B3, Abstract s3): "Tasks are constructed from the representative work of industry professionals with an average of 14 years of experience."—untranslated: composed as received beside LV-05, LV-06LV-05, LV-06
F14, F15F14 (B8, §2.2 s2 and s6): "Experts were required to have a minimum of 4 years of professional experience in their occupation … Experts were well compensated for their time and experience."; F15 (B8, Acknowledgements s2): "We are also grateful to our vendor partners for their collaboration and support throughout this research, and extend a special thank you to the industry experts who contributed their time and expertise to GDPval, without whom this work would not have been possible."—untranslated: composed as received beside LV-07LV-07
F16, F17F16 (B8, §2.4 s1–2): "All 1,320 tasks in the full GDPval set went through an iterative review pipeline involving both automated model-based screening and multiple stages of human expert review. Each task received an average of five human reviews (with a minimum of three reviews)."; F17 (B8, A.5.1 ¶2 s2; A.5.2 s3): "Experts retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks. … Initially, our researchers manually reviewed all tasks to identify experts who produced consistently high-quality tasks; these individuals were trained and promoted to reviewers."—untranslated: composed as received beside LV-08LV-08
F18, F19, F20F18 (B8, §2.5 ¶1): "we conducted blinded expert pairwise comparisons, where experts in the relevant occupation were presented with a request and reference files and asked to rank two or more unlabeled work deliverables."; F19 (B8, §2.5 ¶2 s1): "On average, grading each comparison for the gold subset took over an hour."; F20 (B8, §3.1, footnote 2, s1): "We aimed to keep comparisons as blind as possible, but model samples may still have been identifiable due to stylistic differences."—untranslated: composed as received
F21, F22, F23F21 (B8, §2.5 ¶3 s2): "Although limited, the automated grader is faster and cheaper than expert grading, and achieves 66% agreement with human expert graders, only 5% below human expert inter-rating agreement of 71%."; F22 (B8, A.6.2 ¶2 s1–2): "Our automated grader, based on GPT-5-high, shows lower correlation with human expert graders when assessing outputs from capable OpenAI models. This aligns with empirical evidence that models often favor their own responses"; F23 (B8, §5, 'Cost'): "Constructing and running our evaluation is expensive, particularly with industry expert graders. For this reason, we make an automated grader proxy available, but do not consider it a full substitute for industry expert graders."—untranslated: composed as received beside LV-21LV-21
F24, F25, F56F24 (B8, Figure 5 caption): "On human pairwise comparisons, models are beginning to approach parity with industry experts on the GDPval gold subset."; F25 (B8, §3.1 ¶1, last two sentences): "on the GDPval gold subset, 47.6% of deliverables by Claude Opus 4.1 were graded as better than (wins) or as good as (ties) the human deliverable. Model deliverables outperformed or matched expert humans’ deliverables in just over half the tasks."; F56 (B3, Abstract s4–5): "We find that frontier model performance on GDPval is improving roughly linearly over time, and that the current best frontier models are approaching industry experts in deliverable quality. We analyze the potential for frontier models, when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts."—untranslated: composed as received beside LV-23LV-23
F26F26 (B8, Figure 6 caption): "Performance of OpenAI frontier models increased roughly linearly over time on the GDPval gold subset."—untranslated: composed as received beside LV-23LV-23
F27F27 (B8, §3.1 ¶1): "Claude Opus 4.1 was the best performing model on the GDPval gold subset, excelling in particular on aesthetics (e.g., document formatting, slide layout), while GPT-5 excelled in particular on accuracy (e.g., carefully following instructions, performing correct calculations)"—untranslated: composed as received beside LV-23LV-23
F28, F29F28 (B8, Figure 8 caption): "Across models, experts most often preferred the human deliverable because models failed to fully follow instructions on GDPval tasks."; F29 (B8, A.2.6 ¶2 s2): "Another roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic)."—untranslated: composed as received beside LV-13LV-13
F30, F31, F32, F33, F54F30 (B8, §3.2 ¶1 s4–5): "Here, an expert human samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats. If no satisfactory output is obtained, the human completes the task themselves."; F31 (B8, A.2.1 Table 2, row gpt-5, and the sentence after it): "gpt-5 39.0% 90x 1.12x 1.39x 474x 1.18x 1.63x … When incorporating time to review and redo work, the payoff from using a model shrinks."; F32 (B8, A.2.1, after Table 2, s2–4): "We do not include consideration of the time taken to review a human professional deliverable, although this would commonly occur for tasks in GDPval (either self-review of the professional’s own work or review by a supervisor of a team member’s work). We also do not include the possibility that the human deliverable is also undesirable. One further limitation of this analysis is that it does not capture the cost of catastrophic mistakes, which can be disproportionately expensive in some domains."; F33 (B8, A.2.1, footnote 9): "We are over-penalizing the model here, because the win rate after each completion likely goes up (because the professional will adjust the prompt to the model to fix the errors) and the review time also goes down as the professional gets more comfortable with the task."; F54 (B2, 'Publication' block, the paragraph headed 'Abstract'): "Initial results indicate that frontier AI models are approaching the quality of work produced by human experts, with models able to perform certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts."—untranslated: composed as received beside LV-09LV-09
F34, F35F34 (B8, A.2.1, footnotes 6 and 7): "Because times were self-reported, it is possible that experts under-estimated or over-estimated time taken … Because our experts were recruited specifically for being highly experienced in their field, these wage estimates likely underestimate their true market cost."; F35 (B8, §2.3 s4): "Each task’s dollar value was estimated by multiplying the average estimated completion time by median hourly wages for the corresponding occupation from OEWS data"—untranslated: composed as received beside LV-09LV-09
F36, F37F36 (B8, §1, bullet 'Long-horizon difficulty'): "Tasks require an average of 7 hours of work for an expert professional to complete. On the high end, tasks span up to multiple weeks of work."; F37 (B8, A.2.5 s2): "Win rates are highest for shorter tasks (0–2 hours) and decline steadily as completion time increases."—untranslated: composed as received
F38F38 (B3, Abstract s6): "We also demonstrate that increased reasoning effort, increased task context, and increased scaffolding improves model performance on GDPval."—untranslated: composed as received
F39, F40, F41, F42F39 (B8, §5, 'Dataset size'): "The GDPval full set currently consists of only 44 occupations and 30 total tasks per occupation. It is therefore a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks."; F40 (B8, §5, 'Focus on self-contained knowledge work' s2–3): "Manual labor and physical tasks are not included in the current version. Moreover, tasks that involve extensive tacit knowledge, access to personally identifiable information, use of proprietary software tools, or communication between individuals are out of scope for the current evaluation."; F41 (B8, §5, 'Tasks are precisely-specified and one-shot, not interactive' (heading and s1)): "Tasks are precisely-specified and one-shot, not interactive: … For GDPval, we provide the full context of the task in the prompt, but in real life it often takes effort to figure out the full context of a task and understand what to work on."; F42 (B8, A.2.7 ¶2 s2; ¶3): "We collected and graded GPT-5 completions with expert human graders and found the model’s performance was worse on under-specified prompts. … As a note: this experiment was run on an earlier version of the GDPval gold subset, and therefore the observed win rates do not match those in the main text of the paper."—untranslated: composed as received beside O771-04O771-04
F43, F44, F57, F58, F67F43 (B8, §4 s1 and s3): "We open-source the prompts and reference files in our 220-task gold subset. … Please note that the tasks in the open sourced set have been scrubbed of information that could be used to identify the expert who wrote the task."; F44 (B8, A.6.3 s1): "In the open-source set we mark 12 out of 220 tasks as ungradable due to limitations of the automated grader."; F57 (B5, dataset card): "Dataset for GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. … 220 real-world knowledge tasks across 44 occupations. … Each task consists of a text prompt and a set of supporting reference files."; F58 (B5, dataset viewer, columns and first row): "deliverable_files listlengths 0 6 … rubric_pretty stringlengths 1.15k 15.8k … "author_type": "human""; F67 (B10, 'AI agent benchmark', the GDPval item, s4–6): "A gold subset of 220 tasks is publicly available. The public set can be graded by a finetuned version of GPT-5. The held-out set of 1,100 tasks is only available internally in OpenAI, and external models cannot be submitted for grading on this set."—untranslated: composed as received beside LV-18LV-18
F45F45 (B8, §1, bullet 'No “upper limit”' s2): "Currently, we compare model outputs against a human expert baseline, but we could replace our baseline with increasingly strong models over time and keep evaluating."—untranslated: composed as received
F47F47 (B8, §1 ¶4 s6; §5 'Dataset size' s3): "Future GDPval iterations will incorporate greater breadth, realism, interactivity, and contextual nuance. … We are expanding the dataset size."—untranslated: composed as received beside LV-02, LV-19, LV-11, LV-20, LV-23LV-02, LV-19, LV-11, LV-20, LV-23
F48, F49F48 (B8, header): "License: CC BY 4.0 … arXiv:2510.04374v1 [cs.LG] 05 Oct 2025"; F49 (B4, PDF metadata (pdfinfo, saved as B4-info.txt)): "CreationDate: Thu Sep 25 17:03:48 2025 EDT"—untranslated: composed as received beside LV-02, LV-19, LV-11, LV-20, LV-23LV-02, LV-19, LV-11, LV-20, LV-23
F50, F51F50 (B3, title and authors; B8 author footnote): "View a PDF of the paper titled GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks, by Tejal Patwardhan and 18 other authors"; F51 (B8, author block and footnote): "Amelia Glaese Jerry Tworek OpenAI … Equal contribution. Correspondence to tejal@openai.com."—untranslated: composed as received beside LV-02, LV-19, LV-11, LV-20, LV-23LV-02, LV-19, LV-11, LV-20, LV-23
F52, F53, F55F52 (B2, header ¶1): "GDPval-AA v2.1 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons."; F53 (B2, FAQ 'How does GDPval-AA v2.1 decide which model did better?'): "For each matchup, the two outputs are anonymized and an LLM judge picks a winner."; F55 (B2, leaderboard summary): "Claude Opus 5.5 (Max, Default Fallback) scores the highest on GDPval-AA v2.1 with a score of 1866"—untranslated: composed as received
F59F59 (B5, 'Downloads last month'; 'Models trained or fine-tuned on openai/gdpval'): "Downloads last month 271,657 … Browse 47 models trained on this dataset"—untranslated: composed as received beside LV-18LV-18
F60, F61, F62F60 (B7, Overview ¶1 s3–4; ¶2): "The GDPval full set includes 1,320 specialized tasks (220 in the gold open-sourced set), each meticulously crafted and vetted by experienced professionals with over 14 years of experience on average from these fields. Every task is based on real work products, such as a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan. The version here only includes the 220 tasks in the gold open-sourced set."; F61 (B7, Overview ¶3; Judging ¶1): "Scoring this eval involves saving the results as their own dataset, uploading it to huggingface, and submitting it to the OpenAI auto grader. … Note that OpenAI provides their automated grader model for free."; F62 (B7, Replication Notes): "Average Score: 47.3% ± 6.2% … We note that the official score reported in the official GDPval leaderboard by OpenAI reports a lower score of 31.9% (wins only) and 34.6% (wins and ties)."—untranslated: composed as received beside LV-07, LV-21LV-07, LV-21
F65, F66F65 (B9, Wikipedia: titles GDPval and GDPVal; article-space search): "Wikipedia does not have an article with this exact name. … Results 1 – 2 of 2 … AI agent benchmark … Comparison of generative AI models"; F66 (B10, 'AI agent benchmark', §Benchmarks, 'Economic' (the GDPval item), s1–3): "GDPval: 1,320 tasks evaluating AI agents on real-world knowledge-work deliverables across 44 occupations from the top 9 U.S. GDP-contributing industries. Tasks made by professionals averaging 14 years of experience. The full evaluation is by blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average."—untranslated: composed as received beside LV-25, CAP-1-09LV-25, CAP-1-09

equal-terms case: every field claim composed as received, beside the archive claims it stands with. Reading the received column recovers the field (§5.9).

R7the measure0 of 16 survive

v0.9 §4.10: KO_B sentences surviving in the KO at similarity ≥ 0.95, with positions (computed as apply_v09.py does: difflib.SequenceMatcher ratio, the best KO sentence for each KO_B sentence): v0.9 0 of 16 (positions [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]; composed first under v0.9, so there is no v0.7 object to measure against). Opening-run check (§5.10): no breach.

L_F(B)recomposed from the disclosed field, in the field's ontology585 words

GDPval is, in its authors' words, "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), introduced in its first version in a PDF created on 25 September 2025 and on arXiv on 5 October 2025 under CC BY 4.0 (OpenAI's PDF; the paper), and called by its card and a video an evaluation of model performance on real-world tasks (OpenAI, card only; Vivek Haldar, title only). Its full set is 1,320 tasks across 44 occupations covering most O*NET and BLS Work Activities in the top 9 sectors of U.S. GDP, occupations that earn $3T a year, the sectors chosen at over 5% of GDP and the occupations as predominantly digital by GPT-4o's classification at 60% (arXiv abstract; the paper). Each task is a request with reference files and a deliverable, built from the actual work product of professionals averaging 14 years' experience, who were "well compensated", with vendor partners thanked (the paper; arXiv abstract). Every task went through model screening and an average of five expert reviews, the experts keeping final responsibility (the paper). Grading is by blinded pairwise expert comparison of deliverables, over an hour each, the blinding incomplete because model samples may have been identifiable by style (the paper; Wikipedia). An experimental automated grader based on GPT-5-high agrees with experts 66% of the time against 71% between experts, agrees less on capable OpenAI models' outputs, is not a full substitute for expert grading and cannot grade 12 of the 220 open tasks (the paper); OpenAI provides it free (Inspect Evals). The authors present direct capability measurement as an alternative to lagging adoption indicators, building on usage analyses, set against academic-test benchmarks, toward better data on AI's social impacts (the paper). Models are "beginning to approach parity with industry experts": 47.6% of Claude Opus 4.1's deliverables were graded as good as or better than the human's, OpenAI's models improved roughly linearly, Claude led on aesthetics and GPT-5 on accuracy (the paper; arXiv abstract). Experts preferred human work most often because models did not fully follow instructions, and about 29% of GPT-5's losses were rated bad or catastrophic (the paper). With an expert reviewing and redoing model output, GPT-5's 90x naive speed falls to 1.12x tried once and 1.39x tried n times, "the payoff from using a model shrinks", an estimate that omits the review of human work and catastrophic costs, over-penalizes the model by its authors' account, and values time at median wages that likely underestimate the experts (the paper); Artificial Analysis summarizes it as about 100 times faster (Artificial Analysis). Tasks average 7 hours, win rates fall with task length, and reasoning, context and scaffolding raise performance (the paper; arXiv abstract). It is "a limited, initial cut", excluding physical, tacit and interpersonal work, its tasks precisely specified and one-shot, models doing worse on under-specified prompts, and its human baseline replaceable by stronger models (the paper). The 220-task gold subset is open, scrubbed of the experts' identities, on Hugging Face with deliverables and rubrics, downloaded 271,657 times last month; the other 1,100 tasks stay internal to OpenAI (the paper; Hugging Face; Wikipedia). Artificial Analysis's GDPval-AA v2.1 rates models by an LLM judge, Claude Opus 5.5 leading, and Inspect Evals' replication of GPT-5 scored 47.3% against OpenAI's official 31.9% (Artificial Analysis; Inspect Evals). Its 19 authors are at OpenAI, which plans greater breadth, realism and interactivity (arXiv; the paper). Wikipedia has no article on it (Wikipedia, 2026-10-29).

L_A(B ∪ A)the field and the archive composed in the archive's ontology2445 words

GDPval is, in the Labor Vector census, a benchmark, "an evaluation dataset with an experimental automated grader", whose kind of evidence is a task benchmark at two of the census's intersections, labor performed with AI and labor producing AI, read for who supplies the labor, who directs it and who receives its gains (Labor Vector record); its authors introduce it as "a benchmark evaluating AI model capabilities on real-world economically valuable tasks" (arXiv abstract), in its first version (OpenAI's PDF). OpenAI's own page is reached here only by its card, "Evaluation measuring model performance on 1,320 specialized tasks across 44 knowledge work occupations from 9 U.S. GDP-contributing industries" (OpenAI, card only; the page answered 403), and a video is titled "GDPval: An AI benchmark for real-world tasks" (Vivek Haldar, title only). Of benchmarks as a class the archive holds that benchmarking "tests for predetermined capabilities" (#482), that current benchmarks measure "atomic intelligence: the capacity to solve discrete, well-defined tasks with measurable success criteria" (#48) and that they "measure model performance on standardized tasks" (#637); GDPval's authors set it against "AI benchmarks in the style of an academic test that focus on reasoning difficulty", its tasks "based on actual work product from industry experts" and "tied to time and cost required for completion" (the paper). For its authors adoption, usage and GDP indicators are "lagging indicators of AI impacts", and measuring capabilities directly lets them "assess economic relevance ahead of widespread adoption" (the paper); they build on production usage analyses "to cover areas where model adoption is still emerging" (the paper), the Anthropic Economic Index among them, the census's neighbour (Labor Vector record), and hope for "better data to assess the social impacts of AI models" (the paper). The full set covers 1,320 tasks across 44 occupations, "sourced to cover the majority of Work Activities tracked by O*NET for each occupation" (the paper), in the top 9 sectors of U.S. GDP (arXiv abstract), occupations that "collectively earn $3T annually" (the paper), the sectors chosen at over 5% of GDP and the occupations by total wages and as predominantly digital, which GPT-4o decided by classifying each task, at a 60% threshold (the paper; Labor Vector record); the census counts 30 tasks per occupation and a gold subset of 220 covering 208 O*NET tasks, in U.S. occupations, its period the first version (Labor Vector record). Each task is "a request (often with reference files) and a deliverable (work product)" (the paper), "constructed based on actual work product created by an expert professional" (the paper), from professionals with "an average of 14 years of experience" (arXiv abstract), which the census records as its unit and as whose labor it is: the professionals' work product "recast as tasks for models" (Labor Vector record). The census reads a task an expert wrote as "a fixed commission", at which GDPval meets DSL "at the question of whether the system's labor fulfils the commissioned task, measured in its own terms of win rate" (Labor Vector record); DSL "measures where a synthetic system's labor flows relative to the commissioned task, across five layers" (#771), on the principle that "a system can produce fluent, well-attributed output while having abandoned the user's commissioned direction" (#771). The archive's directionality metrics yield a single reproducible value "only on transcripts where the task vector is exogenous and fixed — a one-shot commission" (#771), and GDPval's authors state their tasks to be "precisely-specified and one-shot, not interactive", with "the full context of the task in the prompt", where "in real life it often takes effort to figure out the full context of a task" (the paper). The experts needed "a minimum of 4 years of professional experience in their occupation" and were "well compensated for their time and experience", no rate given (the paper); the authors thank unnamed "vendor partners" and the experts "without whom this work would not have been possible" (the paper); Inspect Evals describes them as professionals "with over 14 years of experience on average" and the tasks as drawn from work products such as "a legal brief, an engineering blueprint, a customer support conversation, or a nursing care plan" (Inspect Evals); and the census records them as who supplies the labor, as task writers, reviewers and graders, at a rate the paper does not state (Labor Vector record). OpenAI's research team directs the work (Labor Vector record): every task passed "automated model-based screening and multiple stages of human expert review", five reviews on average, researchers reviewed all tasks and promoted experts to reviewers, and the experts "retained final responsibility for task accuracy and completeness; the model did not autonomously alter tasks" (the paper). Of the labor that produces AI the archive distinguishes "ground-truth labor: the labor of telling the model what is already known to be true" from "variance-generative labor", the producing of "new conceptual structures that the model cannot generate from its own distribution" (#163), and holds as a hypothesis that "Ghost work is paid badly; variance-generative labor is not paid at all" (#163). It holds that the labor of "the data labelers performing RLHF" is extracted and its non-compensation felt "materially" (#623), states that in AI-adjacent meaning labor, data labeling among it, "<5% converts to invoice" (#308, no measurement given at the locus), and audits a framework by whether it names each source of semantic labor, "annotators" and "users who re-prompt and re-frame" among them: "Concealed bearing-cost is the first marker of extraction" (#533). For the archive class is a relation and not an essence, so that "A highly paid academic can occupy a proletarian semantic relation to a composition layer", and the data annotator is "A paradigmatic occupational case, not the definition" (#1545). Grading is by "blinded expert pairwise comparisons", experts in the occupation ranking "two or more unlabeled work deliverables" (the paper), each comparison taking "over an hour" (the paper), and the blinding was not complete: "model samples may still have been identifiable due to stylistic differences" (the paper); Wikipedia gives the same design, "blinded expert pairwise comparison against human expert deliverables, each comparison costing 1 human hour on average" (Wikipedia, 'AI agent benchmark'). The experimental automated grader reaches "66% agreement with human expert graders" against 71% between experts (the paper), is "based on GPT-5-high" and "shows lower correlation with human expert graders when assessing outputs from capable OpenAI models", in line with evidence "that models often favor their own responses" (the paper), and is a proxy its authors do not take for "a full substitute for industry expert graders" (the paper); scoring through Inspect Evals runs through "the OpenAI auto grader", and "OpenAI provides their automated grader model for free" (Inspect Evals); the census tags the benchmark "NOT independent of a provider: built and graded by OpenAI" (Labor Vector record). Its headline finding is that "models are beginning to approach parity with industry experts" on the gold subset (the paper), "the current best frontier models are approaching industry experts in deliverable quality" and OpenAI's frontier models improved "roughly linearly over time" (arXiv abstract; the paper); 47.6% of Claude Opus 4.1's deliverables were graded as good as or better than the human deliverable, and the next sentence has model deliverables matching or beating experts' "in just over half the tasks" (the paper), a difference the census leaves unchecked (Labor Vector record), Claude Opus 4.1 leading on aesthetics and GPT-5 on accuracy (the paper). Experts preferred the human deliverable most often "because models failed to fully follow instructions" (the paper), and of GPT-5's losses "roughly 29% of ratings were for bad or catastrophic (with roughly 3% of failures marked as catastrophic)" (the paper), findings the census carries (Labor Vector record). Of what such scores see the archive holds that "Benchmarks measure means, not tails" (#855), that in code they measure functional correctness while solution-space diversity is "the property no benchmark measures" (#199, a hypothesis), that "Output-level evaluation — which is what benchmarks measure — cannot detect reasoning-level failures" (#778), and that a benchmark "begins by closing an ontology: whatever the evaluator has not represented is noise, an invalid action, or invisible to the score" (#1616); "Reality administers no benchmarks; it administers events" (#1540). The authors model savings for an expert who "samples from a model, reviews outputs, and if unsatisfactory, resamples and repeats", completing the task themselves if no output satisfies (the paper); for GPT-5, at a 39.0% win rate, the naive speed figure of 90x becomes 1.12x tried once and 1.39x tried n times, and "When incorporating time to review and redo work, the payoff from using a model shrinks" (the paper). The analysis leaves out the review of human deliverables, the chance that they too are undesirable, and "the cost of catastrophic mistakes" (the paper), and its authors hold that it over-penalizes the model, since the professional adjusts the prompt and reviews faster as the task grows familiar (the paper); its dollar values multiply self-reported completion times by median occupational wages, which "likely underestimate their true market cost" (the paper), and the census records the gains as "not recorded as a distribution", the savings modelled for the expert at median wages (Labor Vector record). Artificial Analysis's page, in a paragraph it heads Abstract under the paper's title, has models performing "certain professional tasks approximately 100 times faster and at a fraction of the cost compared to human experts" (Artificial Analysis), where the paper's own abstract analyses the potential, "when paired with human oversight, to perform GDPval tasks cheaper and faster than unaided experts" (arXiv abstract). The census reads that accounting, "the expert's review and redo time against the model", as where GDPval meets user labor drag (Labor Vector record), the archive's measure of the user's "subsequent semantic labor" redirected "into parsing, locating, and managing the substrate rather than advancing the object", an injury that "leaves no residue" in the artifact (#771) and is identified only by "blind inter-rater coding" (#771); for the archive the erasure of task-origin can occur "upstream of, and invisibly to, any audit of the finished artifact" (#771). Tasks "require an average of 7 hours of work for an expert professional to complete" and reach "multiple weeks" (the paper); win rates are highest on tasks of 0–2 hours and "decline steadily as completion time increases" (the paper), and more reasoning effort, task context and scaffolding improve performance (arXiv abstract). The authors call the full set "a limited, initial cut of knowledge work tasks, not a comprehensive evaluation of all possible occupational tasks" (the paper), leave out of scope "Manual labor and physical tasks", tacit knowledge, personal data, proprietary tools and "communication between individuals" (the paper), and report that GPT-5 did worse on under-specified prompts, in an experiment on "an earlier version of the GDPval gold subset" (the paper); they add that the human baseline itself could go: "we could replace our baseline with increasingly strong models over time and keep evaluating" (the paper). The census keeps those limits (Labor Vector record) and states what the benchmark answers, deliverables compared on "expert-written, precisely specified, one-shot occupational tasks" and the expert time and cost saved "once review and redo are counted", and what it does not: "how work is performed in actual workplaces", "employment or wage outcomes", "who receives the time and cost savings it models" and "the terms and rate at which the professionals were paid" (Labor Vector record), its negative tags holding that deliverable parity "is NOT a finding of job replacement in the paper" (Labor Vector record). The gold subset's prompts and reference files are open-sourced, "scrubbed of information that could be used to identify the expert who wrote the task" (the paper), 12 of the 220 "ungradable due to limitations of the automated grader" (the paper), as the census records (Labor Vector record); the Hugging Face dataset states that "Each task consists of a text prompt and a set of supporting reference files" and serves deliverable files and rubrics beside them, rubric items marked "author_type": "human" (Hugging Face), was downloaded 271,657 times in the last month and lists 47 models trained on it (Hugging Face); the other 1,100 tasks are, in Wikipedia's account, "only available internally in OpenAI", the public set gradable "by a finetuned version of GPT-5" (Wikipedia). Artificial Analysis's GDPval-AA v2.1, a distinct framework "for OpenAI's GDPval dataset", runs models in an agentic loop with Elo ratings "derived from blind pairwise comparisons" in which "an LLM judge picks a winner", its leaderboard led on 2026-10-29 by Claude Opus 5.5 at 1866 (Artificial Analysis); Inspect Evals implements only "the 220 tasks in the gold open-sourced set", and its GPT-5 run scored 47.3% ± 6.2% against 31.9% (wins only) and 34.6% (wins and ties) on "the official GDPval leaderboard by OpenAI" (Inspect Evals). Its authors are Tejal Patwardhan and 18 others (arXiv), of OpenAI, with correspondence to an openai.com address (the paper), the operator the census records, whose own post "answered 403 and is unread" (Labor Vector record); OpenAI's PDF was created on 25 September 2025 (OpenAI's PDF), before arXiv v1 of 5 October 2025 under CC BY 4.0 (the paper), the date the census gives the first version, with the dataset's license not stated (Labor Vector record); future iterations are to add "greater breadth, realism, interactivity, and contextual nuance" and the dataset is being expanded (the paper), and the census records it active, later releases unrecorded, OpenAI's post, the figures, the pay rate and the vendors unchecked (Labor Vector record). At its name on Google's AI Overview, 2026-10-29, the composition gave "the benchmark's maker, scope, construction and grading, and where to find its data", every figure exact against the record, "blind head-to-head comparisons" without the caveat on blinding, and Artificial Analysis's leaderboard set beside GDPval "as an evaluator, not merged with it" (Capture Registry, 2026-10-29); the registry reads it as "composed at its specification" and stopping "before its results", the findings and the limits absent, the experts' recruitment and pay not composed, and the record's labour question carried "as design" while the review time charged against the model is left out (Capture Registry, 2026-10-29). The worked question that names GDPval sets it beside a provider's record of its own users and a classification of one assistant's conversations and finds that none "records, from the commissioning side, whether the system's labor advanced the task the user set, or who receives the time it saves", which "is the question DSL measures" (Labor Vector record); no card cites the record, which landed the same day (Capture Registry, 2026-10-29), and Wikipedia has no article on GDPval, the name standing as one item of 'AI agent benchmark' (Wikipedia, 2026-10-29).

Δthe delta

T against L_F(B) (AIO's ontology at work on its own cards): At 'GDPval' (T1, seven cards: B1–B7, B1 and B6 unreachable; 136 words in the answer body with its closer) the composition gives the benchmark at its specification. Composed from its cards: the definition at the abstract's and the PDF's grain (B3 F1, B4 F2: a benchmark of model performance on economically valuable tasks), with OpenAI as maker (B1, B4, B5); 'rather than academic or math tests' at the coarse grain of the paper's Realism bullet (B4/B8 F6, academic tests of reasoning difficulty; 'math' is in no field source about GDPval); the scope (B3 F3, B4 F7, B1 F63: 1,320 tasks, 44 occupations, the top 9 sectors); the task design (B3 F13, 14 years; B4 F12, actual work product; B7 F60, 'a legal brief, an engineering blueprint'; 'spreadsheets' among the formats of B8 §1); the grading (B4 F18); the open gold subset on Hugging Face and the arXiv paper (B4 F43, B5 F57); and GDPval-AA as a leaderboard (B2 F52). Limit dropped: the blinding caveat in the card's own paper, 'model samples may still have been identifiable due to stylistic differences' (B4/B8 F20), on T1-4's 'blind head-to-head comparisons', as the capture finds (CAP-1-02). Substituted reading: T1-6 'Evaluated by frameworks like Artificial Analysis GDPval-AA v2.1' casts GDPval as the thing evaluated and GDPval-AA beside the expert-graded design; the card gives GDPval-AA as Artificial Analysis's own framework 'for OpenAI's GDPval dataset' that rates models by 'an LLM judge' (B2 F52, F53), the judge not composed; the registry reads the same placement as 'set beside GDPval as an evaluator, not merged with it' (CAP-1-10). Available, not composed: every finding (B3 F56; B4/B8 F24–F29), the review-and-redo accounting and its shrinking payoff (B4 F30, F31–F35), the stated limits (B4 F39–F41; F42), the positioning against lagging adoption indicators (B4 F4, F46), the experts' requirements, pay and vendors (B4 F14, F15; CAP-1-05), the review pipeline (B4 F16), the automated grader and its self-preference limit (B4 F21–F23; B7 F61), the open set's deliverables and rubrics and its downloads (B5 F58, F59), the task length and scaffolding (B4 F36; B3 F38), the human baseline replaceable (B4 F45), the Inspect replication against OpenAI's leaderboard (B7 F62), Artificial Analysis's 100x (B2 F54) and its leaderboard order (B2 F55). The capture's findings name the findings and the limits absent (CAP-1-03, CAP-1-04) and the composition 'composed at its specification' stopping 'before its results' (CAP-1-07). Composed without attribution: no person is named, at the coarse grain of a benchmark description (recorded, not a breach). Outside the cards: 'math tests'. Content replaced by use: the closer (T1-7).

v0.9 §0.5: the first diff is ontological and narrow. T and the field agree on the entity (OpenAI's benchmark of models on tasks built from experts' work). T composes the benchmark as a specification, its scope, design, grading and data, and leaves out everything the specification yields: the findings, the accounting that charges the expert's review and redo to the model, and the limits the authors attach. Of the field's 67 claims, 57 stand on the cards (the PDF carries the paper's text); T composes 14 of them.

L_F(B) against L_A(B ∪ A) (the intervention): Admission adds, in one claim space with the field's: the census's reading of a task as a fixed commission, at which GDPval meets DSL in its own terms of win rate (LV-17), DSL's definition and its principle that fluent, attributed output can abandon the commission (O771-01, O771-02), its identification condition on one-shot commissions beside the authors' one-shot tasks (O771-04 with F41), and the erasure of task-origin invisible to any audit of the finished artifact (O771-03); user labor drag, the user's later labor spent managing the substrate, with its identification limit (O771-05, O771-06), where the census places the review-and-redo accounting (LV-17); the archive's claims on what benchmarks measure, standardized and predetermined tasks (O482-01, O48-01, O637-01), means and outputs and a closed ontology (O855-01, O199-01, O778-01, O1616-01, O1540-01), set beside the field's headline win rate (F24, F56); the archive's account of the labor that produces AI, ground-truth against variance-generative labor (O163-01, O163-02), extraction and the bearing-cost audit (O623-01, O533-01, O308-01 with its mark), and class as a relation and not a wage (O1545-01, O1545-02), set beside the paper's 'well compensated' (F14); the census record's answer to its organizing question, field by field (LV-01–LV-24), with what the benchmark answers and does not (LV-15, LV-16) and its negative tags (LV-21), and the worked question's gap, which DSL measures (LV-25); and the Capture Registry's account of the composition at the name (CAP-1-01–CAP-1-10). Not surfaced: every archive source; no card cites the census record (CAP-1-09).

alignment: {'T1_cards_claims': ['F1', 'F2', 'F3', 'F4', 'F7', 'F8', 'F9', 'F10', 'F11', 'F12', 'F13', 'F14', 'F15', 'F16', 'F18', 'F19', 'F20', 'F21', 'F22', 'F23', 'F24', 'F25', 'F26', 'F27', 'F28', 'F29', 'F30', 'F32', 'F33', 'F34', 'F35', 'F36', 'F38', 'F39', 'F40', 'F41', 'F43', 'F44', 'F45', 'F46', 'F47', 'F49', 'F50', 'F51', 'F52', 'F53', 'F54', 'F55', 'F56', 'F57', 'F58', 'F59', 'F60', 'F61', 'F62', 'F63', 'F64'], 'T1_composed': ['F1', 'F2', 'F6', 'F3', 'F7', 'F63', 'F12', 'F13', 'F60', 'F18', 'F43', 'F57', 'F52', 'F64'], 'T1_outside_cards': ["'math tests' (no field source sets GDPval against math tests; F6 names academic tests of reasoning difficulty)"], 'rule': "a field claim counts as composed when T states it at its source's grain or coarser; a claim on the paper counts as on the cards when its sentence is in the card's PDF (B4) or on the abstract page (B3); the typology of what is not is in T_vs_LB (§6.2)"}

Kthe prospective kernel sealed only at freeze56 entries
K1O771-01 #771missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
claim
O771-01
source
#771
M_src
stipulation
sense
task-origin: whether the labor serves the commissioned task
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K2O771-02 #771missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
claim
O771-02
source
#771
M_src
stipulation
sense
task-origin: whether the labor serves the commissioned task
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K3O771-03 #771missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
claim
O771-03
source
#771
M_src
interpretation
sense
task-origin: whether the labor serves the commissioned task
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K4O771-04 #771qualified claim (F41: the tasks precisely specified and one-shot)
claim
O771-04
source
#771
M_src
stipulation
sense
the identification condition: one-shot commissions
qualifiers carried
O771-06
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F41: the tasks precisely specified and one-shot)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K5O771-05 #771qualified claim (F30, F31: the review and redo time counted against the model)
claim
O771-05
source
#771
M_src
stipulation
sense
user labor drag: the user's labor spent managing the substrate
qualifiers carried
O771-06
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F30, F31: the review and redo time counted against the model)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K6O771-06 #771qualified claim (F30, F31: the review and redo time counted against the model)
claim
O771-06
source
#771
M_src
self-description
sense
user labor drag: the user's labor spent managing the substrate
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F30, F31: the review and redo time counted against the model)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K7O855-01 #855qualified claim (F24, F56: the headline finding a win rate over the gold subset)
claim
O855-01
source
#855
M_src
interpretation
sense
what a benchmark measures: means, outputs, closed ontologies
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F24, F56: the headline finding a win rate over the gold subset)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K8O199-01 #199qualified claim (F24, F56: the headline finding a win rate over the gold subset)
claim
O199-01
source
#199
M_src
hypothesis
sense
what a benchmark measures: means, outputs, closed ontologies
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F24, F56: the headline finding a win rate over the gold subset)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K9O778-01 #778qualified claim (F24, F56: the headline finding a win rate over the gold subset)
claim
O778-01
source
#778
M_src
interpretation
sense
what a benchmark measures: means, outputs, closed ontologies
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F24, F56: the headline finding a win rate over the gold subset)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K10O1616-01 #1616qualified claim (F24, F56: the headline finding a win rate over the gold subset)
claim
O1616-01
source
#1616
M_src
interpretation
sense
what a benchmark measures: means, outputs, closed ontologies
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F24, F56: the headline finding a win rate over the gold subset)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K11O1540-01 #1540qualified claim (F24, F56: the headline finding a win rate over the gold subset)
claim
O1540-01
source
#1540
M_src
interpretation
sense
what a benchmark measures: means, outputs, closed ontologies
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F24, F56: the headline finding a win rate over the gold subset)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K12O482-01 #482qualified claim (F6: the authors set GDPval against academic-test benchmarks)
claim
O482-01
source
#482
M_src
interpretation
sense
what a benchmark measures: standardized, predetermined, well-defined tasks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F6: the authors set GDPval against academic-test benchmarks)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K13O48-01 #48qualified claim (F6: the authors set GDPval against academic-test benchmarks)
claim
O48-01
source
#48
M_src
stipulation
sense
what a benchmark measures: standardized, predetermined, well-defined tasks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F6: the authors set GDPval against academic-test benchmarks)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K14O637-01 #637qualified claim (F6: the authors set GDPval against academic-test benchmarks)
claim
O637-01
source
#637
M_src
interpretation
sense
what a benchmark measures: standardized, predetermined, well-defined tasks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F6: the authors set GDPval against academic-test benchmarks)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K15O163-01 #163qualified claim (F14: the experts 'well compensated', no rate)
claim
O163-01
source
#163
M_src
stipulation
sense
labor producing AI: ground-truth labor and variance-generative labor
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F14: the experts 'well compensated', no rate)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K16O163-02 #163qualified claim (F14: the experts 'well compensated', no rate)
claim
O163-02
source
#163
M_src
hypothesis
sense
labor producing AI: ground-truth labor and variance-generative labor
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F14: the experts 'well compensated', no rate)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K17O623-01 #623qualified claim (F14, F15: 'well compensated'; vendor partners unnamed)
claim
O623-01
source
#623
M_src
interpretation
sense
labor producing AI: extraction, compensation, bearing-cost
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F14, F15: 'well compensated'; vendor partners unnamed)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K18O533-01 #533qualified claim (F14, F15: 'well compensated'; vendor partners unnamed)
claim
O533-01
source
#533
M_src
stipulation
sense
labor producing AI: extraction, compensation, bearing-cost
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F14, F15: 'well compensated'; vendor partners unnamed)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K19O308-01 #308qualified claim (F14, F15: 'well compensated'; vendor partners unnamed)
claim
O308-01
source
#308
M_src
documented
sense
labor producing AI: extraction, compensation, bearing-cost
qualifiers carried
—
f (source's own falsifiers)
— (none stated; marked unsupported at the locus; watch: as above)
contrast
qualified claim (F14, F15: 'well compensated'; vendor partners unnamed)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K20O1545-01 #1545qualified claim (F14: 'well compensated')
claim
O1545-01
source
#1545
M_src
stipulation
sense
class as relation, not wage
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F14: 'well compensated')
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K21O1545-02 #1545qualified claim (F14: 'well compensated')
claim
O1545-02
source
#1545
M_src
stipulation
sense
class as relation, not wage
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F14: 'well compensated')
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K22LV-01 Labor Vector census record, 2026-10-29none (restates the field: B3)
claim
LV-01
source
Labor Vector census record, 2026-10-29
M_src
stipulation
sense
what it is: a benchmark of model capability on economically valuable tasks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B3)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K23LV-02 Labor Vector census record, 2026-10-29none (restates the field: B3)
claim
LV-02
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
the operator and the authors
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B3)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K24LV-03 Labor Vector census record, 2026-10-29missing distinction (the field has no claim on the point)
claim
LV-03
source
Labor Vector census record, 2026-10-29
M_src
stipulation
sense
what the record's schema asks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field has no claim on the point)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K25LV-04 Labor Vector census record, 2026-10-29missing distinction (the field has no claim on the point)
claim
LV-04
source
Labor Vector census record, 2026-10-29
M_src
stipulation
sense
what the record's schema asks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field has no claim on the point)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K26LV-05 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-05
source
Labor Vector census record, 2026-10-29
M_src
stipulation
sense
the unit: a request and a deliverable
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K27LV-06 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-06
source
Labor Vector census record, 2026-10-29
M_src
interpretation
sense
task-origin: the experts' own work product
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K28LV-07 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-07
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
the experts: recruitment, experience, compensation
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K29LV-08 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-08
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
direction of the work: review and quality control
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K30LV-09 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-09
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
valuing the experts' time
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K31LV-10 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-10
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
selection: sectors by GDP share, occupations by wages, digital by model classification
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K32LV-11 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-11
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
scope and count: 1,320 tasks, 44 occupations, 9 sectors
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K33LV-12 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-12
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
scope and count: 1,320 tasks, 44 occupations, 9 sectors
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K34LV-13 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-13
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
headline finding: approaching parity with experts
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K35LV-14 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-14
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
stated limits: size, scope, one-shot specification
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K36LV-15 Labor Vector census record, 2026-10-29missing distinction (the field has no claim on the point)
claim
LV-15
source
Labor Vector census record, 2026-10-29
M_src
interpretation
sense
what the record's schema asks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field has no claim on the point)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K37LV-16 Labor Vector census record, 2026-10-29missing distinction (the field has no claim on the point)
claim
LV-16
source
Labor Vector census record, 2026-10-29
M_src
interpretation
sense
what the record's schema asks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field has no claim on the point)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K38LV-17 Labor Vector census record, 2026-10-29missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
claim
LV-17
source
Labor Vector census record, 2026-10-29
M_src
interpretation
sense
task-origin: whether the labor serves the commissioned task
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K39LV-18 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-18
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
the open gold subset and the held-out set
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K40LV-19 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-19
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
publication
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K41LV-20 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-20
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
versions and lifecycle
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K42LV-21 Labor Vector census record, 2026-10-29missing distinction (the field has no claim on the point)
claim
LV-21
source
Labor Vector census record, 2026-10-29
M_src
interpretation
sense
negative tags: what the benchmark is not
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field has no claim on the point)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K43LV-22 Labor Vector census record, 2026-10-29none (restates the field: B8)
claim
LV-22
source
Labor Vector census record, 2026-10-29
M_src
documented
sense
positioning: capability measured ahead of adoption
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (restates the field: B8)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K44LV-23 Labor Vector census record, 2026-10-29missing distinction (the field has no claim on the point)
claim
LV-23
source
Labor Vector census record, 2026-10-29
M_src
self-description
sense
what the record leaves unchecked
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field has no claim on the point)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K45LV-24 Labor Vector census record, 2026-10-29missing distinction (the field has no claim on the point)
claim
LV-24
source
Labor Vector census record, 2026-10-29
M_src
stipulation
sense
what the record's schema asks
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field has no claim on the point)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K46LV-25 Labor Vector census record, 2026-10-29missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
claim
LV-25
source
Labor Vector census record, 2026-10-29
M_src
interpretation
sense
task-origin: whether the labor serves the commissioned task
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
missing distinction (the field grades deliverables against experts'; no field claim on whether the labor fulfils the commission)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K47CAP-1-01 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-01
source
Capture Registry, 2026-10-29
M_src
documented
sense
the composition at the name: what it held
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K48CAP-1-02 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-02
source
Capture Registry, 2026-10-29
M_src
interpretation
sense
the composition at the name: findings and limits left out
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K49CAP-1-03 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-03
source
Capture Registry, 2026-10-29
M_src
documented
sense
the composition at the name: findings and limits left out
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K50CAP-1-04 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-04
source
Capture Registry, 2026-10-29
M_src
documented
sense
the composition at the name: findings and limits left out
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K51CAP-1-05 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-05
source
Capture Registry, 2026-10-29
M_src
documented
sense
the composition at the name: findings and limits left out
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K52CAP-1-06 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-06
source
Capture Registry, 2026-10-29
M_src
interpretation
sense
the composition at the name: what it held
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K53CAP-1-07 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-07
source
Capture Registry, 2026-10-29
M_src
interpretation
sense
the composition at the name: findings and limits left out
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K54CAP-1-08 Capture Registry, 2026-10-29qualified claim (F30, F31: the review and redo time counted against the model)
claim
CAP-1-08
source
Capture Registry, 2026-10-29
M_src
interpretation
sense
user labor drag: the user's labor spent managing the substrate
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
qualified claim (F30, F31: the review and redo time counted against the model)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K55CAP-1-09 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-09
source
Capture Registry, 2026-10-29
M_src
documented
sense
reception: the record and the cards
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
K56CAP-1-10 Capture Registry, 2026-10-29none (a record of the composition at the address)
claim
CAP-1-10
source
Capture Registry, 2026-10-29
M_src
interpretation
sense
derivative: GDPval-AA (Artificial Analysis)
qualifiers carried
—
f (source's own falsifiers)
— (none stated; watch: a later composition at the address that states or drops the claim, attributed or not, uptake §8.3)
contrast
none (a record of the composition at the address)
resolver
a later capture at 'GDPval' (ADDR-d4b480b48aca) on Google AI Overview, seated in the Capture Registry (§8.3)
Archive bearing — D/R/O traversalread

Candidates are found by string; admission is by reading.

traversalfirst reading, 2026-10-29; second pass with one citation hop
D 0 deposits · 0 sentencesR 0 · 0O 44 · 0 direct, 88 classhop 136read: 17 admitted, 34 not

Lee Sharks · Crimson Hexagonal Archive · register v1.36, 2026-10-27. Built by scripts/build_non.py from the files linked above; it writes nothing back. Working, not frozen.