{"address":{"q":"Data Provenance Initiative","record":"data-provenance-initiative-aio-20261029"},"analysis":{"analysis":"As with the revived Papers with Code on the MMRS side, the composition is ahead of the record on the entity's current extent: the census formalized the initiative from its first release and its repository, and the Overview read its present home page. The capture is a correction to be made from the entity's own page.","d":"THE DATA PROVENANCE INITIATIVE, BY NAME, ON GOOGLE'S AI OVERVIEW, SIGNED OUT, 2026-10-29: 'a multi-institutional research project—led in part by researchers from MIT—that audits the datasets, licenses, and web domains used to train artificial intelligence models.' Four platforms (the Data Provenance Explorer over 1,800+ datasets, a Web Consent Explorer, Open Model Evolution, an AI Observatory) and three findings (license omission over 70% and error over 50% on hosting platforms, closed datasets concentrated in creative tasks and lower-resource languages, a rise of non-commercial licences driven by synthetic data). Four cards: the initiative's home page ('4,000+ audited datasets; 14,000 tracked web domains'), the arXiv paper, MIT Sloan, Nature Machine Intelligence (1,858 datasets across 44 collections).","d_full":"THE DATA PROVENANCE INITIATIVE, BY NAME, ON GOOGLE'S AI OVERVIEW, SIGNED OUT, 2026-10-29: 'a multi-institutional research project—led in part by researchers from MIT—that audits the datasets, licenses, and web domains used to train artificial intelligence models.' Four platforms (the Data Provenance Explorer over 1,800+ datasets, a Web Consent Explorer, Open Model Evolution, an AI Observatory) and three findings (license omission over 70% and error over 50% on hosting platforms, closed datasets concentrated in creative tasks and lower-resource languages, a rise of non-commercial licences driven by synthetic data). Four cards: the initiative's home page ('4,000+ audited datasets; 14,000 tracked web domains'), the arXiv paper, MIT Sloan, Nature Machine Intelligence (1,858 datasets across 44 collections).","findings":["THE FINDINGS RIGHT. 'license omission of 70%+ and error rates of 50%+' is the record's quote from the founding paper. (Repaired 2026-10-29: the finding had given the README as its source.) Checked 2026-10-29 against the provenance census record of the same day and the sources it saved.","THE OPERATOR. The record quotes the initiative's own description, 'a multi-disciplinary volunteer effort'; the composition's 'led in part by researchers from MIT' follows the MIT Sloan card, and 'multi-institutional' is in no card. The record names the founding paper's authors (Shayne Longpre and 16 others); the composition names no one.","THE RECORD AT TOO COARSE A GRAIN. The record has one explorer and the first release's count (44 collections, 1,800+ datasets). The home-page card gives four platforms and '4,000+ audited datasets; 14,000 tracked web domains'. The /non row read dataprovenance.org the same day: 'Four public measurement systems' (the Data Provenance Explorer, the Web Consent Explorer, Open Model Evolution, the AI Observatory) and '4,000+ / datasets audited', '14,000 / web domains tracked'. The card's figures hold; the record is to carry them with their quotes.","THE NEGATIVE TAGS. 'NOT legal advice' and 'describing a collection does not establish which model used it' are not composed; the composition does not claim otherwise."],"match":"BROAD MATCH","mt":"THE DATA PROVENANCE INITIATIVE ON THE OVERVIEW — 'LED IN PART BY RESEARCHERS FROM MIT'; FOUR PLATFORMS WHERE THE RECORD HAS ONE; THE 70% OMISSION AND 50% ERROR RATES RIGHT; THE HOME PAGE'S 4,000+ DATASETS IN A CARD","notes":{"prompts":"One query. Sent by the operator on 2026-10-29 with thirteen others, one per entity of the provenance meta-aggregator, from the strings supplied at his request ('shoot me those search strings. strings on the bare entity, no follow questions'); pasted in one message separated by '$'. The paste was lost to a context compaction of this session and re-pasted by the operator the same day; the second paste is the one seated."},"originator":{"basis":"Not the archive's: Data Provenance Initiative; its operator, as the record gives it: the Data Provenance Initiative (volunteer effort; paper by Shayne Longpre and 16 other authors). Formalized by the provenance meta-aggregator record of 2026-10-29 (https://www.provenanceerasure.org/aggregators/data-provenance-initiative/), editor Lee Sharks; the record's fields carry verbatim quotes from the sources it saved with sha256, each with its basis (the operator's own page or a third-party account). Recorded 2026-10-29; repaired the same day.","entity_type":"dataset","name":"the Data Provenance Initiative (volunteer effort; paper by Shayne Longpre and 16 other authors)","relation":"external","spxi_treatment":"partial"},"reading":"The innocent reading first. The Overview identifies the project, lists its tools and reports what its audits found, from its own site, its paper and the press."},"measures":{"archive_controlled_cites":0,"cite_list":[{"n":1,"note":null,"rel":"third_party","site":"Data Provenance Initiative","snip":"Platforms: Data Provenance Initiative (maps AI datasets, web policies, model markets, real-world use). Metrics: 4,000+ audited datasets; 14,000 tracked web doma...","title":"Data Provenance Initiative","url":null},{"n":2,"note":null,"rel":"third_party","site":"arXiv","snip":"Audit: 1800+ text datasets traced and audited by legal and machine learning experts to evaluate licensing, lineage, and attribution in AI training. Findings: Co...","title":"[2310.16787] The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI","url":null},{"n":3,"note":null,"rel":"third_party","site":"MIT Sloan","snip":"Problem: Untraced AI training datasets cause legal, copyright, regulatory, bias, and quality risks due to poor documentation and high license error rates (>50%)","title":"Bringing transparency to the data used to train artificial intelligence | MIT Sloan","url":null},{"n":4,"note":null,"rel":"third_party","site":"Nature","snip":"Audit: 1,858 text/finetuning datasets across 44 collections evaluated for licensing, attribution, and lineage via the Data Provenance Initiative and DPExplorer.","title":"A large-scale audit of dataset licensing and attribution in AI | Nature Machine Intelligence","url":null}],"cites":4,"per":0.5,"per_note":"MIT named; no person; no identifier; the initiative's home page and paper cited.","per_v":{"author":false,"id":false,"inst":true,"src":true}},"observation":{"auth":"signed out, incognito","date":"2026-10-29","ev":"paste","q":"Data Provenance Initiative","q_kind":"the name of an entity of the provenance meta-aggregator (Data Provenance Initiative; record https://www.provenanceerasure.org/aggregators/data-provenance-initiative/, compiled 2026-10-29), the bare entity string supplied at the operator's request ('strings on the bare entity, no follow questions') and run by him the same day, to supply the composition layer's account of the entity for its record and its /non row. A baseline: the record landed on provenanceerasure.org the same day (commit 8a9f153) and no card cites it. NEW address.","s":"Machine Reception","surface":"Google AI Overview","transcript":"[Google AI Overview at the query, signed out and incognito (operator defaults of 2026-10-01 and 2026-10-14), 2026-10-29. The paste is the Overview expanded (the 'AI Mode Conversation' view, with the source cards that follow the answer); the 'You said' line carries the query twice, run together, and is given once here. Inline links are Google redirect URLs, kept as pasted.]\n\n[QUERENT] Data Provenance Initiative\n\n[SYSTEM] The [Data Provenance Initiative](https://www.google.com/goto?url=CAESVwHrOzAVIs_bBUxr7ducYJKxi6mO2H8PFkOqeMYxYB9BZOGggLJumXN31RQqX2ghNJ2wlF1SU0MHuuvc_7Cgibm_RylNPmv9oYEqoRYseVd3oqu3XZKm1g) is a multi-institutional research project—led in part by researchers from MIT—that audits the datasets, licenses, and web domains used to train artificial intelligence models. [[1](https://www.google.com/goto?url=CAESVwHrOzAVIs_bBUxr7ducYJKxi6mO2H8PFkOqeMYxYB9BZOGggLJumXN31RQqX2ghNJ2wlF1SU0MHuuvc_7Cgibm_RylNPmv9oYEqoRYseVd3oqu3XZKm1g), [2](https://www.google.com/goto?url=CAESqQEB6zswFet87RYIXm3SERz1TKVCghLBNX4HMFbQtx13MaYHj6MDNSA37R0c7ew2FcMKY0uwIb26y1xAvejdbImAK8Q_LTnWNuKmR8qSQN1qnnLlzv64o7kxJszSYWKAOBs9at3KmLXdTqZoVWPghfJ919vtbMoyd1dBDa3EUPb04SLEOewf5nEuwcxJIK3tbZkdhSMe8_ovQaLxzbMOag7gPLlf4I31cXOb)]\nKey Tools & Platforms\n\n* Data Provenance Explorer: An interactive tool that audits and filters over 1,800+ text and fine-tuning datasets across dozens of collections to inspect sources, lineages, and license types.\n* Web Consent Explorer: A system that tracks thousands of web domains for consent signals and web policies.\n* Open Model Evolution: A monitor tracking open-source and open-weight AI models for transparency and concentration.\n* AI Observatory: A comparative tool for analyzing real-world AI conversation sets. [[1](https://www.google.com/goto?url=CAESWAHrOzAVPyKzeU_QMzOf7n22XqYG0KKTgi5Z-PwZ7GxKjlEAW5l2-E65BnypNfmou0_MVC8loaXdqlCyTHI5KJIgRZIQWyxmBUnIZGUKIrLwEMZt9FHY3wc), [2](https://www.google.com/goto?url=CAESqQEB6zswFet87RYIXm3SERz1TKVCghLBNX4HMFbQtx13MaYHj6MDNSA37R0c7ew2FcMKY0uwIb26y1xAvejdbImAK8Q_LTnWNuKmR8qSQN1qnnLlzv64o7kxJszSYWKAOBs9at3KmLXdTqZoVWPghfJ919vtbMoyd1dBDa3EUPb04SLEOewf5nEuwcxJIK3tbZkdhSMe8_ovQaLxzbMOag7gPLlf4I31cXOb), [3](https://www.google.com/goto?url=CAESVwHrOzAVIs_bBUxr7ducYJKxi6mO2H8PFkOqeMYxYB9BZOGggLJumXN31RQqX2ghNJ2wlF1SU0MHuuvc_7Cgibm_RylNPmv9oYEqoRYseVd3oqu3XZKm1g)]\n\nMajor Findings from Audits\n\n* License Omission & Errors: Widely used hosting platforms exhibit over a 70% license omission rate and more than 50% error rates in how they categorize dataset licenses. [[1](https://www.google.com/goto?url=CAESWAHrOzAVPyKzeU_QMzOf7n22XqYG0KKTgi5Z-PwZ7GxKjlEAW5l2-E65BnypNfmou0_MVC8loaXdqlCyTHI5KJIgRZIQWyxmBUnIZGUKIrLwEMZt9FHY3wc)]\n* Composition Divides: Commercially closed or restrictive datasets often monopolize creative tasks, specialized topics, and lower-resource languages. [[1](https://www.google.com/goto?url=CAESWAHrOzAVPyKzeU_QMzOf7n22XqYG0KKTgi5Z-PwZ7GxKjlEAW5l2-E65BnypNfmou0_MVC8loaXdqlCyTHI5KJIgRZIQWyxmBUnIZGUKIrLwEMZt9FHY3wc)]\n* Non-Commercial Spikes: There has been a significant rise in non-commercial and academic-only licenses, frequently driven by synthetic data generated via commercial APIs. [[1](https://www.google.com/goto?url=CAESagHrOzAVZJUi_F28cTVTNDyVGy5o6we0skjvkFWVNKew85OiArqy4UZIJs_HB3iYrw47y7IANWFpbJwvPeNn-_KvDgyFI0xIBWwUWyPbE1ituE9-vBcMmutKajo2Z4cvEYj9YNkbsKyOPKg)]\n\nWould you like to explore how to use the Data Provenance Explorer, or look into specific dataset licenses?\nData Provenance Initiative\nData Provenance Initiative\nPlatforms: Data Provenance Initiative (maps AI datasets, web policies, model markets, real-world use). Metrics: 4,000+ audited datasets; 14,000 tracked web doma...\narXiv\n[2310.16787] The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI\nAudit: 1800+ text datasets traced and audited by legal and machine learning experts to evaluate licensing, lineage, and attribution in AI training. Findings: Co...\nMIT Sloan\nBringing transparency to the data used to train artificial intelligence | MIT Sloan\nProblem: Untraced AI training datasets cause legal, copyright, regulatory, bias, and quality risks due to poor documentation and high license error rates (>50%)\nNature\nA large-scale audit of dataset licensing and attribution in AI | Nature Machine Intelligence\nAudit: 1,858 text/finetuning datasets across 44 collections evaluated for licensing, attribution, and lineage via the Data Provenance Initiative and DPExplorer.","transcript_class":"CAPTURE-TIME VERBATIM RECORD — paste of the Overview expanded, with its source cards","transcript_complete":"complete as supplied","transcript_raw":"The [Data Provenance Initiative](https://www.google.com/goto?url=CAESVwHrOzAVIs_bBUxr7ducYJKxi6mO2H8PFkOqeMYxYB9BZOGggLJumXN31RQqX2ghNJ2wlF1SU0MHuuvc_7Cgibm_RylNPmv9oYEqoRYseVd3oqu3XZKm1g) is a multi-institutional research project—led in part by researchers from MIT—that audits the datasets, licenses, and web domains used to train artificial intelligence models. [[1](https://www.google.com/goto?url=CAESVwHrOzAVIs_bBUxr7ducYJKxi6mO2H8PFkOqeMYxYB9BZOGggLJumXN31RQqX2ghNJ2wlF1SU0MHuuvc_7Cgibm_RylNPmv9oYEqoRYseVd3oqu3XZKm1g), [2](https://www.google.com/goto?url=CAESqQEB6zswFet87RYIXm3SERz1TKVCghLBNX4HMFbQtx13MaYHj6MDNSA37R0c7ew2FcMKY0uwIb26y1xAvejdbImAK8Q_LTnWNuKmR8qSQN1qnnLlzv64o7kxJszSYWKAOBs9at3KmLXdTqZoVWPghfJ919vtbMoyd1dBDa3EUPb04SLEOewf5nEuwcxJIK3tbZkdhSMe8_ovQaLxzbMOag7gPLlf4I31cXOb)]\nKey Tools & Platforms\n\n* Data Provenance Explorer: An interactive tool that audits and filters over 1,800+ text and fine-tuning datasets across dozens of collections to inspect sources, lineages, and license types.\n* Web Consent Explorer: A system that tracks thousands of web domains for consent signals and web policies.\n* Open Model Evolution: A monitor tracking open-source and open-weight AI models for transparency and concentration.\n* AI Observatory: A comparative tool for analyzing real-world AI conversation sets. [[1](https://www.google.com/goto?url=CAESWAHrOzAVPyKzeU_QMzOf7n22XqYG0KKTgi5Z-PwZ7GxKjlEAW5l2-E65BnypNfmou0_MVC8loaXdqlCyTHI5KJIgRZIQWyxmBUnIZGUKIrLwEMZt9FHY3wc), [2](https://www.google.com/goto?url=CAESqQEB6zswFet87RYIXm3SERz1TKVCghLBNX4HMFbQtx13MaYHj6MDNSA37R0c7ew2FcMKY0uwIb26y1xAvejdbImAK8Q_LTnWNuKmR8qSQN1qnnLlzv64o7kxJszSYWKAOBs9at3KmLXdTqZoVWPghfJ919vtbMoyd1dBDa3EUPb04SLEOewf5nEuwcxJIK3tbZkdhSMe8_ovQaLxzbMOag7gPLlf4I31cXOb), [3](https://www.google.com/goto?url=CAESVwHrOzAVIs_bBUxr7ducYJKxi6mO2H8PFkOqeMYxYB9BZOGggLJumXN31RQqX2ghNJ2wlF1SU0MHuuvc_7Cgibm_RylNPmv9oYEqoRYseVd3oqu3XZKm1g)]\n\nMajor Findings from Audits\n\n* License Omission & Errors: Widely used hosting platforms exhibit over a 70% license omission rate and more than 50% error rates in how they categorize dataset licenses. [[1](https://www.google.com/goto?url=CAESWAHrOzAVPyKzeU_QMzOf7n22XqYG0KKTgi5Z-PwZ7GxKjlEAW5l2-E65BnypNfmou0_MVC8loaXdqlCyTHI5KJIgRZIQWyxmBUnIZGUKIrLwEMZt9FHY3wc)]\n* Composition Divides: Commercially closed or restrictive datasets often monopolize creative tasks, specialized topics, and lower-resource languages. [[1](https://www.google.com/goto?url=CAESWAHrOzAVPyKzeU_QMzOf7n22XqYG0KKTgi5Z-PwZ7GxKjlEAW5l2-E65BnypNfmou0_MVC8loaXdqlCyTHI5KJIgRZIQWyxmBUnIZGUKIrLwEMZt9FHY3wc)]\n* Non-Commercial Spikes: There has been a significant rise in non-commercial and academic-only licenses, frequently driven by synthetic data generated via commercial APIs. [[1](https://www.google.com/goto?url=CAESagHrOzAVZJUi_F28cTVTNDyVGy5o6we0skjvkFWVNKew85OiArqy4UZIJs_HB3iYrw47y7IANWFpbJwvPeNn-_KvDgyFI0xIBWwUWyPbE1ituE9-vBcMmutKajo2Z4cvEYj9YNkbsKyOPKg)]\n\nWould you like to explore how to use the Data Provenance Explorer, or look into specific dataset licenses?\nData Provenance Initiative\nData Provenance Initiative\nPlatforms: Data Provenance Initiative (maps AI datasets, web policies, model markets, real-world use). Metrics: 4,000+ audited datasets; 14,000 tracked web doma...\narXiv\n[2310.16787] The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI\nAudit: 1800+ text datasets traced and audited by legal and machine learning experts to evaluate licensing, lineage, and attribution in AI training. Findings: Co...\nMIT Sloan\nBringing transparency to the data used to train artificial intelligence | MIT Sloan\nProblem: Untraced AI training datasets cause legal, copyright, regulatory, bias, and quality risks due to poor documentation and high license error rates (>50%)\nNature\nA large-scale audit of dataset licensing and attribution in AI | Nature Machine Intelligence\nAudit: 1,858 text/finetuning datasets across 44 collections evaluated for licensing, attribution, and lineage via the Data Provenance Initiative and DPExplorer.","transcript_read":"READ IN FULL 2026-10-29","turns":1},"slug":"data-provenance-initiative-aio-20261029"}