Data quality · the corpus, measured

How good is the catalog?

A per-topology data-quality report across the datasets we have probed - a measured slice of the larger catalog, not the whole of it. Every probed dataset is graded on ten quality criteria - completeness, validity, consistency, accuracy, and the rest - each split into Coverage (did the check run?) and Conformance (did it pass?). We grade on two planes - declared (the publisher's metadata) and measured (we read the bytes) - so a slice we have not measured can never render green. Across the corpus that gap is real: measured 93% vs declared 71% - reading the bytes catches what the metadata misses.

2,316datasets89%probed93%measured conf.71%declared conf.

Why topology

Topology is a dataset's measured shape - tabular tables, tensor arrays, spatial rasters, composite archives, and the rest. What good looks like differs by shape, so we read quality per topology rather than against one generic checklist. Two views below: how far each shape has travelled up the depth ladder, and where the corpus is strong or weak across the criteria.

Topology to maturity
CatalogedMeasureddepth = measured or deeper
structure
150
100%
sequence
17
100%
tabular
1,176
99%
tensor
277
97%
modal
155
94%
spatial
8
88%
composite
533
60%

Each shape's datasets by depth - cataloged, then measured against the bytes - ordered so the shapes we've measured deepest lead and the still-shallow ones trail.

Measured vs declared

Each quality criterion, twice: how far the publisher's declared metadata conforms, beside what reading the measured bytes actually found. The bar between the two dots is the moat - where they disagree is where reading the bytes earns its keep. Widest gap first.

measureddeclared0 to 100% conformance
Representation
Completeness
Provenance
Validity
Accessibility
Accuracy
Comprehensiveness
Consistency
Integrity
Uniqueness

By topology

tabular

1,176 datasets
coverage
72%
declared conf.
67%
measured conf.
95%
bytes caught declared
8%
Maturity
12 cataloged1,164 measured
Criteria
criterionplanecoverageconformance
Accuracycross-source
1%
92%
Accuracymeasured
3%
100%
Completenessmeasured
100%
87%
Provenancemeasured
100%
100%
Representationmeasured
100%
99%
Uniquenessmeasured
29%
100%
Validitymeasured
85%
87%
Accessibilitydeclared
100%
100%
Completenessdeclared
100%
62%
Provenancedeclared
100%
67%
Representationdeclared
100%
4%
Validitydeclared
56%
100%
Top failing checks
  • M-DEC-003Completeness1,176 failed
  • M-DEC-008Representation1,130 failed
  • M-DEC-007Provenance385 failed
  • M-TAB-016Validity236 failed
  • M-DEC-002Completeness172 failed

composite

533 datasets
coverage
98%
declared conf.
74%
measured conf.
81%
bytes caught declared
-
Maturity
214 cataloged319 measured
Criteria
criterionplanecoverageconformance
Completenessmeasured
100%
100%
Comprehensivenessmeasured
100%
100%
Integritymeasured
100%
100%
Validitymeasured
100%
25%
Accessibilitydeclared
100%
100%
Completenessdeclared
100%
61%
Provenancedeclared
100%
100%
Representationdeclared
100%
14%
Validitydeclared
75%
94%
Top failing checks
  • M-DEC-003Completeness533 failed
  • M-DEC-008Representation457 failed
  • M-COMP-011Validity238 failed
  • M-DEC-002Completeness94 failed
  • M-DEC-004Validity24 failed

tensor

277 datasets
coverage
92%
declared conf.
72%
measured conf.
100%
bytes caught declared
-
Maturity
8 cataloged269 measured
Criteria
criterionplanecoverageconformance
Completenessmeasured
100%
100%
Integritymeasured
100%
100%
Validitymeasured
100%
100%
Accessibilitydeclared
100%
100%
Completenessdeclared
100%
67%
Provenancedeclared
100%
6%
Representationdeclared
100%
95%
Validitydeclared
5%
93%
Top failing checks
  • M-DEC-003Completeness277 failed
  • M-DEC-007Provenance260 failed
  • M-DEC-008Representation13 failed
  • M-DEC-002Completeness1 failed
  • M-DEC-004Validity1 failed

modal

155 datasets
coverage
94%
declared conf.
76%
measured conf.
100%
bytes caught declared
-
Maturity
10 cataloged145 measured
Criteria
criterionplanecoverageconformance
Consistencymeasured
100%
100%
Representationmeasured
100%
100%
Validitymeasured
50%
100%
Accessibilitydeclared
100%
100%
Completenessdeclared
100%
61%
Provenancedeclared
100%
100%
Representationdeclared
100%
24%
Validitydeclared
79%
97%
Top failing checks
  • M-DEC-003Completeness155 failed
  • M-DEC-008Representation118 failed
  • M-DEC-002Completeness26 failed
  • M-DEC-004Validity4 failed

structure

150 datasets
coverage
100%
declared conf.
93%
measured conf.
-
bytes caught declared
-
Maturity
0 cataloged150 measured
Criteria
criterionplanecoverageconformance
Accessibilitydeclared
100%
100%
Completenessdeclared
100%
67%
Provenancedeclared
100%
100%
Representationdeclared
100%
100%
Validitydeclared
100%
100%
Top failing checks
  • M-DEC-003Completeness150 failed

sequence

17 datasets
coverage
100%
declared conf.
80%
measured conf.
-
bytes caught declared
-
Maturity
0 cataloged17 measured
Criteria
criterionplanecoverageconformance
Accessibilitydeclared
100%
100%
Completenessdeclared
100%
65%
Provenancedeclared
100%
100%
Representationdeclared
100%
35%
Validitydeclared
100%
100%
Top failing checks
  • M-DEC-003Completeness17 failed
  • M-DEC-008Representation11 failed
  • M-DEC-002Completeness1 failed

spatial

8 datasets
coverage
97%
declared conf.
76%
measured conf.
98%
bytes caught declared
-
Maturity
1 cataloged7 measured
Criteria
criterionplanecoverageconformance
Completenessmeasured
100%
100%
Consistencymeasured
100%
100%
Validitymeasured
79%
95%
Accessibilitydeclared
100%
100%
Completenessdeclared
100%
67%
Provenancedeclared
100%
100%
Representationdeclared
100%
13%
Validitydeclared
88%
100%
Top failing checks
  • M-DEC-003Completeness8 failed
  • M-DEC-008Representation7 failed
  • M-SPATIAL-006Validity1 failed