Data quality · the corpus, measured
How good is the catalog?
A per-topology data-quality report across the datasets we have probed - a measured slice of the larger catalog, not the whole of it. Every probed dataset is graded on ten quality criteria - completeness, validity, consistency, accuracy, and the rest - each split into Coverage (did the check run?) and Conformance (did it pass?). We grade on two planes - declared (the publisher's metadata) and measured (we read the bytes) - so a slice we have not measured can never render green. Across the corpus that gap is real: measured 93% vs declared 71% - reading the bytes catches what the metadata misses.
Why topology
Topology is a dataset's measured shape - tabular tables, tensor arrays, spatial rasters, composite archives, and the rest. What good looks like differs by shape, so we read quality per topology rather than against one generic checklist. Two views below: how far each shape has travelled up the depth ladder, and where the corpus is strong or weak across the criteria.
Each shape's datasets by depth - cataloged, then measured against the bytes - ordered so the shapes we've measured deepest lead and the still-shallow ones trail.
Measured vs declared
Each quality criterion, twice: how far the publisher's declared metadata conforms, beside what reading the measured bytes actually found. The bar between the two dots is the moat - where they disagree is where reading the bytes earns its keep. Widest gap first.
By topology
tabular
1,176 datasets- coverage
- 72%
- declared conf.
- 67%
- measured conf.
- 95%
- bytes caught declared
- 8%
| criterion | plane | coverage | conformance |
|---|---|---|---|
| Accuracy | cross-source | 1% | 92% |
| Accuracy | measured | 3% | 100% |
| Completeness | measured | 100% | 87% |
| Provenance | measured | 100% | 100% |
| Representation | measured | 100% | 99% |
| Uniqueness | measured | 29% | 100% |
| Validity | measured | 85% | 87% |
| Accessibility | declared | 100% | 100% |
| Completeness | declared | 100% | 62% |
| Provenance | declared | 100% | 67% |
| Representation | declared | 100% | 4% |
| Validity | declared | 56% | 100% |
- M-DEC-003Completeness1,176 failed
- M-DEC-008Representation1,130 failed
- M-DEC-007Provenance385 failed
- M-TAB-016Validity236 failed
- M-DEC-002Completeness172 failed
composite
533 datasets- coverage
- 98%
- declared conf.
- 74%
- measured conf.
- 81%
- bytes caught declared
- -
| criterion | plane | coverage | conformance |
|---|---|---|---|
| Completeness | measured | 100% | 100% |
| Comprehensiveness | measured | 100% | 100% |
| Integrity | measured | 100% | 100% |
| Validity | measured | 100% | 25% |
| Accessibility | declared | 100% | 100% |
| Completeness | declared | 100% | 61% |
| Provenance | declared | 100% | 100% |
| Representation | declared | 100% | 14% |
| Validity | declared | 75% | 94% |
- M-DEC-003Completeness533 failed
- M-DEC-008Representation457 failed
- M-COMP-011Validity238 failed
- M-DEC-002Completeness94 failed
- M-DEC-004Validity24 failed
tensor
277 datasets- coverage
- 92%
- declared conf.
- 72%
- measured conf.
- 100%
- bytes caught declared
- -
| criterion | plane | coverage | conformance |
|---|---|---|---|
| Completeness | measured | 100% | 100% |
| Integrity | measured | 100% | 100% |
| Validity | measured | 100% | 100% |
| Accessibility | declared | 100% | 100% |
| Completeness | declared | 100% | 67% |
| Provenance | declared | 100% | 6% |
| Representation | declared | 100% | 95% |
| Validity | declared | 5% | 93% |
- M-DEC-003Completeness277 failed
- M-DEC-007Provenance260 failed
- M-DEC-008Representation13 failed
- M-DEC-002Completeness1 failed
- M-DEC-004Validity1 failed
modal
155 datasets- coverage
- 94%
- declared conf.
- 76%
- measured conf.
- 100%
- bytes caught declared
- -
| criterion | plane | coverage | conformance |
|---|---|---|---|
| Consistency | measured | 100% | 100% |
| Representation | measured | 100% | 100% |
| Validity | measured | 50% | 100% |
| Accessibility | declared | 100% | 100% |
| Completeness | declared | 100% | 61% |
| Provenance | declared | 100% | 100% |
| Representation | declared | 100% | 24% |
| Validity | declared | 79% | 97% |
- M-DEC-003Completeness155 failed
- M-DEC-008Representation118 failed
- M-DEC-002Completeness26 failed
- M-DEC-004Validity4 failed
structure
150 datasets- coverage
- 100%
- declared conf.
- 93%
- measured conf.
- -
- bytes caught declared
- -
| criterion | plane | coverage | conformance |
|---|---|---|---|
| Accessibility | declared | 100% | 100% |
| Completeness | declared | 100% | 67% |
| Provenance | declared | 100% | 100% |
| Representation | declared | 100% | 100% |
| Validity | declared | 100% | 100% |
- M-DEC-003Completeness150 failed
sequence
17 datasets- coverage
- 100%
- declared conf.
- 80%
- measured conf.
- -
- bytes caught declared
- -
| criterion | plane | coverage | conformance |
|---|---|---|---|
| Accessibility | declared | 100% | 100% |
| Completeness | declared | 100% | 65% |
| Provenance | declared | 100% | 100% |
| Representation | declared | 100% | 35% |
| Validity | declared | 100% | 100% |
- M-DEC-003Completeness17 failed
- M-DEC-008Representation11 failed
- M-DEC-002Completeness1 failed
spatial
8 datasets- coverage
- 97%
- declared conf.
- 76%
- measured conf.
- 98%
- bytes caught declared
- -
| criterion | plane | coverage | conformance |
|---|---|---|---|
| Completeness | measured | 100% | 100% |
| Consistency | measured | 100% | 100% |
| Validity | measured | 79% | 95% |
| Accessibility | declared | 100% | 100% |
| Completeness | declared | 100% | 67% |
| Provenance | declared | 100% | 100% |
| Representation | declared | 100% | 13% |
| Validity | declared | 88% | 100% |
- M-DEC-003Completeness8 failed
- M-DEC-008Representation7 failed
- M-SPATIAL-006Validity1 failed