Dataset Audit Kit

Problem#

Many ML failures start with the data, not the model. A column disappears after a source change, missing values spike, duplicate rows leak into training, a feature quietly copies the label, or a new batch drifts far from the reference data.

What it does#

• Schema checks against expected columns.
• Missingness summaries by column, with a configurable threshold.
• Duplicate-row detection.
• Label balance, label completeness and label-leakage checks.
• Numeric and categorical drift against a reference dataset, including a Population Stability Index.
• Reports as Markdown, machine-readable JSON, or a standalone HTML file.

Usage#

from dataset_audit_kit import DatasetAuditor

auditor = DatasetAuditor(missing_threshold=0.05, drift_threshold=0.20)
report = auditor.audit_file(
    "train.parquet",
    reference_path="reference.jsonl",
    label_column="target",
    expected_columns=["feature_1", "feature_2", "target"],
)
print(report.to_markdown())

dataset-audit-kit audit data.parquet --reference reference.jsonl --label-column target --json

Engineering#

MIT licensed, tested in CI on Python 3.10+. In --json mode, notices go to stderr so the output can be piped straight into other tools.