# Phase P7 validation — differential-testing harness

Date: 2026-09-28. All work on the Raspberry Pi (no cloud cost).

## Delivered

| Item | Module |
|---|---|
| Tolerance categories EXACT / NEAR / DISCREPANCY / UNKNOWN with the plan §16.3 tolerances; float32 ULP distance; NaN = NaN regardless of sign (K-014) | `compare/core.py` |
| Locus classes (assay, variant type, chromosome group incl. PAR/MT/unmapped, intensity-only) | `compare/core.py` |
| GTC-vs-GTC (every per-locus array, transforms per bin, controls, sample statistics; free-text metadata reported but not counted) and VCF-vs-VCF (per record and FORMAT field, records keyed by CHROM/POS/REF/ALT/ID, classes SNV/MNV/indel, merged sites, multi-allelic) | `compare/records.py` |
| `simurg compare A B` (files or folders paired by name; `--tolerance FIELD=…`, `--fail-on-discrepancy`) → `summary.json`, `discrepancies.tsv.gz` (every discrepancy and unmatched record, never truncated), self-contained `dashboard.html` | `pipeline/compare.py` |
| Nightly run: unit tests, GSA genotype vs gtc2vcf oracle, GSA VCF vs gtc2vcf (semantic) and vs golden (regression), Omni2.5 subset vs vendor GTCs; per-night page + history; 14 nights kept | `scripts/nightly.py`, `scripts/install_nightly.sh` (crontab, 02:30) |

## Acceptance (plan §18 P7: nightly run on P0 + Omni subset produces report)

First complete nightly run on the Pi (2026-09-28, 17 min):

| Step | Compared | Result |
|---|---|---|
| unit | 206 unit tests | PASS |
| gsa_gtc | 36 GSA GTCs from IDATs vs gtc2vcf oracle GTCs, 20 fields, per locus class | **PASS**: 0 discrepancies, 0 unknown (free-text metadata listed separately) |
| gsa_vcf | 36 VCFs vs gtc2vcf (semantic, `validate_p6_vcf.py`) | **PASS**: 0 unexplained |
| gsa_vcf regression | 36 VCFs vs the golden set of the first run: 23 640 768 records | all records present, GT/GS/BAF/LRR EXACT (see note on BAF/LRR of merged sites) |
| omni_gtc | 6 Omni2.5 samples (`--preset autocall-1.6.2`) vs the vendor's AutoCall 1.6.2.2 GTCs, 18 fields | **PASS**: 0 discrepancies, 0 unknown |

Installed as a crontab entry (every night 02:30, `flock` against overlapping runs; `bash scripts/install_nightly.sh
--remove` removes it). History page: `data/nightly/index.html`.

## Notes

* The harness found its own bug while being built: VCF records keyed without ID let duplicate designs overwrite
  each other in `--unsquash-duplicates` output; keys now include the ID and repeated keys are counted, never merged.
* The first regression run reported 51 048 UNKNOWN values: BAF/LRR of multi-allelic merged records, which carry
  `GT:GS` only on *both* sides. A field absent on both sides is now "not applicable" and not counted (GTC and VCF
  alike); a field absent on one side stays UNKNOWN.
* Resource use on the Pi: one Omni2.5 sample genotypes in 93 s at 3.4 GB peak RSS (single process); the GSA VCF
  step memory-maps the 3.1 GB genome cache, so its address-space cap is 12 GB (page cache, not RAM).