# Phase P3 validation report — normalization

Date: 2026-09-27. Development and oracle tests on the Pi; vendor comparison on a self-destructing AWS spot instance
(`run-20260927T220605Z`, 33 min, ≈ $0.15). Summary: `P3_omni2141_summary.json`.

## Delivered

| Item | Module |
|---|---|
| Vendor numerics: percentile, trimmean, IQR, MAD sigma (old/new), bisquare robust fits, sequential sums | `signal/normalization/numerics.py` |
| Six-parameter fit (outliers, offset, rotation, shear, scale, nn12/rr12) and the two-parameter single-channel model; normalization 1.2.0 and 1.1.2 | `signal/normalization/fit.py` |
| Exact and binned (vendor) nearest-neighbour sweeps, vectorised | `signal/normalization/fit.py` |
| Transform application and θ/R | `signal/normalization/apply.py` |
| `simurg normalize` (per-sample transforms + normalized X/Y/θ/R Parquet) | `pipeline/normalize.py` |

## Acceptance (plan §18 P3: transforms within 2 ULP for ≥ 99.9 % of bins)

| Check | Data | Result |
|---|---|---|
| Transforms, normalization 1.2.0 | 36 GSA v4 samples × 48 bins, oracle = gtc2vcf built without FMA | **1728/1728 bins, 0 ULP** (all 10 fields incl. nn12, rr12, taa, tbb) |
| Transforms, normalization 1.1.2 | same samples, 1.1.2 oracle | **1728/1728 bins, 0 ULP** |
| Transforms vs **vendor GTCs** (AutoCall 1.6.2.2) | 2141 Omni2.5 samples × 51 bins | **1.1.2: 109 191/109 191 bins, 0 ULP; all 6 reserved fields identical in 2141/2141 samples** |
| Same vendor GTCs with 1.2.0 | | 1/109 191 bins (expected: that software predates 1.2.0) |
| `ingest` → `normalize` end to end | 18 GSA samples | bit-identical transforms |

The gate is exceeded: every bin is bit-identical (0 ULP), not just within 2 ULP.

## Findings

- The public 1000 Genomes GTCs were produced with normalization **1.1.2**; `simurg normalize --norm-version 1.1.2`
  reproduces them exactly.
- Compilers that fuse `a*b+c` into FMA instructions (gcc on aarch64 by default) shift the last bits of the reference
  C implementation by up to 4 ULP; the vendor's .NET code does not fuse. Oracle GTCs are therefore generated with
  `-ffp-contract=off` (`scripts/make_oracle_gtcs.sh`).

## Performance

| Machine | Normalization fit per sample |
|---|---|
| Raspberry Pi 5, 1 core, GSA (683 k loci) | 1.4 s |
| AWS m7g.4xlarge, 16 workers, Omni2.5 (2.45 M loci), both versions | 2141 samples in 8.5 min (≈ 3.8 s per sample-core for two fits) |