SimurgArray/docs/known_unknowns.md

Known unknowns and findings log

Plan §20 lists the open questions; this file records what the implementation has learned. Newest first.

ID	Date	Topic	Finding	Status
K-013	2026-09-27	Normalization version of public vendor GTCs	The 1000 Genomes Omni2.5 GTCs (AutoCall 1.6.2.2) are reproduced bit-for-bit by normalization 1.1.2 (109 191/109 191 bins, reserved fields included).	resolved
K-012	2026-09-27	FMA and bit-exactness	gcc on aarch64 contracts a*b+c into FMA by default; the gtc2vcf oracle then differs from the vendor by up to 4 ULP in some bins. Oracles are built with -ffp-contract=off. Any future C/Rust port (P9) must also forbid FMA contraction in exact mode.	resolved; constraint for P9
K-011	2026-09-27	Plan U1 (channel order)	Infinium II X = Red, Y = Grn confirmed on 2141 vendor GTCs (the GTC PDF's "Raw X = green" wording is wrong).	resolved
K-010	2026-09-27	Ingest throughput	On AWS, ingest of 2141 Omni samples was I/O-bound (gp3 125 MB/s baseline): 18.9 min, 4 of 16 cores busy. For P9 benchmarks provision gp3 throughput or use instance-store NVMe.	open (P9)
K-009	2026-09-27	Missing-intensity rule	3 loci (in 3 of 2141 vendor GTCs, AutoCall 1.6.2.2) keep one channel's intensity where we zero both. Hypothesis: old AutoCall applies the NBeads ≥ 2 rule per channel. Confirm with the bead counts of 5767788121_R01C01 (rs178512), 5826354152_R02C01 (SNP12-29082770), 6268201099_R02C01 (rs17133206); if confirmed, add a --missing-rule per-channel compatibility option.	open
K-008	2026-09-27	Vendor GTC layout	2141 vendor GTC v3 files (AutoCall 1.6.2.2) re-encode byte-identically with the canonical v3 order, so the writer reproduces the vendor layout (no padding or reordering).	resolved
K-007	2026-09-27	Demo data	The "GDA v1 Demo 36" download is a GenomeStudio project (.bsc + Data/*.bin), not IDATs. The GSA v4 PGx demo (idats_Demo_36.zip) is used as the P0 IDAT fixture instead. Bioconductor hapmap370k is no longer in the current release.	resolved (plan §17 updated)
K-006	2026-09-27	Sample sheet	Recognised input columns (Sample_ID, Sample_Name, …) collide case-insensitively with output columns sample_id/sample_name (IN-005). Recognised columns are exempt from the reserved-name check and are passed through verbatim.	decision, revisit in P4 (OUT-060 pass-through)
K-005	2026-09-27	Sample sheet	Illumina's GSA v4 template sheet has an empty Sample_Name column → E_SHEET_NAME_EMPTY (IN-010 behaviour as specified).	as specified
K-004	2026-09-27	GTC scanner data	PMT gains are 4 bytes read as int32 by BeadArrayFiles and written as float by gtc2vcf. We keep the raw 4 bytes (byte-exact round-trip) and expose the int32 view.	resolved
K-003	2026-09-27	Assay type from sequences	gtc2vcf's rule for inferring Infinium I colour from probe/source sequences (needed only for BPM locus entries < v6) agrees with the explicit BPM value for 99.71 % of GSA v4 Infinium I loci. The BPM is authoritative whenever it carries the assay type.	documented; test threshold 99.5 %
K-002	2026-09-27	EGT data block (plan U3)	GSA v4 and GDA v1 EGTs are data-block v9 with 15 extra floats per record; the first extra float is the intensity threshold (non-zero for 16 015 GDA loci). The reader tries candidate record sizes and accepts the one whose trailing cluster counts match. The Omni2.5 EGT is dbv 7 with 15 extra floats (structure confirmed); whether its first extra float is an intensity threshold (gtc2vcf) or unused (BeadArrayFiles) is still open.	partly resolved
K-001	2026-09-27	IDAT MidBlock (plan U2)	In GSA v4 (BeadChip 48x4), HumanCNV370 (2x12) and 450k (12x8) IDATs the MidBlock count equals N and the section is 4 bytes per entry (int32, as in illuminaio). gtc2vcf reads uint8 but never uses the values. We infer the element size from the section length.	resolved for these chips + BeadChip 4x10 (Omni2.5, 2141 files)