P6 validated against GTCtoVCF too: zero unexplained genotype differences

- GTCtoVCF (Apache-2.0, 36 GSA samples, merged sites): 23.66 M identical calls; every difference named
  (intensity-only probes merged by the oracle, its opposite-strand indel bug EDGE-012, normalisation-based
  grouping, OUT-018 ploidy); indel representations identical, including 144 gtc2vcf cannot resolve
- validate_p6_vcf.py: rule-based categories (--bpm for [I/D]/[D/I] detection)
- docs: validation P6 complete, README status
This commit is contained in:
kemal committed 2026-09-28 13:20:44 +01:00
1 parent c1699b4465
commit d625ef23c3
5 files changed
+2866 -18

No files matched your search

+26 -9
View File
@@ -31,20 +31,37 @@ Both sides left-aligned with `bcftools norm -f`, compared per probe and sample a
|---|---|---|
| identical calls | 23 656 010 | |
| identical no-calls | 102 077 | |
| our merged multi-allelic site differs from the single-probe call | 2 414 | e.g. an Infinium II [G/T] probe calls GG while another probe at the site shows G/C: C shares G's dye, so the probe cannot see it — our site genotype is consistent with every probe |
| our merged site missing, probe called | 3 945 | the probes at the site contradict each other (`W_MERGE_INCONSISTENT`) or leave several genotypes (`W_MERGE_AMBIGUOUS`) |
| our merged site called, probe no-call | 65 | other probes at the site resolve it |
| het in a haploid context → missing (OUT-018) | 14 | male X non-PAR and MT heterozygous calls |
| female Y → missing (OUT-018) | 11 | |
| indels gtc2vcf leaves unresolved (`REF=I/D`) that we resolve | 144 probes | checked against GTCtoVCF below |
| **unexplained differences** | **0** | |
| `grouping_differs` | 6 424 | the probe belongs to a SimurgArray multi-allelic site (kept merged even with `--unsquash-duplicates`, OUT-034); the site genotype uses every probe (e.g. an Infinium II [G/T] probe calls GG while the site is G/C: C shares G's dye) or is missing when the probes contradict each other |
| `ours_missing_het_in_haploid` | 16 | male X non-PAR and MT heterozygous calls → missing (OUT-018) |
| `ours_missing_haploid_Y` | 9 | Y of female samples → missing (OUT-018) |
| `representation_diff` | 5 184 (144 probes) | indels gtc2vcf leaves unresolved (`REF=I/D`); our representation agrees with GTCtoVCF (below) |
| **unexplained** | **0** | |
Summary: `P6_gsa36_vs_gtc2vcf.json`.
Locus sets: gtc2vcf writes 22 400 intensity-only loci that SimurgArray skips by design (OUT-019), and 49 indels it
cannot resolve either (`W_SKIP_INDEL_REF` for us). Nothing we write is missing from gtc2vcf.
### Genotypes vs GTCtoVCF (Apache-2.0), 36 samples, merged sites
### Genotypes vs GTCtoVCF (Apache-2.0), 36 samples, merged sites (default squashing)
(filled in below)
GTCtoVCF 1.2.1 run under Python 3 with two local compatibility patches (K-022), alt-contig loci filtered.
| Category | Count (probe × sample) | Explanation |
|---|---|---|
| identical calls | 23 656 575 | |
| identical no-calls | 109 760 | |
| `oracle_site_includes_probes_we_skip` | 2 917 | GTCtoVCF merges intensity-only CNV probes (e.g. `ilmnseq_CYP2C8_…_CNV`) into SNV sites; their alleles make its call ambiguous. SimurgArray skips intensity-only loci (OUT-019) |
| `oracle_opposite_strand_indel_bug` | 177 | `[I/D]` and `[D/I]` duplicate indel probes all heterozygous: we write `0/1`, GTCtoVCF 1.2.1 `./.` — the reference bug named in OUT-020/EDGE-012 |
| `grouping_differs` | 155 | indel probes designed 1–3 bp apart normalise to the same variant: we merge them (OUT-030), GTCtoVCF groups by manifest position; the probes disagree, so the merged site is missing |
| `ours_missing_het_in_haploid` | 17 | OUT-018 |
| `ours_missing_haploid_Y` | 9 | OUT-018 |
| `oracle_diploid_candidates_in_haploid_context` | 2 | tri-allelic male-X site: we enumerate haploid candidates (OUT-031 step 3) and get one call; GTCtoVCF always enumerates diploid ones |
| representation differences | 0 | every indel (including the 144 gtc2vcf cannot resolve) is written identically after left-alignment |
| **unexplained** | **0** | |
Probe sets: GTCtoVCF also writes the 22 400 intensity-only loci; it drops the 3 CYP2D6 indels whose `SourceSeq`
has the dash on the right (patched to skip instead of crash), which SimurgArray writes. Summary:
`P6_gsa36_vs_gtctovcf.json`.
### PLINK vs plink2 (GPL, subprocess oracle, built from source)