Add sanitized Phase-0 research archive under docs/research
Research notes R1-R4, documentation-study files A-E (source of the functional spec's `src:` tags) and PGx coverage CSVs, rewritten to use neutral naming and current project decisions (plan §2).
This commit is contained in:
1 parent
336a6b8226
commit
854cf95029
12 files changed
+7748
No files matched your search
@@ -0,0 +1,648 @@
|
||||
# R1 — the reference product: public-source product dossier
|
||||
|
||||
> **Archived Phase-0 study (sanitized copy, see `README.md`).** Written 2026-09-27, before the decisions in
|
||||
> `SIMURGARRAY_MASTER_PLAN.md` §2. The analysed vendor product is called "the reference product" and its CLI
|
||||
> `refcli`; its documentation URLs are replaced by `[ref-docs:…]` tokens. Where this file speaks of a clean room,
|
||||
> legal clearance, or running/benchmarking the vendor binary, plan §2 supersedes it: personal research; MIT/BSD/Apache
|
||||
> code may be reused with attribution; GPL tools are subprocess oracles only; vendor binaries are never run for
|
||||
> comparison, inspected or decompiled; oracles are published vendor-produced GTCs, gtc2vcf and BeadArrayFiles.
|
||||
|
||||
Compiled 2026-09-27 for a reimplementation feasibility study.
|
||||
**Only public sources were used.** No binaries were downloaded, run, or inspected.
|
||||
|
||||
Tag legend:
|
||||
- **[PUBLIC DOCUMENTATION]**: stated in an Illumina (or other named public) source. The URL is given.
|
||||
- **[INFERRED]**: a direct logical consequence of public statements. The reasoning is shown.
|
||||
- **[HYPOTHESIS]**: a plausible guess that still needs validation.
|
||||
|
||||
### Primary source: the official reference-product documentation site
|
||||
|
||||
- Site: [ref-docs:home]. It is also mirrored under [ref-docs:mirror].
|
||||
- Every page also has a Markdown copy: append `.md` to the page URL. The site index is [ref-docs:llms.txt].
|
||||
- Versioned docs live under `v1.0` to `v1.3`. The root path is v1.4, the latest release.
|
||||
|
||||
Short aliases used in this report:
|
||||
|
||||
| Alias | URL |
|
||||
|---|---|
|
||||
| `[DOC-HOME]` | [ref-docs:readme] |
|
||||
| `[DOC-APPS]` | [ref-docs:our-features] |
|
||||
| `[DOC-LOCAL]` | [ref-docs:local-analysis] |
|
||||
| `[DOC-IN]` | [ref-docs:input-files] |
|
||||
| `[DOC-OUT]` | [ref-docs:output-files] |
|
||||
| `[DOC-QC]` | [ref-docs:local-analysis/qc-report] |
|
||||
| `[DOC-FAQ]` | [ref-docs:faq] |
|
||||
| `[RN-1.0]` | [ref-docs:release-notes-1.0.0] |
|
||||
| `[RN-1.1]` | [ref-docs:release-notes-1.1.0] |
|
||||
| `[RN-1.2]` | [ref-docs:release-notes-1.2.0] |
|
||||
| `[RN-1.3]` | [ref-docs:release-notes-1.3.0] |
|
||||
| `[RN-1.4]` | [ref-docs:release-notes-1.4.0] |
|
||||
| `[DOC-REV]` | [ref-docs:document-revision-history] |
|
||||
| `[DOC-V10-LOCAL]` | [ref-docs:v1.0/local-analysis] |
|
||||
| `[DL]` | [ref-docs:downloads] |
|
||||
|
||||
---
|
||||
|
||||
## 1. What the reference product is
|
||||
|
||||
### 1.1 Identity
|
||||
|
||||
- **Name and scope.** "[product name] secondary analysis is a … bioinformatics software for Illumina Infinium array-based assays." [PUBLIC DOCUMENTATION] `[DOC-HOME]`
|
||||
- **Two packages.**
|
||||
- A local package with a command-line interface: "no specialized server or hardware required".
|
||||
- A cloud package with a GUI, running in the vendor cloud platform.
|
||||
- [PUBLIC DOCUMENTATION] `[DOC-HOME]`, `[DOC-FAQ]`
|
||||
- **Binary name.**
|
||||
- The CLI executable is **`refcli`** (`refcli.exe` on Windows). It sits in the `refcli` subfolder of the unpacked installer.
|
||||
- Version check: `refcli version`.
|
||||
- Example install path in the docs: `/usr/local/bin/<refcli>-linux-x64-DAv1.4.0/<refcli>/`.
|
||||
- [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, Illumina Knowledge 000010127 ([ref-docs:kb-000010127])
|
||||
- **Installer file names** (from the download page):
|
||||
- `…<Product>v1.4LinuxInstaller-1.4-<refcli>-linux-x64-DAv1.4.0-rc2-sha.9b051a4b….tar.gz` (about 60 MB)
|
||||
- `…-<refcli>-win-x64-DAv1.4.0-rc2-sha.9b051a4b….zip` (about 57 MB)
|
||||
- Both dated May 4, 2026. Downloading requires a MyIllumina login and a customer or instrument serial number.
|
||||
- [PUBLIC DOCUMENTATION] `[DL]`, Knowledge 000010127
|
||||
|
||||
### 1.2 Implementation technology
|
||||
|
||||
- **.NET application.**
|
||||
- "[product] v1.4 is a self-contained .NET 9 application … Linux users may need to install native OS-level dependencies." [PUBLIC DOCUMENTATION] `[RN-1.4]`, `[DOC-LOCAL]` (Linux Dependencies section)
|
||||
- A config file `refcli.runtimeconfig.json` exists. The docs suggest setting `System.Globalization.Invariant=true` in it as a workaround. [PUBLIC DOCUMENTATION] `[RN-1.0]`…`[RN-1.4]`
|
||||
- **Internal namespaces.** Stack traces published in every release note show:
|
||||
- `ArrayAnalysis.CLI.App`
|
||||
- `ArrayAnalysis.Core.GtcFileLoader` (source path `/src/ArrayAnalysis.Core/GtcFileLoader.cs`)
|
||||
- `ArrayAnalysis.Services.GtcFactory`
|
||||
- `ArrayAnalysis.Services.GtcToVcfService`
|
||||
- [PUBLIC DOCUMENTATION] `[RN-1.0]`
|
||||
- Inference: the `refcli` code base descends directly from the vendor's earlier array CLI (ACLI) code base. [INFERRED]
|
||||
|
||||
### 1.3 Release history
|
||||
|
||||
| Version | Date | Key content | Source |
|
||||
|---|---|---|---|
|
||||
| **v1.0.0** | Dec 2023 | Local CLI: genotyping (IDAT→GTC→VCF), PGx CNV, native PGx star-allele caller with quality scores and CPIC/DPWG annotation. PGx validated only on GDA-ePGx (`GDA_PGx…E2`). | `[RN-1.0]` |
|
||||
| Genotyping Cloud 1.0.0 | Mar 2024 | Cloud genotyping; Final Report and Locus Summary outputs. | [ref-docs:release-notes-1.0.0-cloud-genotype] |
|
||||
| Methylation QC Cloud 1.0.0 | May 2024 | Cloud-only methylation QC: 21 control metrics, detection p-values, beta values, M-values. | [ref-docs:release-notes-1.0.0-cloud-methylqc] |
|
||||
| **v1.1.0** | Sep 2024 | EX PGx chips (GSAv4-ePGx, GCRA-ePGx). More PGx genes. A custom EGT can be used with the stock CN model (warning only). JSON schema renames. | `[RN-1.1]` |
|
||||
| **v1.2.0** | Feb 2025 | Whole-genome **cytogenetics** CNV/LOH calling, VCF output, BAF/LRR bedgraph, ISCN 2020 / cytoband annotation JSON. New `pgx` root command. New license server `[ref-license-server]`. | `[RN-1.2]` |
|
||||
| **v1.3.0** | Aug 2025 | Mosaic-fraction estimation. PAR/sex-chromosome improvements. GenomeStudio-compatible sample sheet. Failed samples stay in `gt_sample_summary`. `--smoothing` removed from gtc-to-bedgraph. | `[RN-1.3]` |
|
||||
| Methylation QC Cloud 1.0.1 | Feb 2026 | Maintenance release. | [ref-docs:release-notes-1.0.1-cloud-methylqc] |
|
||||
| **v1.4.0** | May 2026 (latest) | **QC Report** (`qc call`, `qc report`; "replacing the GenomeStudio control dashboard"). GAINLOH calling. Better mosaic detection. PGx SV star alleles and a new CSV set. `--use-infI-nc-info` for multi-allelic variant (MAV) calling. .NET 9. | `[RN-1.4]` |
|
||||
|
||||
All rows above are [PUBLIC DOCUMENTATION]. Additional notes:
|
||||
|
||||
- The v1.0.0 cloud pipeline versions were deprecated in Q1 2026. [PUBLIC DOCUMENTATION] `[RN-1.0]`
|
||||
- Documentation revisions run from 01 (Dec 2023) to 14 (Aug 2026). Revision 14 renames the vendor cloud platform's data layer. [PUBLIC DOCUMENTATION] `[DOC-REV]`
|
||||
|
||||
### 1.4 Relationship to other Illumina tools
|
||||
|
||||
**The vendor's earlier array CLI (ACLI)**
|
||||
- Illumina calls it "the predecessor to [product]". [PUBLIC DOCUMENTATION] Illumina Knowledge 000003079 (https://knowledge.illumina.com/microarray/general/microarray-general-reference_material-list/000003079)
|
||||
- "ACLI was obsolesced in December 2024, and support ended in July 2025." [PUBLIC DOCUMENTATION] (same source)
|
||||
- ACLI v2.1 subcommands: `genotype call`, `genotype gtc-to-vcf`, `copy-number call`, `copy-number train`. Options such as `--bpm-manifest`, `--cluster-file`, `--idat-folder` match `refcli`'s option names. [PUBLIC DOCUMENTATION] https://support-docs.illumina.com/ARR/ArrayAnalysisCLI/Content/ARR/IMA/RunAnalysis_Local.htm and https://sapac.support.illumina.com/array/array_software/ima-array-analysis-cli.html
|
||||
- The reference product's v1.0 also had top-level `copy-number` and `star-allele` commands. v1.2 regrouped them under `pgx`. [PUBLIC DOCUMENTATION] `[DOC-V10-LOCAL]`, `[RN-1.2]`
|
||||
|
||||
**IAAP (Illumina Array Analysis Platform) CLI and AutoConvert**
|
||||
- These are earlier .NET GenCall front-ends (IDAT→GTC). [PUBLIC DOCUMENTATION] https://emea.support.illumina.com/downloads/iaap-genotyping-cli.html
|
||||
- Third-party gtc2vcf (MIT license) reports that "AutoConvert 2.0, IAAP-CLI, and ACLI … perform version 1.2.0 of the normalization step and seem to produce the exact same results", while the original AutoConvert uses normalization 1.1.2. [PUBLIC DOCUMENTATION – third party] https://github.com/freeseek/gtc2vcf
|
||||
- The reference product probably kept normalization 1.2.0 (or a successor) and GenTrain/GenCall-compatible calling. It still reads EGT cluster files and writes GTC v5, and GTCs from AutoConvert/Beeline are accepted by `gtc-to-vcf` as "legacy GTCs". [HYPOTHESIS], supported by `[DOC-OUT]`
|
||||
|
||||
**GenomeStudio 2.0**
|
||||
- It is still the tool for **creating custom cluster files (EGT)**. The reference product does not cluster; it only consumes EGTs. [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, section "Optimizing cluster files"
|
||||
- The reference-product QC Report "replaces GenomeStudio control dashboard". [PUBLIC DOCUMENTATION] `[DOC-FAQ]`, `[RN-1.4]`
|
||||
- The cloud Final Report and Locus Summary are "similar to the one available in GenomeStudio". [PUBLIC DOCUMENTATION] `[DOC-OUT]`
|
||||
- For PGx cluster review, "GenomeStudio auxiliary files" (`*_GS_import*.zip`) are provided. [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, `[DL]`
|
||||
|
||||
**GTCtoVCF (github.com/Illumina/GTCtoVCF, Apache-2.0, Python 2.7)**
|
||||
- Its README says: "As of May 2023, it is now recommended to use … [product] for GTC to VCF conversion." [PUBLIC DOCUMENTATION] https://github.com/Illumina/GTCtoVCF
|
||||
- `refcli genotype gtc-to-vcf` reuses GTCtoVCF's option names and near-verbatim documentation text:
|
||||
- `--unsquash-duplicates`, `--auxiliary-loci`, `--filter-loci`, `--disable-genome-cache`
|
||||
- the "Squashing duplicates", "Genome cache", "Auxiliary loci" and "combined genotype" paragraphs, and the dbSNP mapping conventions
|
||||
- Inference: the refcli implementation is a port or evolution of GTCtoVCF logic. That makes GTCtoVCF (open source, Apache-2.0) a strong **legal reference implementation** for the VCF conversion layer. [INFERRED] Compare `[DOC-LOCAL]` with the GTCtoVCF README.
|
||||
|
||||
**The vendor cloud platform**
|
||||
- The cloud edition of the reference product runs in the vendor cloud platform UI. Data is stored on the vendor cloud platform.
|
||||
- An iScan can stream IDATs to the cloud directly.
|
||||
- Analysis types in the vendor cloud platform dropdown:
|
||||
- "[product] – Genotyping and QC"
|
||||
- "– PGx – CNV calling"
|
||||
- "– PGx – Star allele annotation"
|
||||
- "– Cytogenetics analysis"
|
||||
- "- Cytogenetics analysis + tertiary interpretation" (cloud only)
|
||||
- "– Methylation – QC" (cloud only)
|
||||
- Tertiary-interpretation platform integration uses vendor cloud-platform SNS notifications.
|
||||
- [PUBLIC DOCUMENTATION] `[DOC-APPS]`, [ref-docs:cloud-analysis], [ref-docs:cloud-analysis/cytogenetics-analysis]
|
||||
- Cloud sample limits: max 1152 samples for genotyping, cytogenetics and methylation; max 384 for the PGx types. [PUBLIC DOCUMENTATION] `[DOC-APPS]`
|
||||
|
||||
**Relationship to the vendor's NGS secondary-analysis product of the same brand**
|
||||
- The reference product shares only branding with it: no FPGA, a different code base, CPU only.
|
||||
- The developer article states: "The software runs on CPUs and does not require a specialized server or FPGA hardware."
|
||||
- [PUBLIC DOCUMENTATION] [ref-docs:dev-article-1.0-pgx]
|
||||
|
||||
---
|
||||
|
||||
## 2. Supported arrays and species
|
||||
|
||||
All of this section is [PUBLIC DOCUMENTATION] from `[DOC-APPS]` ("Product & Analysis Compatibility") unless noted.
|
||||
|
||||
### 2.1 Genotyping
|
||||
|
||||
"Any Infinium genotyping array, including human (including custom and semi-custom) and non-human diploid species".
|
||||
|
||||
Tested manifests:
|
||||
|
||||
| Array | Manifest | Genome build |
|
||||
|---|---|---|
|
||||
| ASA | ASA-24v1-0_E2 | GRCh37/38 |
|
||||
| BovineSNP50 v3 | BovineSNP50_v3_A1 | UMD3 (non-human) |
|
||||
| CGA | CGA-24v1-0_A1 | GRCh37/38 |
|
||||
| GCRA | GCRA-24v1-0_20068388_B2 | GRCh37/38 |
|
||||
| GDA | GDA-8v1-0_D2 | GRCh37/38 |
|
||||
| GSA v3 | GSA-24v3-0_A2 | GRCh37/38 |
|
||||
| GSA v4 (Infinium EX) | GSA-48v4-0_20085471_D2 | GRCh37/38 |
|
||||
| JSA | JSA-24v1-0_B2 | GRCh37/38 |
|
||||
| PRSbooster | PRSbooster_20083382_A | GRCh37 |
|
||||
|
||||
Species support:
|
||||
- Non-human diploid species work if the user supplies a reference FASTA. For non-human data, turning off VCF output is recommended on cloud.
|
||||
- "Genotyping only supports diploid organisms. Polyploid genotyping is currently not supported." `[RN-1.1]`…`[RN-1.4]`
|
||||
|
||||
### 2.2 PGx (CNV and star alleles)
|
||||
|
||||
Supported only for these commercial ePGx products, all on GRCh38:
|
||||
- GDA-ePGx `GDA_PGx-8v1-0_20042614_G4`
|
||||
- GSAv4-ePGx `GSA-PGx-48v4-0_20079540_E2`
|
||||
- GCRA-ePGx `GCRA-PGx-24v1-0_20084467_C2`
|
||||
|
||||
Semi-custom arrays get PGx calls only for the commercial backbone content. `[DOC-LOCAL]` (semi-custom section)
|
||||
|
||||
### 2.3 Cytogenetics (CNV/LOH)
|
||||
|
||||
Validated arrays:
|
||||
- GDA Cyto-8 (`GDACyto-8v1-0_20047166_E2`)
|
||||
- GSA Cyto-24 (`GSACyto-24v1_20044998_C2`)
|
||||
- CytoSNP-850K v1.4, iScan (`_iScan_B2`) and NextSeq550 (`_NS550_B2`, from v1.3)
|
||||
|
||||
Details:
|
||||
- Builds GRCh37 and GRCh38.
|
||||
- Constitutional samples only; "oncology samples not supported". `[RN-1.2]`, `[RN-1.4]`
|
||||
- The feature list says "any human genotyping array" `[RN-1.2]`, but custom products need a cyto model file from Tech Support. `[DOC-IN]`
|
||||
|
||||
### 2.4 Methylation
|
||||
|
||||
- **In scope, but only as cloud "Methylation QC".** It is **not available in the local CLI** as a methylation pipeline. `[DOC-APPS]`
|
||||
- Supported arrays: EPIC v1 (B5 manifest), EPIC v2 (`EPIC-8v2-0_A2`), EPIC-Xtra (custom), and MSA (`MSA-48v1-0_20102838_A1`), all GRCh38.
|
||||
- The local `refcli qc call --array-type methylation` does compute methylation **control-probe** QC metrics, and `qc report` can report them. There is no local beta-value, M-value or detection-p pipeline. [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, `[DOC-QC]`
|
||||
- Cloud methylation QC outputs, per sample:
|
||||
- `{Barcode}_{Pos}_ctrl.tsv.gz` (control probes)
|
||||
- `{Barcode}_{Pos}_cgs.tsv.gz` (beta values, M-values, detection p-values)
|
||||
- Cloud methylation QC outputs, per batch:
|
||||
- `sample_qc_summary.{xlsx,tsv}`
|
||||
- plots, including a PCA of beta values
|
||||
- `[DOC-OUT]`
|
||||
|
||||
### 2.5 Instruments
|
||||
|
||||
- IDATs come from iScan. NextSeq550 IDATs are accepted for select arrays. [PUBLIC DOCUMENTATION] `[DOC-IN]`
|
||||
|
||||
---
|
||||
|
||||
## 3. Workflows and algorithms (what is and isn't disclosed)
|
||||
|
||||
### 3.1 Genotyping (IDAT → GTC → VCF)
|
||||
|
||||
**What the docs state** [PUBLIC DOCUMENTATION]:
|
||||
- Pipeline: `genotype call` (IDAT + BPM + EGT → GTC v5), then `genotype gtc-to-vcf` (GTC + BPM + CSV manifest + FASTA → per-sample `*.snv.vcf.gz` + `.tbi`).
|
||||
- `--gencall-cutoff` defaults to **0.15** ("GenCall score cutoff to label a NoCall"). `[DOC-LOCAL]`
|
||||
- The GTC holds genotypes, GenCall scores, BAF, LRR and sample QC. It is the "Binary proprietary format", parseable with https://github.com/Illumina/BeadArrayFiles. `[DOC-OUT]`
|
||||
- In the SNV VCF, `FILTER` is always `PASS`. FORMAT fields are `GT:GS:BAF:LRR`. Genotypes are ploidy-adjusted: haploid on Y, MT and non-PAR X in males. `[DOC-OUT]`
|
||||
- BAF and LRR are oriented REF=A / ALT=B relative to the reference. They may therefore be 1 − the GenomeStudio value. `[DOC-OUT]`
|
||||
- Multi-allelic / duplicate-site merging:
|
||||
- Enumerate the candidate genotypes from the queried alleles.
|
||||
- Filter out genotypes inconsistent with each assay's call, respecting design limits. For example, Infinium II cannot tell A/T or C/G apart.
|
||||
- Exactly one survivor becomes the call; zero or more than one gives a no-call.
|
||||
- For merged records, `GS` is the minimum GS across assays. MAV records carry no BAF or LRR.
|
||||
- `--use-infI-nc-info` (v1.4) treats Infinium I no-calls as "missing signal" rather than "probe failure", which removes alleles. It requires a cluster file with "zeroed-out" MAV assays.
|
||||
- `[DOC-OUT]`, section "Note on Multi-Allelic Variants"
|
||||
- Probe IDs in the `ID` field are joined with "," for SNPs and ";" for indels. `[DOC-OUT]`
|
||||
|
||||
**Algorithms not disclosed for the reference product itself:** GenCall normalization and calling.
|
||||
- The GTC stores per-sample `NormalizationTransform` records: version, offset_x/y, scale_x/y, shear, theta. The public BeadArrayFiles code shows how they are **applied**: subtract offset → rotate by theta → shear → scale → clamp ≥0. It then computes R = x+y and θ = atan2(y,x)·2/π. [PUBLIC DOCUMENTATION] https://github.com/Illumina/BeadArrayFiles/blob/develop/module/GenotypeCalls.py
|
||||
- How the transform parameters are **fit** is only described qualitatively in the tech notes (see §8). [PUBLIC DOCUMENTATION]
|
||||
|
||||
### 3.2 PGx copy number (targeted)
|
||||
|
||||
**Scope** [PUBLIC DOCUMENTATION] [ref-docs:pharmacogenomics-pgx-reference/pgx-cnv-coverage]
|
||||
- 7 genes across 10 regions (GRCh38): GSTM1, UGT2B17, CYP2E1, SULT1A1, CYP2A6 (intron.7, exon.1), CYP2D6 (exon.9, intron.2, p5), and GSTT1 (on `22_KI270879v1_alt`).
|
||||
|
||||
**Batch-based calling** [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`
|
||||
- Needs ≥24 samples, with ≥22 passing LogRDev < 0.2. ≥96 are recommended; about 500 GDA-ePGx samples fit per batch on the reference hardware.
|
||||
- Error text when too few pass: "{n} sample(s) passed QC. Requires at least {m} samples". `[DOC-OUT]`
|
||||
|
||||
**Model file and training** [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, `[DOC-IN]`
|
||||
- Requires a manifest-revision-specific **CN model `.dat`**.
|
||||
- The model is trainable with `pgx copy-number train`:
|
||||
- needs ≥96 samples, of which ≥90 pass LogRDev ≤ 0.2; ≥150 are recommended, with diminishing returns beyond 3,000
|
||||
- needs `--platform LCG|EX`
|
||||
- needs a `.msk` mask file next to the BPM, plus the FASTA
|
||||
- The trainer "is designed with the assumption that the copy number distribution resembles the standard population distributions".
|
||||
- v1.4 turns an EGT mismatch between GTC and CN model into a hard error. v1.1–1.3 only warned. `[RN-1.4]`
|
||||
|
||||
**Output** [PUBLIC DOCUMENTATION] `[DOC-OUT]`
|
||||
- `*.cnv.vcf.gz` with `<CNV>` records, `FORMAT CN:NR`. CN is capped at 5 ("5 or 5+"). NR is the "aggregated normalized intensity".
|
||||
- QUAL is Phred-scaled, capped at 60. `Q7` filter below 7; `SampleQuality` filter when LogRDev > 0.2.
|
||||
- Header keys `##OverallPloidy=` and `##GCCorrect=True`, which implies GC correction of intensities.
|
||||
- Batch files: `cn_batch_summary.csv` (gain/loss/neutral per region) and a CN summary file.
|
||||
|
||||
**Algorithm** [HYPOTHESIS]
|
||||
- Not disclosed. The developer article gives accuracy only: CYP2D6 CNV "98.3%–99.0%" versus qPCR on 288 samples.
|
||||
- Likely shape: batch-normalized, GC-corrected, probe-aggregated normalized R (the "NR" field), compared against per-region reference distributions in the `.dat` model (population-trained). Plausibly a Gaussian-mixture or likelihood model over CN states 0–5.
|
||||
|
||||
### 3.3 PGx star alleles
|
||||
|
||||
**Commands and licensing** [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`
|
||||
- `pgx star-allele call` (VCF folder with `*.snv.vcf.gz` + `*.cnv.vcf.gz`, plus a PGx database `.zip`) → `star_alleles.dat`.
|
||||
- Then `pgx star-allele annotate --guidelines CPIC|DPWG` → CSV and JSON.
|
||||
- **Star-allele calling is the only paid, license-server-gated local step.** It is charged per sample through `--license-server-url https://user:pass@[ref-license-server]` (v1.2+), or `[ref-license-server-legacy]` for v1.0/1.1.
|
||||
|
||||
**Data sources** [PUBLIC DOCUMENTATION] [ref-docs:pharmacogenomics-pgx-reference/pgx-allele-definitions-and-pgx-guidelines]
|
||||
- PharmVar 6.1, PharmGKB snapshot 2024-05-16, UGT nomenclature, legacy CYP nomenclature, CPIC 1.38.0, DPWG June 2023.
|
||||
- CPIC phenotypes cover 17 genes. Activity scores are given for CYP2C9, CYP2D6 and DPYD.
|
||||
- Coverage: "2400+ targets for over 50 genes" `[DOC-HOME]`; the product page says "1700+ targets for 50+ genes".
|
||||
|
||||
**Scoring and reporting** [PUBLIC DOCUMENTATION] `[RN-1.0]`, `[DOC-OUT]`
|
||||
- Quality score is a "negative log likelihood" in v1.0.
|
||||
- Later, "a probability score … based on the SNV and CNV inputs, input quality, and population frequencies" (0–1), plus a `Raw Score` that excludes population frequency.
|
||||
- Ranked alternative solutions; hybrid alleles; allele-specific CN.
|
||||
- Alternatives are shown if their score is ≥50% of the top score.
|
||||
- No novel alleles are called.
|
||||
|
||||
**PGx database format** [PUBLIC DOCUMENTATION] `[DOC-IN]`
|
||||
- "each line … a single probe ID mapping to a variant's HGVS tag". The public database zip is named `DAv1.4.0-rc2-pgx-mapping-sha.b164b4aa….zip`. `[DL]`
|
||||
|
||||
**v1.4 CSV outputs** [PUBLIC DOCUMENTATION] `[DOC-OUT]`
|
||||
- `diplotype_summary.csv`
|
||||
- `diplotype_annotations.csv`
|
||||
- `supporting_variants.csv`
|
||||
- `unresolved_variants.csv`
|
||||
- `variants_not_covered_on_array.csv`
|
||||
- plus a per-sample star-allele JSON. Its fields are listed in `[DOC-OUT]`.
|
||||
|
||||
**Algorithm** [HYPOTHESIS]
|
||||
- Not disclosed. It is described only as "a new and native PGx star allele caller" (developer article).
|
||||
- Possible approach: a haplotype-pair likelihood search over PharmVar definitions. It would take CN constraints from the CNV VCF and error or missing-data tolerance, and use population-frequency priors, which would explain the "Score" versus "Raw Score" split.
|
||||
- The v1.x error "The model has been changed since the solution was last computed" is typical of Google OR-Tools / a MIP/CP solver. That suggests an **optimization (ILP/CP-SAT) formulation**. [HYPOTHESIS]
|
||||
|
||||
### 3.4 Cytogenetics CNV/LOH (genome-wide)
|
||||
|
||||
**Commands** [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`
|
||||
- `cyto call --cn-model cyto_model.dat` (GTC → `*.cnv.vcf.gz` + `cyto.cnv.dat`).
|
||||
- `cyto annotate --annotation-db CytoAnnotateData_*.zip` (→ per-sample JSON with ISCN 2020 notation, cytobands, RefSeq/Ensembl genes, QC).
|
||||
|
||||
**Model file** [PUBLIC DOCUMENTATION] `[DOC-IN]`
|
||||
- The cyto model "primarily contains probe GC content information and is tied to the probe sequence/manifest".
|
||||
- It does **not** need retraining when the EGT changes.
|
||||
|
||||
**Parameters** [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`
|
||||
|
||||
| Event type | Default minimum probes | Default minimum size |
|
||||
|---|---|---|
|
||||
| DEL | 10 | 25 kb |
|
||||
| DUP | 10 | 50 kb |
|
||||
| LOH | 500 | 3000 kb |
|
||||
| GAINLOH | 300 | 500 kb |
|
||||
| mosaic DEL (mdel) | 10 | 25 kb |
|
||||
| mosaic DUP (mdup) | 10 | 50 kb |
|
||||
| mosaic LOH (mloh) | 500 | 3000 kb |
|
||||
|
||||
Other parameters:
|
||||
- `--smoothing 5` (probes on each side)
|
||||
- `--max-mosaic-fraction 1.0`
|
||||
- `--min-qual 20` (annotate)
|
||||
- Minimum probe thresholds below 10 are reset to 10.
|
||||
|
||||
**Behaviours documented** [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, `[DOC-OUT]`, `[RN-1.2]`…`[RN-1.4]`
|
||||
- CN up to 4+.
|
||||
- QUAL = −10·log10(1 − P(called state)), capped at 60; `Q10` filter.
|
||||
- LogRDev > 0.2 → `SampleQuality` flag. LogRDev > 0.3 → only whole-chromosome events are reported.
|
||||
- Mosaic events are detected down to about 15% for one-copy mosaic states. `MF` holds the mosaic fraction.
|
||||
- LOHTYPE ∈ {AOH, CNLOH, GAINLOH}.
|
||||
- LOH < 500 kb flanked by copy-neutral regions is removed (v1.4).
|
||||
- Variants must match the expected LRR sign (v1.4).
|
||||
- Effective size excludes gaps larger than 150× the median probe spacing.
|
||||
- An unknown sex is resolved by the cyto caller or else assumed male.
|
||||
- `GT` is hardcoded to `1/1`.
|
||||
|
||||
**Algorithm** [HYPOTHESIS]
|
||||
- Not disclosed. The outputs (per-state posterior → Phred QUAL; joint LRR+BAF states including LOH, GAINLOH and mosaic fractions; GC correction; smoothing window) are consistent with an **HMM or segmentation + state-likelihood model over LRR and BAF**, in the family of PennCNV / cnvPartition / the reference product "cytogenetics modality".
|
||||
- cnvPartition is Illumina's older published recursive-partition approach (see §8).
|
||||
|
||||
### 3.5 QC (v1.4)
|
||||
|
||||
[PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, `[DOC-QC]`, `[DOC-OUT]`
|
||||
- `qc call --array-type genotyping|methylation` → `controls.raw_metrics.csv`, `controls.qc_metrics.csv`. These files are also emitted automatically by `genotype call`.
|
||||
- `qc report` → `<Product>_QC_Report_YYYY_MM_DD.html` (self-contained, interactive) and `<Product>_QC_table_YYYY_MM_DD.{xlsx|csv}`.
|
||||
|
||||
### 3.6 Other items
|
||||
|
||||
- **Locus Report / Final Report / Locus Summary.**
|
||||
- Cloud-only outputs: `gtc_final_report.csv` (GenomeStudio-style) and `locus_summary.csv`, with columns Call_Freq, Het_Excess_Freq, ChiTest_P100, Cluster_Sep, AA/AB/BB θ and R mean/SD, and so on.
|
||||
- No locus report exists in the local CLI.
|
||||
- [PUBLIC DOCUMENTATION] `[DOC-OUT]`, `[DOC-APPS]`
|
||||
- **Cluster file generation.** Not supported; it is done in GenomeStudio 2.0. [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`
|
||||
- **BedGraph.** `genotype gtc-to-bedgraph` → `*.LRR.bedgraph`, `*.BAF.bedgraph` for IGV. [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, `[RN-1.2]`
|
||||
|
||||
---
|
||||
|
||||
## 4. Command-line interface (v1.4, exact)
|
||||
|
||||
Syntax: `refcli [module] [sub-module] [command] [required] [optional]`. Options are case-sensitive. [PUBLIC DOCUMENTATION] `[DOC-LOCAL]`
|
||||
|
||||
The following commands are all [PUBLIC DOCUMENTATION] from `[DOC-LOCAL]`:
|
||||
|
||||
```
|
||||
refcli version | help
|
||||
refcli qc call --array-type {genotyping|methylation} --csv-manifest F (--idat-folder D | --sample-sheet F) [--output-folder] [--debug] [--json-log]
|
||||
refcli qc report --data D1[,D2..|PARENT] [--config cfg.yaml] [--label L1,..] [--output-folder] [--output-format xlsx|csv]
|
||||
[--threshold-callRate 0.98] [--threshold-logRdev 0.20] [--debug] [--json-log]
|
||||
refcli genotype call --bpm-manifest F.bpm --cluster-file F.egt (--idat-folder D | --sample-sheet F) [--gencall-cutoff 0.15]
|
||||
[--num-threads N] [--output-folder] [--debug] [--json-log]
|
||||
refcli genotype gtc-to-bedgraph --bpm-manifest F.bpm (--gtc-folder D | --sample-sheet F) [--output-folder] ...
|
||||
refcli genotype gtc-to-vcf --bpm-manifest F.bpm --csv-manifest F.csv --genome-fasta-file G.fa (--gtc-folder D | --sample-sheet F)
|
||||
[--auxiliary-loci aux.vcf] [--disable-genome-cache] [--filter-loci list.txt] [--unsquash-duplicates]
|
||||
[--no-bgzip] [--use-infI-nc-info] [--output-folder]
|
||||
refcli pgx copy-number call --cn-model M.dat (--gtc-folder D | --sample-sheet F) [--no-bgzip] [--output-folder]
|
||||
refcli pgx copy-number train --bpm-manifest F.bpm (mask .msk beside it) --genome-fasta-file G.fa --platform {LCG|EX}
|
||||
(--gtc-folder D | --sample-sheet F) [--disable-genome-cache] [--output-folder]
|
||||
refcli pgx star-allele call --database pgx.zip --license-server-url URL --vcf-folder D [--query-license-quota] [--output-folder]
|
||||
refcli pgx star-allele annotate --star-alleles star_alleles.dat [--guidelines CPIC|DPWG] [--output-folder]
|
||||
refcli cyto call --cn-model cyto_model.dat (--gtc-folder D | --sample-sheet F) [--no-bgzip] [--smoothing 5]
|
||||
[--min-{del,dup,loh,gainloh,mdel,mdup,mloh}-{probes,size} ...] [--max-mosaic-fraction 1.0] [--output-folder]
|
||||
refcli cyto annotate --annotation-db CytoAnnotateData.zip --vcf-folder D [--min-* ...] [--min-qual 20] [--output-folder]
|
||||
```
|
||||
|
||||
Common flags on every command: `--debug`, `--json-log`, `--help`, `--version`.
|
||||
|
||||
Behaviour notes:
|
||||
- Output folders are overwritten on re-run.
|
||||
- A `--idat-folder` is searched recursively.
|
||||
- Duplicate IDAT pairs per barcode_position are unsupported.
|
||||
|
||||
### 4.1 Inputs
|
||||
|
||||
[PUBLIC DOCUMENTATION] `[DOC-IN]`
|
||||
|
||||
| Input | Extension | Used by |
|
||||
|---|---|---|
|
||||
| IDAT (Grn/Red pair per `Barcode_RxxCxx`) | `.idat` | genotype call, qc call |
|
||||
| BPM manifest | `.bpm` | genotype call/gtc-to-vcf/gtc-to-bedgraph, pgx cn train |
|
||||
| CSV manifest (with SourceSeq, RefStrand; dbSNP-style 1-based mapping) | `.csv` | gtc-to-vcf, qc call |
|
||||
| Cluster file | `.egt` | genotype call |
|
||||
| PGx CN model | `.dat` | pgx copy-number call |
|
||||
| Cyto model | `.dat` | cyto call |
|
||||
| Mask | `.msk` (beside the BPM) | pgx copy-number train |
|
||||
| PGx database | `.zip` | star-allele call |
|
||||
| Cyto annotation database (RefSeq + Ensembl) | `.zip` | cyto annotate |
|
||||
| Genome FASTA + `.fai`; contigs "1", not "chr1" | `.fa` | gtc-to-vcf, pgx cn train |
|
||||
| QC config | `.yaml` / `.yml` | qc report |
|
||||
|
||||
Sample sheet (CSV):
|
||||
- Keys are `SentrixBarcode_A,SentrixPosition_A`. On cloud they are `beadChipName,sampleSectionName`.
|
||||
- Optional `[Header]` with `RootFolder` and `[Data]` sections. The format is compatible with GenomeStudio sheets, and `[Manifests]` is ignored.
|
||||
- Column names are case-insensitive, and reserved names are rejected.
|
||||
- Values are limited to 500 characters.
|
||||
- Sample-name precedence: sheet `Sample_Name` > IDAT/GTC metadata > `Barcode_Position`.
|
||||
|
||||
Public product files: GRCh37/GRCh38 FASTA zips, per-product BPM/CSV/EGT, CN/cyto model and database zips, QC YAML templates, and demo IDATs.
|
||||
- Examples: GDA-ePGx demo data set (iScan), 15.4 GB; GCRA/GSA-PGx `idats_Demo_36.zip`.
|
||||
- [PUBLIC DOCUMENTATION] `[DL]`
|
||||
- Note: model and database files are Illumina data files; their licence terms were not reviewed.
|
||||
|
||||
### 4.2 Outputs
|
||||
|
||||
[PUBLIC DOCUMENTATION] `[DOC-OUT]`
|
||||
|
||||
**Per sample**
|
||||
- `.gtc` (GTC v5)
|
||||
- `*.snv.vcf.gz` + `.tbi`. Header `##source=refcli 1.4.0`, VCFv4.1. Tabix output is not byte-identical to `bcftools index`.
|
||||
- `*.LRR.bedgraph`, `*.BAF.bedgraph`
|
||||
- PGx `*.cnv.vcf.gz`
|
||||
- cyto `*.cnv.vcf.gz`; header `##source=refcli 1.4.0 Cyto`, `##product=…`
|
||||
- cyto annotation `*.json`
|
||||
- star-allele JSON
|
||||
- methylation (cloud only): `_ctrl.tsv.gz`, `_cgs.tsv.gz`
|
||||
|
||||
**Per batch**
|
||||
- `gt_sample_summary.csv` and `.json`. Columns: `Sample ID, Sample Name, Sample Folder, Autosomal Call Rate, Call Rate, Log R Ratio Std Dev, Sex Estimate, [TGA_Ctrl_5716 Norm R], <samplesheet columns>`.
|
||||
- `controls.raw_metrics.csv`, `controls.qc_metrics.csv`
|
||||
- `<Product>_QC_Report_*.html`, `<Product>_QC_table_*.xlsx|csv`
|
||||
- `cn_batch_summary.csv`, CN summary
|
||||
- `star_alleles.dat`
|
||||
- the five PGx CSVs (§3.3)
|
||||
- `cyto.cnv.dat`
|
||||
- cloud only: `gtc_final_report.csv`, `locus_summary.csv`, `sample_qc_summary.{xlsx,tsv}`
|
||||
|
||||
**Logs**
|
||||
- Warnings and errors go to the terminal, optionally as JSON (`--json-log`).
|
||||
- Error strings are catalogued in `[DOC-OUT]`, for example "Skipping intensity only locus", "Failed to combine genotypes due to inconsistencies …", and "Cluster file (…) is not the same as CN Model Cluster file".
|
||||
|
||||
---
|
||||
|
||||
## 5. QC metrics and thresholds
|
||||
|
||||
### 5.1 Functional per-sample metrics (`gt_sample_summary`)
|
||||
|
||||
[PUBLIC DOCUMENTATION] `[DOC-OUT]`, `[DOC-QC]`, `[DOC-FAQ]`
|
||||
- Metrics: Autosomal Call Rate, Call Rate, Log R Ratio Std Dev (LogRDev), Sex Estimate (M/F/Unknown), and TGA_Ctrl_5716 Norm R.
|
||||
- TGA_Ctrl_5716 Norm R is present on PGx products only. It is the normalized R of one probe; a value < 1 means the targeted gene amplification (TGA) failed.
|
||||
- Failed samples get call rates of 0 and NaN for LogRDev (v1.3+).
|
||||
|
||||
**Documented thresholds** [PUBLIC DOCUMENTATION]
|
||||
|
||||
| Metric / rule | Value | Source |
|
||||
|---|---|---|
|
||||
| `qc report` default callRate | 0.98 (PASS if Autosomal Call Rate ≥ threshold) | `[DOC-LOCAL]`, `[DOC-QC]` |
|
||||
| `qc report` default logRDev | 0.20 (PASS if ≤) | `[DOC-LOCAL]`, `[DOC-QC]` |
|
||||
| Generic genotyping YAML example logRDev | 0.15 | `[DOC-QC]` |
|
||||
| PGx CNV: per-sample pass | LogRDev < 0.2 (≤ 0.2 for training) | `[DOC-LOCAL]` |
|
||||
| PGx CNV: batch minimum | ≥ 22 of ≥ 24 passing | `[DOC-LOCAL]` |
|
||||
| PGx star-allele: review triggers | LogRDev > 0.2, call rate < 0.99, TGA control < 1.0 | `[DOC-LOCAL]` |
|
||||
| Cyto: low-quality flag | LogRDev > 0.2 → SampleQuality | `[DOC-LOCAL]` |
|
||||
| Cyto: whole-chromosome-only mode | LogRDev > 0.3 | `[DOC-LOCAL]` |
|
||||
|
||||
**Not defined in the reference-product docs**
|
||||
- The exact **Sex Estimate** algorithm.
|
||||
- [HYPOTHESIS] It likely mirrors GenCall/GenomeStudio "GenderEst" (X heterozygosity plus Y intensity, with an autosomal call-rate gate). A public QC SOP describes GenomeStudio as estimating sex only when autosomal call rate > 0.97: https://khp-informatics.github.io/COPILOT/GenomeStudio_genotyping_SOP.html
|
||||
- Needs validation against GTC outputs.
|
||||
- p10 GC / p50 GC ("GC10"/"GC50") and p05/p50/p95 percentiles.
|
||||
- These are GTC fields (see BeadArrayFiles TOC IDs 1009, 1011, 1014, 1015), but they are not reported in `gt_sample_summary`. [PUBLIC DOCUMENTATION] https://github.com/Illumina/BeadArrayFiles
|
||||
- "p99 GC" was not found in any reference-product document.
|
||||
- LogRDev computation details, such as which loci are included and whether outliers are trimmed. The cyto JSON defines it as "Standard deviation of the Log R ratio values for all probes". [PUBLIC DOCUMENTATION] `[DOC-OUT]`
|
||||
|
||||
### 5.2 Control-probe QC metrics (`controls.qc_metrics.csv`)
|
||||
|
||||
[PUBLIC DOCUMENTATION] `[DOC-OUT]`
|
||||
|
||||
Definitions:
|
||||
- `bkg_grn` = max green intensity of Extension(A|T).
|
||||
- `bkg_red` = max red intensity of Extension(C|G).
|
||||
- Offset x = 3000.
|
||||
|
||||
Genotyping metrics:
|
||||
|
||||
| Metric | Formula |
|
||||
|---|---|
|
||||
| RestorationQC | Restoration green / (bkg_grn + x) |
|
||||
| StainingGreenQC | Biotin(High) / Biotin(Bgnd) |
|
||||
| StainingRedQC | DNP(High) / DNP(Bgnd) |
|
||||
| ExtensionGreenQC | min Ext(C,G) green / max Ext(A,T) green |
|
||||
| ExtensionRedQC | min Ext(A,T) red / max Ext(C,G) red |
|
||||
| Hyb High/Med and Hyb Med/Low | green ratios |
|
||||
| TargetRemovalIQC | (bkg_grn + x) / TargetRemoval green |
|
||||
| StringencyQC | String(PM) red / String(MM) red |
|
||||
| NonSpecificBinding Green / Red | (bkg + x) / max NSB(Bgnd) |
|
||||
| NonPolymorphic Green / Red | min NP(C,G) / max NP(A,T) green; min NP(A,T) / max NP(C,G) red |
|
||||
|
||||
Methylation adds:
|
||||
- TargetRemovalII
|
||||
- Specificity I green/red, Specificity II, Specificity II background
|
||||
- Bisulfite Conversion I green/red (with background variants), Bisulfite Conversion II (with background variant)
|
||||
- that is, the 21 BeadArray Controls Reporter metrics.
|
||||
|
||||
Genotyping YAML template example values `[DOC-QC]`:
|
||||
|
||||
| Threshold | Example value |
|
||||
|---|---|
|
||||
| staining | 5 |
|
||||
| extension | 5 |
|
||||
| targetRemovalI | 5 |
|
||||
| hyb ratios | 1.25 |
|
||||
| NP | 5 |
|
||||
| stringency | 1.5 |
|
||||
| NSB | 5 |
|
||||
| tgaControl | 1 |
|
||||
| restoration | null (1 when the FFPE Restore kit is used) |
|
||||
|
||||
- Chemistry-specific templates: `config_genotyping.yaml`, `config_genotyping_EX.yaml`, `config_methylation_EPIC.yaml`, `config_methylation_MSA.yaml` (downloads on `[DL]`).
|
||||
- Methylation cloud defaults (EPIC / MSA) are tabulated at [ref-docs:cloud-analysis/methylation-qc]. Includes BgCorrectionOffset 3000 and PvalThreshold 0.05.
|
||||
- Methylation algorithm [PUBLIC DOCUMENTATION] from tech note M-GL-02644 ([ref-docs:methylation-qc-tech-note-M-GL-02644]):
|
||||
- **Noob** background correction
|
||||
- **nonlinear** dye-bias correction
|
||||
- detection p-values using "Negative controls for null, log transformed signal"
|
||||
- GenomeStudio, by contrast, uses background subtraction, linear dye bias, and a negative-control null.
|
||||
|
||||
---
|
||||
|
||||
## 6. Hardware, OS, performance
|
||||
|
||||
[PUBLIC DOCUMENTATION] `[DOC-LOCAL]`, `[DOC-FAQ]`, developer article
|
||||
|
||||
- **CPU only.** "does not require a specialized server or FPGA hardware". The local package is "CPU-based".
|
||||
- **Recommended machine.**
|
||||
- 8 cores.
|
||||
- Memory: **32 GB** in v1.4; 16 GB in v1.0 `[DOC-V10-LOCAL]`.
|
||||
- ≥30 GB disk; `cyto annotate` needs ≥5 GB free.
|
||||
- **Operating systems.** Windows 10+ (win10-x64); CentOS 7+ / Ubuntu 20.04+ (linux-x64).
|
||||
- .NET 9 self-contained, needing native .NET dependencies.
|
||||
- Should not run as admin.
|
||||
- No ARM build is listed. [INFERRED] from the runtime IDs listed (win10-x64, linux-x64).
|
||||
- **Docker.**
|
||||
- No official Docker image is documented for the reference product. [INFERRED] from its absence in the docs.
|
||||
- The cloud runs in the vendor cloud platform; container use there is not documented.
|
||||
- IAAP had a Docker-based "Orchestrated Workflow". [PUBLIC DOCUMENTATION] https://emea.support.illumina.com/downloads/iaap-genotyping-orchestrated-workflow.html
|
||||
- **Performance claims.**
|
||||
- "Genotyping VCF in as little as 35 seconds per sample". `[DOC-HOME]`
|
||||
- "all PGx analysis results in ~1 minute per sample". `[DOC-HOME]`
|
||||
- Accuracy >99.5%. `[DOC-HOME]`
|
||||
- PGx CNV >95%. `[DOC-HOME]`
|
||||
- CYP2D6 call rate >97%. `[DOC-HOME]` The product page says >98%.
|
||||
- **Threading and scale.**
|
||||
- `--num-threads` exists on `genotype call`.
|
||||
- More than 1K samples per sample sheet slows things down (known issue).
|
||||
- Windows and Linux star-allele results differ slightly: 6 field differences in 1,576 samples, with concordant top solutions. `[RN-1.3]`
|
||||
- **Network requirement.**
|
||||
- Internet access is needed for the license check, **only for `pgx star-allele call`**.
|
||||
- Everything else runs offline. [INFERRED] Only star-allele call is quota-gated per `[DOC-LOCAL]` and `[DOC-APPS]` ("Local: No cost download" for genotyping, CNV and cyto).
|
||||
|
||||
---
|
||||
|
||||
## 7. Licensing / EULA
|
||||
|
||||
- **Product-specific EULA text** was not found at a public URL. It is presumably shown in the installer or on MyIllumina download. [HYPOTHESIS] Action: obtain it legally and have counsel review it before any benchmarking with the real binary. *(Superseded: the plan does not run, benchmark or inspect the vendor binary; see plan §2.)*
|
||||
- **Commercial model.**
|
||||
- Local genotyping, PGx CNV and cyto are a "No cost download".
|
||||
- Local star-allele calling uses per-sample quota.
|
||||
- Cloud uses vendor cloud credits, plus a per-sample PGx charge and a vendor cloud-platform subscription.
|
||||
- [PUBLIC DOCUMENTATION] `[DOC-APPS]`, product page [ref-docs:product-page]
|
||||
- **Representative vendor EULA language** [PUBLIC DOCUMENTATION] the vendor's sequencing-pipeline v4.3 EULA, [ref-docs:vendor-sequencing-eula-v4.3]. This is the same family and is likely similar for the reference product [HYPOTHESIS]:
|
||||
> "Licensee may not modify, improve, translate, reverse engineer, decompile, disassemble, or create derivative works of the Software or otherwise attempt to … (b) derive the source code or the underlying ideas, algorithms, structure, or organization form of the Software."
|
||||
- The same EULA says Licensee may not "publish" the Software; the restriction is on the Software itself, not on results.
|
||||
- No explicit benchmarking or publication-of-results clause was found in the passages reviewed. [PUBLIC DOCUMENTATION] same PDF.
|
||||
- **Implications** [INFERRED]
|
||||
- A clean-room effort should rely only on public docs, public tech notes, open-source Illumina code, and observed input/output formats.
|
||||
- It must not decompile the .NET assemblies. .NET is easily decompilable, which makes this contractually sensitive.
|
||||
- Running the licensed binary to generate comparison outputs is probably permitted "internal use", but the "derive … underlying ideas, algorithms" clause is broad. That needs legal review. *(Superseded: the plan does not run, benchmark or inspect the vendor binary; see plan §2.)*
|
||||
- **Open-source Illumina pieces usable as references** (§9):
|
||||
- BeadArrayFiles: BSD-2-style licence text in its README.
|
||||
- GTCtoVCF: Apache-2.0.
|
||||
|
||||
---
|
||||
|
||||
## 8. Illumina public algorithm documents (URLs verified live, HTTP 200)
|
||||
|
||||
| Document | URL | Relevance |
|
||||
|---|---|---|
|
||||
| Illumina GenCall Data Analysis Software (tech spotlight) | https://www.illumina.com/Documents/products/technotes/technote_gencall_data_analysis_software.pdf | Per-array self-normalization (channel intensity, background, dye crosstalk). GenTrain clustering with neural-net estimation of missing clusters. Bayesian calling. GenCall score = GenTrain score × data-to-model fit, penalised by the DNA score. GC10/GC50. Sensitive range 0.2–0.7. |
|
||||
| Infinium Genotyping Data Analysis (tech note) | https://www.illumina.com/Documents/products/technotes/technote_infinium_genotyping_data_analysis.pdf (alt: https://support.illumina.com/content/dam/illumina-marketing/documents/services/technote_infinium_genotyping_data_analysis.pdf) | Normalization, clustering, QC workflow, and cluster-editing practice. Cited by the reference product docs `[DOC-LOCAL]`. |
|
||||
| GenTrain 2.0 | https://www.illumina.com/Documents/products/technotes/technote_gentrain2.pdf | Three-step θ/R clustering: preliminary, model proposals, HWE-aware scoring. |
|
||||
| GenTrain 3.0 (M-GL-01258) | https://www.illumina.com/content/dam/illumina/gcs/assembled-assets/marketing-literature/gentrain-tech-note-m-gl-01258/gentrain-tech-note-m-gl-01258.pdf | Normalization "bins" (Inf I and Inf II separate). A two-parameter model for Inf I bins with < 192 loci, versus six parameters otherwise. Limits the penalty on near-axis homozygous clusters. |
|
||||
| X-chromosome SNP cluster generation | https://www.illumina.com/documents/products/technotes/technote_x_chromosome_snp_cluster_generation.pdf | Sex-aware clustering. |
|
||||
| DNA Copy Number Analysis Algorithms (cnvPartition basis) | https://www.illumina.com/documents/products/technotes/technote_cnv_algorithms.pdf | LRR/BAF definitions; recursive-partition CNV. |
|
||||
| Cytogenetic analysis tech note | https://www.illumina.com/Documents/products/technotes/technote_cytoanalysis.pdf | LRR/BAF cytogenetics. |
|
||||
| cnvPartition in GenomeStudio (training PDF) | https://support.illumina.com/content/dam/illumina-support/courses/GenomeStudio-cnvPartition/story_content/external_files/GenomeStudio_cnvPartition.pdf | |
|
||||
| CNV plug-ins for BeadStudio | https://www.illumina.com/content/dam/illumina-marketing/documents/products/technotes/technote_cnv_plug_ins.pdf | |
|
||||
| Custom cluster file creation for improved copy number analysis (M-GL-02142) | https://www.illumina.com/content/dam/illumina/gcs/assembled-assets/marketing-literature/custom-cluster-file-tech-note-m-gl-02142/custom-cluster-file-tech-note-m-gl-02142.pdf | Cited by the reference product for EGT/CN-model retraining. |
|
||||
| [product] Methylation QC (M-GL-02644) | [ref-docs:methylation-qc-tech-note-M-GL-02644] | Noob, nonlinear dye bias, detection-p method. |
|
||||
| iSelect / Infinium I vs II design | https://www.illumina.com/Documents/products/technotes/technote_iselect_design.pdf | Needed for MAV logic. |
|
||||
| Infinium Controls training guide | https://support.illumina.com/content/dam/illumina-support/courses/eval-inf-controls/story_content/external_files/Infinium_Controls_Training_Guide.pdf | Control-probe semantics. |
|
||||
| Developer article: [product] 1.0 PGx | [ref-docs:dev-article-1.0-pgx] | Accuracy numbers; CPU-only. |
|
||||
|
||||
- Tech-note summaries in the table are [PUBLIC DOCUMENTATION], extracted from the PDFs with pdftotext.
|
||||
- **Key gap:** none of these gives equations for the normalization-transform **fitting**, the GenCall data-to-model score, or GenTrain's neural-net cluster imputation.
|
||||
- Published third-party descriptions and reimplementations of Illumina-like calling exist: gtc2vcf (MIT), which re-implements BAF/LRR from EGT; and the WARP Illumina genotyping pipeline, https://broadinstitute.github.io/warp/docs/Pipelines/Illumina_Genotyping_Arrays_Pipeline/README (it uses iaap-cli). [PUBLIC DOCUMENTATION]
|
||||
|
||||
---
|
||||
|
||||
## 9. Illumina GitHub repositories relevant to arrays
|
||||
|
||||
The GitHub API was rate-limited (unauthenticated), so repos were checked through their public HTML/raw pages. A complete org listing still needs to be done (`gh api orgs/Illumina/repos --paginate` with a token).
|
||||
|
||||
| Repo | Licence | Relevance |
|
||||
|---|---|---|
|
||||
| **Illumina/BeadArrayFiles** — https://github.com/Illumina/BeadArrayFiles | BSD-style (text in README, "Copyright (c) 2020, Illumina") | Python parsers for **GTC (v5 spec: `docs/GTC_File_Format_v5.pdf`)**, **BPM**, **EGT**, and LocusAggregate. `NormalizationTransform.normalize_intensities` and `rect_to_polar` give the exact application of the stored affine normalization. The README recommends the reference product for making GTCs. [PUBLIC DOCUMENTATION] |
|
||||
| **Illumina/GTCtoVCF** — https://github.com/Illumina/GTCtoVCF | Apache-2.0 | Semi-archived since May 2023 in favour of the reference product. Same option set and merge semantics as `refcli genotype gtc-to-vcf` (§1.4). Includes unit tests (`tests/test_class.py`) on genotype merging. [PUBLIC DOCUMENTATION] |
|
||||
| Illumina/IlluminaHumanMethylationEPICv2manifest, Illumina/IlluminaHumanMethylationMSAmanifest | not checked | R/Bioconductor-style manifests for EPIC v2 and MSA; useful for methylation QC. [PUBLIC DOCUMENTATION] (the repos exist) |
|
||||
| Illumina/Cyrius | "Other" licence (not SPDX-identified) | CYP2D6 caller for **WGS**, not arrays. Conceptual reference for CYP2D6 star-allele and CN logic only. |
|
||||
| Illumina/canvas, Illumina/SMNCopyNumberCaller, Illumina/Gauchian, Illumina/ExpansionHunter, Illumina/hap.py, Illumina/witty.er | "Other" licence (not SPDX-identified) | NGS tools. hap.py and witty.er are useful for benchmarking VCF concordance. |
|
||||
|
||||
Related non-Illumina open source:
|
||||
- freeseek/gtc2vcf (MIT): IDAT/BPM/EGT/GTC → VCF with BAF/LRR recomputation. https://github.com/freeseek/gtc2vcf
|
||||
|
||||
---
|
||||
|
||||
## 10. Biggest unknowns and risks for an independent reimplementation
|
||||
|
||||
1. **GenCall internals** (normalization fitting, per-locus calling likelihood, GenCall score formula, DNA-score penalty, sex estimation).
|
||||
- Only qualitative tech notes are public.
|
||||
- Reproducing bit-compatible GTC genotypes and GC scores will need empirical calibration against reference outputs.
|
||||
- The GTC stores the per-sample normalization transform, which helps with verification.
|
||||
2. **PGx CNV model** (`.dat` format and algorithm) and **cyto model** (`.dat`: "primarily … GC content"). Both formats and both algorithms are undocumented.
|
||||
3. **Star-allele caller**: scoring function, priors, solver, hybrid-allele handling. The PGx database zip format is only loosely described ("probe ID → HGVS" lines).
|
||||
4. **Cyto CNV/LOH caller**: the segmentation or HMM approach, the mosaic-fraction estimator, GC correction, and QUAL calibration are all undocumented.
|
||||
5. **LogRDev, Autosomal Call Rate and Sex Estimate**: the exact locus sets and formulas are not specified.
|
||||
6. **EULA**:
|
||||
- The product-specific terms are not public.
|
||||
- The vendor-family EULA forbids deriving "underlying ideas, algorithms" from the Software.
|
||||
- Comparison benchmarking should be legally cleared. *(Superseded: the plan does not run, benchmark or inspect the vendor binary; see plan §2.)*
|
||||
- .NET decompilation must be strictly avoided.
|
||||
7. **Clearly reproducible from public material**: VCF conversion (GTCtoVCF), GTC/BPM/EGT I/O (BeadArrayFiles), control-probe QC formulas (fully tabulated in `[DOC-OUT]`), methylation Noob/dye-bias/beta/M-value methods (open literature plus the tech note), and output schemas (VCF headers, CSV/JSON fields).
|
||||
Reference in new issue
Block a user