kemal/SimurgArray
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c71f1b63332658e8742dd26a688de5b4cac03e63
SimurgArray/SIMURGARRAY_FUNCTIONAL_SPEC.md
T
kemal 194c087b24 PGx star alleles: pgx star-allele call (PGX-011..021), probe masks vs 1000G, GeT-RM validation (K-029) 
- simurg/pgx/star.py: solver-free diplotype enumeration incl. CN forms (deletion, xN), wildcards, array-equivalence
  classes, raw_score/score with CPIC frequency priors; calibrated error model (spec PGX-012 amended).
- pipeline/pgx_star.py: DB gene models, SNV + PGx CN VCF evidence, masks, diplotype_summary.tsv + pgx_calls.json.
- scripts/pgx: derive_masks.py (array vs 1000G 30x over HTTPS, GS calibration), getrm_truth.py,
  validate_star_getrm.py.
- GSA v4 PGx demo vs GeT-RM (16 CEPH): 100/109 gene calls; CYP2D6/2C19/2C9 41/45 (docs/validation/V2_pgx_star.md).
2026-09-29 14:08:03 +01:00

4143 lines
239 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SimurgArray — Functional Specification
> Companion to `SIMURGARRAY_MASTER_PLAN.md` (the "plan"). The plan defines algorithms, architecture, data sources
> and the phased roadmap. **This document defines externally visible behaviour**: commands, options, validation,
> file schemas, QC semantics, messages, edge cases and acceptance tests. Version 1.0 · 2026-09-27.
>
> It consolidates a documentation-level study of a commercial Infinium array-analysis product (the "reference
> product"), whose public documentation was read in full. That study is archived under `_phase0_archive/site_analysis/`
> as files A–E. About 200 suggested additions from their gap analyses were deduplicated and merged here.
> No vendor binary was inspected (plan §2). Documented behaviour is emulated only where it helps users.
> Documentation errors are listed in §12 and are **never** turned into requirements.
---
## 0. Conventions
### 0.1 Item format
Every requirement has a stable ID, a trace tag and optional source tags:
`- **CLI-021** \`§14 · P4\` **Title.** Requirement text … (src: A-G14, C-G1)`
- **ID prefixes:** `SCOPE` scope, `CLI` command line, `IN` inputs, `OUT` genotyping outputs, `QC` control QC and the
QC report, `CYTO` genome-wide CNV/LOH, `PCN` targeted PGx copy number, `PGX` star alleles and phenotypes, `MET`
methylation control QC, `EDGE` edge cases and regression tests, `PERF` performance, `OQ` open questions.
- **Trace tag:** the master-plan section(s), then the roadmap phase (P0–P10, or `v2` for the plan's v2 modules).
The table in §13 is generated from these tags.
- **Source tags** (`src:`) point at gap-analysis items in the archived study. `A-G5` means file A, item G5;
`B-12` means file B, item 12; and so on for C, D and E. `plan` means the item restates or sharpens the plan.
§13.2 checks that every gap item is covered.
- **Keywords:** MUST, SHOULD and MAY as in RFC 2119.
- **Behaviour markers:**
- **[EMU]**: the reference product's documented behaviour, emulated on purpose.
- **[DIFF]**: we deliberately do it differently. The reason is always given.
- **[OURS]**: the reference documents nothing here, so this is SimurgArray's own definition.
- **[INF]** and **[HYP]**: as in plan §0 (our inference; a hypothesis still to be tested).
### 0.2 Naming conventions (SimurgArray-wide)
- CLI flags: lowercase kebab-case only. No mixed case and no case-sensitive look-alikes.
- File names: lowercase snake_case. Per-sample files are named `<sample>.<kind>.<ext>`, where `<sample>` is the
*output stem* defined in IN-012.
- TSV/CSV column names and JSON keys: snake_case. Reference-style column names are available only through the
explicit compatibility options (OUT-066, QC-009).
- Numbers are always formatted locale-independently: `.` as the decimal separator and no thousands separators
(EDGE-001).
- Timestamps in our outputs use ISO-8601 UTC (`2026-09-27T10:15:02Z`). Legacy formats are written only inside
GTC fields, for interoperability (OUT-001).
- Genomic coordinates are 1-based and fully closed in every TSV and JSON output. VCF follows the VCF 4.3
conventions exactly (OUT-041, CYTO-029).
### 0.3 Relationship to the plan
Where this spec and the plan disagree about **I/O, naming, options or messages**, this spec wins. Where they disagree
about **algorithms or numerics**, the plan wins. Each such conflict found during implementation MUST be logged in
`docs/known_unknowns.md`, and both documents updated.
---
## 1. Scope & compatibility targets
- **SCOPE-001** `§1 · P0` **Purpose.** SimurgArray reproduces the functional envelope of a modern Infinium
secondary-analysis pipeline, locally and offline:
- IDAT → GTC genotyping with control-probe QC and a sample summary;
- GTC → SNV VCF and bedGraph;
- a multi-dataset interactive QC report;
- genome-wide CNV/LOH/mosaic calling with ISCN annotation (v2);
- targeted PGx copy number and star-allele diplotypes with CPIC/DPWG phenotypes (v2);
- methylation-array control QC (v2).
(src: plan)
- **SCOPE-002** `§1 · P0` **Emulation targets [EMU].** SimurgArray emulates these reference behaviours, because
users and downstream tools depend on them:
1. **Workflow shape:**
- genotype → VCF;
- genotype (or control-only QC) → QC report;
- genotype → VCF → targeted CN → star alleles → phenotype;
- genotype → CNV/LOH → annotation.
2. **Content of the files that move between steps:**
- control raw and QC metric tables;
- the sample summary, with sheet columns passed through;
- the SNV VCF (FORMAT `GT:GS:BAF:LRR`, merged multi-probe records);
- the CNV VCF information content;
- the PGx tables.
3. **Numeric defaults:**
- GenCall cutoff 0.15;
- QC 0.98 / 0.20;
- per-type CNV size and probe minimums;
- QUAL cap 60 and filters Q10 / Q7;
- PGx batch 24/22 and training 96/90;
- alternative solutions at ≥ 50 % of the top score;
- low-GS review at 0.15.
4. **Control-probe metric formulas and QC status semantics** (two status axes, `null` disables a check).
5. **Sample-sheet semantics:** GenomeStudio-compatible sections, case-insensitive columns, name precedence,
RootFolder.
(src: A-G1, C-G1)
- **SCOPE-003** `§1 · P0` **Deliberate differences [DIFF].** SimurgArray MUST differ from the reference product in
each of the following. Each has its own item; this table is the index.
| # | Reference behaviour | SimurgArray behaviour | Item |
|---|---|---|---|
| 1 | silently overwrites the output folder | refuses unless `--force`/`--resume` | CLI-010 |
| 2 | duplicate IDAT pairs: one picked, OS-dependent | error `E_DUP_IDAT` (or a deterministic opt-in rule) | IN-022 |
| 3 | CN probe thresholds < 10 silently reset | reset with a warning | CYTO-008 |
| 4 | CNV VCF: `REF=N`, positive SVLEN for DEL, `GT=1/1` placeholder, ID/POS disagreements | VCF-4.3-correct padding-base POS, FASTA REF, signed SVLEN, meaningful GT, consistent IDs; `--compat` modes available | CYTO-029–CYTO-031, PCN-011 |
| 5 | JSON numbers as strings, `"."` sentinels, stray key punctuation | typed JSON, `null`, JSON-Schema validated | OUT-075 |
| 6 | output order and PGx scores differ by OS | byte-identical outputs across OS, CPU and thread count | OUT-076, PGX-029 |
| 7 | licence server; results discarded when over quota | offline; computed results are never discarded | CLI-012, PGX-033 |
| 8 | mixed-case, case-sensitive flags | lowercase kebab-case | CLI-002 |
| 9 | unknown sex → silently male in CN calling | explicit policy + flag | CYTO-024 |
| 10 | no ISCN for LOH/mosaic/≤ 1 kb; first event sometimes unannotated | ISCN for every reported event | CYTO-034 |
| 11 | bedGraph smoothing corrupted LRR | raw tracks are never smoothed | OUT-052 |
| 12 | own tabix writer without count metadata | htslib index with counts | OUT-038 |
| 13 | cluster file names restricted to `[A-Za-z0-9_-]` | any name accepted | IN-045 |
| 14 | BPM/CSV/EGT/IDAT mismatches silent or per-sample | pre-flight validation errors | IN-042–IN-044, IN-027 |
| 15 | semi-custom arrays: PGx must run twice | single run; PGx uses whatever mapped content exists | PGX-030 |
| 16 | per-step inconsistencies (raw vs effective size; GAINLOH counted in some places only) | one consistent rule, documented | CYTO-011, CYTO-033 |
- **SCOPE-004** `§1.3 · P0` **Out of scope.** The following are not provided and MUST NOT be emulated:
- cloud platform features: workgroups, credits, sharing, notifications, scanner-to-cloud upload, case creation
on interpretation platforms;
- licence servers or quota;
- polyploid genotyping ("diploid only", as in the reference);
- oncology CNV interpretation (`annotation_type` is always `constitutional`);
- novel star-allele discovery.
The local equivalents of two cloud features *are* in scope as optional v2 conveniences: the scan-registration
gate (IN-026) and the planned-run watcher (CLI-037). (src: A-G13, D-26, D-27)
- **SCOPE-005** `§1 · P0` **Species and builds.**
- Human GRCh37 and GRCh38 for genotyping, VCF, QC and CNV/LOH.
- PGx is GRCh38-only in v2.0; see PCN-015.
- Non-human diploid organisms are supported for genotyping, GTC, QC and bedGraph, given a user-supplied FASTA and
a *species profile* (IN-065). The bovine UMD3 build (BovineSNP50) is the non-human acceptance case.
- A VCF for a non-human species needs a FASTA whose contig names match the manifest.
(src: C-G22, A-G62)
- **SCOPE-006** `§16 · P8` **Product test matrix.** Validation (plan §16.5) MUST include at least one dataset per
manifest family:
- genotyping: ASA, CGA, GCRA, GDA, GSA v3, GSA v4 (EX chemistry), JSA, PRS-type (GRCh37 only), BovineSNP50;
- PGx: GDA-ePGx, GSAv4-ePGx, GCRA-ePGx (all GRCh38);
- cyto: GDA-Cyto, GSA-Cyto, CytoSNP-850K (both iScan and NextSeq 550 IDATs);
- methylation: EPIC v1, EPIC v2, MSA.
Each is run on GRCh37 and GRCh38 where the product supports both. Families with no public data are recorded as
gaps in the validation report. (src: C-G28)
- **SCOPE-007** `§1 · P10` **Product-file compatibility matrix.** The user guide MUST publish a matrix linking
manifest revision ↔ cluster file ↔ PGx CN model ↔ cyto model ↔ PGx DB ↔ annotation bundle. It lists, for each
combination, which checks are errors and which are warnings (IN-049). `simurg inspect --compat <files…>` prints
the same matrix for the files the user supplies. (src: A-G63)
- **SCOPE-008** `§3 · P10` **User runtime requirements.**
- **Platforms:** Linux x86-64 and aarch64, macOS arm64. Windows is best-effort (via WSL) and not a release gate.
- **Resources:**
- Genotyping and QC: 4 cores and 8 GB RAM minimum; 8 cores and 32 GB recommended.
- Disk: ≥ 30 GB free for GSA-size batches of 1 000 samples, plus the annotation bundle (CYTO-036).
- PGx CN batches: memory scales with the batch, as documented in PCN-016.
- **Operation:**
- No elevated privileges needed; nothing is written outside the output, cache and tmp directories.
- No network access at run time (offline-first). Downloads happen only in explicit `fetch`/`build` helper scripts.
(src: A-G12, A-G13, A-G43)
- **SCOPE-009** `§19 · P9` **Batch limits.** There is no hard sample cap. Every command MUST be tested at:
- ≥ 1 152 samples for genotyping, cyto and methylation;
- ≥ 384 samples for PGx;
- 10 000 synthetic samples for genotyping and QC.
Memory must stay flat in batch size wherever the computation is per-sample. (src: D-31, D-9)
- **SCOPE-010** `§11 · v2` **Semi-custom and custom arrays.**
- Genotyping, QC, VCF and CNV work on any Infinium genotyping manifest.
- PGx uses every probe that maps to a PGx-DB variant, whichever part of the design it comes from. A second
"commercial-files" run is never required [DIFF].
- Warnings are issued when a gene's commercial backbone probes are missing (PGX-030). (src: A-G42, D-20)
---
## 2. Command-line interface (`simurg`)
### 2.1 Structure and global behaviour
- **CLI-001** `§14 · P1` **Command tree.** Plan §14's flat verbs are canonical. Grouped verbs are subcommands.
```
simurg
├── version # also: simurg --version
├── inspect <file>... [--compat]
├── ingest | normalize # plan §14 (developer/debug steps)
├── genotype # IDAT → GTC + controls + sample summary (P4)
├── gtc-to-vcf # GTC → <sample>.snv.vcf.gz (P6)
├── export bedgraph | plink | finalreport | locus-summary (P6)
├── qc
│ ├── controls # IDAT + manifest → control tables, no cluster file (P5)
│ ├── samples | loci | contamination | relatedness (P5)
│ ├── report # multi-dataset HTML + table (P5)
│ └── suggest-thresholds (P5)
├── cn # alias: cyto (v2)
│ ├── build-model
│ ├── call
│ └── annotate
├── pgx (v2)
│ ├── copy-number call | train
│ ├── star-allele call
│ ├── phenotype # alias: annotate
│ ├── coverage
│ └── db build
├── methyl (v2)
│ ├── qc
│ └── convert-manifest
├── train # ArrayTrain (plan §12) (v2)
├── pipeline run | plan | watch (P6; watch v2)
├── compare (P7)
└── synth (P1+)
```
Every group and leaf accepts `--help`. `simurg <group>` with no leaf prints the group help and exits with 2.
(src: A-G1, A-G44, plan)
- **CLI-002** `§14 · P1` **Syntax rules.**
- Options are lowercase kebab-case. Boolean options come in `--x/--no-x` pairs where a default-on switch exists.
- List-valued options accept either a comma list with no spaces (`a,b`; surrounding spaces are trimmed with a
warning) or a repeated flag.
- Paths may contain spaces and non-ASCII characters.
- Unknown options are errors (exit 2) and come with a "did you mean" suggestion.
(src: A-G11)
- **CLI-003** `§14 · P10` **Migration aliases.** A documented table maps reference-style verbs to simurg verbs:
- `genotype call` → `genotype`
- `genotype gtc-to-vcf` → `gtc-to-vcf`
- `genotype gtc-to-bedgraph` → `export bedgraph`
- `qc call` → `qc controls`
- `cyto call|annotate` → `cn call|annotate`
- `pgx copy-number call|train` → identical
- `pgx star-allele call` → identical
- `pgx star-allele annotate` → `pgx phenotype`
The two-word forms are accepted as hidden aliases, with an informational note. Mixed-case reference flags (for
example the threshold flags) are *not* accepted; the error message names the simurg equivalent. (src: A-G1)
- **CLI-004** `§14 · P1` **Global options** (accepted by every leaf command).
| Option | Type | Default | Validation / notes |
|---|---|---|---|
| `--threads N` | int | `0` = auto: physical cores available to the process, respecting CPU affinity and cgroup quota | 0–1024; results MUST NOT depend on N (plan §15) |
| `--log-level` | enum `debug\|info\|warning\|error` | `info` | |
| `--debug` | flag | off | sets `debug` level **and** includes tracebacks in error records |
| `--log-json` (alias `--json-log`) | flag | off | JSON Lines on stderr, schema in CLI-007 |
| `--log-file PATH` | path | `<out>/logs/<command>-<run_id>.log` | always JSON Lines, regardless of `--log-json` |
| `--quiet` / `--no-progress` | flag | progress bars only when stderr is a TTY | |
| `--seed N` | int | `42` | recorded in the run manifest |
| `--config FILE` | YAML | none | see CLI-019 |
| `--force` | flag | off | allows replacing existing outputs (CLI-010) |
| `--resume` | flag | off | reuses completed per-sample outputs whose input hashes match (plan §14 content-addressed cache) |
| `--tmp-dir DIR` | dir | `<out>/.simurg_tmp` | removed on success |
| `--help`, `--version` | flag | | `--version` prints `SimurgArray <semver>+<git sha>` |
(src: A-G2, A-G3, A-G7, plan)
- **CLI-005** `§14, §15 · P1` **Option precedence.**
1. Explicit CLI option.
2. The command's section in the `--config` YAML.
3. The named profile (QC-024), if any.
4. The built-in default.
The only environment variables read are `SIMURG_CACHE_DIR` (IN-064) and `NO_COLOR`. The *effective* value of
every option and threshold, with its source (`cli|yaml:<path>|profile:<name>|default`), MUST be recorded:
- in `run_manifest.json`;
- in VCF headers (`##simurg_options=`);
- in JSON outputs (a `parameters` object);
- in the QC table's Thresholds sheet (QC-045).
(src: A-G30, E-47)
- **CLI-006** `§14 · P1` **Input-set rule.** Commands that take a sample set accept a folder (`--idats DIR` or
`--gtcs DIR`), a sheet (`--sheet FILE`), or both.
- **Sheet only:** the folder is the sheet's `RootFolder` (IN-008). If it has none → `E_NO_INPUT_FOLDER`.
- **Folder only:** every discovered sample is processed.
- **Both:** the sheet selects and annotates samples; the CLI folder overrides `RootFolder`, with an info message
if they differ.
- **Neither:** `E_NO_INPUT` (exit 2).
(src: A-G5, plan)
- **CLI-007** `§14 · P1` **Logging.**
- Human-readable logs go to stderr. With `--log-json`, each record is one JSON object per line:
```json
{"schema":"simurg.log/1","ts":"2026-09-27T10:15:02.123Z","level":"warning",
"code":"W_MERGE_AMBIGUOUS","msg":"Failed to combine genotypes (ambiguous) - rs1 (InfiniumII): AA, rs1_dup (InfiniumI): TT",
"command":"gtc-to-vcf","run_id":"01J9…","step":"merge","sample_id":"204753010023_R01C01",
"locus":"rs1","file":null,"count":1,"elapsed_ms":8123,"exc":null}
```
- `exc` holds the formatted traceback only with `--debug`.
- `code` is a stable catalogue ID (CLI-016); `msg` is free text that may change between versions.
- Unknown fields MUST be ignored by consumers.
(src: A-G2, A-G3)
- **CLI-008** `§14 · P1` **Warning rate limiting.**
- Each warning code is printed at most 20 times per sample, then once as a summary: `W_… repeated N more times`.
- Every occurrence is still written to the audit tables (plan §8.2), and the run summary counts them per code.
(src: B-5)
- **CLI-009** `§14 · P1` **Run summary.** Each command ends with a summary:
- samples succeeded / failed / skipped;
- counts per warning code;
- output paths;
- wall time.
It goes to stderr and into `run_manifest.json` (`summary` object). (src: E-18, plan)
- **CLI-010** `§14 · P1` **Overwrite policy [DIFF].**
- If any file the command would write already exists, the command stops before doing any work: `E_OUTPUT_EXISTS`
lists up to 10 paths, exit 2.
- `--force` replaces them.
- `--resume` keeps outputs whose recorded input hashes and parameters match, and recomputes the rest.
- Files not produced by the current command are never deleted.
- Exception: `qc controls` MAY add its control tables to an existing genotype output folder that lacks them
(QC-014).
(src: A-G4, E-6)
- **CLI-011** `§14 · P1` **Atomic writes.** Every output is written to `<name>.part` in the same directory, then
fsynced and renamed. An interrupted run MUST NOT leave a truncated file under its final name. (src: plan §14)
- **CLI-012** `§14 · P1` **Never discard results [DIFF].** Per-sample outputs are finalized as soon as each sample
completes. A later failure (another sample, batch step or report) does not remove them. Batch files (summary, QC
tables) are rewritten at the end, and are also written if the run fails part-way. (src: A-G10)
- **CLI-013** `§14, §15 · P1` **Threading.**
- Parallelism is per sample (a process pool) plus, inside a sample, vectorized/Rust threads (P9). The total never
exceeds `--threads`.
- bgzip compression uses htslib worker threads taken from the same budget.
- Changing `--threads` MUST NOT change any output byte (EDGE-022).
(src: A-G7)
- **CLI-014** `§14 · P1` **Exit codes.**
| Code | Meaning |
|---|---|
| 0 | success; warnings are allowed |
| 1 | internal error (a bug). The message asks for a `--debug` rerun |
| 2 | usage or pre-flight validation error; no per-sample work started. Examples: bad option, missing file, schema or mismatch errors, output exists, batch minimum not met |
| 3 | partial failure: ≥ 1 sample failed and ≥ 1 succeeded. Outputs for the successful samples are complete, and the failed samples are listed in the summary |
| 4 | the run completed but no sample succeeded (extension of plan §14) |
| 130 / 143 | interrupted by SIGINT / SIGTERM (CLI-015) |
(src: A-G10, plan)
- **CLI-015** `§14 · P1` **Interruption.** On SIGINT/SIGTERM:
1. Stop scheduling new work.
2. Let in-flight atomic writes finish or abort them; either way no partial final files.
3. Write `run_manifest.json` with `status: "interrupted"`.
4. Exit with 130/143.
A later `--resume` continues the run. (src: plan)
- **CLI-016** `§14 · P1` **Message catalogue.** Every error and warning has a stable code. The catalogue
(`docs/messages.md`, generated from `simurg/core/errors.py`) is part of the acceptance tests (EDGE-*). Minimum set:
| Code | Level | Trigger |
|---|---|---|
| `E_NO_INPUT`, `E_NO_INPUT_FOLDER` | error | CLI-006 |
| `E_OUTPUT_EXISTS` | error | CLI-010 |
| `E_FILE_NOT_FOUND`, `E_FILE_UNREADABLE` | error | any missing or unreadable input, e.g. a manifest path |
| `E_SHEET_*` (`DUP_COLUMN`, `RESERVED`, `TOO_LONG`, `BARCODE_FORMAT`, `NAME_EMPTY`, `NAME_COLLISION`, `DUP_SAMPLE`, `NO_KEY_COLUMNS`, `EMPTY`, `DELIMITER`) | error | IN-001 – IN-016 |
| `W_SHEET_EMPTY_COLUMN`, `W_SHEET_BARCODE_REPAIRED`, `W_SHEET_ROOTFOLDER_OVERRIDDEN` | warning | IN-004, IN-007, IN-008 |
| `E_DUP_IDAT`, `W_IDAT_UNPAIRED`, `E_IDAT_FORMAT`, `E_IDAT_ENCRYPTED`, `W_IDAT_NAME_MISMATCH` | error / warning | IN-020 – IN-023 |
| `E_IDAT_MANIFEST_MISMATCH`, `W_IDAT_LOW_ADDRESS_COVERAGE` | error / warning | IN-027 |
| `E_GTC_CORRUPT` (sample skipped), `W_GTC_FOREIGN`, `E_GTC_UNSUITABLE` | warning / error | IN-031 – IN-033 |
| `E_BPM_CSV_MISMATCH`, `E_EGT_MANIFEST_MISMATCH`, `E_EGT_LOAD`, `W_EGT_NAME` | error / warning | IN-042 – IN-045 |
| `E_FAI_MISMATCH`, `E_CONTIG_MISMATCH`, `E_BUILD_MISMATCH`, `E_PGX_ALT_CONTIG_MISSING` | error | IN-060 – IN-063 |
| `W_REF_NOT_QUERIED`, `W_SKIP_UNMAPPED`, `W_SKIP_INTENSITY_ONLY`, `W_SKIP_INDEL_CONTEXT`, `W_SKIP_INDEL_REF`, `W_INDEL_PARTIAL_MATCH`, `W_SKIP_EXCLUDED` | warning | OUT-019 |
| `W_MERGE_INCONSISTENT`, `W_MERGE_AMBIGUOUS`, `W_MERGE_ALL_NC`, `W_INF1_NC_CONFLICT` | warning | OUT-031 |
| `E_AUX_INVALID` | error | OUT-035 |
| `W_CTRL_MISSING`, `W_CTRL_ZERO_DIVISOR` | warning | QC-007, QC-008 |
| `E_QC_*` (`NO_DATASETS`, `DATASET_INCOMPLETE`, `LABEL_COUNT`, `LABEL_DUPLICATE`, `MIXED_TYPES`, `FORMAT`, `CONFIG_MISSING`, `CONFIG_EXT`, `CONFIG_PARSE`, `CONFIG_STRUCTURE`, `THRESHOLD_KEY`, `THRESHOLD_VALUE`, `THRESHOLD_RANGE`) and `W_QC_LABEL_SANITIZED`, `W_QC_UNMATCHED_SAMPLES` | error / warning | QC-020 – QC-043 |
| `E_MODEL_MANIFEST_MISMATCH`, `E_MODEL_CLUSTER_MISMATCH` / `W_MODEL_CLUSTER_MISMATCH`, `E_MODEL_VERSION` | error / warning | IN-050, IN-051 |
| `E_PGX_BATCH_TOO_SMALL`, `W_PGX_BATCH_BELOW_RECOMMENDED`, `E_PGX_TRAIN_TOO_SMALL` | error / warning | PCN-004, PCN-005 |
| `E_PGX_VCF_UNSUITABLE`, `E_PGX_VCF_UNINDEXED`, `E_PGX_CN_MISSING` | error | PGX-011 |
| `W_CN_PROBE_FLOOR`, `W_CN_THRESHOLD_EXCEEDS_GENOME`, `I_CN_SEX_RESOLVED`, `W_CN_SEX_UNRESOLVED`, `E_CN_SIDECAR_MISSING`, `E_CN_VCF_UNINDEXED` | mixed | CYTO-* |
| `E_METH_*` and `W_METH_*` | mixed | MET-015 |
(src: B-33, C-G21, A-G56, B-5)
- **CLI-017** `§14 · P1` **Paths.**
- Relative paths on the CLI resolve against the CWD. Relative paths inside a sheet or config resolve against that
file's directory.
- Symlinks are followed, with loop detection.
- Output paths are printed as absolute paths.
(src: E-4)
- **CLI-018** `§13 · P1` **Output layout.**
- Default `--layout nested` (plan §13): per-sample files go under `<out>/samples/<sample>/`, batch files under
`<out>/`.
- `--layout flat` writes everything into `<out>/`, as the reference does.
- Every downstream command discovers inputs recursively, so both layouts work interchangeably. This includes the
star-allele step, which needs SNV and CN VCFs "in one folder" in the reference. (src: A-G44, plan)
- **CLI-019** `§14 · P1` **Configuration file.**
- `--config` takes YAML with an optional `version: 1`. It has one top-level section per command (`genotype:`,
`gtc_to_vcf:`, `qc:`, `cn:`, `pgx:`, `methyl:`) plus `runtime:`.
- Keys are the snake_case option names. Unknown keys → `E_CONFIG_KEY` (exit 2).
- `pipeline run` uses the same file, extended with `inputs:`, `reference:` and `outputs:` (plan §14 example). In
`outputs:`, `gtc`, `vcf`, `bedgraph`, `plink`, `final_report`, `locus_summary`, `qc_report`: true/false.
- `pipeline plan` warns when `final_report: true` and loci × samples > 1e8, and suggests turning it off (about
400 GSA-size samples ≈ 50 GB).
(src: D-30, plan)
### 2.2 Commands
- **CLI-020** `§14 · P1` **`simurg inspect <file>...`**
- Prints the header/TOC of IDAT, BPM, CSV manifest, EGT, GTC, sheet, VCF, CN model, PGx DB or annotation bundle.
- `--json` gives machine-readable output.
- `--compat` prints the compatibility matrix for the given files (SCOPE-007).
(src: plan)
- **CLI-021** `§14 · P4` **`simurg genotype`** (IDAT → GTC + control tables + sample summary).
| Option | Type | Default | Validation |
|---|---|---|---|
| `--idats DIR` | dir | – | CLI-006; recursive discovery, IN-020 |
| `--sheet FILE` | CSV | – | IN-001 … |
| `--bpm FILE` | BPM | **required** | IN-040 |
| `--egt FILE` | EGT | **required** | IN-043/IN-044 |
| `--csv FILE` | CSV manifest | optional | if given, checked against the BPM (IN-042) and used for control names (QC-001) |
| `--out DIR` | dir | **required** | CLI-010 |
| `--cutoff X` | float | `0.15` | 0 ≤ X ≤ 1; GenCall score < X → no-call |
| `--norm-version` | enum `1.2.0\|1.1.2` | `1.2.0` | plan §7.2 |
| `--exact/--clean` | flag | `--exact` | plan §0 |
| `--gtc/--no-gtc` | flag | on | off keeps calls only in the Parquet output |
| `--controls/--no-controls` | flag | on | writes `controls_raw.tsv`, `controls_qc.tsv` (QC-009, QC-011) |
| `--summary-format` | enum `tsv\|csv\|both` | `tsv` | plus `.json` always |
| `--compat-column-names` | flag | off | OUT-066 |
| `--on-duplicate-idat` | enum `error\|first\|newest` | `error` | IN-022 |
| `--assay-control-probes` | list | `TGA_Ctrl_5716` | OUT-062 |
| `--species-profile` | name/file | `human` | IN-065 |
Outputs: GTC (OUT-001), `sample_summary.{tsv,json}` (OUT-060), control tables (QC-009/011), `run_manifest.json`.
(src: A-G14, A-G7, plan)
- **CLI-022** `§14, §8 · P6` **`simurg gtc-to-vcf`.**
| Option | Type | Default | Validation / notes |
|---|---|---|---|
| `--gtcs DIR` / `--sheet FILE` | dir / CSV | – | CLI-006 |
| `--bpm FILE` | BPM | **required** | |
| `--csv FILE` | CSV manifest | **required** for indels | must contain `SourceSeq`. If absent, indels are skipped with `W_SKIP_INDEL_CONTEXT` and a one-time error-level notice |
| `--fasta FILE` | FASTA | **required** | IN-060 |
| `--out DIR` | dir | **required** | |
| `--no-bgzip` | flag | off | OUT-010 |
| `--genome-cache/--no-genome-cache` | flag | cache on | IN-064 |
| `--auxiliary-loci FILE` | VCF | – | OUT-035 |
| `--exclude-loci FILE` (alias `--filter-loci`) / `--include-loci FILE` | text | – | OUT-036 |
| `--unsquash-duplicates` | flag | off | OUT-034 |
| `--inf1-nocall-informative` | flag | off | OUT-037 |
| `--merge` | flag | off | multi-sample BCF (OUT-040) |
| `--drop-ambiguous` | flag | off | plan §8.2 |
| `--info-fields` | list | none | OUT-012 |
| `--extra-format` | list `X,Y,NORMX,NORMY,THETA,R` | none | plan §8.3 |
| `--filter-gs X` | float | off | adds FILTER `LowGS` when GS < X |
| `--contig-alias` | enum `strict\|auto\|FILE` | `strict` | IN-061 |
| `--ploidy-policy` | enum `sex-aware\|diploid` | `sex-aware` | OUT-018 |
| `--merge-snv-indel` | flag | off | OUT-021 |
(src: A-G18, A-G19, A-G20, A-G9, A-G8, plan)
- **CLI-023** `§14 · P6` **`simurg export …`**
- `export bedgraph --gtcs/--sheet --bpm [--no-gzip]`: OUT-050.
- `export plink --vcfs|--gtcs … [--multiallelic split|drop]`: OUT-043.
- `export finalreport --gtcs --bpm --csv [--matrix]`: OUT-071.
- `export locus-summary --gtcs --bpm --egt`: OUT-072.
None of them re-genotypes: they read GTCs plus the manifest. (src: A-G26)
- **CLI-024** `§14, §9 · P5` **`simurg qc controls`** (control-only QC, no cluster file).
| Option | Type | Default | Validation |
|---|---|---|---|
| `--idats DIR` / `--sheet FILE` | | | CLI-006 |
| `--csv FILE` or `--bpm FILE` | manifest | one required | control definitions (QC-001) |
| `--array-type` | enum `genotyping\|methylation` | **required** | selects the metric set (QC-005/QC-006) |
| `--out DIR` | dir | **required** | QC-014 exception to CLI-010 |
| `--bg-offset X` | float | `3000` | ≥ 0 |
(src: A-G27, C-G2)
- **CLI-025** `§14, §9 · P5` **`simurg qc samples | loci | contamination | relatedness`.** These follow plan
§9.2–§9.4 and §7.9 and write `sample_qc.tsv`, `locus_summary.tsv`, `contamination.tsv` and `relatedness.tsv`.
They take `--gtcs`/`--sheet`, the manifest and `--out`. `relatedness` also takes `--pedigree-from-sheet` (plan
§9.4). (src: plan)
- **CLI-026** `§14, §9.5 · P5` **`simurg qc report`.**
| Option | Type | Default | Validation |
|---|---|---|---|
| `--data` | list of dirs | **required** | one folder, a comma list, or one parent folder (QC-041) |
| `--label` | list | folder names | count must equal the number of datasets (QC-042) |
| `--config FILE` | YAML | none | QC-020/QC-021 |
| `--profile NAME` | enum (QC-024) | none | |
| `--threshold-call-rate X` | float | `0.98` | 0 ≤ X ≤ 1 |
| `--threshold-lrr-sd X` | float | `0.20` | X ≥ 0 |
| `--threshold KEY=VALUE` | repeatable | – | any QC-020 key; `VALUE` may be `null` |
| `--output-format` | enum `xlsx\|csv\|both` | `xlsx` | anything else → `E_QC_FORMAT` |
| `--report-name STEM` | str | `qc_report_YYYY_MM_DD` / `qc_table_YYYY_MM_DD` | QC-044 |
| `--drop-metadata COLS` / `--keep-metadata COLS` / `--no-metadata` | list / flag | keep all | QC-046 |
| `--split-by` | enum `none\|dataset\|plate\|run` | `none` | QC-057 |
| `--out DIR` | dir | CWD | |
(src: A-G28, A-G29, A-G30, A-G31, C-G6, C-G17, C-G20)
- **CLI-027** `§9 · P5` **`simurg qc suggest-thresholds --data … [--k 4] [--min-samples 30] --out FILE.yaml`**:
QC-026. (src: C-G8)
- **CLI-028** `§10 · v2` **`simurg cn call`** (alias `cyto call`).
| Option | Type | Default | Range / validation |
|---|---|---|---|
| `--gtcs DIR` / `--sheet FILE` | | | CLI-006; GTCs must qualify (IN-031) |
| `--model FILE` | cyto model | **required** | IN-051 |
| `--summary FILE` | sample_summary.tsv | auto-discovered next to the GTCs | source of sex and LRR SD |
| `--fasta FILE` | FASTA | optional; recommended | REF padding base (CYTO-029) |
| `--out DIR` | dir | **required** | |
| `--min-del-probes` / `--min-del-size` | int / kb | 10 / 25 | probes 0–250000 (floored to 10, CYTO-008); size 0–250000 kb |
| `--min-dup-probes` / `--min-dup-size` | int / kb | 10 / 50 | same |
| `--min-loh-probes` / `--min-loh-size` | int / kb | 500 / 3000 | same |
| `--min-gainloh-probes` / `--min-gainloh-size` | int / kb | 300 / 500 | same |
| `--min-mdel-probes` / `--min-mdel-size` | int / kb | 10 / 25 | same |
| `--min-mdup-probes` / `--min-mdup-size` | int / kb | 10 / 50 | same |
| `--min-mloh-probes` / `--min-mloh-size` | int / kb | 500 / 3000 | same |
| `--smoothing K` | int | 5 | 0–1000 probes each side (CYTO-004) |
| `--max-mosaic-fraction F` | float | 1.0 | 0 < F ≤ 1 (CYTO-020) |
| `--min-mosaic-fraction F` | float | 0.10 | 0 ≤ F < 1 (`LowFraction`, CYTO-021) |
| `--filter-qual Q` | int | 10 | 0–60 (FILTER `Q10`) |
| `--cn-cap N` | int | 4 | 3–8 |
| `--gc-correct/--no-gc-correct` | flag | on | |
| `--unknown-sex-policy` | enum `skip-sex-chromosomes\|female\|male\|error` | `skip-sex-chromosomes` | CYTO-024 |
| `--gt-mode` | enum `vcf\|compat` | `vcf` | CYTO-031 |
| `--svlen-mode` | enum `signed\|positive` | `signed` | CYTO-029 |
| `--mask-bed FILE` | BED | built-in centromere/gap mask | CYTO-017 |
| `--no-bgzip` | flag | off | warns that `cn annotate` needs bgzip + index (CLI-039) |
(src: A-G45, A-G47, A-G48, D-2, D-3, C-G24)
- **CLI-029** `§10 · v2` **`simurg cn annotate`** (alias `cyto annotate`).
- Required: `--vcfs DIR` (the `cn call` output) and `--annotation-bundle FILE` (CYTO-036).
- Optional:
- the same 14 `--min-*` options, defaulting to the values recorded by `cn call`; they can only tighten, never
loosen;
- `--min-qual Q` (default 20, 0–60);
- `--size-basis effective|raw` (default `effective`, CYTO-011);
- `--iscn-min-size bp` (default 0);
- `--out DIR`.
(src: A-G49, A-G53, A-G54, D-2)
- **CLI-030** `§10 · v2` **`simurg cn build-model --bpm --fasta [--csv] --out model.simurg-cyto`**: CYTO-038.
(src: A-G52)
- **CLI-031** `§11 · v2` **`simurg pgx copy-number call`.**
| Option | Type | Default | Validation |
|---|---|---|---|
| `--gtcs DIR` / `--sheet FILE` | | | GTCs must qualify (IN-031) |
| `--model FILE` | PGx CN model | **required** | IN-050 |
| `--fasta FILE` | GRCh38 FASTA with alt contigs | **required** | IN-062 |
| `--out DIR` | dir | **required** | recommended: the SNV VCF folder (PCN-014) |
| `--min-batch N` | int | 24 | ≥ 1 |
| `--min-passing N` | int | 22 | ≤ `--min-batch` |
| `--recommended-passing N` | int | 96 | warning only |
| `--lrr-sd-max X` | float | 0.20 | passing means LRR SD **<** X (PCN-004) |
| `--cluster-mismatch` | enum `error\|warn` | `error` | IN-050 |
| `--no-bgzip` | flag | off | |
(src: A-G33, D-11, E-39)
- **CLI-032** `§11 · v2` **`simurg pgx copy-number train`.**
| Option | Type | Default | Validation |
|---|---|---|---|
| `--gtcs` / `--sheet` | | | |
| `--bpm FILE` | BPM | **required** | |
| `--csv FILE` | CSV manifest | optional | source of `Assay Format` |
| `--fasta FILE` | FASTA | **required** | |
| `--mask FILE` | TSV | `<bpm-stem>.mask.tsv` next to the BPM if present | IN-052 |
| `--platform` | enum `auto\|LCG\|EX` | `auto` = from the CSV `Assay Format` heading | error if `auto` and no CSV |
| `--min-samples N` / `--min-passing N` | int | 96 / 90 | |
| `--max-samples N` | int | 3000 | seeded random subset above this, with a note |
| `--train-lrr-sd-max`, `--train-call-rate-min`, `--train-assay-control-min` | float | 0.20 / 0.99 / 1.0 | exclusion rules (PCN-005) |
| `--genome-cache/--no-genome-cache` | flag | on | |
| `--out DIR` | dir | CWD | |
(src: A-G34, A-G35, A-G36)
- **CLI-033** `§11 · v2` **`simurg pgx star-allele call`.**
- Required: `--vcfs DIR[,DIR]` (SNV + PGx CN VCFs; recursive; symlinks allowed), `--db FILE` and `--out DIR`.
- Optional:
- `--population NAME` (prior; default `global`);
- `--max-candidates N` (10);
- `--feasible-ratio R` (0.5);
- `--low-gs X` (0.15);
- `--no-cn` (run without CN VCFs; CN-dependent genes are reported as `not_assessed`);
- `--allow-foreign-vcf`.
- Writes `<sample>.pgx_calls.parquet` plus preliminary TSVs.
(src: A-G39, A-G41, B-28, E-40)
- **CLI-034** `§11 · v2` **`simurg pgx phenotype`** (alias `annotate`).
- Required: `--calls DIR`.
- Optional: `--guideline cpic|dpwg` (default `cpic`) and `--out DIR`.
- Writes the TSV set (PGX-026) and `<sample>.pgx.json`. It can be re-run with another guideline without
re-calling. (src: A-G41, B-28, D-16)
- **CLI-035** `§11 · v2` **`simurg pgx db build --sources DIR --manifests BPM[,…] --out FILE`**: PGX-002.
**`simurg pgx coverage --bpm --db`**: PGX-035. (src: A-G40)
- **CLI-036** `§1.2 · v2` **`simurg methyl qc`.**
- Required: `--idats`/`--sheet`, `--manifest CSV` and `--out`.
- Optional:
- `--profile epic|msa|custom`;
- `--threshold KEY=VALUE`, `--config`;
- `--pval-threshold` (0.05);
- `--bg-offset` (3000);
- `--beta-offset` (0);
- `--min-prop-passing` (off);
- `--pca-probes` (10000);
- `--pca-max-samples-full` (100);
- `--ffpe` (sets restoration to 1).
- **`simurg methyl convert-manifest`**: MET-003.
(src: C-G26, D-21, D-22)
- **CLI-037** `§14 · P6 (watch: v2)` **`simurg pipeline run|plan|watch config.yaml`.**
- `plan` = dry run: it resolves inputs, validates everything (IN-049) and prints the step DAG, output sizes and
warnings.
- `run` executes, with `--resume` support.
- `watch` (planned-run mode): polls `inputs.idat_dir` until every sheet sample has an IDAT pair; with
`--scan-metrics`, a sample whose scan registration fails counts as failed (IN-026).
- It **aborts** if failed samples > `max_failed_scans` (default 0).
- Otherwise it runs with the passing samples.
- It writes `sample_status.tsv` (`sample_id, status ∈ {ready, failed, pending}, reason`).
(src: D-27, D-30)
- **CLI-038** `§14 · P7, P1` **`compare`, `synth`, `train`** follow plan §14, §16.4 and §12. (src: plan)
- **CLI-039** `§14 · P6` **Cross-step guards.**
- Using `--unsquash-duplicates` or an exclude/include list in `gtc-to-vcf` records this in the VCF header;
`pgx star-allele call` refuses such VCFs (PGX-011).
- `pipeline plan` also warns *before* running when the config combines these options with a PGx step.
- `cn call --no-bgzip` warns if the configured pipeline contains `cn annotate`.
(src: A 1.14, A-G39, A-G9)
- **CLI-040** `§14 · P10` **Workflow recipes** in the user guide, each as copy-pasteable commands plus a
`pipeline` YAML:
1. genotyping (IDAT → GTC → VCF);
2. QC report (genotype or control-only, then the report; multi-run trends);
3. PGx (genotype → VCF → CN → star alleles → phenotype; optional CN training);
4. cytogenetics (genotype → cn call → cn annotate → bedGraph);
5. semi-custom arrays (single run, SCOPE-010);
6. methylation QC.
Each recipe states the recommended sample gates (QC-035). (src: A-G64, A-G65)
---
## 3. Inputs
### 3.1 Sample sheet
- **IN-001** `§6.6 · P1` **Format.**
- The sheet is a CSV file: UTF-8 (a BOM is tolerated), comma-delimited, RFC 4180 quoting, LF or CRLF line endings.
- Optional sections: `[Header]` (key,value rows), `[Manifests]` (**ignored**) and `[Data]` (the table).
- A sheet with no section markers is valid: its first non-empty line is the header row.
- Section names are matched case-insensitively.
- The file is rejected with `E_SHEET_DELIMITER` and a hint in two cases:
- semicolon- or tab-delimited files (e.g. from European Excel);
- JSON sheets.
- An empty `[Data]` section, or a sheet with no rows, → `E_SHEET_EMPTY`.
(src: A-G58, E-8, B-31)
- **IN-002** `§6.6 · P1` **Key columns.**
- Required: `SentrixBarcode_A` and `SentrixPosition_A`.
- Also accepted, as aliases for the same pair: `beadChipName` and `sampleSectionName`.
- Missing both forms → `E_SHEET_NO_KEY_COLUMNS`.
- Optional recognised columns:
- `Sample_ID`, `Sample_Name`, `Sample_Plate`, `Sample_Well`, `Sample_Group`;
- reported sex: `Gender`, `Sex` or `demographicSex`;
- `Parent1`, `Parent2`, `Replicate`.
(src: A-G59, E-8, plan §6.6)
- **IN-003** `§6.6 · P1` **Case-insensitive columns.**
- Header names are matched after Unicode case-folding and trimming.
- Two columns that become equal after folding are an error: `E_SHEET_DUP_COLUMN`, with the message "Duplicate
column '<name>' (column names are case-insensitive). Rename or remove one of: <orig1>, <orig2>".
(src: A-G56, E-9)
- **IN-004** `§6.6 · P1` **Empty header cells.** Columns whose header is empty are ignored, with one
`W_SHEET_EMPTY_COLUMN` warning. Examples: `SentrixBarcode_A,SentrixPosition_A,,` and a single trailing comma.
Such columns MUST NOT count as duplicates of each other, and MUST NOT produce empty-key fields in any output.
(src: E-9)
- **IN-005** `§6.6 · P1` **Reserved column names.** A sheet column must not collide (case-insensitively) with any
column that SimurgArray itself writes into the sample summary. That covers:
- every OUT-060 column name;
- their reference-style spellings under OUT-066, such as `Sex Estimate`, `Call Rate`, `Log R Ratio Std Dev` and
`Sample Folder`.
A collision → `E_SHEET_RESERVED`. With `--namespace-sheet-columns`, all pass-through columns are prefixed
`sheet_` instead. (src: A-G56, E-9)
- **IN-006** `§6.6 · P1` **Length limit.** Column names and cell values are limited to 500 characters (Unicode code
points). A longer one → `E_SHEET_TOO_LONG`, naming the row, the column and its length. (src: A-G56, E-9)
- **IN-007** `§6.6 · P1` **Barcode and position validation.**
- **Barcode:** after trimming it must match `^[0-9]{10,14}$`.
- Values like `204753010023.0` are repaired to `204753010023`, with `W_SHEET_BARCODE_REPAIRED`.
- Scientific notation (`2.04753E+11`, `2,04753E+11`) cannot be repaired, because precision is lost. It is
rejected with `E_SHEET_BARCODE_FORMAT` and the hint "format the SentrixBarcode_A column as a number with 0
decimals (or as text) before saving".
- **Position:** must match `^[Rr][0-9]{2}[Cc][0-9]{2}$` and is normalised to upper case.
(src: A-G56, E-9)
- **IN-008** `§6.6 · P1` **RootFolder.**
- `[Header] RootFolder,<path>` names the IDAT (or GTC) root. A relative path resolves against the sheet's
directory.
- A CLI folder overrides it (`W_SHEET_ROOTFOLDER_OVERRIDDEN`, logged at info level when the two paths are equal).
- Other `[Header]` keys are kept verbatim in `run_manifest.json` under `sheet_header`.
(src: A-G57, E-8)
- **IN-009** `§6.6 · P1` **Sample identity.** `sample_id` is always `<barcode>_<position>`. A duplicate
`sample_id` among sheet rows → `E_SHEET_DUP_SAMPLE`. `--on-duplicate-sample drop` MAY instead keep the first row
and warn, as the methylation pipeline of the reference does. (src: B-31, D-24)
- **IN-010** `§6.6 · P1` **Sample-name precedence [EMU].**
1. The sheet's `Sample_Name`, if the column exists.
2. The sample name stored in the IDAT (field 406), or in the GTC (field 10).
3. `sample_id`.
Rules for `Sample_Name` when present:
- It must be non-empty in **every** row (`E_SHEET_NAME_EMPTY`).
- It must not equal a *different* sample's `sample_id` (`E_SHEET_NAME_COLLISION`).
- It must be unique across the sheet (`E_SHEET_NAME_COLLISION`) [OURS: required so that output file names are
unique].
(src: A-G16, E-10)
- **IN-011** `§6.6 · P1` **Per-command resolution.**
- Names are resolved independently by each command, and the sheet given to that command is the source of truth.
- The name inside a GTC is **never** rewritten: `genotype` stores the IDAT name, or empty.
- A later command given a sheet uses the sheet's names in its outputs: VCF sample column, summaries, JSON.
(src: A-G16, E-10)
- **IN-012** `§13 · P1` **Output stem.** `<sample>` in file names is the resolved sample name after sanitisation:
- characters outside `[A-Za-z0-9._-]` become `_`;
- leading dots are stripped;
- length is capped at 200.
If two samples sanitise to the same stem → `E_SHEET_NAME_COLLISION`. The unsanitised name is kept inside the files.
(src: plan §13)
- **IN-013** `§6.6 · P1` **Metadata pass-through.**
- Every sheet column that is not a key column is copied **verbatim** into `sample_summary.{tsv,json}`: the header
as written in the sheet, the values as strings, sheet column order preserved.
- They are placed after the fixed columns (OUT-060).
- The QC report and QC table receive them from the summary (QC-040).
(src: A-G15, E-11, B-9)
- **IN-014** `§6.6 · P1` **Reported sex.**
- Values from `Gender`, `Sex` or `demographicSex` are normalised: `M`/`Male`/`1` → `M`, `F`/`Female`/`2` → `F`,
everything else (including empty) → `U`.
- If more than one of these columns is present and they disagree → warning, and the first in the order `Sex`,
`Gender`, `demographicSex` wins.
- The result goes into `sex_reported`, and is compared with `sex_est` in `sex_check` (OUT-060).
(src: A-G59)
- **IN-015** `§6.6 · P1` **Plate, well and group.**
- `Sample_Plate` and `Sample_Well` from the sheet override the IDAT fields 408/409. They drive plate heatmaps
(QC-052) and the plate/well columns of `controls_qc.tsv`.
- `Sample_Group` is the default colour field in plots: control PCA (QC-060) and methylation PCA (MET-013).
(src: C-G3, A-G59, A-G60)
- **IN-016** `§6.6 · P1` **Sheet vs files.**
- A sheet row with no matching IDAT/GTC gets a summary row with status `MISSING_INPUT` and counts toward exit 3/4.
- Files not listed in the sheet are ignored; an info message gives their count.
(src: plan)
- **IN-017** `§6.6, §19 · P1` **Scale.** Parsing and matching are O(n) in rows plus files. A 10 000-row sheet
against 10 000 IDAT pairs must parse and match in < 2 s (PERF-007). (src: E-7)
- **IN-018** `§6.6 · P5` **Parent/replicate columns.** `Parent1`, `Parent2` and `Replicate` feed relatedness and
Mendelian checks (plan §9.4). Values refer to `Sample_ID`, `Sample_Name` or `sample_id`, resolved in that order.
(src: plan)
### 3.2 IDAT and GTC discovery
- **IN-020** `§6.1 · P1` **Recursive IDAT discovery.**
- `--idats DIR`, or the RootFolder, is searched recursively for `<barcode>_<position>_{Grn,Red}.idat` and
`.idat.gz`.
- Matching of the channel token is case-insensitive.
- Symlinks are followed, with cycle detection.
- Directory listing order MUST NOT influence any result: paths are sorted bytewise.
(src: A-G6, E-5)
- **IN-021** `§6.1 · P1` **Pairing.** A sample needs exactly one Grn and one Red file. A lone channel →
`W_IDAT_UNPAIRED`, and the sample fails with `MISSING_INPUT`. (src: A-G6)
- **IN-022** `§6.1 · P1` **Duplicate pairs [DIFF].**
- Two or more pairs that resolve to the same `sample_id` are **not allowed**, for example rescans in different
subfolders.
- Default `--on-duplicate-idat error` → `E_DUP_IDAT` before any processing. It lists every path of every duplicated
sample.
- `first` picks the bytewise-smallest path, and `newest` the most recent IDAT scan date (field 300), with ties
broken by path. Both are deterministic and OS-independent, and log a warning.
(src: A-G6, D-28, E-5)
- **IN-023** `§6.1 · P1` **IDAT validation.**
- Checked: magic, version 3, TOC offsets inside the file, and equal N and address sets in Red and Grn.
- Encrypted (version 1) IDATs → `E_IDAT_ENCRYPTED`.
- A barcode/position inside the IDAT (402/404) that disagrees with the file name → `W_IDAT_NAME_MISMATCH`, and the
file name wins.
(src: plan §6.1)
- **IN-024** `§6.1, §9 · P1` **IDAT metadata extraction.** For the control table and the report, extract:
- imaging date (the scan entry of RunInfo 300), converted to ISO-8601;
- scanner ID and scanner software version (RunInfo/scanner fields);
- plate (408), well (409), sample ID (406), chip type (403), manifest (401).
Missing fields become empty. An IDAT with no scan time → `W_IDAT_NO_SCAN_TIME` (also MET-015). (src: C-G3, B-12)
- **IN-025** `§6.1 · P1` **NextSeq 550 IDATs.**
- IDATs from NextSeq 550 scans must parse. Different RunInfo layouts and missing fields are tolerated.
- Tests use CytoSNP-850K NS550 data when it is available.
- QC thresholds carry a caveat for this scanner (MET-018).
(src: A-G61, C-G24)
- **IN-026** `§14 · v2` **Scan-registration gate [OURS].**
- `--scan-metrics FILE` reads the scanner's per-section metrics file (`metrics.txt`). A section with stripe
registration < 0.75 is marked `failed`, and its sample is excluded, with reason `scan_registration`.
- The status goes into `sample_status.tsv` (CLI-037) and `sample_summary.tsv`.
(src: D-26)
- **IN-027** `§6, §7.1 · P2` **IDAT ↔ manifest check [DIFF].** Before processing, for each chip type:
- compute the fraction of BPM probe addresses present in the IDAT;
- coverage < 0.99 → `E_IDAT_MANIFEST_MISMATCH` (exit 2), with the chip type and manifest name;
- 0.99 ≤ coverage < 1 → `W_IDAT_LOW_ADDRESS_COVERAGE`;
- also compare IDAT field 401/403 with the BPM name, warning only.
The reference silently produces bad calls in this case. (src: E-3, D-29)
- **IN-030** `§6.5 · P6` **GTC discovery.**
- `--gtcs DIR` is searched recursively for `*.gtc`. The sample key is the GTC Sentrix ID (field 1016); for v3/v4
files, the file-name stem.
- Duplicate keys → `E_DUP_GTC`.
- Sheet matching follows IN-016.
(src: A-G6)
- **IN-031** `§6.5 · P6` **GTC acceptance.**
- `gtc-to-vcf`, `export bedgraph` and QC accept GTC v3–v5 from any producer. A GTC whose `autocall_version` string
is not SimurgArray's, or that lacks v5 fields, gets `W_GTC_FOREIGN` once per batch.
- `cn call` and `pgx copy-number call|train` need BAF/LRR (v ≥ 4), normalization transforms, and the cluster-file
name. Anything else → `E_GTC_UNSUITABLE`.
(src: A-G22, B-34)
- **IN-032** `§6.5 · P4` **Corrupt GTC [DIFF].** A GTC that fails validation is skipped: bad magic, truncation,
out-of-range TOC, or an unreadable string. That sample gets `E_GTC_CORRUPT` at warning level and a `FAILED` summary
row; the others continue, and the exit code is 3. The reference aborts the whole run instead. (src: E-2)
- **IN-033** `§6.5 · P1` **Tolerant string decoding.**
- All .NET 7-bit-length strings are decoded as UTF-8 with replacement characters.
- A length prefix that runs past EOF is corruption (IN-032), never an uncaught exception.
- Locale-corrupted date strings are preserved as bytes, and decoded for display only.
(src: E-1)
### 3.3 Manifest, cluster file and model compatibility
- **IN-040** `§6.2 · P1` **BPM.**
- Versions 3–5 with locus-entry versions 4/6/7/8 (plan §6.2) are supported.
- Manifest names of any length are supported (tested at > 80 chars), as are paths with spaces and non-ASCII
characters.
- The manifest name and a SHA-256 go into provenance.
(src: E-4)
- **IN-041** `§6.3 · P1` **CSV manifest.**
- `[Heading]`, `[Assay]` and `[Controls]` sections are parsed.
- `Assay Format` (e.g. `LCG`, `EX`) is read from `[Heading]`. It supplies the platform default (CLI-032) and the QC
profile selection (QC-025).
- `SourceSeq` is required for indel REF/ALT resolution.
- A missing `[Controls]` or `[Assay]` section, or missing required columns, gives the errors of MET-002 when the
CSV is used for QC.
(src: A-G23, A-G36, D-24)
- **IN-042** `§6.3 · P1` **BPM ↔ CSV consistency [DIFF].** When both are supplied:
- locus-name sets must be equal;
- counts must be equal;
- Chr/MapInfo/SNP/AddressA/AddressB must agree for every locus.
Any difference → `E_BPM_CSV_MISMATCH` with counts and the first 10 examples. `--allow-manifest-mismatch` downgrades
this to a warning and uses the BPM values. (src: E-3)
- **IN-043** `§6.4 · P4` **BPM ↔ EGT consistency [DIFF].**
- Every non-control BPM locus name must be present in the EGT. Missing names → `E_EGT_MANIFEST_MISMATCH`
(pre-flight, exit 2), with the count and the first 10 names. The reference instead skips each sample with a
dictionary-key error.
- EGT loci not in the BPM are ignored, with an info count.
- A difference between the EGT's `manifest_name` and the BPM name → warning.
(src: E-3, B-33)
- **IN-044** `§6.4 · P1` **EGT load errors.** Any of the following → `E_EGT_LOAD`, naming the cause:
- a corrupt file;
- an unsupported `version` or `data_block_version`;
- a record-count disagreement.
(src: B-33)
- **IN-045** `§6.4, §12 · P1` **Cluster-file names.**
- Any file name is accepted as input [DIFF].
- ArrayTrain *outputs* whose names contain characters outside `[A-Za-z0-9_-]` get `W_EGT_NAME`, as a portability
notice for other tools.
(src: A-G17, E-4)
- **IN-046** `§6.3, §8.1 · P6` **Manifest mapping conventions** are validated and converted to VCF representation:
- positions are 1-based on the plus strand;
- PAR loci are reported on X coordinates (chr `XY` → X PAR);
- an insertion is anchored at the base immediately 5′ of the insertion;
- a deletion is at its most 5′ deleted base;
- `Chr=0` or `MapInfo=0` means unmapped.
Indels are left-normalised against the FASTA, and the anchor base is added as VCF requires. (src: A-G23, E-19)
- **IN-047** `§15 · P1` **Product-file provenance.** Every product file (BPM, CSV, EGT, models, DB, bundle) is
recorded in `run_manifest.json` by name, size and SHA-256. Vendor files are never copied into outputs. (src: plan)
- **IN-048** `§6 · v2` **Mask file.** See IN-052. **Assay-format platform codes** (`LCG`, `EX`) are stored in model
headers (IN-050). (src: A-G36)
- **IN-049** `§6, §14 · P4` **Pre-flight check table.** `pipeline plan` and every command run these checks before
starting per-sample work.
| Check | Severity | Code | Override |
|---|---|---|---|
| sheet structure, keys, reserved, length, barcode format, names | error | `E_SHEET_*` | – |
| duplicate IDAT pairs / GTCs | error | `E_DUP_IDAT`, `E_DUP_GTC` | `--on-duplicate-idat` |
| IDAT address coverage vs BPM < 0.99 | error | `E_IDAT_MANIFEST_MISMATCH` | – |
| IDAT address coverage < 1.0 | warning | `W_IDAT_LOW_ADDRESS_COVERAGE` | – |
| BPM vs CSV mismatch | error | `E_BPM_CSV_MISMATCH` | `--allow-manifest-mismatch` |
| BPM loci missing from EGT | error | `E_EGT_MANIFEST_MISMATCH` | – |
| EGT manifest name ≠ BPM name | warning | `W_EGT_MANIFEST_NAME` | – |
| FASTA/.fai inconsistent | error | `E_FAI_MISMATCH` | – |
| FASTA contigs vs manifest chromosomes | error (strict) | `E_CONTIG_MISMATCH` | `--contig-alias auto` |
| genome build (manifest vs FASTA lengths) | error | `E_BUILD_MISMATCH` | – |
| PGx: alt contig for GSTT1 missing | error | `E_PGX_ALT_CONTIG_MISSING` | – |
| CN/cyto model vs manifest probe set | error | `E_MODEL_MANIFEST_MISMATCH` | – |
| PGx CN model vs GTC cluster file | error (default) / warning | `E_/W_MODEL_CLUSTER_MISMATCH` | `--cluster-mismatch warn` |
| model schema version unsupported | error | `E_MODEL_VERSION` | – |
| PGx batch size / passing count | error / warning | `E_PGX_BATCH_TOO_SMALL`, `W_PGX_BATCH_BELOW_RECOMMENDED` | `--min-batch`, `--min-passing` |
| star-allele VCFs unsquashed / filtered / unindexed | error | `E_PGX_VCF_UNSUITABLE`, `E_PGX_VCF_UNINDEXED` | `--allow-foreign-vcf` (foreign only) |
| cn annotate: sidecar or index missing | error | `E_CN_SIDECAR_MISSING`, `E_CN_VCF_UNINDEXED` | – |
(src: A-G63, B-20, E-3)
### 3.4 Models, masks and databases
- **IN-050** `§11 · v2` **PGx CN model file** (our own format, `.simurg-pgxcn`). It is a zip that contains:
- `header.json`:
- `schema_version`, `software_version`;
- `manifest_name`, `manifest_revision`, `probe_set_sha256`;
- `cluster_file_name`, `cluster_file_sha256`;
- `genome_build`, `fasta_contigs_sha256`, `platform` (`LCG|EX`), `mask_sha256`;
- `regions_bed_version`, `n_training_samples`, `training_exclusions`, `created`.
- `probes.parquet`: per-probe reference statistics.
- `regions.bed`.
- `mixture.json`: the per-region CN mixture parameters.
Compatibility rules:
- The manifest probe set must match → otherwise `E_MODEL_MANIFEST_MISMATCH`.
- The cluster file must match the one recorded in the input GTCs:
- by default the mismatch is an error, `E_MODEL_CLUSTER_MISMATCH`, whose message tells the user to retrain the
model (PCN-005);
- `--cluster-mismatch warn` downgrades it to `W_MODEL_CLUSTER_MISMATCH`, logging "Cluster file (<gtc egt>)
differs from the CN model's cluster file (<model egt>)".
The reference changed this rule across versions: error, then warning, then error again. Both modes are tested
(EDGE-026). (src: A-G37, B-20, D-11, E-39)
- **IN-051** `§10 · v2` **Cyto model file** (`.simurg-cyto`).
- Contents: `header.json` (schema version, manifest name/revision, probe-set hash, build, FASTA hash, GC window
sizes) plus `probes.parquet` (per-probe GC content at 1 kb / 100 kb / 1 Mb, and optional reference LRR
statistics).
- It is independent of the cluster file, and it is built by us (CYTO-038).
- A probe-set mismatch → `E_MODEL_MANIFEST_MISMATCH`.
- An older `schema_version` → `E_MODEL_VERSION`, with a rebuild hint.
(src: A-G52)
- **IN-052** `§11 · v2` **Probe mask (open format).**
- A TSV with `probe_name<TAB>reason`; `#` starts a comment.
- It is discovered as `<bpm-stem>.mask.tsv` next to the BPM, or given with `--mask`.
- It is used by PGx CN training and calling, and its SHA-256 is recorded in the model.
- A vendor `.msk` file is **not parsed**, because its format is undocumented (OQ-033).
(src: A-G35)
- **IN-053** `§11 · v2` **PGx DB bundle.** A SQLite file named `pgx_db-<schema>-<sha12>.sqlite`, where `sha12` is
the first 12 hex digits of the content SHA-256. Its `meta` table holds the source versions (PGX-001), the builder
software version and the full SHA-256. See PGX-002. (src: A-G40, D-15)
- **IN-054** `§10.4 · v2` **Annotation bundle.**
- A versioned zip `simurg-cyto-annot-<build>-<date>-<sha12>.zip`, built by `scripts/build_annotation_bundle.sh`
from public sources (CYTO-036).
- `cn annotate` checks free disk space before extraction, needing ≥ 2 × the bundle's uncompressed size.
(src: A-G53)
- **IN-055** `§9 · P5` **YAML inputs.**
- The extension must be `.yaml` or `.yml` (`E_QC_CONFIG_EXT`).
- A file named `*.yaml.txt` or `*.yml.txt` gets a targeted hint about editors appending `.txt`.
- Parse errors report the line and column (`E_QC_CONFIG_PARSE`).
(src: C-G6)
### 3.5 Reference FASTA and genome cache
- **IN-060** `§8 · P6` **FASTA + index.**
- The FASTA may be plain or bgzip-compressed; `.fai` (and `.gzi`) are expected next to it.
- A missing `.fai` is built into the cache directory (IN-064), never next to a read-only input, with an info
message.
- A `.fai` whose contig lengths or offsets disagree with the FASTA → `E_FAI_MISMATCH`. The check samples 3 contigs
and verifies line lengths.
(src: B-33, A-G24)
- **IN-061** `§8.2 · P6` **Contig naming.**
- Default `--contig-alias strict`: manifest chromosome names (`1`…`22`, `X`, `Y`, `XY`, `MT`) must exist verbatim
as FASTA contigs, except that `XY` maps to X.
- `auto` applies the alias table `1↔chr1`, `MT↔chrM`, `22_KI270879v1_alt↔chr22_KI270879v1_alt`.
- `FILE` loads a two-column alias TSV.
- The VCF uses FASTA contig names.
(src: A-G24, E-22)
- **IN-062** `§11 · v2` **Alt contigs for PGx.**
- PGx commands require a GRCh38 FASTA that contains the GSTT1-carrying alt contig (`chr22_KI270879v1_alt` or
`22_KI270879v1_alt`). If it is absent → `E_PGX_ALT_CONTIG_MISSING`, with a hint to use an "analysis set with
alts".
- Genome-wide steps are unaffected by alt contigs: loci on alt contigs are emitted only when the manifest maps
probes there.
(src: D-10)
- **IN-063** `§8 · P6` **Build detection.**
- The build is inferred from the chr1 length in the `.fai` (249 250 621 → GRCh37; 248 956 422 → GRCh38) and
compared with the manifest `GenomeBuild`. A disagreement → `E_BUILD_MISMATCH`.
- Non-human builds are taken from the species profile (IN-065).
- Written as `##simurg_genome_build=GRCh38`.
(src: B-6)
- **IN-064** `§8, §19 · P6` **Genome cache.**
- Default: the needed contigs are loaded once into memory (a NumPy uint8 array per contig, about 3.1 GB for human),
shared read-only with worker processes through mmap of a cache file at
`$SIMURG_CACHE_DIR/genomes/<fasta-sha256>/`.
- `--no-genome-cache` uses indexed random access through htslib, for low-memory machines or small manifests.
- Results MUST be identical in both modes.
(src: A-G8)
- **IN-065** `§1, §8 · P6` **Species profile.** A YAML file that lists:
- build name;
- autosome names;
- sex-chromosome names;
- haploid contigs (e.g. MT);
- PAR intervals;
- the sex system (XY/ZW/none).
`human` (GRCh37, GRCh38) and `bovine_umd3` are built in. It controls ploidy (OUT-018), autosomal call rate, the LRR
SD locus set and the sex estimate. (src: C-G22)
---
## 4. Genotyping outputs
### 4.1 GTC
- **OUT-001** `§6.5 · P4` **GTC v5 writer.**
- All TOC entries of plan §6.5 are written, in ascending ID order.
- Strings use .NET 7-bit varint length prefixes and are **ASCII-only** for every field SimurgArray generates.
- Date fields (200, 201) are formatted with an invariant culture: `M/d/yyyy h:mm:ss tt`, with ASCII digits and
`AM`/`PM`.
- The output must be byte-identical under `LC_ALL` = `C`, `tr_TR.UTF-8`, `ar_SA.UTF-8`, `ja_JP.UTF-8` and
`de_DE.UTF-8` (EDGE-001).
- Non-ASCII sample names from an IDAT are written as UTF-8 with a correct byte-length prefix.
(src: E-1)
- **OUT-002** `§6.5 · P4` **GTC content rules.**
- Field 10 (sample name) is the IDAT name, never the sheet name (IN-011).
- Field 100 is the cluster file *name*. Its SHA-256 is recorded in `run_manifest.json` and in the per-sample
Parquet, so that CN models can compare it (IN-050).
- Field 300 is `SimurgArray <version>`.
- Fields 1006–1015 (call rate, gender, LogR dev, p10GC, extended data, BAF, LRR, percentiles) are always filled.
(src: A-G16, E-39)
- **OUT-003** `§6.5, §16 · P4` **Interoperability.** Written GTCs must parse without warnings in BeadArrayFiles,
gtc2vcf and Picard, and must round-trip byte-identically through SimurgArray's own reader and writer. (src: plan)
### 4.2 SNV VCF
- **OUT-010** `§8.3, §13 · P6` **File naming.**
- The default output is `<sample>.snv.vcf.gz` with a `<sample>.snv.vcf.gz.tbi` index. The `.snv` infix keeps SNV
VCFs apart from `.cyto.cnv.vcf.gz` and `.pgx_cn.vcf.gz` files in the same folder.
- With `--no-bgzip`, the output is a plain `<sample>.snv.vcf` and no index is written. A warning is logged, because
`pgx star-allele call` and `cn annotate` need compressed, indexed input.
- `--merge` writes a single `cohort.snv.bcf` with a `.csi` index.
(src: A-G25, A-G9)
- **OUT-011** `§8.3 · P6` **Exact header** (default options). Lines appear in exactly this order:
```
##fileformat=VCFv4.3
##source=SimurgArray <semver>
##reference=file://<absolute FASTA path>
##simurg_genome_build=<GRCh37|GRCh38|species build>
##simurg_manifest=<bpm name>;sha256=<hex>
##simurg_csv_manifest=<csv name>;sha256=<hex>
##simurg_cluster_file=<egt name from GTC>
##simurg_options=unsquash_duplicates=false;exclude_loci=none;include_loci=none;inf1_nocall_informative=false;auxiliary_loci=none;ploidy_policy=sex-aware;contig_alias=strict
##simurg_run=<run_id>
##contig=<ID=1,length=248956422>
… (one line per FASTA contig that carries ≥ 1 manifest locus, in .fai order; identical for all samples of a run)
##FILTER=<ID=PASS,Description="All filters passed">
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
##FORMAT=<ID=GS,Number=1,Type=Float,Description="GenCall score. For records merged from several assays, the minimum GenCall score over the contributing assays">
##FORMAT=<ID=BAF,Number=1,Type=Float,Description="B allele frequency, oriented so that 1 means homozygous ALT (REF=A, ALT=B relative to the reference genome)">
##FORMAT=<ID=LRR,Number=1,Type=Float,Description="Log R ratio">
##SAMPLE=<ID=<sample name>,SentrixID=<barcode_position>>
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT <sample name>
```
- The header contains no `fileDate` and no timestamps, so reruns stay byte-identical (plan §15).
- The `##simurg_*` lines record the effective options (CLI-005).
- The `##simurg_csv_manifest` line is omitted when no CSV manifest was supplied.
- Optional INFO/FORMAT header lines appear only when the matching options are used.
(src: B-6, A-G39, plan §8.3)
- **OUT-012** `§8.3 · P6` **Record defaults [EMU].**
- By default QUAL is `.`, FILTER is `PASS` and INFO is `.`. Call quality is carried only in `FORMAT/GS`.
- `--info-fields ASSAY_TYPE,NORM_BIN,GENTRAIN,CLUSTER_SEP,ILMN_STRAND,PROBE_IDS` adds those annotations. This
changes plan §8.3, where INFO was always on; it is now opt-in.
- `--filter-gs X` adds `##FILTER=<ID=LowGS,…>` and sets FILTER to `LowGS` when GS < X.
(src: B-1, B-2)
- **OUT-013** `§8.3 · P6` **Number formatting.** GS uses 4 decimal places (`%.4f`); BAF and LRR use 8 significant
digits (`%.8g`). NaN is written as `.`. Formatting is locale-independent and deterministic, using the
shortest-round-trip-safe routine from `simurg.core.fmt`. (src: B-7)
- **OUT-014** `§8.3 · P6` **Record order.**
- Records are sorted by contig in `.fai` order, then POS, then REF, then ALT, then ID.
- Output order is identical for every sample of a batch.
- The sample column holds the resolved sample name (IN-010). The `##SAMPLE` line maps it to the Sentrix ID.
(src: B-6, E-27)
- **OUT-015** `§8.1, §8.3 · P6` **REF/ALT assignment.**
- REF is always the FASTA base (or sequence) at POS. ALT alleles come **a priori** from the manifest: the probe's
alleles are mapped to the plus strand (plan §8.1), with REF removed. They never depend on observed data.
- Multiple ALT alleles are listed in order of first appearance, following BPM order of the contributing probes and
then allele order within each probe.
- When REF is not among a probe's queried alleles, the record is still written and `W_REF_NOT_QUERIED` is raised.
ALT then lists all queried alleles, so no sample can be `0/0` for this probe alone. This is typical for MNVs; see
OUT-035.
(src: B-5, plan §8.3)
- **OUT-016** `§8.3 · P6` **BAF/LRR orientation.**
- For single-probe biallelic records, BAF is re-oriented so that it refers to ALT. Let A and B be the probe's
plus-strand alleles:
- if A = REF and B = ALT, write BAF unchanged;
- if A = ALT and B = REF, write 1 − BAF.
- LRR is never changed.
- Test fixtures MUST include a locus where A = ALT (EDGE-017). This intentionally differs from legacy A/B-oriented
reports, where BAF may be 1 − BAF.
(src: E-21, B §3.3)
- **OUT-017** `§8.3 · P6` **Unmerged probes carry full FORMAT.** Single-probe records carry `GT:GS:BAF:LRR` (and
extra fields if requested). Merged records carry `GT:GS` only; see OUT-032. (src: B-3)
- **OUT-018** `§8.3 · P6` **Ploidy.** With `--ploidy-policy sex-aware` (the default):
- Haploid genotypes are written:
- on MT for everyone;
- on Y and non-PAR X for samples with `sex_est = M`.
- Diploid genotypes are written:
- on PAR loci (X coordinates) for everyone;
- on autosomes.
- A heterozygous call in a haploid context is written as `.` (missing) and counted in the audit table as
`het_in_haploid`.
- Y loci of `F` samples are written as haploid `.` (missing).
- `U` samples are treated as diploid on X and haploid on Y/MT, and the header records `unknown_sex_policy=diploid_x`.
- `--ploidy-policy diploid` writes diploid genotypes everywhere except MT.
(src: E-20, B §3.2)
- **OUT-019** `§8.3 · P6` **Skip taxonomy.** Each skipped locus produces one warning (rate-limited, CLI-008) and one
audit row. Totals per code appear in the run summary.
| Code | Condition |
|---|---|
| `W_SKIP_UNMAPPED` | `Chr = 0`, or position 0 or missing |
| `W_SKIP_INTENSITY_ONLY` | `Intensity_Only = 1` (CN probe; kept in bedGraph LRR) |
| `W_SKIP_INDEL_CONTEXT` | indel whose inserted or deleted sequence cannot be determined (no CSV, no SourceSeq, malformed brackets) |
| `W_SKIP_INDEL_REF` | indel whose reference allele cannot be determined from the FASTA |
| `W_INDEL_PARTIAL_MATCH` | source sequence flanks match the genome only partially (the record is still written if the unambiguous part suffices, otherwise skipped) |
| `W_SKIP_EXCLUDED` | probe named in `--exclude-loci` or absent from `--include-loci` |
| `W_SKIP_CONTIG` | manifest chromosome absent from the FASTA under the active alias policy |
(src: B-5, E-18)
- **OUT-020** `§8.1 · P6` **Indels.**
- REF and ALT are resolved from the CSV `SourceSeq`/`TopGenomicSeq` against the FASTA, left-normalised and anchored
(IN-046).
- When the manifest's deletion normalises to an insertion against the reference, or the other way round, the
reference-consistent representation is written. The genotype is remapped, and `flipped_indel=true` goes into the
audit table.
- Duplicate indel probes designed on opposite strands (`[D/I]` and `[I/D]`) whose calls are both heterozygous MUST
merge to `0/1`, never `./.`. The reference had this bug until its v1.4; see the regression test in EDGE-012.
(src: B-4, E-14, E-19)
- **OUT-021** `§8.3 · P6` **SNV and indel at the same position.**
- These are written as separate records by default, which is the reference behaviour.
- `--merge-snv-indel` merges them into one multi-allelic record using the OUT-031 algorithm. Probes that query
different variant types never constrain each other's alleles, except through the allele sets they query.
(src: E-15)
### 4.3 Multi-probe merging ("squashing")
- **OUT-030** `§8.3 · P6` **Site grouping.** After plus-strand mapping and normalisation, probes whose variant
representations share CHROM, POS and REF form one *site*. For indels the normalised span must also be identical.
The site's ALT set is the union of all queried non-REF alleles (OUT-015). Duplicate designs of the same variant, and
tri-allelic designs, both become one site. (src: B §3.1, E-12)
- **OUT-031** `§8.3 · P6` **Genotype combination algorithm** (exact).
1. **Allele classes.** For a probe *p* with queried plus-strand alleles `{a_p, b_p}`, each allele *z* at the site
maps to a probe class `c_p(z) ∈ {A, B, ∅}`:
- **Infinium II (SNV):** the extended base determines the dye: A/T give red, C/G give green. So
`c_p(z) = A` if `z` has the same dye group as `a_p`, and `B` if it has the same dye group as `b_p`. Every base
falls into one of the two classes, because Infinium II never queries an A/T or C/G pair.
- **Infinium I (SNV), and every indel probe [INF]:** `c_p(z) = A` if `z = a_p`, `B` if `z = b_p`, and `∅`
otherwise (no extension, so no signal).
2. **Predicted call.** For a candidate genotype `g`, take the multiset of classes of its alleles (1 allele if
haploid, 2 if diploid) and drop the `∅` entries:
- `{A}` or `{A,A}` → `AA`;
- `{B}` or `{B,B}` → `BB`;
- `{A,B}` → `AB`;
- empty → `NC` (no signal).
3. **Candidates.** Enumerate every unordered genotype over `{REF} ∪ ALT` with the site's ploidy (OUT-018).
4. **Constrain.** Keep `g` only if, for every probe whose call is not NC, the predicted call equals the observed
call. A predicted `NC` never equals an observed call. **An NC probe constrains nothing by default**; it is
presumed to be a probe failure.
5. **Optional Infinium-I no-call rule** (`--inf1-nocall-informative`). Let `S` be the survivors of step 4. For each
Infinium I probe with an NC call, `S' = {g ∈ S : c_p(z) = ∅ for all z ∈ g}`. The NC is read as "missing signal",
so `g` carries none of the probe's queried alleles.
- If `S'` is non-empty, `S := S'`.
- Otherwise keep `S`, and raise `W_INF1_NC_CONFLICT`, because other assays show the allele.
- The exact semantics are [INF]; see OQ-042.
6. **Outcome.**
- `|S| = 1` → call it.
- `|S| = 0` → `./.` plus `W_MERGE_INCONSISTENT`.
- `|S| > 1` → `./.` plus `W_MERGE_AMBIGUOUS`.
- All probes NC → `./.` plus `W_MERGE_ALL_NC`.
The message text is `Failed to combine genotypes (<reason>) - <probe> (<InfiniumI|InfiniumII>): <AA|AB|BB|NC>, …`,
with the probes in BPM order. It is also written to the audit table.
Required tests (EDGE-011):
- Infinium II `[A/G]` = AA with Infinium I `[T/G]` = (T,T) → `./.`, ambiguous between AT and TT.
- Infinium I `[T/A]` giving the T-homozygous call, with Infinium II `[A/G]` = NC → `./.`, ambiguous; *not* a T/T
call.
- A pair that yields 0 survivors → inconsistent.
- All probes NC.
- The Infinium-I NC option both on and off.
(src: B-3, E-12, E-13, A-G20, plan §8.3)
- **OUT-032** `§8.3 · P6` **Merged record content.**
- FORMAT is `GT:GS` only. BAF and LRR are *omitted* from the FORMAT string, not written as `.`.
- GS is the minimum GS over **all** contributing probes, including NC probes [INF, OQ-041].
- QUAL is `.`, FILTER is `PASS` and INFO is `.`, unless `--info-fields` is set.
(src: B-3, E-17)
- **OUT-033** `§8.3 · P6` **Merged record ID.** The names of all contributing probes, in BPM order, joined by `,` for
SNV sites and by `;` for indel sites. (src: B §3.1, E-17)
- **OUT-034** `§8.3 · P6` **`--unsquash-duplicates`.**
- Each probe gets its own record with full FORMAT, so the site is not merged.
- **Exception:** a site whose union allele set is multi-allelic (≥ 2 ALT) is always merged, even with this flag.
- The header records `unsquash_duplicates=true`.
- For assay validation only; PGx refuses such files (PGX-011).
(src: A-G21, E-25)
- **OUT-035** `§8.3 · P6` **Auxiliary loci (MNV restoration).** `--auxiliary-loci FILE.vcf` is a VCF whose records
have `ID` equal to a manifest probe name, and `CHROM/POS/REF/ALT` equal to the true variant.
Validation (each failure → `E_AUX_INVALID` naming the record; exit 2):
- (a) exactly one ALT (not multi-allelic);
- (b) it is an MNV: `len(REF) = len(ALT) ≥ 2` and REF ≠ ALT;
- (c) the probe is the **only** assay at the site (no duplicates);
- (d) the bases the probe interrogates match the probe's designed alleles, one each, which fixes the allele
mapping: the **first** base of REF and ALT on the plus strand for plus-strand designs (for example,
`ATGC[AT/CG]GTAA` designed as `ATGC[A/C]NNNN`: A ↔ AT, C ↔ CG), the **last** base for minus-strand designs,
because the probe extends from the other side (as GTCtoVCF);
- (e) REF equals the FASTA.
The aux record replaces the probe's REF/ALT, and its genotype is mapped through (d). The final report's Top alleles
for MNV probes are **never** reverse-complemented by this mapping (EDGE-016). (src: A-G18, E-16)
- **OUT-036** `§8.3 · P6` **Locus filter lists.**
- `--exclude-loci FILE` (alias `--filter-loci`; the reference's wording "generates" is a documentation error,
OQ-002) and `--include-loci FILE` take plain text: one probe name per line, UTF-8, `#` comments, blank lines
ignored.
- Unknown names produce one warning with a count.
- The list's SHA-256 is recorded in the header and in provenance.
(src: A-G19)
- **OUT-037** `§8.3 · P6` **`--inf1-nocall-informative`** (default off) implements OUT-031 step 5. The user guide
MUST state three things:
- it raises call rate *and* false positives;
- it assumes a cluster file that clusters the "zeroed-out" multi-allelic assays;
- it is recorded in the header.
(src: A-G20, B-3, E-13)
- **OUT-038** `§8.3 · P6` **Index.**
- Indexes are written with htslib: tabix `.tbi`, or `.csi` when any contig exceeds 2²⁹ bp.
- The index includes the count metadata (the pseudo-bin), so `bcftools index --nrecords` and `--stats` work.
- Indexes written by third-party tools are accepted downstream.
(src: E-23)
- **OUT-039** `§16 · P6` **Validation.** CI runs `bcftools view -H`, `bcftools index --stats` and a VCF-4.3
validator on golden outputs. Any error fails the build. (src: E-24)
- **OUT-040** `§8.3 · P6` **Multi-sample BCF** (`--merge`). The same records and FORMAT rules as the per-sample
VCFs. Sample order is sheet order, or sorted `sample_id` when there is no sheet. (src: plan)
- **OUT-041** `§8.3 · P6` **VCF version.**
- SimurgArray writes `VCFv4.3` consistently, with `;`-separated FILTER lists.
- It never claims 4.1 while using 4.3 syntax, which is the inconsistency noted in the reference (OQ-044).
(src: E-24)
- **OUT-042** `§8.2 · P6` **Ambiguous-strand loci.** Plan §8.2 applies unchanged: an ambiguous A/T or C/G locus
without RefStrand gets `confidence=low` in the audit. `--drop-ambiguous` removes those loci. (src: plan)
- **OUT-043** `§8.4 · P6` **PLINK export.**
- `simurg export plink` splits multi-allelic sites into biallelic records by default (`--multiallelic split`,
equivalent to `bcftools norm -m-`), or drops them (`--multiallelic drop`).
- Merged records with no BAF/LRR are fine for PLINK.
- The user guide shows the equivalent manual pipeline: `bcftools merge` → `norm -m-` → `plink2 --vcf`.
(src: B-8)
### 4.4 bedGraph
- **OUT-050** `§13 · P6` **bedGraph tracks.** Two files per sample: `<sample>.baf.bedgraph.gz` and
`<sample>.lrr.bedgraph.gz` (`--no-gzip` gives plain `.bedgraph`).
- **Line 1:** `track type=bedGraph name="<sample> BAF" description="SimurgArray <ver>" visibility=full autoScale=off viewLimits=0:1`.
For the LRR track, use `viewLimits=-2:2`.
- **Rows:** `chrom<TAB>start0<TAB>end<TAB>value`, with `start0 = POS − 1` and `end = POS`. Values use `%.6g`.
- **Order:** manifest chromosomes in `.fai` order, or manifest order when there is no FASTA; then position.
- **Excluded:**
- from both tracks: NaN values, unmapped loci and zero-intensity loci;
- from BAF only: intensity-only loci (they stay in LRR).
- **Probes at the same position:** the mean of their finite values is written, so bedGraph never overlaps [OURS].
- **Orientation:** BAF is written in the manifest A/B orientation. bedGraph has no REF/ALT, and this matches other
bedGraph producers.
(src: B-32, A-G26)
- **OUT-051** `§13 · P6` **bedGraph without re-genotyping.** `export bedgraph` reads GTC + BPM (+ sheet), and does not
need an EGT or FASTA. The chromosome naming option `--chrom-style manifest|ucsc` defaults to `manifest`.
(src: A-G26)
- **OUT-052** `§13 · P6` **No smoothing of raw tracks [DIFF].**
- Raw BAF/LRR tracks are never smoothed.
- `--smoothed-lrr K` MAY add a *separate* `<sample>.lrr_smoothed.bedgraph.gz`: a moving median over 2K+1 probes
within each chromosome.
- A golden test asserts that the raw track equals the GTC values bit-for-bit after formatting. The reference once
corrupted LRR through its smoothing option.
(src: E-28, D-3)
### 4.5 Sample summary
- **OUT-060** `§13, §9.2 · P4 (QC columns P5)` **`sample_summary.tsv` and `.json`.** One row per sample in the input
set: processed, failed and missing samples all get rows. Columns come in this fixed order:
| # | Column | Type | Definition |
|---|---|---|---|
| 1 | `sample_id` | str | `barcode_position` |
| 2 | `sample_name` | str | resolved name (IN-010) |
| 3 | `sample_folder` | str | IDAT directory relative to the input root |
| 4 | `status` | enum | `OK`, `FAILED`, `MISSING_INPUT`, `EXCLUDED_SCAN` |
| 5 | `fail_reason` | str | message code + short text; empty if OK |
| 6 | `autosomal_call_rate` | float | plan §7.7 |
| 7 | `call_rate` | float | plan §7.5 step 10 |
| 8 | `logr_dev` | float | plan §7.6 |
| 9 | `sex_est` | enum M/F/U | plan §7.8 |
| 10 | `sex_reported` | enum M/F/U | IN-014 |
| 11 | `sex_check` | enum `match`, `mismatch`, `unknown` | |
| 12 | `gc10`, `gc50` | float | plan §7.7 |
| 13 | `n_calls`, `n_nocalls`, `n_intensity_only` | int | |
| 14 | `p05_x`, `p50_x`, `p95_x`, `p05_y`, `p50_y`, `p95_y` | int | raw intensity percentiles |
| 15 | `het_rate` | float | autosomal heterozygosity over calls (P5) |
| 16 | `contamination` | float | plan §9.3 (P5; empty if not run) |
| 17 | `<probe>_norm_r` | float | one column per assay-control probe present (OUT-062) |
| 18 | `functional_status`, `control_status`, `control_flag_codes`, `flags` | str | QC-030/031/033 (P5; empty before P5) |
| 19… | sheet columns | str | verbatim, sheet order (IN-013) |
(src: B-9, A-G14, A-G15, E-11, C-G11, plan §13)
- **OUT-061** `§13 · P4` **Failed samples [EMU].** A sample that fails IDAT → GTC still gets a row:
- `status=FAILED` and `fail_reason` set;
- `autosomal_call_rate = call_rate = 0`;
- `logr_dev`, `<probe>_norm_r`, `gc10` and `gc50` are NaN (`NaN` in TSV, `null` in JSON);
- sex is `U`;
- its sheet columns are still passed through.
(src: E-2, B-9)
- **OUT-062** `§9.1, §13 · P4` **Assay-control locus metric.**
- For each probe named in `--assay-control-probes` (default `TGA_Ctrl_5716`, the targeted-amplification control
of ePGx arrays) that exists in the manifest, `<probe>_norm_r` is the normalized R `xn + yn` of that locus (plan
§7.3–§7.4) [INF, OQ-032].
- The column is omitted when no listed probe exists.
- Values < 1.0 add the flag `assay_control_fail` (QC-033).
(src: A-G32, B-9, C-G10, E-26)
- **OUT-063** `§13 · P4` **Precision.** Rates and LRR SD use 8 decimal places (`%.8f`); intensities are integers.
NaN is written as `NaN` in TSV. (src: B §2.8)
- **OUT-064** `§13 · P4` **JSON form.** `sample_summary.json` is an array of flat objects: same keys and order as the
TSV, numbers as JSON numbers, NaN as `null`, sheet values as strings. (src: B-9)
- **OUT-065** `§13 · P4` **CSV form.** `--summary-format csv|both` writes `sample_summary.csv` with the same columns,
for spreadsheet users. (src: A-G14)
- **OUT-066** `§13 · P4` **Compatibility column names.** `--compat-column-names` renames the first columns to
reference-style headers:
- `Sample ID`, `Sample Name`, `Sample Folder`;
- `Autosomal Call Rate`, `Call Rate`, `Log R Ratio Std Dev`;
- `Sex Estimate`;
- `<probe> Norm R`.
It also writes `gt_sample_summary.csv` as an alias file. Our own tools read either form. (src: A-G14, C-G1)
### 4.6 Other genotyping outputs
- **OUT-070** `§9.1, §13 · P5` **Control tables.** `simurg genotype` writes `controls_raw.tsv` and
`controls_qc.tsv` (QC-009, QC-011) into the batch folder by default. The genotype output folder is therefore a
complete QC-report dataset (QC-040). (src: A-G14, C-G1)
- **OUT-071** `§6.7 · P6` **Final report export.** `simurg export finalreport` writes a long-format TSV with one row
per locus × sample. Columns:
- `SNP Name`, `SNP`, `Sample ID`;
- `Allele1 - Top`, `Allele2 - Top`, `Allele1 - Forward`, `Allele2 - Forward`, `Allele1 - Plus`, `Allele2 - Plus`;
- `GC Score`, `GT Score`, `Log R Ratio`, `B Allele Freq`, `Chr`, `Position`.
Allele rules:
- Top alleles come from the TOP-strand base calls.
- MNV probes are never reverse-complemented (EDGE-016).
- For intensity-only loci, BAF is NaN and LRR uses the cluster-mean expected R (plan §7.6).
`--matrix` gives a matrix layout. It warns (`W_LARGE_OUTPUT`) when loci × samples > 1e8, with a size estimate.
(src: B-10, C-G23, E-16)
- **OUT-072** `§7.9 · P6` **Locus summary.** `locus_summary.tsv` has these columns, in order:
- `locus_name`, `locus_id`;
- `n_nocalls`, `n_calls` (both counted over **samples**), `call_freq`;
- `aa_freq`, `ab_freq`, `bb_freq`, `minor_freq`;
- `gentrain_score`, `gc50`, `gc10`;
- `het_excess`, `chi_test_p100`, `cluster_sep`;
- `aa_t_mean`, `aa_t_sd`, `ab_t_mean`, `ab_t_sd`, `bb_t_mean`, `bb_t_sd`;
- `aa_r_mean`, `aa_r_sd`, `ab_r_mean`, `ab_r_sd`, `bb_r_mean`, `bb_r_sd`;
- `plus_minus_strand` (`+`, `-`, `U`).
`het_excess` follows plan §7.9, the k-regularised legacy form. The simpler form `(f_ab − 2pq)/(2pq + ε)` is written
as `het_excess_simple`, and OQ-030 records the discrepancy between the two. (src: B-11, C-G23)
- **OUT-073** `§15 · P1` **Run manifest.** `run_manifest.json` follows plan §15, plus:
- `summary` (CLI-009);
- `effective_options` with sources (CLI-005);
- `sheet_header` (IN-008);
- `skipped_loci_by_code`;
- `failed_samples[]`, each with a code;
- `schema_versions` of every output.
(src: E-47, plan)
- **OUT-074** `§8.2, §15 · P6` **Audit tables.** `audit/*.parquet` holds:
- one row per harmonisation decision (plan §8.2);
- one row per merge decision, with the probes, calls, candidate count and outcome code;
- one row per skip (OUT-019);
- one row per indel flip (OUT-020).
(src: B-3, B-5)
- **OUT-075** `§15 · P6` **Schema discipline [DIFF].**
- Every JSON and TSV output has a versioned schema (`schema` field in JSON; `#simurg_schema=` comment line at the
top of TSVs, which `--no-comment-lines` suppresses). JSON-Schema files are kept under `schemas/`.
- CI validates the golden outputs against them.
- Numbers are numbers; missing values are `null` in JSON and empty or `NaN` in TSV, as documented per column.
- Keys contain no punctuation other than `_`.
- A schema change bumps its version and is logged in `CHANGELOG.md` under "Output schema changes".
(src: E-48, B-18, B-35)
- **OUT-076** `§15 · P6` **Deterministic ordering.** Every output is sorted by documented keys:
- samples by input order (sheet order, or sorted `sample_id`);
- loci by genome order;
- genes by name;
- solutions by rank, then lexical tie-breakers.
No output depends on hash-map iteration order, directory listing order, the OS or the thread count. Golden tests
run on x86-64 and aarch64 (plan U11). (src: E-27, E-5)
- **OUT-077** `§15 · P6` **Default-change changelog.** Any change to a default value of any option or threshold is
listed in `CHANGELOG.md` under "Default changes", with old → new. The reference changed CNV minimum sizes across
minor versions without a machine-readable record. (src: E-47)
---
## 5. QC
### 5.1 Control-probe metrics
- **QC-001** `§9.1 · P5` **Control definitions.**
- Controls are read from the BPM control config (`addr[:addr…],Category,Color,Name`) or from the CSV `[Controls]`
section (`Address,Type,Color_Channel,Name`).
- When both manifests are given, the BPM wins, and differences produce one warning.
- Each control address gives a raw `green = Grn.Mean[addr]` and `red = Red.Mean[addr]`, using the raw IDAT means
with no normalization.
(src: A-G27, plan §9.1)
- **QC-002** `§9.1 · P5` **Canonical control names.**
- Names are normalised before matching: case-fold, collapse whitespace, remove spaces inside parentheses, and treat
`Bgnd` and `Bkg` as the same token.
- They are then mapped to canonical IDs through `simurg/qc/control_names.yaml`. Examples: `Extension (A)` →
`ext_A`; `Biotin (High)` → `biotin_high`; `DNP (Bgnd)` → `dnp_bkg`; `NP (A)` → `np_A`; `String (PM)` →
`string_pm`; `NSB (Bgnd) Red` → `nsb_bkg`; `Hyb (High)` → `hyb_high`.
- Methylation control names (`BS Conversion I C1`, `GT Mismatch 1 (PM)`, …) are mapped the same way.
- When several beads share one canonical ID and a formula uses that probe as a single value, the **mean** is used
[OURS]. Formulas that say min or max take the min or max over all matching beads.
(src: C-G4)
- **QC-003** `§9.1 · P5` **Backgrounds and offset.**
- `bkg_grn` = the maximum **green** intensity over `Extension (A)` and `Extension (T)`.
- `bkg_red` = the maximum **red** intensity over `Extension (C)` and `Extension (G)`.
- Offset `x = 3000`, configurable with `--bg-offset` or YAML `bgOffset`, value ≥ 0.
(src: B-13, E-44)
- **QC-004** `§9.1 · P5, v2` **Metric catalogue.** There are 24 metrics. Table columns:
- **G** = computed for genotyping arrays; **M** = computed for methylation arrays.
- **YAML key** is the threshold key used in QC-020.
- Every metric uses direction `below`: a value < threshold is flagged.
| # | Column (`controls_qc.tsv`) | YAML key | Formula | G | M |
|---|---|---|---|---|---|
| 1 | `restoration` | `restoration` | Restoration green / (bkg_grn + x) | ✓ | ✓ |
| 2 | `staining_green` | `stainingGreen` | Biotin (High) green / Biotin (Bkg) green | ✓ | ✓ |
| 3 | `staining_red` | `stainingRed` | DNP (High) red / DNP (Bkg) red | ✓ | ✓ |
| 4 | `extension_green` | `extensionGreen` | min(Ext C, Ext G) green / max(Ext A, Ext T) green | ✓ | ✓ |
| 5 | `extension_red` | `extensionRed` | min(Ext A, Ext T) red / max(Ext C, Ext G) red | ✓ | ✓ |
| 6 | `hyb_high_med` | `hybGreenHighMed` | Hyb (High) green / Hyb (Medium) green | ✓ | ✓ |
| 7 | `hyb_med_low` | `hybGreenMediumLow` | Hyb (Medium) green / Hyb (Low) green | ✓ | ✓ |
| 8 | `target_removal_1` | `targetRemovalI` | (bkg_grn + x) / Target Removal (1) green | ✓ | ✓ |
| 9 | `target_removal_2` | `targetRemovalII` | (bkg_grn + x) / Target Removal 2 green | – | ✓ |
| 10 | `stringency` | `stringency` | String (PM) red / String (MM) red (channel configurable, OQ-009) | ✓ | – |
| 11 | `nsb_green` | `nonSpecificBindingGreen` | (bkg_grn + x) / max NSB (Bkg) green | ✓ | – |
| 12 | `nsb_red` | `nonSpecificBindingRed` | (bkg_red + x) / max NSB (Bkg) red | ✓ | – |
| 13 | `np_green` | `npGreen` | min(NP C, NP G) green / max(NP A, NP T) green | ✓ | ✓ |
| 14 | `np_red` | `npRed` | min(NP A, NP T) red / max(NP C, NP G) red | ✓ | ✓ |
| 15 | `specificity_1_green` | `specificityIGreen` | min(GT Mismatch 1–3 PM) green / max(GT Mismatch 1–3 MM) green | – | ✓ |
| 16 | `specificity_1_red` | `specificityIRed` | min(GT Mismatch 4–6 PM) red / max(GT Mismatch 4–6 MM) red | – | ✓ |
| 17 | `specificity_2` | `specificityII` | min(Specificity 1–3 red) / max(Specificity 1–3 green) | – | ✓ |
| 18 | `specificity_2_bg` | `specificityIIBg` | (bkg_grn + x) / max(Specificity 1–3 green) (OQ-012) | – | ✓ |
| 19 | `bs_conv_1_green` | `bisulfiteConversionIGreen` | min(BS Conv I C1, C2) green / max(BS Conv I U1, U2) green | – | ✓ |
| 20 | `bs_conv_1_green_bg` | `bisulfiteConversionIGreenBg` | (bkg_grn + x) / max(BS Conv I U1, U2) green | – | ✓ |
| 21 | `bs_conv_1_red` | `bisulfiteConversionIRed` | min(BS Conv I C3–C5) red / max(BS Conv I U3–U5) red | – | ✓ |
| 22 | `bs_conv_1_red_bg` | `bisulfiteConversionIRedBg` | (bkg_red + x) / max(BS Conv I U3–U5) red (OQ-012) | – | ✓ |
| 23 | `bs_conv_2` | `bisulfiteConversionII` | min(BS Conv II 1–4 red) / max(BS Conv II 1–4 green) | – | ✓ |
| 24 | `bs_conv_2_bg` | `bisulfiteConversionIIBg` | (bkg_grn + x) / max(BS Conv II 1–4 green) | – | ✓ |
Notes:
- This corrects plan §9.1, where NSB red must use `bkg_red`.
- Restoration and Target Removal II are new relative to the plan.
- Genotyping has 13 control metrics plus the assay-control metric (`assayControl`, OUT-062). Methylation has 21
(MET-006).
(src: B-13, D-25, C-G5, E-44)
- **QC-005** `§9.1 · P5` **Genotyping metric set** = rows 1–8 and 10–14 of QC-004, plus `assay_control_norm_r`, which
is copied from the sample summary when present. (src: A-G32, D-25)
- **QC-006** `§9.1 · v2` **Methylation metric set** = the 21 rows with M ✓. The same computation code is shared with
MET-006. (src: C-G26, D-21)
- **QC-007** `§9.1 · P5` **Missing controls.**
- Columns that do not apply to the array type are **omitted** from `controls_qc.tsv`.
- When a control a metric needs is absent from the manifest, the metric is `NaN` and `W_CTRL_MISSING` is raised
once per batch. The warning lists the missing control types or names, for example on custom chips.
- These metrics are marked *not evaluable*. They are excluded from pass/fail and listed in `na_metrics` (QC-031).
(src: B-31, D-24)
- **QC-008** `§9.1 · P5` **Degenerate values.**
- A zero denominator raises `W_CTRL_ZERO_DIVISOR`. With a positive numerator the value is `+Inf` (written `Inf`),
which passes every `below` check: signal over a perfectly clean background is not a failure (old scanners write
background-subtracted intensities of exactly 0; K-021). `0/0` gives `NaN`.
- Unlike QC-007, a NaN caused by the *signal* (the probe exists on the chip) counts as out of range, so the sample
is flagged with the metric's code.
- Bead addresses listed by the manifest but absent from the IDAT are skipped; a control none of whose addresses is
on the chip is *missing* (QC-007), not a signal failure.
- Negative intensities cannot occur (uint16 input).
(src: plan) [OURS]
- **QC-009** `§9.1 · P5` **`controls_raw.tsv`** is long format, with one row per control bead per sample:
- columns: `sample_id, category, bead_address, control_name, color_label, green_intensity, red_intensity`;
- row order: sample order, then manifest control order.
`--compat-column-names` writes `controls.raw_metrics.csv` instead, with columns `SampleId, Category, Beadtype,
Control, Color, GreenIntensity, RedIntensity`. All controls are written, including NEGATIVE and NORM. The exclusions
of QC-010 apply only to the report. (src: B-12, C-G1)
- **QC-010** `§9.5 · P5` **Report exclusion list.** Raw-probe views and the QC table's raw-intensity columns leave
out:
- the categories NEGATIVE and NORM;
- the methylation-only `NP (G) 1`–`NP (G) 5`;
- `Biotin (5K)` and `DNP (20K)`.
Derived metrics are still computed from every probe they need. The list is configurable in
`control_names.yaml`. (src: C-G4)
- **QC-011** `§9.1 · P5` **`controls_qc.tsv`** has one row per sample. Columns, in order:
- `sample_id`, `array_type`;
- `imaging_date` (ISO-8601), `sample_plate`, `sample_well`, `scanner_id`, `scanner_version`: taken from the IDAT
(IN-024), with plate and well overridden by the sheet;
- `bg_offset`;
- the applicable metric columns, in QC-004 order.
`--compat-column-names` writes `controls.qc_metrics.csv` with `SampleId, ImagingDate, SamplePlate, SampleWell,
ScannerId, ScannerVersion, RestorationQC, StainingGreenQC, …`. Its date format is `M/D/YYYY h:mm:ss AM/PM`.
(src: B-12, C-G1, C-G3)
- **QC-012** `§9, §14 · P5` **Control-only QC path.** `simurg qc controls` produces QC-009 and QC-011 from IDATs plus
a manifest. It needs no cluster file and does no genotyping, and it works for both genotyping and methylation
arrays. The resulting folder is a valid QC-report dataset with functional QC = `NA`. (src: A-G27, C-G2)
- **QC-013** `§9, §13 · P5` **Genotype emits controls.** See OUT-070. (src: A-G14)
- **QC-014** `§14 · P5` **Adding controls to an existing genotype folder.** `qc controls --out <folder>` MAY write
into a folder that already contains a `sample_summary`, for example from an older SimurgArray version or an external
converter, without `--force`, as long as the control tables themselves do not exist yet. The sample IDs must match
the summary; unmatched IDs produce a warning. This mirrors the reference's legacy path. (src: C §1.2)
### 5.2 Threshold configuration
- **QC-020** `§9 · P5` **Threshold YAML schema.** Two top-level spellings are accepted:
- `qc:` → `report:`
- `QC:` → `Report:`, for files exported by other tools.
```yaml
qc:
profile: genotyping # optional; see QC-024
report:
callRate: 0.98 # number enables; null (or empty) disables
logRDev: 0.20
stainingGreen: 5
stringency: {threshold: 2, direction: below} # mapping form allows direction
restoration: null
assayControl: 1 # alias: tgaControl
bgOffset: 3000
```
- Keys: `callRate`, `logRDev`, `assayControl`, `bgOffset`, and every YAML key in QC-004.
- Aliases: `tgaControl` = `assayControl`; the plan's snake_case `call_rate_min` = `callRate` and `logr_dev_max` =
`logRDev`.
- `direction` is `below` (flag if value < threshold) or `above` (flag if value > threshold). Defaults are `above` for
`logRDev` and `below` for every other key.
- A value exactly equal to the threshold passes.
(src: A-G30, C-G5, E-44)
- **QC-021** `§9 · P5` **Validation.** Each of these errors exits with code 2:
- the file must exist (`E_QC_CONFIG_MISSING`: "QC config file was provided but does not exist: <path>");
- the extension must be `.yaml` or `.yml` (`E_QC_CONFIG_EXT`);
- it must be valid YAML (`E_QC_CONFIG_PARSE`);
- the wrapper `qc`/`QC` → `report`/`Report` must be present (`E_QC_CONFIG_STRUCTURE`);
- only known keys are allowed (`E_QC_THRESHOLD_KEY`, which lists the valid keys);
- values must be numeric or null (`E_QC_THRESHOLD_VALUE`);
- `callRate` must be in [0, 1] and every other threshold ≥ 0 (`E_QC_THRESHOLD_RANGE`).
The same checks apply to `--threshold KEY=VALUE` and to YAML uploaded into the HTML report (QC-055). (src: C-G6,
C-G21)
- **QC-022** `§9 · P5` **Disabled checks.** A `null` or empty value disables a check. Disabled checks are excluded
from both status axes, from threshold lines, from heatmap binary colouring and from the table's conditional
formatting. (src: A-G30, C 1.8)
- **QC-023** `§9 · P5` **Precedence and built-ins.**
- Order, highest first: CLI (`--threshold-call-rate`, `--threshold-lrr-sd`, `--threshold KEY=VALUE`), then the
`--config` YAML, then `--profile` (or the YAML's `profile:`), then built-in.
- Built-in (no profile) [EMU]: `callRate = 0.98` and `logRDev = 0.20` are enabled. **Every other threshold is
unset.**
- Example: the YAML sets `callRate: 0.90` and the CLI sets 0.95 → 0.95 is used, with source `cli`.
- Each effective threshold is recorded with its source (CLI-005, QC-045).
(src: A-G30, C-G6)
- **QC-024** `§9 · P5, v2` **Shipped profiles** (`simurg/qc/profiles/*.yaml`) are *starting points for review*, not
truths. The documentation says they came from the centre and spread of background distributions in a small set of
good and failed datasets, and that they should be recalibrated with QC-026. Values where the reference's own
documents disagree are listed in OQ-007/OQ-008.
| Key | `genotyping` (LCG/HTS) | `genotyping_ex` (EX; provisional, OQ-046) | `genotyping_example` (alternative values) | `methylation_epic` | `methylation_msa` |
|---|---|---|---|---|---|
| callRate / logRDev | 0.98 / 0.20 | 0.98 / 0.20 | 0.98 / 0.15 | – | – |
| stainingGreen / Red | 5 / 5 | 5 / 5 | 5 / 5 | 5 / 5 | 5 / 5 |
| extensionGreen / Red | 5 / 5 | 5 / 5 | 5 / 5 | 5 / 5 | 5 / 5 |
| hybGreenHighMed / MediumLow | 1 / 1 | 1 / 1 | 1.25 / 1.25 | 1 / 1 | 1 / 1 |
| targetRemovalI / II | 1 / – | 1 / – | 5 / – | 1 / 1 | 1 / 1 |
| restoration | null (1 with `+ffpe`) | null | null | 0 (1 with `+ffpe`) | 0 (1 with `+ffpe`) |
| stringency | 2 | 2 | 1.5 | – | – |
| nonSpecificBindingGreen / Red | 1 / 1 | 1 / 1 | 5 / 5 | – | – |
| npGreen / npRed | 5 / 5 | 5 / 5 | 5 / 5 | 5 / 5 | 2.5 / 3 |
| assayControl | 1 | 1 | 1 | – | – |
| bisulfiteConversionIGreen / Red | – | – | – | 1 / 1 | 1 / 1 |
| bisulfiteConversionIGreenBg / RedBg | – | – | – | 1 / 1 | 0.5 / 0.5 |
| bisulfiteConversionII / IIBg | – | – | – | 1 / 1 | 0.5 / 0.5 |
| specificityIGreen / Red / II / IIBg | – | – | – | 1 / 1 / 1 / 1 | 1 / 1 / 1 / 1 |
| bgOffset | 3000 | 3000 | 3000 | 3000 | 3000 |
- The suffix `+ffpe` (e.g. `--profile methylation_epic+ffpe`) sets `restoration: 1`.
- Methylation profiles also carry `pvalThreshold: 0.05` (MET-011).
(src: C-G7, B-14, D-21, A-G30)
- **QC-025** `§9 · P5` **Profile auto-selection.** `--profile auto` picks the profile from the array type (dataset
detection, QC-043) and the manifest `Assay Format`: `EX` → `genotyping_ex`, and everything else → `genotyping`. For
methylation it goes by manifest name: EPIC → `methylation_epic`, MSA → `methylation_msa`. The report shows which
profile was used for each dataset. (src: B-14, C-G7)
- **QC-026** `§9 · P5` **Data-driven threshold suggestion.** `simurg qc suggest-thresholds` reads one or more
datasets. For every enabled or known metric it computes a robust centre `m` (the median) and a spread
`s = 1.4826·MAD` over samples, on the log scale for ratio metrics.
- It suggests `exp(m − k·s)` for `below` keys and `exp(m + k·s)` for `above` keys, with k = 4 by default.
- It needs ≥ 30 samples per metric; otherwise the metric is left unset, with a note.
- Output: a YAML file (QC-020 schema, with comments showing m, s, n and the % flagged at both the current and the
suggested value) and a TSV of the same numbers.
(src: C-G8)
### 5.3 QC status semantics
- **QC-030** `§9.2 · P5` **Functional status.** Values: `PASS`, `FAIL` or `NA`.
- `PASS` iff every enabled functional check passes: `autosomal_call_rate ≥ callRate` **and** `logr_dev ≤ logRDev`.
- `FAIL` if any enabled check fails. Failed samples (OUT-061) are `FAIL`.
- `NA` when no genotyping summary exists: control-only datasets and methylation.
- Overall call rate is reported but never drives the status.
(src: C-G9, C-G11)
- **QC-031** `§9.2 · P5` **Control status.** Values: `CLEAR` or `FLAGGED`.
- `FLAGGED` iff any enabled control metric is out of range, including `assayControl`.
- `control_flag_codes` lists the failing YAML keys, `;`-joined in QC-004 order (e.g. `npRed;stringency`).
- `na_metrics` lists metrics that could not be evaluated (QC-007).
- The two status axes are independent: a sample can be `PASS` and `FLAGGED`. The documentation explains how to read
the combination: biological or sample failures versus technical assay steps.
(src: C-G9, C 1.8)
- **QC-032** `§9.2 · P5` **Methylation pass.** `pass_qc = 1` iff every enabled methylation control metric passes;
`fail_codes` lists the failures (MET-008). Probe detection rate never affects it unless `--min-prop-passing` is
set. (src: D-21, D-22)
- **QC-033** `§9.2 · P5` **Sample QC flags** (`flags` column, `;`-joined, informational):
- `low_call_rate`, `high_lrr_sd`: mirror the functional checks;
- `sex_mismatch`;
- `het_outlier` (|z| > 3 within the batch);
- `contaminated` (plan §9.3 estimate > 0.02 [OURS]);
- `assay_control_fail` (OUT-062);
- `duplicate_of:<id>`, `unexpected_relatedness` (plan §9.4).
(src: plan §9.2)
- **QC-034** `§9.2 · P5` **Assay-control metric in QC.** `assayControl` (default threshold 1 in the genotyping
profiles) takes part in control status, in the trend statistics (QC-054) and in the PGx review gate (QC-035).
(src: C-G10, A-G32)
- **QC-035** `§9.2, §11 · P5` **Recommended downstream gates.** These are documented in the recipes (CLI-040) and
applied as `pgx_review` and `cn_training_exclude` flags:
- PGx review: `logr_dev > 0.2`, `call_rate < 0.99`, or `assay_control_norm_r < 1.0`.
- CN training: the same three conditions (PCN-005).
- Cyto: `logr_dev > 0.2` → SampleQuality; `> 0.3` → whole-chromosome-only mode (CYTO-014).
(src: A-G65, A-G34)
### 5.4 QC report — inputs, outputs, datasets
- **QC-040** `§9.5 · P5` **Input contract.** The report reads **only** dataset folders that contain:
- `controls_raw.tsv` and `controls_qc.tsv` (or their compat CSV names): required;
- `sample_summary.tsv` (or `.csv`, or the compat `gt_sample_summary.csv`): optional but recommended, because it
enables functional QC, call-rate and LRR SD views, heatmaps and metadata.
It never touches raw data, so reports can be regenerated or merged later. Joins use `sample_id`. IDs present in one
file but not the other → `W_QC_UNMATCHED_SAMPLES`, listing up to 20 of them. Those samples appear with the fields
that are available. (src: C-G1, A-G28, C 1.13)
- **QC-041** `§9.5 · P5` **`--data` forms.**
1. A folder that contains the required files.
2. A comma list of such folders.
3. A parent folder, searched recursively for every subfolder containing both control files. Discovered folders are
sorted by path.
Error cases:
- A missing path → `E_QC_NO_DATASETS`: "Input data folder does not exist: <path>".
- No valid folder → "No valid dataset folders detected. Each folder must include at least: controls_raw.tsv,
controls_qc.tsv".
- A listed folder lacking files → `E_QC_DATASET_INCOMPLETE`: "Dataset <label> is missing required file(s): …".
(src: A-G28, C-G17, C-G21)
- **QC-042** `§9.5 · P5` **Labels.**
- `--label` assigns labels in list or discovery order.
- Count mismatch → `E_QC_LABEL_COUNT`: "Number of labels (2) does not match number of datasets (3)."
- With no `--label`, each dataset is labelled with its folder name. Repeated folder names get `_2`, `_3`, … suffixes.
- Labels are sanitised to `[A-Za-z0-9_.-]`, with other characters replaced by `_`. Each changed label raises
`W_QC_LABEL_SANITIZED` ("Label sanitized: My Label<1> → My_Label_1").
- Labels that are duplicates after sanitisation → `E_QC_LABEL_DUPLICATE`.
- The label identifies the dataset in every view, table and export.
(src: C-G17, A-G28)
- **QC-043** `§9.5 · P5` **Dataset types.**
- A dataset is `methylation` if its `controls_qc.tsv` has `array_type=methylation`, or if its controls include
bisulfite-conversion categories; otherwise it is `genotyping`.
- Mixing the two types → `E_QC_MIXED_TYPES`, which lists both groups.
- Allowed mixes: PGx and non-PGx datasets (assay-control columns are empty for non-PGx), legacy and EX chips, and
different versions of one family.
(src: C-G18)
- **QC-044** `§9.5 · P5` **Output files.**
- `qc_report_YYYY_MM_DD.html` and `qc_table_YYYY_MM_DD.{xlsx|csv}`, dated from the run date in UTC.
- `--report-name STEM` sets a fixed stem, for reproducible file names.
- HTML content is byte-deterministic given identical inputs and options, apart from one embedded generation
timestamp.
(src: C-G19, B-15)
- **QC-045** `§9.5 · P5` **QC table layout.**
- **xlsx** (default) has four sheets:
1. **Samples:** one row per sample. Column order:
- `dataset`, `sample_id`, `sample_name`;
- `functional_status`, `control_status`, `control_flag_codes`, `na_metrics`;
- the functional metrics (autosomal call rate, call rate, LRR SD, sex estimate, assay control);
- every derived control metric;
- selected raw control intensities in wide form, one column per `<control_name>_<channel>` (excluding
QC-010);
- metadata columns, **alphabetically ordered**.
Formatting: conditional fills (PASS blue, FAIL orange, out-of-range functional metric orange, out-of-range
control metric light amber, flag-code cell amber), a frozen header, autofilter.
2. **Thresholds:** `key, description, threshold, direction, enabled, source (default|profile:<n>|yaml:<path>|cli)`.
3. **Datasets:** `label, path, type, n_samples, profile`.
4. **About:** software version, generation time and input file hashes.
- **csv:** the Samples sheet only, values only.
- `--output-format both` writes both formats.
(src: A-G29, C-G19, B-15)
- **QC-046** `§9.5 · P5` **Metadata privacy.**
- The report and the table carry every sheet column found in `sample_summary`. The HTML shows a notice when
metadata is present, and the docs warn about sharing.
- `--drop-metadata COLS`, `--keep-metadata COLS` (an allowlist) or `--no-metadata` remove columns before anything is
embedded.
- `--redact-paths` blanks `sample_folder`.
- Editing `sample_summary.tsv` and re-running the report also works, because the report reads only files (QC-040).
(src: A-G31, C-G20)
### 5.5 QC report — HTML specification
- **QC-047** `§9.5 · P5` **Technology.**
- The report is one self-contained HTML file: all JS and CSS inlined (bundled Plotly.js, MIT) and **no network
requests**.
- Data is embedded as compressed JSON and decoded in the page.
- It must work offline in current Firefox and Chromium.
- Colours are colour-blind-safe. Every colour pairs with a text label or legend.
(src: C-G12)
- **QC-048** `§9.5 · P5` **Chrome.**
- **Header:** "SimurgArray QC Report", software version, generation date, dataset count, sample count, and the
metadata notice (QC-046).
- **Left navigation:** Control Dashboard · Automated QC · Sample Heatmaps · Trend Analysis · ⚙ QC Thresholds.
- **Plot toolbar:** PNG export, zoom, pan, box select, lasso select, zoom in/out, autoscale, reset axes.
(src: C-G12)
- **QC-049** `§9.5 · P5` **View A: Control Dashboard.**
- **Controls:** Dataset (one at a time); Sort plots by (default Sentrix barcode, then any metric or metadata
field); Colour palette; Facet by (None by default, or an eligible metadata field, QC-056).
- **Top card:** "Autosomal Call Rate", a horizontal box plot of the dataset with individual outlier points. Shown
only when functional metrics exist.
- **Cards:** one per control category × channel, two per row. Categories: Staining, Extension, Target Removal,
Hybridization, Restoration, Stringency, Non-Specific Binding, Non-Polymorphic, Assay control; for methylation
also Bisulfite Conversion I/II and Specificity I/II.
- **Card header:** the title, plus an *expected behaviour* sentence written by us. Example: "Staining – Green:
Biotin (High) should be strong; DNP (High/Bkg) and Biotin (Bkg) should be near background."
- **Plot:** x = samples (categorical, in sort order, tick labels hidden); y = raw intensity from 0. One trace per
control probe, each probe with a fixed colour and a legend entry.
- **Hover:** sample ID, barcode, position, dataset, autosomal call rate (if available) and the value.
(src: C-G12)
- **QC-050** `§9.5 · P5` **View B: Automated QC.**
- **Dataset selector.**
- **Two status histograms**, horizontal bars labelled with count and percentage (e.g. "76 (40%)"):
- Functional: FAIL (orange), PASS (blue). Hidden when the status is NA.
- Control: FLAGGED (amber), CLEAR (blue).
- **Sample table.**
- Columns: Dataset, Sample ID, Functional QC Status, Control QC Flag Codes, Autosomal Call Rate, Overall Call
Rate, LRR SD, Sex Estimate (shown as Male/Female/Unknown), one column per derived metric, then metadata.
- Features:
- per-column sort;
- a column show/hide menu;
- per-column footer filters (text for strings, min–max range for numbers);
- global search;
- pagination (10 per page by default; 10/25/50/100/All);
- "Showing a to b of N entries";
- Export ▾ (xlsx, csv);
- "Select none".
- Cell colouring as in QC-045.
- Export writes the selected rows if any are selected, otherwise the currently filtered rows.
- **Per-metric scatter cards.**
- Header: the metric name and its formula as help text (QC-004 text, e.g. "(Bkg Red + offset) / highest NSB
Red").
- Axes: x = samples in the current view ("Sample ID"); y = the value.
- Points are coloured by functional status by default (PASS blue, FAIL orange, NA grey), and can be recoloured by
an eligible metadata field.
- Sort by autosomal call rate, LRR SD, any metadata field or any metric.
- A dashed threshold line, annotated "Flag if below X" or "Fail if above X"; none when disabled.
(src: C-G12, C-G13)
- **QC-051** `§9.5 · P5` **Linked selection.**
- A box or lasso selection in any scatter (Views A and B) highlights the same samples in every scatter and filters
the table.
- Double-clicking a plot clears the selection.
- Selecting table rows highlights the matching points; "Select none" clears it.
- Plot selection, table search and column filters combine with AND.
- Moving or resizing a selection over 12 000 points must not freeze the page for more than 200 ms (EDGE-028). The
reference has a known freeze here.
(src: C-G13, E-45)
- **QC-052** `§9.5 · P5` **View C: Sample Heatmaps.**
- **Controls:** Dataset; Colour scale (Continuous | Binary); Group by (Chip | Plate); Max chips per page (default 2,
chip mode only); Sort by (barcode, plate, or metric mean); Plate selector (plate mode); paging with Prev/Next
and "1 / N".
- **Panels:** two metrics side by side for every chip or plate: "Autosomal Call Rate (higher is better)" and "LRR
SD (lower is better)". Each has the subtitle "Cut-off = X", or "Cut-off disabled".
- **Chip layout:**
- rows R01…Rnn on the y-axis and columns C01…Cmm on the x-axis; the panel title is the chip barcode;
- the grid dimensions come from a built-in chip-format table keyed by IDAT chip type or manifest (e.g. 24-sample
= 12×2, 48-sample = 12×4, 8-sample = 8×1);
- if the chip type is unknown, the grid uses the maximum observed row and column;
- empty positions are drawn hatched, labelled "missing".
- **Plate layout:** 8×12 (rows A–H, columns 1–12), keyed by `sample_well`, with the plate name as title. The sheet
wins over the IDAT (IN-015). Samples with no plate are listed below the panels.
- **Cells:** annotated with the value (3 decimals for call rate, 2 for LRR SD).
- Continuous colouring: call rate light→dark green over the data min–max; LRR SD dark→light blue.
- Binary colouring: call rate orange if below the cut-off, dark green otherwise; LRR SD blue "Below cut-off",
yellow "Above cut-off", with a two-swatch legend.
- **Hover:** sample ID, name, position or well, and the value.
- Heatmap controls must stay aligned for ≥ 50 plates or barcodes (EDGE-028).
(src: C-G14, C-G12, E-45)
- **QC-053** `§9.5 · P5` **Chip completeness.** Per dataset:
- `n_chips`;
- `n_complete_chips`: every position of the chip format filled;
- `n_missing_positions` = Σ (expected positions − observed).
These use the chip-format table of QC-052. (src: C-G15)
- **QC-054** `§9.5 · P5` **View D: Trend Analysis.**
- **Summary table**, one row per dataset, sortable, with grouped headers:
- *Scan:* date range (`YYYY-MM-DD to YYYY-MM-DD`, or a single date), # samples, # complete / # chips, # missing
positions.
- *Autosomal Call Rate:* min, median, mean (SD), # < threshold.
- *LRR SD:* mean (SD), median, # > threshold.
- *Assay control:* mean (SD), # < threshold. Blank for non-PGx datasets.
- *Sex estimate:* # M, # F, # U.
- **Plots**, with Group by (dataset (default) | chip) and Sort by:
1. Autosomal call rate: horizontal box plot per group, with a dashed red threshold line.
2. LRR SD: box plot with a threshold line.
3. Assay control: box plot with a threshold line (PGx datasets only).
4. Sample count: bar chart.
5. Sex estimates: grouped bars (Male blue, Female orange, Unknown grey).
- **Box plots:** Q1–Q3 box, median line, whiskers to the most extreme points within 1.5 × IQR, and outliers drawn.
This rule is stated explicitly, not left to library defaults.
- A single dataset still gets this view.
(src: C-G15, C-G12)
- **QC-055** `§9.5 · P5` **View E: QC Thresholds (modal).**
- **Layout:** two sections, "Sample QC metrics" (Threshold, **Fail if** below/above) and "Control QC metrics"
(Threshold, **Flag if** below/above). Columns: threshold input, direction dropdown, metric, description (formula).
A blank input disables the check.
- **Buttons:**
- **Apply** recomputes both status axes client-side and refreshes every visual (histograms, table colours,
threshold lines, heatmap cut-offs, trend counts).
- **Download YAML** exports the current thresholds in the QC-020 schema, so the file can be reused with
`--config`.
- **Upload YAML** loads a file into the fields, validated as in QC-021, with inline errors.
- Changes last for the session only and are never written back into the HTML.
(src: C-G12)
- **QC-056** `§9.5 · P5` **Metadata eligibility.**
- A metadata field is offered for Colour by or Facet by when it has 2–20 unique non-empty values.
- Faceting shows up to 12 facet panels. Extra levels are grouped as "Other", with a note [DIFF]; the reference
collapses anything above 6.
- Hover text shows up to 200 characters per field, then "…" [DIFF]; the reference truncates at 40. The field list in
hover is capped at 12 fields.
(src: C-G13, E-45)
- **QC-057** `§9.5 · P5` **Scale behaviour.**
- **> 1 000 samples** in a dataset: control and metric plots are split into tabs, one per category.
- **> 12 000:** scatters switch to WebGL rendering (`scattergl`) and also offer a density (2-D histogram) mode.
They are **not disabled** [DIFF]. A notice explains the mode.
- **> 50 000:** density mode by default.
- **> 20 000 in total** across datasets: a warning in the CLI and in the page.
- `--split-by dataset|plate|run` writes one report per group, plus an index page.
- Switching from a large dataset to a small one must not need a reload; the reference needs one.
(src: C-G16, E-45)
- **QC-058** `§9.5 · P5` **Multi-dataset behaviour.** Each view has a dataset selector: one dataset at a time, except
Trend Analysis, which shows all of them. Dataset labels (QC-042) prefix sample IDs in hover and in exports. A report
built from one dataset still shows every view. (src: A-G28, C-G17)
- **QC-059** `§9.5 · P5` **Sex view.** The plan's sex plot is kept as a card inside Trend Analysis: X vs Y median R per
sample, coloured by `sex_est`, with the symbol showing `sex_check`. (src: plan §9.5)
- **QC-060** `§9.5 · P5` **Control-metric PCA (optional card).** A PCA of the log-transformed, standardised derived
control metrics across the samples of a dataset. It is coloured by `Sample_Group`, or by any eligible field, and is
available for genotyping and methylation alike. With fewer than 3 metrics or fewer than 10 samples, the card shows
"insufficient data". (src: A-G60)
- **QC-061** `§9.5 · P5` **Robust CSV/TSV reading.** Input files are UTF-8 (a BOM is allowed), comma- or
tab-delimited as the extension says, with RFC-4180 quoting. Parse errors name the file, the line and the column.
(src: C 1.13)
- **QC-062** `§9.5 · P5` **Methylation datasets in the report.**
- Views A, B and D work with the methylation metric set. There is no functional status; `pass_qc` is shown as the
control axis.
- Heatmaps colour by the fraction of probes passing detection, when `methyl_qc_summary.tsv` is present.
(src: C-G26)
- **QC-063** `§16 · P5` **Acceptance tests.** Every error and warning in QC-021/041/042/043, plus `E_QC_FORMAT`
("Invalid --output-format value. Only csv, xlsx or both are accepted."), has a CLI test that asserts the code and
the exit status (EDGE-028). (src: C-G21)
- **QC-064** `§9.3, §9.4 · P5` **Contamination and relatedness** stay as in plan §9.3–§9.4. Their outputs join
`sample_summary` (`contamination`) and the `flags` column (QC-033), and appear as optional table columns in the
report. (src: plan)
---
## 6. Cytogenetics: genome-wide CNV / LOH / mosaicism (v2)
### 6.1 Scope, inputs, pre-processing
- **CYTO-001** `§10 · v2` **Scope.**
- Constitutional analysis only (`annotation_type = "constitutional"`).
- The genome is assumed to be mostly diploid. Triploid or near-haploid genomes are a documented limitation: events
may be missed.
- Any human genotyping array is accepted. Sensitivity depends on probe density, and the report states the median
probe spacing.
(src: D-7, E-35)
- **CYTO-002** `§10 · v2` **Inputs.**
- GTCs that satisfy IN-031.
- A cyto model (IN-051).
- The batch `sample_summary.tsv`, which supplies sex and LRR SD.
- Optionally a FASTA, for REF padding bases.
- Nothing depends on the cluster file beyond the GTC contents.
(src: A-G52)
- **CYTO-003** `§10.1 · v2` **GC-wave correction.**
- LRR is regressed on the model's GC covariates (plan §10.1), and the residuals are used.
- The on/off state is written as `##GCCorrect=true|false` and as JSON `gc_correct`.
- The correction factor (GCWF) goes into the JSON.
(src: A-G52)
- **CYTO-004** `§10.2 · v2` **LRR smoothing.**
- `--smoothing K` (default 5, range 0–1000) applies a moving median over 2K+1 consecutive probes of the same
chromosome. At chromosome ends the window is truncated, never wrapped. K = 0 disables smoothing.
- Smoothed LRR is used **only** as segmentation input and for the visual track (OUT-052).
- Stored LRR, `NR`, `LRD` and all statistics use unsmoothed values.
(src: A-G48, D-3, E-28)
- **CYTO-005** `§10.2 · v2` **State space.**
- Constitutional states (plan §10.2): CN0, CN1, CN2, CN2-LOH, CN3, CN3-LOH (gain-LOH), CN≥4.
- Mosaic states: loss, gain and CN-LOH, each with cell fraction f.
- Haploid-region states (male X/Y outside PAR): CN0, CN1, CN2, plus mosaic states between them.
- CN is capped by `--cn-cap` (default 4, meaning "4 or more").
(src: A-G45, D-1, E-31)
### 6.2 Event taxonomy and reporting thresholds
- **CYTO-006** `§10.2 · v2` **Event types** (emulating the reference header definitions; see OQ-016/OQ-035).
| Call type | ALT | Subtype (`LOHTYPE`) | Rule | Min probes / size (default) |
|---|---|---|---|---|
| DEL | `<DEL>` | – | CN < baseline (2 for autosomes/PAR/female X; 1 in haploid regions), non-mosaic | 10 / 25 kb |
| DUP | `<DUP>` | – | CN > baseline, non-mosaic | 10 / 50 kb |
| LOH | `<LOH>` | `AOH` | non-mosaic absence of heterozygosity with CN = 2 | 500 / 3000 kb |
| LOH | `<LOH>` | `AOH` (constitutional gain-LOH) or `GAINLOH` (mosaic gain-LOH) | absence of heterozygosity with CN ≥ 3 | 300 / 500 kb (the `gainloh` thresholds) |
| mosaic DEL | `<DEL>` + `MOSAIC` | – | mosaic loss | 10 / 25 kb |
| mosaic DUP | `<DUP>` + `MOSAIC` | – | mosaic gain, including male chrY duplication | 10 / 50 kb |
| mosaic LOH | `<LOH>` + `MOSAIC` | `CNLOH` | mosaic copy-neutral LOH | 500 / 3000 kb |
LOH at CN = 1 (hemizygous) is part of the DEL call and is not reported separately. (src: A-G45, B-16, D-1, E-31)
- **CYTO-007** `§10.2 · v2` **Per-type thresholds.**
- The 14 options `--min-{del,dup,loh,gainloh,mdel,mdup,mloh}-{probes,size}` of CLI-028 are applied independently.
- Sizes are given in kb on the CLI and stored in bp internally [DIFF]. The JSON shows `*_bp` integers, so units
can never be wrong.
- Ranges are 0–250 000 (probes and kb alike).
(src: A-G45, B-17, D-2, E-29, E-38)
- **CYTO-008** `§10.2 · v2` **Probe floor [DIFF].** Any `--min-*-probes` value < 10 is raised to 10, and
`W_CN_PROBE_FLOOR` is logged once for each option. The reference resets such values silently. (src: A-G46, E-29)
- **CYTO-009** `§10.2 · v2` **Thresholds that exceed the genome.** When a size threshold exceeds the sample's
effective genome size, every event of that type is filtered out. This is not an error: `W_CN_THRESHOLD_EXCEEDS_GENOME`
is logged and an empty result is written. (src: E-29)
- **CYTO-010** `§10.2 · v2` **Effective size.**
- Let the probes inside an event be sorted by position, with spacings `s_i`.
- Let `m` be the median inter-probe spacing of that chromosome over all manifest probes that are used [INF,
OQ-039].
- A spacing with `s_i > 150·m` is a *gap*.
- Effective size = `(end − start + 1) − Σ_gaps (s_i − m)`.
- The effective genome size is the sum of the chromosome effective spans.
(src: B §3.11, D-7)
- **CYTO-011** `§10.2 · v2` **Size basis [DIFF].**
- `cn call` filters by **effective** size.
- `cn annotate --size-basis` defaults to `effective` too, which keeps the two steps consistent. `raw` is available.
The reference filtered by effective size at call and raw size at annotate.
- The chosen basis is written to the sidecar and the JSON.
- A regression test uses an event that spans a centromere gap (EDGE-024).
(src: D-2, E-30)
- **CYTO-012** `§10.2 · v2` **QUAL.**
- `QUAL = min(60, max(0, round(−10·log10(1 − P))))`, where `P` is the mean over the event's probes of the
posterior marginal probability of the called state class, from forward–backward [OURS, OQ-040].
- If P = 1, QUAL is 60.
- The VCF writes an integer. The JSON `quality` is a float with 1 decimal.
(src: B-16, D-5)
- **CYTO-013** `§10.2 · v2` **Filters.**
- VCF FILTER values: `Q10` (QUAL < `--filter-qual`, default 10), `SampleQuality` (sample `logr_dev` > 0.2),
`LowFraction` (mosaic MF < `--min-mosaic-fraction`), or `PASS`. They are `;`-joined when several apply.
- Reporting gate: `cn annotate --min-qual` (default 20) removes events with QUAL < min-qual from the JSON and
ISCN. They stay in the VCF.
- Events with QUAL 0 are kept in the VCF, and reach the JSON whenever min-qual ≤ 0 (EDGE-024).
(src: A-G49, A-G50, B-17, D-5, E-29)
- **CYTO-014** `§10.2 · v2` **Sample-level noise gating.**
- `logr_dev > 0.2`: every record gets FILTER `SampleQuality`, and the JSON gets `qc_flags: ["high_noise"]`.
- `logr_dev > 0.3`: *whole-chromosome-only* mode. Only events that cover ≥ 90 % of a chromosome's effective span
are called and reported [OURS, OQ-052]. The JSON gets `qc_flags: ["whole_chromosome_only"]`.
(src: A-G50, E-33)
- **CYTO-015** `§10.2 · v2` **Post-filters**, applied in this order:
1. **LRR direction consistency.** A gain needs ≥ 60 % of its probes with LRR > 0; a loss needs ≥ 60 % with
LRR < 0 (`--lrr-direction-min-frac`, default 0.6 [HYP]). LOH has no requirement. Failing events are dropped and
counted.
2. **Short LOH.** LOH < 500 kb whose immediate neighbours on both sides are copy-neutral (CN2, non-LOH) is removed.
3. **Merging.** Adjacent events of the same type, CN and mosaic state, separated by ≤ 3 probes and by a gap ≤ 10 %
of their combined length, are merged [OURS].
4. **Per-type thresholds** (CYTO-007).
(src: E-33, plan §10.2)
- **CYTO-016** `§10.2 · v2` **Excess events.** More than 10 DEL+DUP+LOH events on one chromosome sets sample flag
`excess_events` in the JSON and in `cn_sample_summary.tsv`, with the chromosome list, for visual review. (src:
A-G51, E-33)
- **CYTO-017** `§10.2 · v2` **Centromeres and gaps.**
- A built-in mask BED per build covers centromeres, the acrocentric p-arms and large assembly gaps. It is
replaceable with `--mask-bed`.
- Events are trimmed to the unmasked sequence.
- Events that are ≥ 50 % masked are dropped.
- Trimmed events carry the JSON flag `centromere_trimmed`.
(src: D-6, E-35)
### 6.3 Mosaicism
- **CYTO-018** `§10.3 · v2` **Mosaic fraction (MF)** [OURS, based on LIT (MoChA, PennCNV-style models)]. With BAF
split `d` = |BAF_het-band − 0.5| over heterozygous-informative probes, and allelic imbalance measured on phased or
unphased BAF:
- mosaic loss: `f = 2d/(0.5 + d)`;
- mosaic gain: `f = 2d/(0.5 − d)`;
- mosaic CN-LOH: `f = 2d`.
In haploid regions, or with too few heterozygous probes, MF is estimated from LRR as
`f = (2^{(LRR_event − LRR_base)/γ} − 1)·sign`, with the per-sample compression γ fitted from the constitutional
state means. MF is clipped to [0, 1]. (src: A-G47, D-4)
- **CYTO-019** `§10.3 · v2` **MF not estimable.** With fewer than 20 informative probes for the method in use, MF is
`.` in the VCF and `null` in the JSON [EMU: the reference writes NaN]. (src: A-G47, E-32)
- **CYTO-020** `§10.3 · v2` **Promotion.** A mosaic event with MF ≥ `--max-mosaic-fraction` (default 1.0, so off) is
*promoted*:
- it becomes a constitutional call, without the `MOSAIC` flag;
- it gains the `HIGHFRACTION` flag;
- MF is kept;
- the ID prefix loses `Mosaic`;
- the constitutional thresholds of its type apply.
(src: A-G47, D-4, E-32)
- **CYTO-021** `§10.3 · v2` **Low fraction [DIFF].** A mosaic event with MF < `--min-mosaic-fraction` (default 0.10
[HYP, OQ-020]) is **kept** with FILTER `LowFraction`. It is excluded from ISCN, but listed in the JSON with
`filters: ["LowFraction"]`. There is no hard cut-off; the reference's old 5 % cut-off was removed. (src: D-4, E-32)
- **CYTO-022** `§10.3, §16 · v2` **Sensitivity target.** Mosaic MF ≥ 15 % is detectable for one-copy-difference
states:
- diploid CN2↔CN1 and CN2↔CN3;
- haploid CN1↔CN0 and CN1↔CN2, including male chrY duplication.
The validation (plan §16.4 synthetic) simulates MF = 5, 10, 15, 20 and 30 % at 1, 5 and 20 Mb, and reports
sensitivity. The gate is ≥ 90 % sensitivity at 15 % / 5 Mb on GSA-density data. (src: A-G55, D-4, E-32)
### 6.4 Sex, ploidy, summary statistics
- **CYTO-023** `§10.2 · v2` **PAR.** PAR1/PAR2 probes are called as diploid for both sexes. Manifest `XY` loci are
mapped to X PAR positions (IN-046). A missing PAR mapping in a manifest must not make PAR uncallable: PAR intervals
come from the species profile. (src: D-6, E-34)
- **CYTO-024** `§10.2 · v2` **Sex resolution [DIFF].**
- The sex used for baselines is: `sex_est` from the summary, else the sheet's reported sex.
- If it is `U`, the caller estimates sex from the median LRR of chrX and chrY and the X heterozygosity rate, and
logs `I_CN_SEX_RESOLVED` ("Sample <id> sex updated from U to <M|F>").
- If sex is still unresolved, `--unknown-sex-policy` decides:
- `skip-sex-chromosomes` (default): no X/Y events; flag `sex_unresolved`;
- `female` or `male`: assume that sex; flag `sex_assumed`;
- `error`: the sample fails.
- The sample is never assumed male silently.
(src: D-6, E-34)
- **CYTO-025** `§10.2 · v2` **Overall ploidy.**
- `overall_ploidy` = the length-weighted mean CN over all called contigs (autosomes, X and Y, using effective
lengths), with baseline CN per the sex model. Males therefore sit at ≈ 1.95 without events.
- Header: `##OverallPloidy=<%.3f>`.
(src: B-22)
- **CYTO-026** `§10.2 · v2` **Copy-number median.** `copy_number_median` = the length-weighted median CN over all
segments, with no minimum segment length. The per-chromosome `copy_number_mean` is length-weighted too. (src: B §3.2,
E-37)
- **CYTO-027** `§10.2 · v2` **Event statistics.**
- `NR` = median over the event's probes of `2^{LRR}` (≈ CN/2) [INF].
- `LRD` = SD of the unsmoothed LRR in the event.
- `bafDev` = SD of BAF over the event's probes, excluding homozygous calls.
- `percent_het` = the percentage of called probes in the event that are heterozygous.
(src: B §2.2)
### 6.5 CNV VCF
- **CYTO-028** `§10, §13 · v2` **File.** `<sample>.cyto.cnv.vcf.gz` with its `.tbi` (bgzip by default; CLI-028).
Records sorted by `.fai` or manifest contig order, then POS. (src: B-16, A-G25)
- **CYTO-029** `§10 · v2` **Coordinates [DIFF, VCF-correct 1-based].**
- For an event whose first affected base is `S` and last affected base is `E` (1-based, inclusive):
- `POS = S − 1`: the padding base, VCF 4.3 §5.3 (if S = 1, `POS = 1` and the padding base is the base after the
event, per the spec);
- `REF` = the FASTA base at POS, or `N` only when no FASTA was supplied (a warning is logged);
- `INFO/END = E`;
- `INFO/SVLEN`:
- `signed` mode (default): `−(E − S + 1)` for DEL and `+(E − S + 1)` for DUP and LOH;
- `--svlen-mode positive`: always positive.
- `ID = <Prefix>:<CHROM>:<S>-<E>`, with CHROM exactly as the VCF CHROM field.
- Prefixes: `DEL`, `DUP`, `AOH`, `GAINLOH`, `CNLOH`, `MosaicDEL`, `MosaicDUP`.
- Tests assert that `S = POS + 1` and that the ID coordinates equal POS + 1 and END for every record (EDGE-025).
- Every TSV and JSON uses `start = S` (1-based), and never a 0-based start.
(src: E-36, B-16)
- **CYTO-030** `§10 · v2` **Exact header.**
```
##fileformat=VCFv4.3
##source=SimurgArray <semver> cn
##reference=file://<fasta or "none">
##simurg_genome_build=GRCh38
##simurg_manifest=<name>;sha256=<hex>
##simurg_cyto_model=<file>;sha256=<hex>;schema=<n>
##simurg_thresholds=min_del_probes=10;min_del_size_bp=25000;min_dup_probes=10;min_dup_size_bp=50000;min_loh_probes=500;min_loh_size_bp=3000000;min_gainloh_probes=300;min_gainloh_size_bp=500000;min_mdel_probes=10;min_mdel_size_bp=25000;min_mdup_probes=10;min_mdup_size_bp=50000;min_mloh_probes=500;min_mloh_size_bp=3000000;smoothing=5;max_mosaic_fraction=1.0;min_mosaic_fraction=0.10;filter_qual=10;size_basis=effective;cn_cap=4
##simurg_sex_used=<M|F|U>;source=<summary|sheet|resolved|assumed|unresolved>
##OverallPloidy=1.953
##GCCorrect=true
##contig=<ID=1,length=248956422> (all contigs of the called set, .fai order)
##ALT=<ID=DEL,Description="Copy number loss">
##ALT=<ID=DUP,Description="Copy number gain">
##ALT=<ID=LOH,Description="Absence/loss of heterozygosity (AOH/LOH/ROH)">
##FILTER=<ID=PASS,Description="All filters passed">
##FILTER=<ID=Q10,Description="Quality below 10">
##FILTER=<ID=SampleQuality,Description="Sample flagged as potentially low quality: LRR SD above 0.2">
##FILTER=<ID=LowFraction,Description="Mosaic event with estimated mosaic fraction below 0.10">
##INFO=<ID=END,Number=1,Type=Integer,Description="Last base of the event (1-based, inclusive)">
##INFO=<ID=SVTYPE,Number=1,Type=String,Description="DEL, DUP or LOH">
##INFO=<ID=SVLEN,Number=1,Type=Integer,Description="Event length in bp; negative for deletions">
##INFO=<ID=EFFLEN,Number=1,Type=Integer,Description="Effective length in bp (gaps above 150x median probe spacing replaced by the median spacing)">
##INFO=<ID=PROBE,Number=1,Type=Integer,Description="Number of probes in the event">
##INFO=<ID=MOSAIC,Number=0,Type=Flag,Description="Mosaic event">
##INFO=<ID=HIGHFRACTION,Number=0,Type=Flag,Description="Mosaic event promoted to constitutional because its mosaic fraction reached the maximum">
##INFO=<ID=LOHTYPE,Number=1,Type=String,Description="AOH (constitutional, copy-neutral or gain), CNLOH (mosaic copy-neutral), GAINLOH (mosaic copy-gain)">
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype (see SimurgArray docs, CYTO-031)">
##FORMAT=<ID=CN,Number=1,Type=Integer,Description="Copy number; the maximum value 4 means 4 or more">
##FORMAT=<ID=NR,Number=1,Type=Float,Description="Aggregated normalized intensity (median 2^LRR; about CN/2)">
##FORMAT=<ID=LRD,Number=1,Type=Float,Description="Standard deviation of LRR within the event">
##FORMAT=<ID=MF,Number=1,Type=Float,Description="Estimated mosaic fraction; missing if not mosaic or not estimable">
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT <sample name>
```
- The numbers in the FILTER and CN descriptions follow the effective options.
- FORMAT is always `GT:CN:NR:LRD:MF`; MF is `.` when not applicable.
- NR has 4 decimals, LRD 4 and MF 3.
(src: B-16, A-G47, A-G50, D-5)
- **CYTO-031** `§10 · v2` **GT semantics [DIFF].** With `--gt-mode vcf` (the default):
| Event | Diploid region | Haploid region |
|---|---|---|
| DEL CN1 | `0/1` | – |
| DEL CN0 | `1/1` | `1` |
| DUP CN3 | `0/1` | – |
| DUP CN ≥ 4 | `./1` | `1` (CN ≥ 2) |
| LOH (any subtype) | `1/1` | – |
| mosaic events | `0/1` | `1` |
`--gt-mode compat` writes `1/1` for every event, as the reference does; the header records the mode.
(src: E-36, D-5)
- **CYTO-032** `§10 · v2` **Sidecar and segments.**
- `cn call` also writes `cn_call_manifest.json` in the output folder, which `cn annotate` requires. It holds:
- the list of samples;
- effective thresholds;
- model and summary hashes;
- per-sample QC: `logr_dev`, `sex_used`, flags.
- It also writes `<sample>.cn_segments.parquet`: every segment, including those below threshold, with posteriors.
`annotate` can then re-filter without re-calling.
- Annotate errors:
- missing sidecar → `E_CN_SIDECAR_MISSING`;
- VCFs not bgzipped and indexed → `E_CN_VCF_UNINDEXED`.
(src: A-G44, A-G54)
### 6.6 Per-sample JSON report (`cn annotate`)
- **CYTO-033** `§10.4 · v2` **`<sample>.cyto.json`** (schema `simurg.cyto/1`; typed JSON; `null` for missing values;
percentages 0–100 as numbers; sizes in bp).
```json
{
"schema": "simurg.cyto/1",
"software_version": "0.9.0",
"annotation_bundle": {"file": "…zip", "sha256": "…", "sources": [
{"name": "RefSeq", "version": "GCF_000001405.40-RS_2023_10", "release_date": "2023-10-07"},
{"name": "Ensembl", "version": "112", "release_date": "2024-05-14"},
{"name": "UCSC cytoBand", "version": "hg38", "release_date": null}]},
"reference_genome": "file://…", "genome_build": "GRCh38",
"annotation_type": "constitutional", "iscn_version": "ISCN 2020",
"sample_id": "204753010023_R01C01", "sample_name": "NA12878",
"gc_correct": true, "gcwf": 0.012,
"parameters": {"min_del_probes": 10, "min_del_size_bp": 25000, "…": "…", "min_qual": 20, "size_basis": "effective"},
"overall_ploidy": 1.953, "call_rate": 0.9941, "logr_dev": 0.159, "logr_dev_robust": 0.046,
"baf_dev": {"aa": 0.0161, "ab": 0.0441, "bb": 0.0168},
"num_loh_over_1mb": 2, "num_loh_over_8mb": 1, "total_size_loh_over_1mb": 16992771,
"percent_loh": 0.55, "copy_number_median": 2.0,
"sex_estimate": "M", "sex_source": "summary",
"qc_flags": [],
"iscn_traditional": "del(1)(p36.33p36.33),dup(9)(p24.3q21.11)",
"iscn_microarray": "arr[GRCh38] 1p36.33(1567924_1632825)x1,9p24.3q21.11(14326_69444970)x3,16p11.2q11.2(31382412_46974423)x2 hmz",
"chromosomes": [ { "id": "1", "size": 248956422, "effective_size": 230000000,
"percent_het": 11.57, "has_mosaicism": false,
"lrr_median": 0.001, "lrr_mean": 0.002, "lrr_dev": 0.15,
"num_loh_over_1mb": 0, "num_loh_over_8mb": 0, "total_size_loh_over_1mb": 0, "percent_loh": 0.0,
"copy_number_median": 2.0, "copy_number_mean": 1.9924,
"min_lrr": -1.2, "max_lrr": 0.9, "median_mosaic_fraction": null,
"number_del": 1, "number_dup": 0, "number_loh": 0, "number_gainloh": 0, "number_mosaic": 0 } ],
"events": [ { "id": "DEL:1:1567924-1632825", "chrom": "1", "start": 1567924, "end": 1632825,
"call_type": "DEL", "loh_type": null, "mosaic": false, "mosaic_fraction": null, "high_fraction": false,
"copy_number": 1, "quality": 60.0, "filters": ["PASS"],
"size": 64902, "effective_size": 64902, "probe_count": 99,
"percent_het": 0.0, "lrr_median": -0.45, "lrr_dev": 0.21, "baf_dev": 0.03,
"start_cytoband": "p36.33", "end_cytoband": "p36.33",
"iscn_traditional": "del(1)(p36.33p36.33)",
"iscn_microarray": "arr[GRCh38] 1p36.33(1567924_1632825)x1",
"flags": [], "gene_count": 3, "genes": ["CDK11A", "SLC35E2B", "ENSG00000…"],
"dgv_overlap": [] } ]
}
```
Field rules:
- `logr_dev` = population SD of autosomal LRR (plan §7.6).
- `logr_dev_robust` = 1.4826·MAD of autosomal LRR. It takes the place of the reference's undocumented
`medianLogRDev` (OQ-022).
- `baf_dev.{aa,ab,bb}` = SD of BAF over autosomal probes called AA, AB or BB outside events.
- LOH counts and sizes, at both sample and chromosome level, **include** gain-LOH [DIFF]. `number_gainloh` is
reported separately.
- `percent_loh` = total LOH length / effective genome length × 100.
- `size = end − start + 1`.
- Sex is written as `M`/`F`/`U`, the same codes as in the summary.
- `--compat-json` MAY write reference-style key names (camelCase) and string sentinels, for comparison tests only.
(src: B-18, D-7, E-37, A-G53)
- **CYTO-034** `§10.4 · v2` **ISCN 2020 generator.**
- **Microarray form:** `arr[GRCh37|GRCh38] ` followed by comma-joined events, ordered by chromosome (1–22, X, Y)
and then position. Each event is written as
`<chr><band_start>[<band_end>](<S>_<E>)x<CN>`, where `band_end` is omitted when it equals `band_start`.
- LOH: `x2 hmz`; gain-LOH: `x3 hmz`.
- Mosaic events: `x<CN_base>~<CN_event>` (e.g. `x1~2`) [HYP, OQ-034].
- Coordinates are 1-based. Thousands separators are never used.
- **Normal results:**
- `arr[GRCh38](1-22,X)x2` for a female;
- `arr[GRCh38](1-22)x2,(XY)x1` for a male;
- `arr[GRCh38](1-22)x2` when sex is unresolved.
- **Traditional form:** `del(<chr>)(<bands>)` and `dup(<chr>)(<bands>)` for CNVs only. LOH has no karyotype
notation here and is omitted.
- **Coverage:** ISCN is produced for **every** reported event, including the first, LOH and mosaic events
(`--iscn-min-size`, default 0). The reference misses these cases [DIFF].
- The per-sample string contains only events that pass FILTER and min-qual.
- Golden tests compare against hand-checked ISCN strings.
(src: B-19, D-8, E-37)
- **CYTO-035** `§10.4 · v2` **Genes and bands.**
- Genes = every transcript whose span overlaps the event by ≥ 1 bp. Each is named by its HGNC symbol (via RefSeq),
or by its Ensembl gene ID when it has no symbol.
- The list is deduplicated and sorted by transcript start, then name. `gene_count` is the length of the list.
- Cytobands come from UCSC `cytoBand` for the build.
- DGV overlap (plan §10.4) is optional, and uses reciprocal overlap ≥ 50 %.
(src: A-G53, B-18)
- **CYTO-036** `§10.4 · v2` **Annotation bundle build.** `scripts/build_annotation_bundle.sh <build>` downloads the
following public files and packages them with a manifest of versions and hashes (IN-054):
- RefSeq GFF (pinned `GCF_000001405.40` release);
- Ensembl GTF (pinned release, e.g. 112);
- HGNC;
- UCSC cytoBand and the centromere/gap tables;
- DGV (optional).
The bundle is independent of any array product. (src: A-G53, plan §10.4)
- **CYTO-037** `§10 · v2` **Batch outputs.**
- `cn_sample_summary.tsv`, one row per sample. Columns: `sample_id`, `sample_name`, `sample_folder`, `status`,
`logr_dev`, `overall_ploidy`, `gc_correct`, `sex_used`, `sex_source`, `n_del`, `n_dup`, `n_loh`, `n_gainloh`,
`n_mosaic`, `n_filtered`, `qc_flags`.
- The LRR/BAF bedGraph tracks of OUT-050 when `export bedgraph` or `pipeline` requests them.
(src: B-21, C-G24)
- **CYTO-038** `§10 · v2` **Model building.**
- `simurg cn build-model --bpm --fasta` computes, for every non-control probe, its GC fraction in windows of 1 kb,
100 kb and 1 Mb centred on the probe, plus probe-sequence GC when the CSV `SourceSeq` is given.
- It writes IN-051.
- It needs no cluster file. The model is rebuilt only when the manifest or the model schema changes.
(src: A-G52)
- **CYTO-039** `§10, §19 · v2` **Batch scale.** Samples are processed independently and streamed, so memory is
O(1) in batch size. Tests run 1 152 samples; the reference failed at about 200. There is no hidden batch
normalisation. (src: D-9, E-35)
- **CYTO-040** `§10 · v2` **Degenerate samples.** None of these may crash or overflow:
- zero events;
- an all-NaN chromosome;
- a single-probe chromosome;
- a chromosome with no heterozygous probes;
- LRR SD = 0.
Each yields valid, possibly empty, VCF and JSON output (EDGE-024). (src: E-35)
- **CYTO-041** `§10.3 · v2` **MF refinements [OURS].** Two refinements of CYTO-018, from real-data validation
(docs/validation/V2_cn.md, K-027):
- *noise-corrected split:* per-probe expected noise σ_i = the reference's heterozygous BAF spread of the probe ×
the sample's scale (median |BAF − 0.5| / spread over autosomal heterozygous calls, ÷ 0.6745; the sample's
heterozygous BAF SD without a reference spread). Over the event's heterozygous calls, m = median(|BAF − 0.5| /
σ_i); the split in σ units δ solves Φ(m − δ) − Φ(−m − δ) = ½ (δ = 0 when m ≤ 0.6745) and d = median(σ_i)·δ.
Without this, noise alone gave MF ≈ 0.1–0.3 and genomic waves were called as mosaic losses.
- *complete loss:* a mosaic loss whose called genotypes hold fewer than a quarter of the heterozygotes the PFB
predicts (Σ 2p(1−p) ≥ 3) has MF = 1. With the default `--max-mosaic-fraction 1.0` it is therefore promoted
(CYTO-020) to a constitutional DEL with `HIGHFRACTION`. This catches constitutional deletions whose LRR is
compressed towards the mosaic states.
- **CYTO-042** `§10.2 · v2` **Per-probe LRR noise [OURS].** A model built with `--reference-gtcs` stores each
probe's robust LRR SD across the reference (1.4826·MAD after removing each sample's median and the probe's
reference median). `cn call` scales the sample's LRR emission SD per probe by that spread relative to the autosomal
median spread, clipped to [0.5, 3]. Without the column, or with `--no-probe-lrr-sd`, every probe uses the sample SD.
---
## 7. Targeted PGx copy number (v2)
- **PCN-001** `§11 · v2` **Region catalogue.**
- Seven genes, ten target regions. They are defined in a versioned BED shipped with the PGx DB
(`pgx_cn_regions.GRCh38.v<N>.bed`: `chrom, start0, end, region, gene`). The BED is derived from gene annotation
plus probe coverage and validated against the public region descriptions.
| Gene | Regions | Notes |
|---|---|---|
| CYP2D6 | `CYP2D6.p5` (5′ flank/promoter), `CYP2D6.intron.2`, `CYP2D6.exon.9` | Sub-regions separate deletions (*5), duplications (xN), CYP2D7-derived exon-9 conversions (*36-like) and 2D7-2D6 hybrids (*13-like) |
| CYP2A6 | `CYP2A6.exon.1`, `CYP2A6.intron.7` | *4-type deletion and conversion |
| GSTM1 | whole gene | *0 null deletion |
| GSTT1 | whole gene | lies on the GRCh38 **alt contig** `chr22_KI270879v1_alt` (IN-062) |
| UGT2B17 | whole gene | common deletion |
| SULT1A1 | whole gene | copy gain |
| CYP2E1 | whole gene | copy gain |
- The catalogue is extensible: a new BED version plus a retrained model is enough; no code change is needed.
(src: B-23, D-10, C-G25)
- **PCN-002** `§11 · v2` **Genotype every region [EMU].** Each sample gets exactly one record per region in every
output, whether its CN is 2 or not. A region that cannot be assessed (for example all its probes are masked, or it
has too few finite values) gets CN `.`, QUAL 0 and FILTER `NoCall` [OURS]. (src: B-23)
- **PCN-003** `§11 · v2` **Inputs.**
- GTCs that satisfy IN-031.
- The model (IN-050).
- A GRCh38 FASTA with the alt contigs: used for REF padding bases and contig headers, and checked against the
model's FASTA hash.
- The sample summary, which supplies LRR SD and the assay control.
(src: A-G37)
- **PCN-004** `§11 · v2` **Batch rules.** Batch statistics are only valid when enough samples are good.
- Passing means `logr_dev < 0.2` (strict), set by `--lrr-sd-max`.
- Fewer than `--min-batch` samples in total (24), or fewer than `--min-passing` passing samples (22), →
`E_PGX_BATCH_TOO_SMALL`: "<n> sample(s) passed QC. Requires at least <m> samples to proceed." Exit 2.
- Fewer than `--recommended-passing` passing samples (96) → `W_PGX_BATCH_BELOW_RECOMMENDED`.
- Only passing samples build the batch reference.
- Non-passing samples are still genotyped, with FILTER `SampleQuality`.
- Boundary semantics are listed in OQ-004.
(src: A-G33, D-11, C-G25, E-39)
- **PCN-005** `§11 · v2` **Model training.**
- Needs ≥ 96 samples, of which ≥ 90 pass (`logr_dev ≤ 0.2`); otherwise `E_PGX_TRAIN_TOO_SMALL`. The documentation
recommends ≥ 150 samples.
- Above `--max-samples` (3 000) a seeded random subset is used, because returns diminish.
- Samples are excluded automatically if any of these hold:
- `logr_dev > 0.2`;
- `call_rate < 0.99`;
- `assay_control_norm_r < 1.0`.
- Exclusions go to `train_exclusions.tsv` (`sample_id, reason`).
- The model assumes the training cohort's CN distribution looks like a general population's. The documentation
requires the user to validate a retrained model against truth data before relying on it.
- The model must be retrained whenever the cluster file changes (IN-050).
(src: A-G34, E-39)
- **PCN-006** `§11 · v2` **Batch-composition warnings [OURS].** Warnings are raised in two cases:
- the sheet contains a reagent-lot, plate-batch or processing-batch column with more than one value (column names
configurable; defaults `Reagent_Lot`, `Batch`, `AMP_Plate`);
- IDAT scan dates span more than 14 days.
Warnings only, never errors. (src: A-G38)
- **PCN-007** `§11 · v2` **Model compatibility.** See IN-050: a manifest mismatch is an error; a cluster-file
mismatch is an error by default, or a warning with `--cluster-mismatch warn`. The recorded platform (`LCG`/`EX`)
must match the manifest's `Assay Format`, otherwise a warning is issued. (src: A-G37, B-20, D-11, E-39)
- **PCN-008** `§11 · v2` **Mask.** Masked probes (IN-052) are excluded from training and calling. A region left with
fewer than 10 unmasked probes is flagged in the model header. (src: A-G35)
- **PCN-009** `§11 · v2` **Calling algorithm [HYP, plan §11; U10].**
1. Per probe, compute normalized R (plan §7.3–7.4) and GC-correct it using the probe GC stored in the model.
2. Divide by the probe's batch median over passing samples, and take a robust per-sample scale (the median over
non-target reference probes stored in the model).
3. For each region, `NR` = the median over its unmasked probes of the normalized ratio. NR ≈ CN/2.
4. Compute the CN posterior from the region's trained 1-D Gaussian mixture over NR, with components CN 0…5 and
frequency priors from training.
5. `CN = argmax` of the posterior, capped at 5, meaning "5 or more".
6. `QUAL = min(60, round(−10·log10(1 − P_max)))`.
The procedure is robust to outlier samples: batch medians rather than means, and outlier samples (|z| > 5 on
genome-wide NR) are excluded from the batch reference. NR is computed per region and never shared across regions
[DIFF, OQ-027]. (src: A-G33, plan §11)
- **PCN-010** `§11 · v2` **CN-consistent BAF evidence.**
- For SNP probes inside a region, the expected BAF for k alternate copies out of n is k/n.
- The per-region BAF likelihood of each CN state is computed and stored in `pgx_calls` for the star-allele caller
(PGX-012).
- It is not used to change CN by default; `--bafs-in-cn` is experimental.
(src: B-30, plan §11)
- **PCN-011** `§11, §13 · v2` **PGx CN VCF.** One file per sample, `<sample>.pgx_cn.vcf.gz` with its `.tbi`. The
header:
```
##fileformat=VCFv4.3
##source=SimurgArray <semver> pgx-cn
##reference=file://<fasta>
##simurg_genome_build=GRCh38
##simurg_manifest=<name>;sha256=<hex>
##simurg_pgx_cn_model=<file>;sha256=<hex>;cluster_file=<name>;regions=<bed version>
##simurg_batch=n_samples=<N>;n_passing=<P>;lrr_sd_max=0.2
##OverallPloidy=<%.3f> (length-weighted mean CN over target regions [INF, OQ-028])
##GCCorrect=true
##contig=<ID=chr1,length=248956422> … (contigs that carry target regions, including chr22_KI270879v1_alt)
##ALT=<ID=CNV,Description="Copy number variable region (targeted genotype)">
##FILTER=<ID=PASS,Description="All filters passed">
##FILTER=<ID=Q7,Description="Quality below 7">
##FILTER=<ID=SampleQuality,Description="Sample LRR SD above 0.2">
##FILTER=<ID=NoCall,Description="Region could not be assessed">
##INFO=<ID=END,Number=1,Type=Integer,Description="Last base of the region (1-based, inclusive)">
##INFO=<ID=SVTYPE,Number=1,Type=String,Description="CNV">
##INFO=<ID=CNVLEN,Number=1,Type=Integer,Description="Region length in bp">
##INFO=<ID=PROBE,Number=1,Type=Integer,Description="Number of probes used for the region">
##INFO=<ID=REGION,Number=1,Type=String,Description="Target region name">
##INFO=<ID=GENE,Number=1,Type=String,Description="Gene symbol">
##FORMAT=<ID=CN,Number=1,Type=Integer,Description="Copy number; 5 means 5 or more">
##FORMAT=<ID=NR,Number=1,Type=Float,Description="Aggregated normalized intensity (about CN/2)">
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT <sample name>
```
Record rules:
- POS is the padding base, `S − 1`; REF is the FASTA base; END is `E`; `CNVLEN = E − S + 1`.
- `ID = CNV:<region>:<CHROM>:<S>-<E>`. The coordinates in the ID are always the same as POS + 1 and END
[DIFF: the reference's IDs sometimes disagree with POS/END, OQ-026].
- QUAL is an integer in 0–60. FILTER is `Q7` if QUAL < 7; `SampleQuality` and `NoCall` are added as they apply.
- FORMAT is `CN:NR`, with no GT [EMU], and NR has 6 decimals.
- Records are sorted by contig order, then POS.
(src: B-23, D-12, A-G25)
- **PCN-012** `§11, §13 · v2` **Batch summary.** `pgx_cn_batch_summary.tsv` has columns
`region, gene, n_gain, n_loss, n_neutral, n_nocall, n_filtered`:
- gain means CN > 2, loss CN < 2, neutral CN = 2;
- `n_filtered` counts samples within the other columns that carry any non-PASS FILTER;
- one row per region, sorted alphabetically, then a final `ALL` totals row;
- `n_gain + n_loss + n_neutral + n_nocall` equals N for every row.
(src: B-24, D-12)
- **PCN-013** `§11, §13 · v2` **Per-sample summary.** `pgx_cn_sample_summary.tsv` has columns:
- `sample_id`, `sample_name`, `sample_folder`, `logr_dev`;
- `passed_batch_qc`, `used_in_reference`;
- one `cn_<region>` column per region;
- `flags`.
It is the "CN summary" file. (src: B-21, C §2.2)
- **PCN-014** `§11 · v2` **Placement.** The docs recommend writing CN VCFs into the SNV VCF folder. `pgx star-allele
call --vcfs` also accepts several folders and pairs files by `sample_id`, which is read from the VCF headers
(`##SAMPLE`), not from file names. (src: A-G39)
- **PCN-015** `§11 · v2` **Genome build.** PGx CN and star-allele calling support GRCh38 only in v2.0. A GRCh37 input
→ `E_BUILD_MISMATCH`, with a note. GRCh37 support is a future item: region liftover, and GSTT1 lies on the primary
chr22 in GRCh37. (src: A-G62, D-10)
- **PCN-016** `§11, §19 · v2` **Memory model.**
- Calling streams per-sample normalized region data. The batch keeps only per-probe batch medians
(O(probes in regions)).
- Training memory is O(n_samples × n_region_probes).
- Documented target: 500 GDA-ePGx samples within 8 GB, and 3 000 within 32 GB.
(src: A-G43)
- **PCN-017** `§11, §16 · v2` **Validation.** On 1000 Genomes array data, with WGS-derived CN truth:
- per-region CN concordance ≥ 95 %;
- CYP2D6 region call rate ≥ 97 %;
- the outlier-robustness test of EDGE-026 (10 % injected outlier samples) must leave the other samples' calls
unchanged in ≥ 99 % of cases.
(src: C-G25, C-G27)
---
## 8. Star alleles and phenotypes (v2)
### 8.1 Database
- **PGX-001** `§11 · v2` **Allele-definition sources, with pinned versions.** Every version is recorded in the DB
`meta` table and in every output.
| Source | Minimum version (parity target) | Use |
|---|---|---|
| PharmVar | 6.1 | star-allele definitions (CYP*, NUDT15, SLCO1B1, …) |
| PharmGKB allele definition tables | snapshot 2024-05-16 or later | non-PharmVar genes, variant-based genes |
| CPIC | 1.38.0 or later | allele function, activity scores, diplotype → phenotype tables |
| DPWG (via PharmGKB) | June 2023 or later | DPWG phenotype tables |
| UGT allele nomenclature (legacy) | 2010-12-21 | UGT1A1/UGT1A4/UGT2B15 alleles |
| Human CYP allele nomenclature (legacy archive) | July 2024 | lettered alleles (CYP1A2 `*1A…`, CYP2E1) |
| Population frequencies | the CPIC/PharmGKB frequency tables of the same release | priors (PGX-014) |
(src: D-15, A-G40, plan §11)
- **PGX-002** `§11 · v2` **DB build.** `simurg pgx db build` produces a SQLite file (IN-053) with these tables:
- `genes(gene, result_basis ∈ {star_allele, haplotype_named, variant_based, tagging_variant})`;
- `alleles(gene, allele, kind ∈ {star_allele, reference, rsid_variant_allele, hgvs_variant, named_haplotype}, function, activity)`;
- `allele_variants(gene, allele, build, chrom, pos, ref, alt, hgvs, rsid)`;
- `probe_map(manifest, probe_name, chrom, pos, ref, alt)` (many probes → one variant);
- `masks(manifest, probe_name, reason)`;
- `limitations(manifest, gene, allele, status, reported_as)`;
- `frequencies(gene, allele, population, freq)`;
- `phenotypes(guideline, gene, diplotype_or_activity, phenotype)`;
- `meta(key, value)`.
The build is deterministic: identical inputs give an identical file hash. (src: A-G40, D-14)
- **PGX-003** `§11 · v2` **Probe-to-variant mapping.**
- SNV VCF records are linked to DB variants through their `ID` probe names (split on `,` and `;`) via `probe_map`.
- The link is then **verified** on CHROM, POS, REF and the ALT set [DIFF].
- A record whose ID matches but whose coordinates disagree → a warning, and the variant counts as not covered.
- Records with no probe ID fall back to position and allele matching.
(src: A 3.8, D-14)
- **PGX-004** `§11 · v2` **Multi-allelic extension.** When a definition names one alternate allele at a site that the
array's merged record shows as multi-allelic, the definition is extended to the site's full allele set. Other alleles
at the site count as "not this allele", not as missing. This is derived automatically from the manifest; the
reference hand-listed 52 extensions in CACNA1S, CFTR, COMT, DPYD and RYR1, and those serve as a test oracle.
(src: D-14, E-40)
- **PGX-005** `§11 · v2` **Build-specific definitions.** Definitions are stored per build. Where the build's reference
base differs from a definition's reference base, the definition is re-expressed. G6PD is the canonical test:
- GRCh37 "Mediterranean Haplotype" = rs2230037 G>A + rs5030868 G>A.
- GRCh38 = rs5030868 G>A with rs2230037 reference A.
- The multi-name allele "Mediterranean, Dallas, Panama, Sassari, Cagliari, Birmingham" is handled the same way.
- rs2230037 is ignored in all other G6PD alleles.
(src: D-14)
- **PGX-006** `§11 · v2` **`*0` whole-gene deletion.** For genes with a whole-gene CN region and no named deletion
allele (e.g. GSTM1, GSTT1, UGT2B17), CN 0 or 1 adds `*0` haplotypes. CYP2D6 uses its named allele `*5`. (src: D-14)
- **PGX-007** `§11 · v2` **Tagging-variant genes.**
| Gene | Allele | Tag |
|---|---|---|
| HLA-A | `*31:01` | rs1061235 A>T |
| HLA-B | `*15:02` | rs144012689 T>A |
| HLA-B | `*57:01` | rs2395029 T>G |
| HLA-B | `*58:01` | rs9263726 G>A |
| IFNL3/4 | – | rs12979860 |
These are reported as `call_type = tagging_variant`, with the fixed caveat "tag reliability varies by population".
(src: D-14)
- **PGX-008** `§11 · v2` **Coverage parity target (gene list and counts).** The archived coverage table, 2 405
allele/variant entries across 58 genes, gives the minimum content our DB must represent. Entry kinds:
- 1 420 rsID variant alleles;
- 674 star alleles;
- 186 named haplotypes;
- 85 HGVS-only variants;
- 40 reference entries.
| Result basis | Genes (entries) |
|---|---|
| star allele (17) | CYP2D6 184 (incl. 14 duplication and 7 hybrid/tandem forms; sub-alleles such as `*4M`), CYP2C9 85, CYP2A6 51, CYP2B6 46, TPMT 46, CYP3A4 45, SLCO1B1 42, CYP1A2 39 (legacy lettered), CYP2C19 34, NUDT15 20, CYP2C8 18, CYP4F2 16, CYP2E1 11, UGT1A1 9 (incl. `*80+*28`, `*80+*37`), UGT1A4 8, UGT2B15 7, CYP3A5 6 |
| named haplotype (2) | G6PD 187, APOE 3 (E2/E3/E4) |
| tagging variant (3) | HLA-B 4, HLA-A 2, IFNL3/4 2 |
| variant based (36) | DPYD 475, RYR1 355, OPRM1 175, CFTR 59, DRD2 56, COMT 46, OPRD1 46, CACNA1C 37, HTR2A 32, OPRK1 24, MT-RNR1 24, TNF 23, BDNF 23, IL6 21, HTR2C 17, GRIN2B 15, MTHFR 13, ITGB3 12, FKBP5 12, ADH1B 10, ANKK1 10, HMGCR 9, ATM 7, KIF6 6, F2 5, CACNA1S 4, LPA 3, F13A1 3, GRIK1 3, GRIK4 3, YEATS4 2, ABCG2 2, F5 2, ALDH2 2, ANK3 2, VKORC1 2 |
- The counts are parity checks, not data to copy. Our DB is built only from public sources (PGX-001).
- The mitochondrial gene MT-RNR1 needs `NC_012920.1:m.` HGVS and haploid genotype handling.
- Suffixed identifiers for several definitions at one rsID (e.g. `rs114096998.2.G>C`) must be supported.
(src: D-17, C-G25)
- **PGX-009** `§11 · v2` **Array limitations are derived, not hard-coded.**
- For every manifest, `simurg pgx coverage` computes which alleles are indistinguishable from each other, or not
callable, given the probes present and not masked.
- Results go to the `limitations` table and are reported as array-equivalents (PGX-017).
- The reference's known limitations are regression oracles:
- APOE E2/E4 not callable on GSAv4/GCRA-ePGx;
- CYP2A6 *5 unsupported on GDA-ePGx;
- CYP4F2 *2 reported as *1, and *4 as *3;
- UGT1A1 *28/*37 TA-repeat not covered;
- UGT2B15 *4/*5 unsupported on GSAv4/GCRA-ePGx;
- CYP2D6 *40/*58 reported as *40;
- NUDT15 *6/*9 reported as *6;
- HLA-B*15:02 absent on GDA-ePGx.
(src: D-14)
- **PGX-010** `§11 · v2` **Masked variants.**
- Per manifest, probes known to perform poorly are masked: their calls are treated as no-calls.
- Alleles defined only by masked variants are never called, and are listed as "not callable (masked)".
- Our masks come from concordance against WGS truth (plan §16.5): a probe with < 95 % genotype concordance over
≥ 100 samples is masked.
- The reference's 24 masked-variant rows (8 genes: CYP1A2, CYP2C19, CYP2C9, CYP2D6, CYP4F2, UGT1A1, OPRM1, COMT;
4 on GDA-ePGx, 10 each on GSAv4/GCRA-ePGx) are a comparison oracle only.
(src: D-14)
### 8.2 Calling
- **PGX-011** `§11 · v2` **Input guard.** `pgx star-allele call` requires all of the following:
1. SNV VCFs made with `unsquash_duplicates=false` and no exclude/include list, as read from `##simurg_options`.
Otherwise `E_PGX_VCF_UNSUITABLE`.
2. Files bgzipped and indexed. Otherwise `E_PGX_VCF_UNINDEXED`.
3. A `pgx_cn` VCF for every sample, unless `--no-cn` is given. Otherwise `E_PGX_CN_MISSING`.
4. The same build and manifest across all inputs.
Notes:
- Symlinked inputs are accepted.
- VCFs from other converters (no `##simurg_options`) are refused unless `--allow-foreign-vcf` is given, in which
case a warning is issued and squashing is assumed.
- Any valid tabix or CSI index is accepted, including one rebuilt with htslib.
(src: A-G39, D-19, E-40)
- **PGX-012** `§11 · v2` **Variant evidence model.** Each DB variant gets one of these states:
- `called` (GT, GS, BAF);
- `no_call` (`./.`);
- `masked`;
- `not_covered` (no probe on the array).
In a region whose CN is n ≠ 2, the variant copy number `k ∈ {0..n}` is the maximum-likelihood value under the
observed BAF ~ N(k/n, σ²), with σ estimated per batch from AB clusters. It is reported as `variant_copy_number`.
GS enters as a per-call error rate: ε = 1e-3 for GS ≥ 0.15 and 0.05 below [OURS, calibrated: after probe masking
(PGX-010), 0 of 9 740 GSA v4 PGx calls at DB variants disagreed with 1000G 30x in any GS bin, so errors are
probe-specific and belong in the mask; the earlier ε(GS) = 0.5·(1 − GS)² made correct low-GS calls such as
CYP2C9 rs1799853 at GS 0.16 lose to the prior; K-029]. (src: B-30, E-41)
- **PGX-013** `§11 · v2` **Candidate enumeration [DIFF: solver-free].**
- For each gene, SimurgArray enumerates diplotypes from the gene's haplotypes: star alleles, `*0`, duplications
`xN`, and the hybrid and tandem structures in the DB, such as `*36+*10`, `*13+*4`, `*68+*4`.
- The enumeration is constrained by the region CN signature. CYP2D6 rules:
- total CN(p5) = number of gene copies;
- CN(exon.9) < CN(p5) implies CYP2D7-exon-9 conversion copies, i.e. `*36`-type (or `*83`, `*4.013`-like)
units;
- CN(p5) < CN(intron.2) or CN(exon.9) implies 2D7-2D6 `*13`-type hybrids.
The full table is in `docs/pgx_cyp2d6_cn_rules.md`, derived from PharmVar structural-variant documentation.
- Branch-and-bound pruning keeps the top 200 candidates.
- The enumeration is deterministic: fixed order, and no LP/MIP solver. The reference's intermittent solver error
and its OS-dependent scores cannot occur here.
(src: E-43, B-26)
- **PGX-014** `§11 · v2` **Scoring.** For each candidate solution s:
- `LL(s) = Σ_v log P(obs_v | s) + Σ_r log P(NR_r | CN_r(s))`.
- Variant term: a genotype likelihood with error ε(GS) (PGX-012), plus a BAF term when the variant CN ≠ 2.
- Region term: from the CN model.
- `no_call`, `masked` and `not_covered` variants contribute 0.
- `raw_score(s) = exp(LL(s)) / Σ_s' exp(LL(s'))`: no prior.
- `score(s) = exp(LL(s) + log π(s)) / Σ_s' exp(LL(s') + log π(s'))`, where π(s) is the product of the two
haplotype frequencies in `--population` (default `global`, with a floor frequency of 1e-5 for unknown ones).
- Both scores lie in [0, 1] and are rounded to 4 decimals in outputs.
- Ranking: `score` descending, then `raw_score`, then the solution string in natural order.
(src: B-26, D-13, E-42, plan §11)
- **PGX-015** `§11 · v2` **Feasible alternatives.**
- Every solution with `score ≥ --feasible-ratio × top score` (default 0.5) is *feasible*.
- The gene's `genotype` field lists all feasible solutions `;`-joined, in rank order.
- `candidate_solutions` lists up to `--max-candidates` (default 10) solutions, feasible or not, with their rank.
(src: B-27, E-42)
- **PGX-016** `§11 · v2` **Low-GS review.** A supporting variant with GS < 0.15 (`--low-gs`) gets `low_gs: true`. The
gene gets the flag `review_alternatives`, and the docs advise checking those clusters for shift. (src: B-27, D-13)
- **PGX-017** `§11 · v2` **Array-equivalent alleles.**
- Two alleles are *array-equivalent in a sample* when no covered, unmasked, called variant separates them.
- For each allele of a solution, the reported allele is the equivalence-class member with the highest population
frequency. Ties are broken by lowest star number, then natural order.
- The other members are listed in `array_equivalent_alleles`.
- Sub-alleles such as `*4.013` collapse onto their core allele (`*4`) and are listed as equivalents [OURS].
(src: B-27, D-13)
- **PGX-018** `§11 · v2` **Background alleles and variant change.**
- For structural or hybrid solutions, `background_allele` names the allele the SV sits on (e.g. `*10` under `*36`).
- `variant_change` describes the observed change: an SV note, a reference call, or "unresolved due to no-call at
<hgvs>".
- Exon-9 conversion alleles other than `*36` (e.g. `*83`) are reported as themselves, not folded into `*36`
(EDGE-027).
(src: B-26, E-40)
- **PGX-019** `§11 · v2` **No-call handling.**
- Two lists are kept apart:
- (a) `variant_no_calls`: variants on the array but no-called or masked in this sample;
- (b) `variants_not_covered`: variants absent from the array. This list is the same for every sample of a
manifest.
- Each carries `alleles_impacted`.
- When no-calls leave several candidates, all of them are ranked with a `variant_change` delta: the variants that
differ from the top solution.
- Candidates whose edit distance from the top exceeds `--max-edit-distance` (default 3) are flagged
`distant_alternative`.
(src: E-41, D-13)
- **PGX-020** `§11 · v2` **Unresolved variants.** An unresolved variant is a defining variant of a solution's allele
that is not `called`. Reasons: `not_on_array`, `no_call`, `masked`. (src: B-25, D-13)
- **PGX-021** `§11 · v2` **Variant-type calls.**
- Variant-based and tagging genes are reported with `call_type = variant`. The genotype uses explicit notation:
`rs9923231 C/T`, or `Reference` when homozygous reference.
- REF>ALT is always written with `>`, never `_` (EDGE-027).
- Multi-definition rsIDs keep their suffix identifiers.
(src: B-29, D-17, E-40)
- **PGX-022** `§11 · v2` **Nomenclature.**
- Duplications: `*2x2`. When the copy count exceeds the CN cap: `*1xN` (N = unknown, ≥ 3).
- Tandems and hybrids: `*36+*10`, `*13+*68x2+*4`.
- Diplotypes: `A/B`, with alleles in natural order unless structure requires otherwise.
(src: B §3.12)
- **PGX-023** `§11 · v2` **Definition sanity tests.** Definition regressions are checked in CI:
- the CYP2A6 `*1` definition must contain no spurious variants (the reference once included an extra delins);
- ABCG2 reference calls get the phenotype "Normal" by default.
(src: E-40)
- **PGX-024** `§11 · v2` **Unknown allele in lookup.** A called allele or diplotype with no guideline entry gives
phenotype `Indeterminate` and activity `null`, and flags `phenotype_not_in_guideline`. It never crashes. The
reference crashed when cluster drift produced a CYP2E1 allele with no entry. (src: E-40)
### 8.3 Phenotype annotation
- **PGX-025** `§11 · v2` **Guidelines.**
- `--guideline cpic` (default) or `dpwg`.
- Phenotype is a direct lookup of the diplotype, or of the activity score, in the chosen guideline's table.
- Coverage:
- CPIC, 17 genes: CACNA1S, CYP2B6, CYP2C19, CYP2C9, CYP2D6, CYP3A5, DPYD, G6PD, MT-RNR1, NUDT15, RYR1, SLCO1B1,
TPMT, UGT1A1, CFTR, IFNL3/4, VKORC1. Activity scores: CYP2C9, CYP2D6, DPYD.
- DPWG, 14 genes: CYP1A2, CYP2B6, CYP2C19, CYP2C9, CYP2D6, CYP3A4, CYP3A5, DPYD, NUDT15, SLCO1B1, TPMT, UGT1A1,
VKORC1, F5. Activity scores: CYP2D6, DPYD.
- Genes outside the chosen guideline get phenotype `null`.
- With several feasible solutions, a phenotype is reported only when all of them agree. Otherwise the gene gets
`Indeterminate` plus the list of per-solution phenotypes.
(src: D-16, A-G41)
### 8.4 Outputs
- **PGX-026** `§11, §13 · v2` **Files.**
- **Per sample:** `<sample>.pgx_calls.parquet` (written by `call`) and `<sample>.pgx.json` (written by
`phenotype`).
- **Per batch** (under `pgx/`):
- `diplotype_summary.tsv`;
- `diplotype_annotations.tsv`;
- `supporting_variants.tsv`;
- `unresolved_variants.tsv`;
- `variants_not_covered.tsv`.
- Every TSV starts with `#` metadata lines: `#simurg_schema=`, `#software_version=`, `#genome_build=GRCh38`,
`#pgx_db=<file>;sha256=`, `#allele_sources=`, `#phenotype_sources=`, `#guideline=`.
(src: B-25, D-18, A-G41, B-28)
- **PGX-027** `§11 · v2` **TSV columns** (in order).
- `diplotype_summary.tsv`: `sample_id, sample_name, solution_id, gene, call_type, solution,
array_equivalent_alleles_1, array_equivalent_alleles_2, no_call_variants, alleles_impacted_by_no_call,
background_alleles, variant_change, score, raw_score, copy_number_solution, flags`
- One row per feasible or candidate solution.
- `copy_number_solution` is written as `CYP2D6.p5=5;CYP2D6.intron.2=5;CYP2D6.exon.9=2`.
- `diplotype_annotations.tsv`: `sample_id, sample_name, gene, guideline, genotype, activity_score, phenotype, flags`
- One row per sample × gene.
- `supporting_variants.tsv`: `sample_id, solution_id, gene, allele, hgvs, rsid, chrom, pos, ref, alts, genotype,
variant_copy_number, gs, baf, probe_ids, low_gs`
- `unresolved_variants.tsv`: `sample_id, gene, solution_id, allele, hgvs, rsid, reason, probe_ids`
- `variants_not_covered.tsv`: `gene, hgvs, rsid, alleles_impacted`
- Depends on the manifest and DB only.
Formatting rules:
- Lists inside a cell are `;`-joined.
- Sorting: sample, gene, solution_id, then allele.
(src: B-25, D-18)
- **PGX-028** `§11 · v2` **JSON schema (`simurg.pgx/1`).**
```json
{"schema":"simurg.pgx/1","software_version":"…","genome_build":"GRCh38",
"pgx_db":{"file":"pgx_db-1-3fa2c0d9e1b4.sqlite","sha256":"…"},
"allele_sources":[{"name":"PharmVar","version":"6.1"}],
"phenotype_sources":[{"name":"CPIC","version":"1.38.0"}],
"guideline":"cpic","sample_id":"…","sample_name":"…",
"genes":[{"gene":"CYP2D6","call_type":"star_allele",
"genotype":"*27/*36x2+*83+*10","feasible_solutions":["*27/*36x2+*83+*10"],
"activity_score":null,"phenotype":"Indeterminate","score":0.5963,"raw_score":0.9981,"flags":[],
"array_equivalent_alleles":[{"allele":"*27","equivalents":["*27"]}],
"supporting_variants":[{"chrom":"chr22","pos":42126611,"ref":"C","alts":["G"],"gt":"0/1","gs":0.8123,
"baf":0.3185,"variant_copy_number":1,"hgvs":"NC_000022.11:g.42126611C>G","rsid":"rs1135840",
"probe_ids":["…"],"allele_ids":["*2","*10"],"low_gs":false}],
"variant_no_calls":[{"hgvs":"…","rsid":"…","probe_ids":["…"],"alleles_impacted":["*4"]}],
"variants_not_covered":[{"hgvs":"…","rsid":null,"alleles_impacted":["*15"]}],
"candidate_solutions":[{"rank":1,"genotype":"*27/*36x2+*83+*10","score":0.5963,"raw_score":0.9981,
"activity_score":null,"phenotype":"Indeterminate",
"alleles":[{"allele":"*27","supporting_variants":[],"unresolved_variant_sites":[],
"array_equivalent_alleles":["*27"],"background_allele":null},
{"allele":"*36x2+*83+*10","supporting_variants":[],"unresolved_variant_sites":[],
"array_equivalent_alleles":[],"background_allele":"*10"}],
"variant_no_calls":[],"alleles_impacted_by_no_calls":[],"variant_change":null,
"copy_number":[{"region":"CYP2D6.p5","cn":5},{"region":"CYP2D6.intron.2","cn":5},{"region":"CYP2D6.exon.9","cn":2}]}],
"alleles_tested":["*1","*2","…"]}]}
```
Rules:
- Numbers are numbers, and missing values are `null`.
- **Both** alleles of every candidate carry full supporting and unresolved details. The reference filled only the
first allele.
- `alleles_tested` is in natural sort order.
- The same information appears in the TSVs, and a CI test checks JSON ↔ TSV equivalence.
(src: B-26, E-41, E-48)
- **PGX-029** `§11, §15 · v2` **Determinism.** Given identical inputs, outputs are byte-identical across OS, CPU
architecture and thread count. Enumeration and summation run in a fixed order, and float formatting is fixed.
Golden tests run on x86-64 and aarch64. The reference showed Linux/Windows differences in scores, unresolved
variants and collapsed alleles. (src: E-27, E-43, D-13)
- **PGX-030** `§11 · v2` **Semi-custom arrays [DIFF].**
- Star-allele calling works on any manifest with `probe_map` entries, so a semi-custom array is handled in one run.
- `pgx coverage` reports, for each gene, any definitions whose commercial-backbone probes are missing. A gene losing
more than 10 % of its covering probes → a warning.
- Add-on content that maps to DB variants is used; other add-on content is ignored.
(src: A-G42, D-20)
- **PGX-031** `§11 · v2` **Novel alleles.** Out of scope. When the best solution still leaves covered, called
variants unexplained (observed ALT not in any allele of the solution), the gene gets the flag
`unexplained_variants`, listing them. It is not reported as a novel allele. (src: D §5.5)
- **PGX-032** `§11, §16 · v2` **Validation targets** (plan §16.5), measured against WGS-derived calls for 1000 Genomes
samples:
- star-allele call rate ≥ 99 % on in-silico data;
- diplotype concordance ≥ 90 % on in-silico data and ≥ 90 % against WGS truth for CYP2D6/CYP2C19/CYP2C9;
- CYP2D6 call rate ≥ 97 %.
(src: C-G25, C-G27)
- **PGX-033** `§11 · v2` **No licence, no network [DIFF].** PGx runs fully offline, with no per-sample quota.
Results are written per sample as they complete (CLI-012). (src: A-G13, A 2.3)
- **PGX-034** `§11 · v2` **`alleles_tested`** is the per-gene list of alleles callable with the manifest, from
PGX-009, in natural order. It is identical for every sample of a manifest. (src: B-26)
- **PGX-035** `§11 · v2` **`simurg pgx coverage --bpm --db [--csv]`** writes `pgx_coverage.tsv`: `gene, allele,
allele_kind, result_basis, status (callable|not_covered|masked|equivalent_to:<allele>|reported_as:<allele>),
covering_probes`. The same information feeds `variants_not_covered.tsv`. The output mirrors the structure of the
archived coverage CSV, so the two can be compared directly. (src: D-14, D-13)
---
## 9. Methylation control QC (v2)
- **MET-001** `§1.2, §9 · v2` **Scope.**
- Control-probe QC and a basic signal summary for Infinium methylation arrays: EPIC v1, EPIC v2, EPIC-Xtra-style
custom arrays, and MSA.
- Runs locally through `simurg methyl qc`; the reference offers this only in its cloud.
- No differential-methylation analysis.
- Genome build is irrelevant: no mapping is used.
(src: C-G26, B-31)
- **MET-002** `§6.3 · v2` **Manifest validation.** A methylation CSV manifest must contain:
- an `[Assay]` section with `IlmnID`, `AddressA_ID`, `AddressB_ID` and `Color_Channel`;
- a `[Controls]` section with exactly 4 columns: `Address, Type, Color_Channel, Name`.
Errors (exit 2):
- `E_METH_NO_ASSAY` ("Probe section not found");
- `E_METH_NO_CONTROLS` ("Missing control probes in manifest");
- `E_METH_MISSING_COLUMNS` ("Missing required columns: …");
- `E_METH_CONTROLS_FORMAT` ("Controls not formatted correctly. Must have 4 columns (Address,Type,Color_Channel,Name)").
(src: D-24, B-31)
- **MET-003** `§6.3 · v2` **Manifest conversion.** `simurg methyl convert-manifest` (and every `methyl qc` run)
writes two files:
- `probes.tsv`: `probe_id, u, m, col`.
- `u = AddressA_ID` and `m = AddressB_ID`.
- `col` is `R` or `G` for Infinium I and `NA` for Infinium II. This is the correct reading of a misprint in the
reference documentation, OQ-014.
- Controls get `probe_id = ctl_<address>`.
- `controls.tsv`: `address, type, color_channel, name, probe_id`.
The layout mirrors the manifest format of the open-source methylation toolkits, so they can be used as oracles.
(src: B-31, D-23)
- **MET-004** `§6.6 · v2` **Sample sheet.**
- IN-001 … IN-016 apply.
- Duplicate `sample_id` → error by default; `--on-duplicate-sample drop` keeps the first row with a warning (IN-009).
- `Sample_Group` colours the PCA. Other columns are passed through to the QC summary.
- Errors: `E_METH_NO_IDATS` ("No IDATs found"), `E_SHEET_EMPTY`, and `E_SHEET_NO_KEY_COLUMNS`. For the last, the
beadChipName/sampleSectionName aliases are named in the message.
(src: B-31, A-G59)
- **MET-005** `§7.1 · v2` **Raw signal assembly.**
- Infinium I: M = channel[AddressB] and U = channel[AddressA], where the channel is the probe's `Color_Channel`.
Out-of-band signal is the other channel.
- Infinium II: M = Grn[AddressA] and U = Red[AddressA].
- Beads with NBeads < 3 are masked (detection p = 1) [OURS].
- Per-sample control table `<sample>_ctrl.tsv.gz`: `address, type, color_channel, name, probe_id, green, red`. The
reference labels these MG/MR/UG/UR; their per-probe meaning is documented [OURS].
(src: B-31, D-23)
- **MET-006** `§9.1 · v2` **21 control metrics.**
- The QC-004 rows marked M: restoration; staining ×2; extension ×2; hybridization ×2; target removal I/II;
bisulfite conversion I green/green-bg/red/red-bg; bisulfite conversion II and II-bg; specificity I green/red;
specificity II and II-bg; non-polymorphic ×2.
- Defined as in QC-004, with `bkg_G`/`bkg_R` as in QC-003 and offset x = 3000 (`--bg-offset`).
- The same code path as genotyping control QC.
- The two documented background definitions disagree (OQ-012). We use the newer ones, and the alternative is
selectable (`--bs-conv-red-bg-set U3-U5|U4-U6`).
(src: D-21, C-G26, E-46)
- **MET-007** `§9.1 · v2` **Thresholds.**
- The profiles `methylation_epic` and `methylation_msa` come from QC-024. The `+ffpe` variant sets restoration
to 1.
- Pass rule: `value ≥ threshold`, the same inclusive rule as genotyping QC. The reference documents `>` for
methylation (OQ-013).
- The documentation reproduces the guidance:
- hybridization, target removal, specificity and the background ratios should not normally be changed;
- staining, extension, BS-conversion C/U ratios and non-polymorphic may need raising for some scanners;
- custom or discontinued chips, and NextSeq 550 IDATs, need user thresholds.
(src: D-21, B-14)
- **MET-008** `§9.2 · v2` **Pass/fail.**
- `pass_qc = 1` iff every enabled, evaluable metric passes.
- `fail_codes` lists the failing keys, `;`-joined.
- `na_metrics` lists metrics that are NA because controls are missing (MET-009).
- The fraction of probes passing detection does not affect `pass_qc` unless `--min-prop-passing X` is given.
(src: D-21, D-22)
- **MET-009** `§9.1 · v2` **Missing controls.** Missing control types or names produce
`W_METH_MISSING_CONTROL_TYPES` or `W_METH_MISSING_CONTROL_NAMES`, with the list. The affected metric becomes NA,
and is excluded from `pass_qc` (QC-007). (src: D-24)
- **MET-010** `§7 · v2` **Signal processing [OURS]: open reimplementations of published methods.** The reference's
formulas are undocumented (OQ-036). Each step is published or an open source, and is validated against an
open-source methylation toolkit run as a subprocess oracle.
1. **Detection p-value:** pOOBAH-style [LIT: Zhou et al. 2018 NAR]. It uses the empirical CDF of out-of-band
Infinium I signal per channel, and gives `p = 1 − ECDF(M + U)`.
2. **Background correction:** noob [LIT: Triche et al. 2013 NAR]. A normal-exponential convolution is fitted per
channel on the out-of-band signal and applied to the in-band signal.
3. **Dye-bias correction:** by default, linear scaling so that the red and green means of the normalization
controls (NORM_A/T red vs NORM_C/G green) match their average. `--dye-bias nonlinear` applies quantile matching
of Infinium I red vs green (as in open-source toolkits).
4. **Beta:** `β = M / (M + U + α)`, with `α = --beta-offset`, default 0 and floored signals ≥ 1. α = 100 gives the
legacy-desktop convention.
5. **M-value:** `log2(β' / (1 − β'))`, where β' is β clipped to [1e-6, 1 − 1e-6].
A failure in any step for a sample → `E_METH_PREP_FAILED` for that sample only. The other samples continue, and
the exit code is 3. (src: D-22, E-46, B-31)
- **MET-011** `§9 · v2` **Detection threshold.**
- `--pval-threshold` defaults to 0.05. A probe passes if p < threshold.
- Reported per sample: `n_probes` (all analytical CG and SNP probes), `n_pass_detection`, `prop_probes_passing`.
- The docs recommend excluding probes with p > 0.05 downstream, and warn that detection rates depend on the method
used (this method is stricter than legacy ones).
(src: D-22)
- **MET-012** `§9 · v2` **Average signals.** Four per-sample means over all analytical probes: `avg_green_raw`,
`avg_red_raw`, `avg_green_norm` and `avg_red_norm`. The `_norm` values are taken after dye-bias correction and noob.
(src: B-31)
- **MET-013** `§13 · v2` **Outputs.**
- **Per sample:**
- `<sample>_ctrl.tsv.gz` (MET-005);
- `<sample>_cgs.tsv.gz`: `probe_id, beta, m_value, detection_p`, with values at `%.6g`.
- **Per batch:**
- `methyl_qc_summary.tsv` and `.xlsx`. Columns: `sentrix_id` (12 digits), `sentrix_position`, `sample_id`, the
pass-through metadata, the 21 metrics, the 4 average signals, `scan_time` (ISO-8601), `n_probes`,
`n_pass_detection`, `prop_probes_passing`, `pass_qc`, `fail_codes`, `na_metrics`. The xlsx highlights failing
cells.
- `methyl_qc_summary.html`, self-contained, as in QC-047, with two plots:
- a histogram of `prop_probes_passing`, coloured by pass status;
- a PCA of betas over all analytical probes. The PCA uses probes finite in every sample. With more than
`--pca-max-samples-full` (100) samples, it uses a seeded random subset of `--pca-probes` (10 000) probes. It
is coloured by `Sample_Group`.
- `methyl_qc_summary.pdf`: optional, `--pdf`.
- `pcs.tsv.gz`: `sample_id, pc1, pc2, sample_group`; `NA` when the sheet has no group.
- `probes.tsv`, `controls.tsv` (MET-003).
- `controls_raw.tsv` and `controls_qc.tsv` (QC-009/011), so that `qc report` accepts the folder.
- No common probes → `W_METH_NO_FEATURES` ("No features, skipping PCA plot"), and the PCA is omitted.
(src: B-31, C-G26, D-23)
- **MET-014** `§9.5 · v2` **Report integration.** Methylation datasets can be loaded into `qc report` (QC-062). They
cannot be mixed with genotyping datasets (QC-043). (src: C-G18)
- **MET-015** `§14 · v2` **Message catalogue.**
| Code | Level | Message |
|---|---|---|
| `E_METH_NO_IDATS` | error | "No IDATs found" |
| `E_SHEET_EMPTY` | error | "No samples in sample sheet" |
| `E_SHEET_DELIMITER` / structural | error | "Sample sheet not correctly formatted" |
| `E_SHEET_NO_KEY_COLUMNS` | error | key columns missing, naming both key forms |
| `E_SHEET_DUP_SAMPLE` / `W_…` | error / warning | duplicate samples (dropped under `--on-duplicate-sample drop`) |
| `E_METH_NO_CONTROLS`, `E_METH_NO_ASSAY`, `E_METH_MISSING_COLUMNS`, `E_METH_CONTROLS_FORMAT` | error | MET-002 |
| `W_METH_MISSING_SAMPLE` | warning | "Missing sample: <id>" (IDATs absent; the sample fails, the run continues) |
| `W_IDAT_NO_SCAN_TIME` | warning | "No scan time available" |
| `E_METH_PREP_FAILED` | error (sample) | "Prep failed" plus the step name |
| `W_METH_MISSING_CONTROL_TYPES` / `_NAMES` | warning | metric set to NA |
| `W_METH_NO_FEATURES` | warning | PCA skipped |
(src: B-31, D-24)
- **MET-016** `§19 · v2` **Scale.**
- Per-sample processing is streamed.
- The PCA uses randomised SVD over at most 10 000 probes when n > 100, so memory is bounded.
- Regression tests run 80, 90, 100 and 110 samples on a > 900 k-probe manifest (EPIC v2 or MSA), the reference's
crash zone, and 1 152 samples.
(src: E-46, D-31)
- **MET-017** `§9 · v2` **Custom manifests.** A manifest whose controls are missing or partial still runs: the affected
metrics are NA (MET-009). Thresholds for custom or discontinued chips must be user-supplied. `suggest-thresholds`
(QC-026) works on methylation datasets. (src: E-46, D-21)
- **MET-018** `§9 · v2` **NextSeq 550 IDATs.** These are processed normally. The report adds a caveat that the shipped
thresholds were not calibrated for this scanner. (src: A-G61)
---
## 10. Edge cases & regression-test catalogue
Every item below is an automated test (plan §16.1). "Ref-bug" marks a case where the reference product had a defect;
SimurgArray must behave correctly.
- **EDGE-001** `§16 · P1` **Locale independence.** The test suite runs under `LC_ALL` = `C.UTF-8`, `tr_TR.UTF-8`
(dotless-i case folding), `de_DE.UTF-8` (decimal comma), `ar_SA.UTF-8` (Arabic-Indic digits) and `ja_JP.UTF-8`.
- GTC, VCF, TSV, JSON and HTML outputs are byte-identical across all locales.
- Sheet column matching is not affected by Turkish case rules.
- Ref-bug: non-ASCII date strings in GTCs broke later reads.
(src: E-1)
- **EDGE-002** `§16 · P1` **GTCs from other tools with bad strings.** GTCs written by other tools whose strings have
a wrong length or invalid UTF-8 (synthesised) → a clean `E_GTC_CORRUPT`, or a decode with replacement characters,
never an uncaught exception. (src: E-1)
- **EDGE-003** `§16 · P4` **Corrupt or truncated GTC** in a batch of 3 → the other 2 are processed, the summary shows
the failed sample, and the exit code is 3. Ref-bug: the whole run aborted. (src: E-2)
- **EDGE-004** `§16 · P1` **Malformed IDATs.** Truncated file, wrong magic, bad TOC offset, encrypted v1, mismatched
Red/Grn N → the specific error, per sample. (src: plan §16.1)
- **EDGE-005** `§16 · P1` **Duplicate IDAT pairs.** The same sample in two subfolders gives `E_DUP_IDAT` with both
paths. The result must be identical under a directory listing shuffled by a test hook, and on Linux and macOS.
`--on-duplicate-idat first|newest` must be deterministic. (src: E-5, D-28)
- **EDGE-006** `§16 · P1` **Long and odd names.** Manifest name > 80 characters (ref-bug: failure); cluster file
names with spaces and `()`; output paths with spaces and non-ASCII characters; paths > 260 characters. (src: E-4)
- **EDGE-007** `§16 · P1` **Sample-sheet matrix.** Each of these is a separate test:
- trailing empty column(s) (ref-bug: a crash, or an empty-key field);
- `Sample_Name` and `sample_name` both present;
- a reserved column (`Sex Estimate`, `call_rate`);
- a 501-character value;
- barcode `2.04753E+11` and barcode `204753010023.0`;
- a BOM, CRLF line endings, quoted commas;
- semicolon delimiters;
- a `[Manifests]` section present;
- a sectionless sheet;
- `RootFolder` absolute, relative, and conflicting with the CLI;
- `beadChipName`/`sampleSectionName` keys;
- an empty `Sample_Name` in one row;
- `Sample_Name` equal to another sample's ID;
- duplicate names;
- a 10 000-row sheet (performance, PERF-007).
(src: E-9, A-G56, A-G57, A-G58, E-7)
- **EDGE-008** `§16 · P4` **Manifest mismatch matrix.** Each of these must be caught before processing starts:
- BPM vs CSV differing by one locus (ref-bug: not detected);
- an EGT built for another manifest (ref-bug: every sample skipped with a dictionary error);
- IDATs from another chip type (ref-bug: silently inaccurate results).
(src: E-3)
- **EDGE-009** `§16 · P6` **FASTA problems.** A `.fai` from another FASTA → `E_FAI_MISMATCH`. A missing `.fai` is
built into the cache. `chr`-prefixed vs bare contig names give an error in strict mode and succeed with
`--contig-alias auto`. The alt contig is named either `22_KI270879v1_alt` or `chr22_KI270879v1_alt`. A GRCh37
FASTA with a GRCh38 manifest → `E_BUILD_MISMATCH`. (src: E-22, A-G24, B-33)
- **EDGE-010** `§16 · P6` **Genome cache on and off** give identical VCFs. (src: A-G8)
- **EDGE-011** `§16 · P6` **Multi-probe merge scenarios** (OUT-031):
- the two published ambiguity scenarios;
- an inconsistent pair;
- all probes NC;
- `--inf1-nocall-informative` on and off, including the conflict fallback;
- the message format and codes;
- GS equal to the minimum;
- no BAF/LRR in the FORMAT;
- ID joining with `,` and `;`.
(src: E-12, E-13, E-17, B-3)
- **EDGE-012** `§16 · P6` **Opposite-strand duplicate indels** (`[D/I]` and `[I/D]`), both heterozygous → `0/1`.
Ref-bug: `./.`. (src: E-14)
- **EDGE-013** `§16 · P6` **SNV and indel at the same position.** Written as separate records by default, and merged
with `--merge-snv-indel`. (src: E-15)
- **EDGE-014** `§16 · P6` **`--unsquash-duplicates` on a multi-allelic duplicated design** → the site stays merged.
(src: E-25, A-G21)
- **EDGE-015** `§16 · P6` **Auxiliary loci.** One test per violated restriction (multi-allelic, not an MNV,
duplicated assay, allele mapping mismatch, REF ≠ FASTA) → `E_AUX_INVALID`. A valid MNV restores its true alleles.
(src: E-16, A-G18)
- **EDGE-016** `§16 · P6` **MNV designs in the final-report export.** Top alleles are not reverse-complemented.
Ref-bug: open issue. (src: E-16)
- **EDGE-017** `§16 · P6` **BAF orientation.** At a locus where the probe's A allele is ALT, the VCF BAF equals
1 − GTC BAF. LRR is unchanged. (src: E-21)
- **EDGE-018** `§16 · P6` **Indel flips.** A manifest deletion that normalises to an insertion is written in the
reference-consistent form with the genotype remapped, and the audit carries `flipped_indel`. Checked against
dbSNP-normalised truth. (src: E-19, B-4)
- **EDGE-019** `§16 · P6` **Skip taxonomy.** Unmapped (`Chr=0`), intensity-only and unresolvable indels each produce
exactly one warning code, and the run-summary counts are correct. (src: E-18, B-5)
- **EDGE-020** `§16 · P6` **Index metadata.** `bcftools index --nrecords` on our `.tbi` equals the record count.
Ref-bug: the index lacked count metadata. The PGx caller accepts an index rebuilt by htslib. (src: E-23)
- **EDGE-021** `§16 · P6` **VCF validity.** Every golden VCF passes `bcftools view` and a VCF-4.3 validator. The
header's declared version matches the syntax used. (src: E-24)
- **EDGE-022** `§15, §16 · P6` **Cross-platform determinism.** The same inputs on Linux x86-64, Linux aarch64 and
macOS arm64, with `--threads` 1, 4 and 16 → byte-identical outputs, excluding `run_manifest.json` timestamps.
Ref-bug: output order and PGx scores differed by OS. (src: E-27, plan §15)
- **EDGE-023** `§16 · P6` **bedGraph values.** The raw tracks equal the GTC values after formatting. A smoothed
track, if requested, is a separate file. Ref-bug: smoothing corrupted LRR. (src: E-28)
- **EDGE-024** `§16 · v2` **Cyto degenerate and regression cases.** Each is a separate test:
- a noisy sample with zero events (ref-bug: overflow);
- an all-NaN chromosome;
- a single-probe chromosome;
- a QUAL-0 event reaching the JSON (ref-bug);
- the first event getting ISCN (ref-bug);
- thresholds shown in the right units (ref-bug);
- a threshold larger than the genome → an empty result and a warning;
- a probe threshold of 5 → floored to 10 with a warning;
- an event spanning a centromere gap: effective vs raw size, consistent between call and annotate;
- 200 and 1 152-sample batches with flat memory (ref-bug);
- unknown sex (ref-bug: silently treated as male);
- female with Y probes;
- PAR events on a manifest without `XY` loci (ref-bug);
- mosaic chrY duplication in a male;
- LOH < 500 kb between copy-neutral neighbours → removed;
- a gain with mostly negative LRR → removed;
- `logr_dev` 0.25 → SampleQuality; 0.35 → whole-chromosome only;
- more than 10 events on one chromosome → `excess_events`.
(src: E-29, E-30, E-31, E-32, E-33, E-34, E-35, A-G46, A-G51)
- **EDGE-025** `§16 · v2` **CNV VCF coordinates.** For every record, `ID` coordinates = (POS + 1, END), REF = the
FASTA base at POS, and the SVLEN sign follows the mode. The same checks apply to `pgx_cn` VCFs. Ref-bug: ID and POS
disagreed; REF was `N`. (src: E-36, B-16)
- **EDGE-026** `§16 · v2` **PGx CN batch and model rules.**
- Batch of 23 → error.
- 24 samples with 21 passing → error.
- 24 samples with 22 passing → OK, with a below-recommended warning.
- A sample with `logr_dev` exactly 0.20 does not pass at calling (strict `<`) but passes in training (`≤`).
- Cluster mismatch under `error` and under `warn` (both reference behaviours).
- Manifest mismatch → error.
- Training with 95 samples → error.
- 10 % injected outlier samples → the other samples' calls are unchanged.
(src: E-39, A-G33, A-G34)
- **EDGE-027** `§16 · v2` **Star-allele regressions.** Each is a separate test:
- an allele absent from the phenotype table (ref-bug: crash);
- ABCG2 reference → Normal;
- CYP2D6 `*83` not folded into `*36`;
- CYP2A6 `*1` definition;
- HGVS and REF>ALT written with `>`, never `_`;
- symlinked VCFs accepted;
- unsquashed, filtered or unindexed VCFs refused;
- JSON: both alleles carry supporting details;
- JSON ↔ TSV equivalence;
- multi-allelic extension;
- G6PD Mediterranean on GRCh37 vs GRCh38;
- many-probes-to-one-variant mapping;
- no LP/MIP solver dependence.
(src: E-40, E-41, E-42, E-43)
- **EDGE-028** `§16 · P5` **QC report error and behaviour tests.**
- Every `E_QC_*` and `W_QC_*` code, including `.yaml.txt`, a JSON config, a missing wrapper, an unknown key, a
non-numeric value, `callRate` 1.2 and a negative threshold.
- Browser-level tests (Playwright, headless):
- 1 001 samples → tabs;
- 12 001 samples → WebGL scatter, not disabled;
- switching from a 12 000-sample dataset to a 500-sample dataset without a reload (ref-bug);
- 15 facet levels;
- 150-character hover text;
- dragging and resizing a selection on 12 000 points without freezing (ref-bug);
- heatmap control alignment with 60 plates (ref-bug);
- "Download YAML" → re-upload round trip;
- export of selected rows only.
(src: C-G21, E-45)
- **EDGE-029** `§16 · v2` **Methylation.** A manifest missing `[Controls]`; a control type missing (the metric is NA
and excluded); 80–110 samples on a > 900 k-probe array (ref-bug: crash); a missing scan time; a prep failure in one
sample, isolated; zero common probes → PCA skipped. (src: E-46, D-24)
- **EDGE-030** `§16 · P1` **Overwrite and resume.** A rerun into the same folder → `E_OUTPUT_EXISTS`. `--force`
replaces the outputs. `--resume` after deleting one sample's outputs recomputes only that sample. (src: E-6, A-G4)
- **EDGE-031** `§16 · P1` **Interruption.** SIGINT during a batch → no `*.part` file left under a final name, a run
manifest with `status=interrupted`, and exit 130. `--resume` completes the batch. (src: plan §14)
- **EDGE-032** `§16 · P6` **GTCs from other producers.** Accepted by `gtc-to-vcf` with `W_GTC_FOREIGN`. Refused by
`cn call` and `pgx copy-number` with `E_GTC_UNSUITABLE`. (src: A-G22, B-34)
- **EDGE-033** `§16 · P1` **IDAT naming.** An IDAT whose internal barcode disagrees with its file name →
`W_IDAT_NAME_MISMATCH`, and the file name is used. (src: plan §6.1)
- **EDGE-034** `§16 · P8` **Non-human diploid.** A BovineSNP50 run on UMD3 with a user FASTA produces GTC, summary
and QC; ploidy follows the bovine profile. The VCF is produced only when the FASTA contigs match. (src: C-G22)
- **EDGE-035** `§16 · P4` **Missing inputs.** A sheet row with no IDATs → a summary row with `MISSING_INPUT`, and
exit 3 (or 4 if every row is missing). (src: plan)
- **EDGE-036** `§16 · P6` **Final-report size guard.** loci × samples > 1e8 → `W_LARGE_OUTPUT`, plus a
`pipeline plan` suggestion to turn the output off. (src: B-10, D-30)
- **EDGE-037** `§16 · P4` **Assay-control column.** A PGx manifest gives a `TGA_Ctrl_5716_norm_r` column. A non-PGx
manifest has no such column. A failed sample has NaN. (src: E-26, B-9)
- **EDGE-038** `§16 · P6` **Unknown sex in the VCF.** A sample with `sex_est = U` gets diploid X and haploid Y/MT,
and the header records the policy. (src: E-20)
- **EDGE-039** `§16 · P1` **Schema conformance.** Every JSON output validates against its JSON Schema. No key
contains `:` or other punctuation. Numbers are never strings. (src: E-48, B-35)
- **EDGE-040** `§16 · P5` **Metadata redaction.** `--drop-metadata` removes the columns from the HTML (checked by
searching the embedded JSON for them) and from the xlsx/csv. (src: A-G31, C-G20)
---
## 11. Performance & benchmark targets
- **PERF-001** `§19 · P6 (Python), P9 (Rust)` **Genotyping throughput** for GSA-size data (~650 k loci), IDAT → GTC
+ SNV VCF:
- Python reference implementation: ≤ 35 s per sample on one core. This matches the reference product's stated
"as little as 35 s per sample".
- After P9: ≥ 20 samples/min/core (plan §18).
- Omni2.5-size data: ≤ 4× the GSA time.
(src: C-G27, plan §18)
- **PERF-002** `§19 · v2` **PGx throughput.** CN plus star alleles plus phenotype: ≤ 60 s per sample amortised on
8 cores, for a 96-sample batch. (src: C-G27)
- **PERF-003** `§19 · v2` **Cyto throughput.** `cn call` plus `annotate`: ≤ 60 s per sample on one core for an 850 k
array, with memory ≤ 2 GB per worker, independent of batch size. (src: D-9)
- **PERF-004** `§19 · P6` **Memory.**
- Genotyping worker: ≤ 1.5 GB for GSA and ≤ 4 GB for Omni5.
- Genome cache: ≤ 3.5 GB, shared (IN-064).
- `--no-genome-cache` mode: ≤ 1 GB.
- PGx CN: see PCN-016.
- QC report generation: ≤ 4 GB for 12 000 samples.
(src: A-G43, A-G8)
- **PERF-005** `§3, §19 · P10` **Reference hardware.** 8 cores, 32 GB RAM, ≥ 30 GB free disk, no GPU. All targets are
measured on this configuration, and on the plan's 16–32-core machine for scaling. (src: A-G12)
- **PERF-006** `§19 · P9` **Scaling.**
- Near-linear throughput up to 32 threads for per-sample steps.
- Batch sizes 10, 100, 1 000 and 10 000 (synthetic), with no super-linear growth in time or memory.
- Tested batch minimums: 1 152 for genotyping, cyto and methylation; 384 for PGx.
(src: D-31, SCOPE-009)
- **PERF-007** `§19 · P1` **Sample sheet.** A 10 000-row sheet parses and matches against 10 000 IDAT pairs in < 2 s.
The reference slowed down badly above 1 000 rows. (src: E-7)
- **PERF-008** `§19 · P5` **QC report.**
- The HTML for 12 000 samples is ≤ 60 MB and becomes interactive in < 10 s on the reference hardware.
- 50 000 samples work in density mode.
- Selection updates take < 200 ms on 12 000 points.
(src: C-G16, E-45)
- **PERF-009** `§16.5 · P8, v2` **Accuracy targets.**
- Genotype concordance vs WGS truth > 99.5 % (a floor). Against vendor GTCs, plan §16.3's ≥ 99.999 % applies.
- PGx CN concordance > 95 %.
- CYP2D6 call rate > 97 %.
- Star-allele in-silico call rate ≥ 99 %, and diplotype accuracy ≥ 90 %.
- Mosaic sensitivity ≥ 90 % at 15 % MF / 5 Mb.
(src: C-G27, C-G25)
- **PERF-010** `§19 · P6` **Large outputs.** The final report and the multi-sample BCF are streamed, and never held
in memory. A 2 000-sample × 2.4 M-locus final report stays within 2 GB RSS. (src: B-10)
- **PERF-011** `§19 · v2` **Methylation.** 1 152 EPIC v2 samples complete within 2 h on the reference hardware, with
PCA memory ≤ 8 GB. (src: E-46)
---
## 12. Open questions and documentation inconsistencies
These are inconsistencies or gaps in the reference documentation, or open choices of ours. **None of them is a
requirement.** Where a decision was needed, the item names it; everything else goes into `docs/known_unknowns.md`
(plan §20).
| ID | Issue | Decision / action |
|---|---|---|
| **OQ-001** | The reference marks the IDAT folder "required" and also says "folder, sheet, or both" | we follow "folder, sheet or both" (CLI-006) |
| **OQ-002** | `--filter-loci` is described as "generates a text file" | treated as an *input* exclude list (OUT-036) |
| **OQ-003** | bedGraph described as LRR-only in one place, LRR + BAF in another | both tracks (OUT-050) |
| **OQ-004** | PGx pass boundary: `< 0.2` (calling) vs `≤ 0.2` (training, recommendations) | both kept as documented and configurable (PCN-004/005, EDGE-026) |
| **OQ-005** | an example command omits a module name; another uses a relative path | doc typos; ignored |
| **OQ-006** | an older doc lists the cyto DB option under a different name; a typo "star-alle" | ignored |
| **OQ-007** | LRR SD default: 0.15 (YAML example) vs 0.20 (CLI/UI) | 0.20 built in; 0.15 only in `genotyping_example` (QC-024) |
| **OQ-008** | control threshold conflicts: target removal 5 vs 1; hyb 1.25 vs 1; stringency 1.5 vs 2; NSB 5 vs 1 | UI values → `genotyping`; example values → `genotyping_example`; calibrate with QC-026 on public data |
| **OQ-009** | stringency channel: red (formula text) vs green (UI) | red by default; `stringencyChannel: green` option; decide from real control data (P5 experiment) |
| **OQ-010** | UI formula for NSB green uses "Bkg Red" | treated as a typo; `bkg_grn` (QC-004) |
| **OQ-011** | the example `controls.qc_metrics` row has NSB green = red and NP ≈ 0.1 | example values judged synthetic; no formulas inferred from them |
| **OQ-012** | methylation backgrounds: BS-conv I red bg U3–U5 vs U4–U6; specificity II bg S1–S4 vs S1–S3 | newer definition by default; alternatives selectable (MET-006) |
| **OQ-013** | methylation pass is "> threshold"; genotyping flags "below" | uniform inclusive rule `value ≥ threshold` passes (QC-020, MET-007) |
| **OQ-014** | converted-manifest doc says `col` is NA for Infinium I | misprint; NA is for Infinium II (MET-003) |
| **OQ-015** | cyto LOH example: CHROM `21`, ID on chr22 | doc error; our IDs always equal CHROM (CYTO-029) |
| **OQ-016** | "GAINLOH" example carries `LOHTYPE=AOH`; LOHTYPE declared `Number=A` | taxonomy per CYTO-006; `Number=1` |
| **OQ-017** | cyto VCF POS convention changed between doc versions (start vs start−1); chromosome token in IDs varies (`chr1` vs `19`) | VCF-correct padding base; IDs use the CHROM string (CYTO-029) |
| **OQ-018** | positive SVLEN for deletions | signed by default; `--svlen-mode positive` (CYTO-029) |
| **OQ-019** | interplay of the minQual 20 reporting gate and the Q10 FILTER | both kept: FILTER in the VCF, gate in JSON/ISCN (CYTO-013) |
| **OQ-020** | `LowFraction` threshold undocumented; old 5 % cut-off removed | default 0.10 [HYP]; calibrate by simulation (CYTO-021) |
| **OQ-021** | `annotationType` documented as an int, shown as a string | string `constitutional` (CYTO-033) |
| **OQ-022** | `medianLogRDev` undocumented | replaced by the defined `logr_dev_robust` (CYTO-033) |
| **OQ-023** | GAINLOH included in sample-level LOH counts but excluded at chromosome level | included at both levels, plus separate counts (CYTO-033) |
| **OQ-024** | reference JSON: numbers as strings, `"."` sentinels, keys such as `"CN:"`, inconsistent casing, a table/example mismatch (`id` vs `hgvs`) | typed JSON, clean snake_case keys (OUT-075, PGX-028) |
| **OQ-025** | a summary JSON example shows a `Sex` key where the CSV has `MetaData1` | doc error; pass-through keys are verbatim sheet headers (IN-013) |
| **OQ-026** | SULT1A1 CN-record ID coordinates differ from POS/END; the region definition changed between versions | ID and POS/END always consistent; region from our BED (PCN-001/011) |
| **OQ-027** | two CYP2A6 regions share the same NR value in an example (per-gene aggregation?) | NR computed per region (PCN-009) |
| **OQ-028** | meaning of `OverallPloidy` in the targeted CN VCF | length-weighted mean over target regions [INF]; revisit with data |
| **OQ-029** | "CN summary" file content undocumented beyond three columns | defined by us (PCN-013) |
| **OQ-030** | het-excess: plan's k-regularised formula vs `(f_ab−2pq)/(2pq+ε)` | both written (OUT-072); test which matches the reference locus summaries |
| **OQ-031** | locus-summary `#No_Calls` described as loci rather than samples | counted over samples (OUT-072) |
| **OQ-032** | exact definition of the assay-control "Norm R" | `xn + yn` of the control locus [INF]; verify against summaries from PGx demo data (P5 experiment) |
| **OQ-033** | the vendor `.msk` and `.dat` model formats are undocumented | not parsed; open formats defined (IN-050/051/052) |
| **OQ-034** | ISCN notation for mosaic microarray results | `x<a>~<b>` [HYP]; check against the ISCN 2020 text before v2 release |
| **OQ-035** | germline vs somatic wording for LOH types on arrays | mosaic ≡ somatic, non-mosaic ≡ constitutional (CYTO-006) |
| **OQ-036** | methylation detection-p method, beta offset, dye-bias method and noob parameters undocumented | open published methods (MET-010); compare against an open-source toolkit |
| **OQ-037** | "proportion of probes passing" listed as adjustable yet "not used for pass/fail" | not used by default; opt-in (MET-008) |
| **OQ-038** | the cloud GenCall cutoff default is not stated | 0.15 everywhere |
| **OQ-039** | whether "median probe spacing" for effective size is per chromosome or genome-wide | per chromosome [INF] (CYTO-010) |
| **OQ-040** | how segment QUAL aggregates probe posteriors | mean marginal posterior [OURS] (CYTO-012) |
| **OQ-041** | whether "min GS" for merged records includes no-call probes | includes them [INF] (OUT-032); verify on reference VCFs if any become available |
| **OQ-042** | exact semantics of "Infinium I no-call as missing signal" | OUT-031 step 5 [INF] |
| **OQ-043** | PAR loci coded `XY` in manifests vs "reported on X" | mapped to X PAR coordinates (IN-046) |
| **OQ-044** | the reference claims VCF 4.1 while using 4.3 FILTER syntax | we write VCF 4.3 consistently (OUT-041) |
| **OQ-045** | cyto model contents beyond GC ("primarily GC") | our model: GC plus optional reference LRR statistics (IN-051) |
| **OQ-046** | EX-chemistry threshold template values not archived | `genotyping_ex` = `genotyping` until calibrated on GSA v4 data |
| **OQ-047** | haploid handling of chrY for female samples in the SNV VCF | haploid missing `.` (OUT-018) |
| **OQ-048** | the cluster-mismatch policy flipped between reference versions | configurable, error by default (IN-050) |
| **OQ-049** | CNV minimum-size defaults changed across versions (0 → 20 kb → 25/50 kb) | latest values adopted; changelog rule (OUT-077) |
| **OQ-050** | a known issue (manifest names > 80 characters) vanished from the notes without being marked fixed | covered by EDGE-006 |
| **OQ-051** | coverage page groups APOE and HLA with the star-allele genes (19); the parsed table classifies them as named-haplotype and tagging | the parsed classification is used (PGX-008) |
| **OQ-052** | the whole-chromosome-only rule at LRR SD > 0.3 has no stated coverage criterion | ≥ 90 % of the effective span [OURS] (CYTO-014) |
| **OQ-053** | the documented batch-composition advice (same reagent lots; a 2-week window) is not enforced by the reference | warnings only (PCN-006) |
| **OQ-054** | IDAT field that holds the scanner ID/version | to be confirmed on real IDATs (P1 experiment, plan U2 family) |
---
## 13. Traceability
### 13.1 Spec item → master plan § → phase
Generated from the item tags (`build.py`). Phases: P0–P10 as in plan §18; `v2` = the plan's v2 modules.
| ID | Title | Plan § | Phase | Sources |
|---|---|---|---|---|
| SCOPE-001 | Purpose | §1 | P0 | plan |
| SCOPE-002 | Emulation targets [EMU] | §1 | P0 | A-G1, C-G1 |
| SCOPE-003 | Deliberate differences [DIFF] | §1 | P0 | plan |
| SCOPE-004 | Out of scope | §1.3 | P0 | A-G13, D-26, D-27 |
| SCOPE-005 | Species and builds | §1 | P0 | A-G62, C-G22 |
| SCOPE-006 | Product test matrix | §16 | P8 | C-G28 |
| SCOPE-007 | Product-file compatibility matrix | §1 | P10 | A-G63 |
| SCOPE-008 | User runtime requirements | §3 | P10 | A-G12, A-G13, A-G43 |
| SCOPE-009 | Batch limits | §19 | P9 | D-9, D-31 |
| SCOPE-010 | Semi-custom and custom arrays | §11 | v2 | A-G42, D-20 |
| CLI-001 | Command tree | §14 | P1 | A-G1, A-G44 |
| CLI-002 | Syntax rules | §14 | P1 | A-G11 |
| CLI-003 | Migration aliases | §14 | P10 | A-G1 |
| CLI-004 | Global options | §14 | P1 | A-G2, A-G3, A-G7 |
| CLI-005 | Option precedence | §14, §15 | P1 | A-G30, E-47 |
| CLI-006 | Input-set rule | §14 | P1 | A-G5 |
| CLI-007 | Logging | §14 | P1 | A-G2, A-G3 |
| CLI-008 | Warning rate limiting | §14 | P1 | B-5 |
| CLI-009 | Run summary | §14 | P1 | E-18 |
| CLI-010 | Overwrite policy [DIFF] | §14 | P1 | A-G4, E-6 |
| CLI-011 | Atomic writes | §14 | P1 | plan |
| CLI-012 | Never discard results [DIFF] | §14 | P1 | A-G10 |
| CLI-013 | Threading | §14, §15 | P1 | A-G7 |
| CLI-014 | Exit codes | §14 | P1 | A-G10 |
| CLI-015 | Interruption | §14 | P1 | plan |
| CLI-016 | Message catalogue | §14 | P1 | A-G56, B-5, B-33, C-G21 |
| CLI-017 | Paths | §14 | P1 | E-4 |
| CLI-018 | Output layout | §13 | P1 | A-G44 |
| CLI-019 | Configuration file | §14 | P1 | D-30 |
| CLI-020 | `simurg inspect <file>...` | §14 | P1 | plan |
| CLI-021 | `simurg genotype` | §14 | P4 | A-G7, A-G14 |
| CLI-022 | `simurg gtc-to-vcf` | §14, §8 | P6 | A-G8, A-G9, A-G18, A-G19, A-G20 |
| CLI-023 | `simurg export …` | §14 | P6 | A-G26 |
| CLI-024 | `simurg qc controls` | §14, §9 | P5 | A-G27, C-G2 |
| CLI-025 | `simurg qc samples \| loci \| contamination \| relatedness` | §14, §9 | P5 | plan |
| CLI-026 | `simurg qc report` | §14, §9.5 | P5 | A-G28, A-G29, A-G30, A-G31, C-G6, C-G17, C-G20 |
| CLI-027 | `simurg qc suggest-thresholds --data … [--k 4] [--min-samples 30] -… | §9 | P5 | C-G8 |
| CLI-028 | `simurg cn call` | §10 | v2 | A-G45, A-G47, A-G48, C-G24, D-2, D-3 |
| CLI-029 | `simurg cn annotate` | §10 | v2 | A-G49, A-G53, A-G54, D-2 |
| CLI-030 | `simurg cn build-model --bpm --fasta [--csv] --out model.simurg-cyto` | §10 | v2 | A-G52 |
| CLI-031 | `simurg pgx copy-number call` | §11 | v2 | A-G33, D-11, E-39 |
| CLI-032 | `simurg pgx copy-number train` | §11 | v2 | A-G34, A-G35, A-G36 |
| CLI-033 | `simurg pgx star-allele call` | §11 | v2 | A-G39, A-G41, B-28, E-40 |
| CLI-034 | `simurg pgx phenotype` | §11 | v2 | A-G41, B-28, D-16 |
| CLI-035 | `simurg pgx db build --sources DIR --manifests BPM[,…] --out FILE` | §11 | v2 | A-G40 |
| CLI-036 | `simurg methyl qc` | §1.2 | v2 | C-G26, D-21, D-22 |
| CLI-037 | `simurg pipeline run\|plan\|watch config.yaml` | §14 | P6 (watch: v2) | D-27, D-30 |
| CLI-038 | `compare`, `synth`, `train` | §14 | P7, P1 | plan |
| CLI-039 | Cross-step guards | §14 | P6 | A-G9, A-G39 |
| CLI-040 | Workflow recipes | §14 | P10 | A-G64, A-G65 |
| IN-001 | Format | §6.6 | P1 | A-G58, B-31, E-8 |
| IN-002 | Key columns | §6.6 | P1 | A-G59, E-8 |
| IN-003 | Case-insensitive columns | §6.6 | P1 | A-G56, E-9 |
| IN-004 | Empty header cells | §6.6 | P1 | E-9 |
| IN-005 | Reserved column names | §6.6 | P1 | A-G56, E-9 |
| IN-006 | Length limit | §6.6 | P1 | A-G56, E-9 |
| IN-007 | Barcode and position validation | §6.6 | P1 | A-G56, E-9 |
| IN-008 | RootFolder | §6.6 | P1 | A-G57, E-8 |
| IN-009 | Sample identity | §6.6 | P1 | B-31, D-24 |
| IN-010 | Sample-name precedence [EMU] | §6.6 | P1 | A-G16, E-10 |
| IN-011 | Per-command resolution | §6.6 | P1 | A-G16, E-10 |
| IN-012 | Output stem | §13 | P1 | plan |
| IN-013 | Metadata pass-through | §6.6 | P1 | A-G15, B-9, E-11 |
| IN-014 | Reported sex | §6.6 | P1 | A-G59 |
| IN-015 | Plate, well and group | §6.6 | P1 | A-G59, A-G60, C-G3 |
| IN-016 | Sheet vs files | §6.6 | P1 | plan |
| IN-017 | Scale | §6.6, §19 | P1 | E-7 |
| IN-018 | Parent/replicate columns | §6.6 | P5 | plan |
| IN-020 | Recursive IDAT discovery | §6.1 | P1 | A-G6, E-5 |
| IN-021 | Pairing | §6.1 | P1 | A-G6 |
| IN-022 | Duplicate pairs [DIFF] | §6.1 | P1 | A-G6, D-28, E-5 |
| IN-023 | IDAT validation | §6.1 | P1 | plan |
| IN-024 | IDAT metadata extraction | §6.1, §9 | P1 | B-12, C-G3 |
| IN-025 | NextSeq 550 IDATs | §6.1 | P1 | A-G61, C-G24 |
| IN-026 | Scan-registration gate [OURS] | §14 | v2 | D-26 |
| IN-027 | IDAT ↔ manifest check [DIFF] | §6, §7.1 | P2 | D-29, E-3 |
| IN-030 | GTC discovery | §6.5 | P6 | A-G6 |
| IN-031 | GTC acceptance | §6.5 | P6 | A-G22, B-34 |
| IN-032 | Corrupt GTC [DIFF] | §6.5 | P4 | E-2 |
| IN-033 | Tolerant string decoding | §6.5 | P1 | E-1 |
| IN-040 | BPM | §6.2 | P1 | E-4 |
| IN-041 | CSV manifest | §6.3 | P1 | A-G23, A-G36, D-24 |
| IN-042 | BPM ↔ CSV consistency [DIFF] | §6.3 | P1 | E-3 |
| IN-043 | BPM ↔ EGT consistency [DIFF] | §6.4 | P4 | B-33, E-3 |
| IN-044 | EGT load errors | §6.4 | P1 | B-33 |
| IN-045 | Cluster-file names | §6.4, §12 | P1 | A-G17, E-4 |
| IN-046 | Manifest mapping conventions | §6.3, §8.1 | P6 | A-G23, E-19 |
| IN-047 | Product-file provenance | §15 | P1 | plan |
| IN-048 | Mask file | §6 | v2 | A-G36 |
| IN-049 | Pre-flight check table | §6, §14 | P4 | A-G63, B-20, E-3 |
| IN-050 | PGx CN model file | §11 | v2 | A-G37, B-20, D-11, E-39 |
| IN-051 | Cyto model file | §10 | v2 | A-G52 |
| IN-052 | Probe mask (open format) | §11 | v2 | A-G35 |
| IN-053 | PGx DB bundle | §11 | v2 | A-G40, D-15 |
| IN-054 | Annotation bundle | §10.4 | v2 | A-G53 |
| IN-055 | YAML inputs | §9 | P5 | C-G6 |
| IN-060 | FASTA + index | §8 | P6 | A-G24, B-33 |
| IN-061 | Contig naming | §8.2 | P6 | A-G24, E-22 |
| IN-062 | Alt contigs for PGx | §11 | v2 | D-10 |
| IN-063 | Build detection | §8 | P6 | B-6 |
| IN-064 | Genome cache | §8, §19 | P6 | A-G8 |
| IN-065 | Species profile | §1, §8 | P6 | C-G22 |
| OUT-001 | GTC v5 writer | §6.5 | P4 | E-1 |
| OUT-002 | GTC content rules | §6.5 | P4 | A-G16, E-39 |
| OUT-003 | Interoperability | §6.5, §16 | P4 | plan |
| OUT-010 | File naming | §8.3, §13 | P6 | A-G9, A-G25 |
| OUT-011 | Exact header | §8.3 | P6 | A-G39, B-6 |
| OUT-012 | Record defaults [EMU] | §8.3 | P6 | B-1, B-2 |
| OUT-013 | Number formatting | §8.3 | P6 | B-7 |
| OUT-014 | Record order | §8.3 | P6 | B-6, E-27 |
| OUT-015 | REF/ALT assignment | §8.1, §8.3 | P6 | B-5 |
| OUT-016 | BAF/LRR orientation | §8.3 | P6 | E-21 |
| OUT-017 | Unmerged probes carry full FORMAT | §8.3 | P6 | B-3 |
| OUT-018 | Ploidy | §8.3 | P6 | E-20 |
| OUT-019 | Skip taxonomy | §8.3 | P6 | B-5, E-18 |
| OUT-020 | Indels | §8.1 | P6 | B-4, E-14, E-19 |
| OUT-021 | SNV and indel at the same position | §8.3 | P6 | E-15 |
| OUT-030 | Site grouping | §8.3 | P6 | E-12 |
| OUT-031 | Genotype combination algorithm | §8.3 | P6 | A-G20, B-3, E-12, E-13 |
| OUT-032 | Merged record content | §8.3 | P6 | B-3, E-17 |
| OUT-033 | Merged record ID | §8.3 | P6 | E-17 |
| OUT-034 | `--unsquash-duplicates` | §8.3 | P6 | A-G21, E-25 |
| OUT-035 | Auxiliary loci (MNV restoration) | §8.3 | P6 | A-G18, E-16 |
| OUT-036 | Locus filter lists | §8.3 | P6 | A-G19 |
| OUT-037 | `--inf1-nocall-informative` | §8.3 | P6 | A-G20, B-3, E-13 |
| OUT-038 | Index | §8.3 | P6 | E-23 |
| OUT-039 | Validation | §16 | P6 | E-24 |
| OUT-040 | Multi-sample BCF | §8.3 | P6 | plan |
| OUT-041 | VCF version | §8.3 | P6 | E-24 |
| OUT-042 | Ambiguous-strand loci | §8.2 | P6 | plan |
| OUT-043 | PLINK export | §8.4 | P6 | B-8 |
| OUT-050 | bedGraph tracks | §13 | P6 | A-G26, B-32 |
| OUT-051 | bedGraph without re-genotyping | §13 | P6 | A-G26 |
| OUT-052 | No smoothing of raw tracks [DIFF] | §13 | P6 | D-3, E-28 |
| OUT-060 | `sample_summary.tsv` and `.json` | §13, §9.2 | P4 (QC columns P5) | A-G14, A-G15, B-9, C-G11, E-11 |
| OUT-061 | Failed samples [EMU] | §13 | P4 | B-9, E-2 |
| OUT-062 | Assay-control locus metric | §9.1, §13 | P4 | A-G32, B-9, C-G10, E-26 |
| OUT-063 | Precision | §13 | P4 | plan |
| OUT-064 | JSON form | §13 | P4 | B-9 |
| OUT-065 | CSV form | §13 | P4 | A-G14 |
| OUT-066 | Compatibility column names | §13 | P4 | A-G14, C-G1 |
| OUT-070 | Control tables | §9.1, §13 | P5 | A-G14, C-G1 |
| OUT-071 | Final report export | §6.7 | P6 | B-10, C-G23, E-16 |
| OUT-072 | Locus summary | §7.9 | P6 | B-11, C-G23 |
| OUT-073 | Run manifest | §15 | P1 | E-47 |
| OUT-074 | Audit tables | §8.2, §15 | P6 | B-3, B-5 |
| OUT-075 | Schema discipline [DIFF] | §15 | P6 | B-18, B-35, E-48 |
| OUT-076 | Deterministic ordering | §15 | P6 | E-5, E-27 |
| OUT-077 | Default-change changelog | §15 | P6 | E-47 |
| QC-001 | Control definitions | §9.1 | P5 | A-G27 |
| QC-002 | Canonical control names | §9.1 | P5 | C-G4 |
| QC-003 | Backgrounds and offset | §9.1 | P5 | B-13, E-44 |
| QC-004 | Metric catalogue | §9.1 | P5, v2 | B-13, C-G5, D-25, E-44 |
| QC-005 | Genotyping metric set | §9.1 | P5 | A-G32, D-25 |
| QC-006 | Methylation metric set | §9.1 | v2 | C-G26, D-21 |
| QC-007 | Missing controls | §9.1 | P5 | B-31, D-24 |
| QC-008 | Degenerate values | §9.1 | P5 | plan |
| QC-009 | `controls_raw.tsv` | §9.1 | P5 | B-12, C-G1 |
| QC-010 | Report exclusion list | §9.5 | P5 | C-G4 |
| QC-011 | `controls_qc.tsv` | §9.1 | P5 | B-12, C-G1, C-G3 |
| QC-012 | Control-only QC path | §9, §14 | P5 | A-G27, C-G2 |
| QC-013 | Genotype emits controls | §9, §13 | P5 | A-G14 |
| QC-014 | Adding controls to an existing genotype folder | §14 | P5 | plan |
| QC-020 | Threshold YAML schema | §9 | P5 | A-G30, C-G5, E-44 |
| QC-021 | Validation | §9 | P5 | C-G6, C-G21 |
| QC-022 | Disabled checks | §9 | P5 | A-G30 |
| QC-023 | Precedence and built-ins | §9 | P5 | A-G30, C-G6 |
| QC-024 | Shipped profiles | §9 | P5, v2 | A-G30, B-14, C-G7, D-21 |
| QC-025 | Profile auto-selection | §9 | P5 | B-14, C-G7 |
| QC-026 | Data-driven threshold suggestion | §9 | P5 | C-G8 |
| QC-030 | Functional status | §9.2 | P5 | C-G9, C-G11 |
| QC-031 | Control status | §9.2 | P5 | C-G9 |
| QC-032 | Methylation pass | §9.2 | P5 | D-21, D-22 |
| QC-033 | Sample QC flags | §9.2 | P5 | plan |
| QC-034 | Assay-control metric in QC | §9.2 | P5 | A-G32, C-G10 |
| QC-035 | Recommended downstream gates | §9.2, §11 | P5 | A-G34, A-G65 |
| QC-040 | Input contract | §9.5 | P5 | A-G28, C-G1 |
| QC-041 | `--data` forms | §9.5 | P5 | A-G28, C-G17, C-G21 |
| QC-042 | Labels | §9.5 | P5 | A-G28, C-G17 |
| QC-043 | Dataset types | §9.5 | P5 | C-G18 |
| QC-044 | Output files | §9.5 | P5 | B-15, C-G19 |
| QC-045 | QC table layout | §9.5 | P5 | A-G29, B-15, C-G19 |
| QC-046 | Metadata privacy | §9.5 | P5 | A-G31, C-G20 |
| QC-047 | Technology | §9.5 | P5 | C-G12 |
| QC-048 | Chrome | §9.5 | P5 | C-G12 |
| QC-049 | View A: Control Dashboard | §9.5 | P5 | C-G12 |
| QC-050 | View B: Automated QC | §9.5 | P5 | C-G12, C-G13 |
| QC-051 | Linked selection | §9.5 | P5 | C-G13, E-45 |
| QC-052 | View C: Sample Heatmaps | §9.5 | P5 | C-G12, C-G14, E-45 |
| QC-053 | Chip completeness | §9.5 | P5 | C-G15 |
| QC-054 | View D: Trend Analysis | §9.5 | P5 | C-G12, C-G15 |
| QC-055 | View E: QC Thresholds (modal) | §9.5 | P5 | C-G12 |
| QC-056 | Metadata eligibility | §9.5 | P5 | C-G13, E-45 |
| QC-057 | Scale behaviour | §9.5 | P5 | C-G16, E-45 |
| QC-058 | Multi-dataset behaviour | §9.5 | P5 | A-G28, C-G17 |
| QC-059 | Sex view | §9.5 | P5 | plan |
| QC-060 | Control-metric PCA (optional card) | §9.5 | P5 | A-G60 |
| QC-061 | Robust CSV/TSV reading | §9.5 | P5 | plan |
| QC-062 | Methylation datasets in the report | §9.5 | P5 | C-G26 |
| QC-063 | Acceptance tests | §16 | P5 | C-G21 |
| QC-064 | Contamination and relatedness | §9.3, §9.4 | P5 | plan |
| CYTO-001 | Scope | §10 | v2 | D-7, E-35 |
| CYTO-002 | Inputs | §10 | v2 | A-G52 |
| CYTO-003 | GC-wave correction | §10.1 | v2 | A-G52 |
| CYTO-004 | LRR smoothing | §10.2 | v2 | A-G48, D-3, E-28 |
| CYTO-005 | State space | §10.2 | v2 | A-G45, D-1, E-31 |
| CYTO-006 | Event types | §10.2 | v2 | A-G45, B-16, D-1, E-31 |
| CYTO-007 | Per-type thresholds | §10.2 | v2 | A-G45, B-17, D-2, E-29, E-38 |
| CYTO-008 | Probe floor [DIFF] | §10.2 | v2 | A-G46, E-29 |
| CYTO-009 | Thresholds that exceed the genome | §10.2 | v2 | E-29 |
| CYTO-010 | Effective size | §10.2 | v2 | D-7 |
| CYTO-011 | Size basis [DIFF] | §10.2 | v2 | D-2, E-30 |
| CYTO-012 | QUAL | §10.2 | v2 | B-16, D-5 |
| CYTO-013 | Filters | §10.2 | v2 | A-G49, A-G50, B-17, D-5, E-29 |
| CYTO-014 | Sample-level noise gating | §10.2 | v2 | A-G50, E-33 |
| CYTO-015 | Post-filters | §10.2 | v2 | E-33 |
| CYTO-016 | Excess events | §10.2 | v2 | A-G51, E-33 |
| CYTO-017 | Centromeres and gaps | §10.2 | v2 | D-6, E-35 |
| CYTO-018 | Mosaic fraction (MF) | §10.3 | v2 | A-G47, D-4 |
| CYTO-019 | MF not estimable | §10.3 | v2 | A-G47, E-32 |
| CYTO-020 | Promotion | §10.3 | v2 | A-G47, D-4, E-32 |
| CYTO-021 | Low fraction [DIFF] | §10.3 | v2 | D-4, E-32 |
| CYTO-022 | Sensitivity target | §10.3, §16 | v2 | A-G55, D-4, E-32 |
| CYTO-023 | PAR | §10.2 | v2 | D-6, E-34 |
| CYTO-024 | Sex resolution [DIFF] | §10.2 | v2 | D-6, E-34 |
| CYTO-025 | Overall ploidy | §10.2 | v2 | B-22 |
| CYTO-026 | Copy-number median | §10.2 | v2 | E-37 |
| CYTO-027 | Event statistics | §10.2 | v2 | plan |
| CYTO-028 | File | §10, §13 | v2 | A-G25, B-16 |
| CYTO-029 | Coordinates [DIFF, VCF-correct 1-based] | §10 | v2 | B-16, E-36 |
| CYTO-030 | Exact header | §10 | v2 | A-G47, A-G50, B-16, D-5 |
| CYTO-031 | GT semantics [DIFF] | §10 | v2 | D-5, E-36 |
| CYTO-032 | Sidecar and segments | §10 | v2 | A-G44, A-G54 |
| CYTO-033 | `<sample>.cyto.json` | §10.4 | v2 | A-G53, B-18, D-7, E-37 |
| CYTO-034 | ISCN 2020 generator | §10.4 | v2 | B-19, D-8, E-37 |
| CYTO-035 | Genes and bands | §10.4 | v2 | A-G53, B-18 |
| CYTO-036 | Annotation bundle build | §10.4 | v2 | A-G53 |
| CYTO-037 | Batch outputs | §10 | v2 | B-21, C-G24 |
| CYTO-038 | Model building | §10 | v2 | A-G52 |
| CYTO-039 | Batch scale | §10, §19 | v2 | D-9, E-35 |
| CYTO-040 | Degenerate samples | §10 | v2 | E-35 |
| CYTO-041 | MF refinements | §10.3 | v2 | K-027 |
| CYTO-042 | Per-probe LRR noise | §10.2 | v2 | K-027 |
| PCN-001 | Region catalogue | §11 | v2 | B-23, C-G25, D-10 |
| PCN-002 | Genotype every region [EMU] | §11 | v2 | B-23 |
| PCN-003 | Inputs | §11 | v2 | A-G37 |
| PCN-004 | Batch rules | §11 | v2 | A-G33, C-G25, D-11, E-39 |
| PCN-005 | Model training | §11 | v2 | A-G34, E-39 |
| PCN-006 | Batch-composition warnings [OURS] | §11 | v2 | A-G38 |
| PCN-007 | Model compatibility | §11 | v2 | A-G37, B-20, D-11, E-39 |
| PCN-008 | Mask | §11 | v2 | A-G35 |
| PCN-009 | Calling algorithm [HYP, plan §11; U10] | §11 | v2 | A-G33 |
| PCN-010 | CN-consistent BAF evidence | §11 | v2 | B-30 |
| PCN-011 | PGx CN VCF | §11, §13 | v2 | A-G25, B-23, D-12 |
| PCN-012 | Batch summary | §11, §13 | v2 | B-24, D-12 |
| PCN-013 | Per-sample summary | §11, §13 | v2 | B-21 |
| PCN-014 | Placement | §11 | v2 | A-G39 |
| PCN-015 | Genome build | §11 | v2 | A-G62, D-10 |
| PCN-016 | Memory model | §11, §19 | v2 | A-G43 |
| PCN-017 | Validation | §11, §16 | v2 | C-G25, C-G27 |
| PGX-001 | Allele-definition sources, with pinned versions | §11 | v2 | A-G40, D-15 |
| PGX-002 | DB build | §11 | v2 | A-G40, D-14 |
| PGX-003 | Probe-to-variant mapping | §11 | v2 | D-14 |
| PGX-004 | Multi-allelic extension | §11 | v2 | D-14, E-40 |
| PGX-005 | Build-specific definitions | §11 | v2 | D-14 |
| PGX-006 | `*0` whole-gene deletion | §11 | v2 | D-14 |
| PGX-007 | Tagging-variant genes | §11 | v2 | D-14 |
| PGX-008 | Coverage parity target (gene list and counts) | §11 | v2 | C-G25, D-17 |
| PGX-009 | Array limitations are derived, not hard-coded | §11 | v2 | D-14 |
| PGX-010 | Masked variants | §11 | v2 | D-14 |
| PGX-011 | Input guard | §11 | v2 | A-G39, D-19, E-40 |
| PGX-012 | Variant evidence model | §11 | v2 | B-30, E-41 |
| PGX-013 | Candidate enumeration [DIFF: solver-free] | §11 | v2 | B-26, E-43 |
| PGX-014 | Scoring | §11 | v2 | B-26, D-13, E-42 |
| PGX-015 | Feasible alternatives | §11 | v2 | B-27, E-42 |
| PGX-016 | Low-GS review | §11 | v2 | B-27, D-13 |
| PGX-017 | Array-equivalent alleles | §11 | v2 | B-27, D-13 |
| PGX-018 | Background alleles and variant change | §11 | v2 | B-26, E-40 |
| PGX-019 | No-call handling | §11 | v2 | D-13, E-41 |
| PGX-020 | Unresolved variants | §11 | v2 | B-25, D-13 |
| PGX-021 | Variant-type calls | §11 | v2 | B-29, D-17, E-40 |
| PGX-022 | Nomenclature | §11 | v2 | plan |
| PGX-023 | Definition sanity tests | §11 | v2 | E-40 |
| PGX-024 | Unknown allele in lookup | §11 | v2 | E-40 |
| PGX-025 | Guidelines | §11 | v2 | A-G41, D-16 |
| PGX-026 | Files | §11, §13 | v2 | A-G41, B-25, B-28, D-18 |
| PGX-027 | TSV columns | §11 | v2 | B-25, D-18 |
| PGX-028 | JSON schema (`simurg.pgx/1`) | §11 | v2 | B-26, E-41, E-48 |
| PGX-029 | Determinism | §11, §15 | v2 | D-13, E-27, E-43 |
| PGX-030 | Semi-custom arrays [DIFF] | §11 | v2 | A-G42, D-20 |
| PGX-031 | Novel alleles | §11 | v2 | plan |
| PGX-032 | Validation targets | §11, §16 | v2 | C-G25, C-G27 |
| PGX-033 | No licence, no network [DIFF] | §11 | v2 | A-G13 |
| PGX-034 | `alleles_tested` | §11 | v2 | B-26 |
| PGX-035 | `simurg pgx coverage --bpm --db [--csv]` | §11 | v2 | D-13, D-14 |
| MET-001 | Scope | §1.2, §9 | v2 | B-31, C-G26 |
| MET-002 | Manifest validation | §6.3 | v2 | B-31, D-24 |
| MET-003 | Manifest conversion | §6.3 | v2 | B-31, D-23 |
| MET-004 | Sample sheet | §6.6 | v2 | A-G59, B-31 |
| MET-005 | Raw signal assembly | §7.1 | v2 | B-31, D-23 |
| MET-006 | 21 control metrics | §9.1 | v2 | C-G26, D-21, E-46 |
| MET-007 | Thresholds | §9.1 | v2 | B-14, D-21 |
| MET-008 | Pass/fail | §9.2 | v2 | D-21, D-22 |
| MET-009 | Missing controls | §9.1 | v2 | D-24 |
| MET-010 | Signal processing [OURS]: open reimplementations of published methods | §7 | v2 | B-31, D-22, E-46 |
| MET-011 | Detection threshold | §9 | v2 | D-22 |
| MET-012 | Average signals | §9 | v2 | B-31 |
| MET-013 | Outputs | §13 | v2 | B-31, C-G26, D-23 |
| MET-014 | Report integration | §9.5 | v2 | C-G18 |
| MET-015 | Message catalogue | §14 | v2 | B-31, D-24 |
| MET-016 | Scale | §19 | v2 | D-31, E-46 |
| MET-017 | Custom manifests | §9 | v2 | D-21, E-46 |
| MET-018 | NextSeq 550 IDATs | §9 | v2 | A-G61 |
| EDGE-001 | Locale independence | §16 | P1 | E-1 |
| EDGE-002 | GTCs from other tools with bad strings | §16 | P1 | E-1 |
| EDGE-003 | Corrupt or truncated GTC | §16 | P4 | E-2 |
| EDGE-004 | Malformed IDATs | §16 | P1 | plan |
| EDGE-005 | Duplicate IDAT pairs | §16 | P1 | D-28, E-5 |
| EDGE-006 | Long and odd names | §16 | P1 | E-4 |
| EDGE-007 | Sample-sheet matrix | §16 | P1 | A-G56, A-G57, A-G58, E-7, E-9 |
| EDGE-008 | Manifest mismatch matrix | §16 | P4 | E-3 |
| EDGE-009 | FASTA problems | §16 | P6 | A-G24, B-33, E-22 |
| EDGE-010 | Genome cache on and off | §16 | P6 | A-G8 |
| EDGE-011 | Multi-probe merge scenarios | §16 | P6 | B-3, E-12, E-13, E-17 |
| EDGE-012 | Opposite-strand duplicate indels | §16 | P6 | E-14 |
| EDGE-013 | SNV and indel at the same position | §16 | P6 | E-15 |
| EDGE-014 | `--unsquash-duplicates` on a multi-allelic duplicated design | §16 | P6 | A-G21, E-25 |
| EDGE-015 | Auxiliary loci | §16 | P6 | A-G18, E-16 |
| EDGE-016 | MNV designs in the final-report export | §16 | P6 | E-16 |
| EDGE-017 | BAF orientation | §16 | P6 | E-21 |
| EDGE-018 | Indel flips | §16 | P6 | B-4, E-19 |
| EDGE-019 | Skip taxonomy | §16 | P6 | B-5, E-18 |
| EDGE-020 | Index metadata | §16 | P6 | E-23 |
| EDGE-021 | VCF validity | §16 | P6 | E-24 |
| EDGE-022 | Cross-platform determinism | §15, §16 | P6 | E-27 |
| EDGE-023 | bedGraph values | §16 | P6 | E-28 |
| EDGE-024 | Cyto degenerate and regression cases | §16 | v2 | A-G46, A-G51, E-29, E-30, E-31, E-32, E-33, E-34, E-35 |
| EDGE-025 | CNV VCF coordinates | §16 | v2 | B-16, E-36 |
| EDGE-026 | PGx CN batch and model rules | §16 | v2 | A-G33, A-G34, E-39 |
| EDGE-027 | Star-allele regressions | §16 | v2 | E-40, E-41, E-42, E-43 |
| EDGE-028 | QC report error and behaviour tests | §16 | P5 | C-G21, E-45 |
| EDGE-029 | Methylation | §16 | v2 | D-24, E-46 |
| EDGE-030 | Overwrite and resume | §16 | P1 | A-G4, E-6 |
| EDGE-031 | Interruption | §16 | P1 | plan |
| EDGE-032 | GTCs from other producers | §16 | P6 | A-G22, B-34 |
| EDGE-033 | IDAT naming | §16 | P1 | plan |
| EDGE-034 | Non-human diploid | §16 | P8 | C-G22 |
| EDGE-035 | Missing inputs | §16 | P4 | plan |
| EDGE-036 | Final-report size guard | §16 | P6 | B-10, D-30 |
| EDGE-037 | Assay-control column | §16 | P4 | B-9, E-26 |
| EDGE-038 | Unknown sex in the VCF | §16 | P6 | E-20 |
| EDGE-039 | Schema conformance | §16 | P1 | B-35, E-48 |
| EDGE-040 | Metadata redaction | §16 | P5 | A-G31, C-G20 |
| PERF-001 | Genotyping throughput | §19 | P6 (Python), P9 (Rust) | C-G27 |
| PERF-002 | PGx throughput | §19 | v2 | C-G27 |
| PERF-003 | Cyto throughput | §19 | v2 | D-9 |
| PERF-004 | Memory | §19 | P6 | A-G8, A-G43 |
| PERF-005 | Reference hardware | §3, §19 | P10 | A-G12 |
| PERF-006 | Scaling | §19 | P9 | D-31 |
| PERF-007 | Sample sheet | §19 | P1 | E-7 |
| PERF-008 | QC report | §19 | P5 | C-G16, E-45 |
| PERF-009 | Accuracy targets | §16.5 | P8, v2 | C-G25, C-G27 |
| PERF-010 | Large outputs | §19 | P6 | B-10 |
| PERF-011 | Methylation | §19 | v2 | E-46 |
| OQ-001 | The reference marks the IDAT folder "required" and also says "folde… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-002 | `--filter-loci` is described as "generates a text file" | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-003 | bedGraph described as LRR-only in one place, LRR + BAF in another | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-004 | PGx pass boundary: `< 0.2` (calling) vs `≤ 0.2` (training, recommen… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-005 | an example command omits a module name; another uses a relative path | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-006 | an older doc lists the cyto DB option under a different name; a typ… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-007 | LRR SD default: 0.15 (YAML example) vs 0.20 (CLI/UI) | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-008 | control threshold conflicts: target removal 5 vs 1; hyb 1.25 vs 1;… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-009 | stringency channel: red (formula text) vs green (UI) | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-010 | UI formula for NSB green uses "Bkg Red" | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-011 | the example `controls.qc_metrics` row has NSB green = red and NP ≈ 0.1 | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-012 | methylation backgrounds: BS-conv I red bg U3–U5 vs U4–U6; specifici… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-013 | methylation pass is "> threshold"; genotyping flags "below" | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-014 | converted-manifest doc says `col` is NA for Infinium I | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-015 | cyto LOH example: CHROM `21`, ID on chr22 | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-016 | "GAINLOH" example carries `LOHTYPE=AOH`; LOHTYPE declared `Number=A` | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-017 | cyto VCF POS convention changed between doc versions (start vs star… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-018 | positive SVLEN for deletions | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-019 | interplay of the minQual 20 reporting gate and the Q10 FILTER | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-020 | `LowFraction` threshold undocumented; old 5 % cut-off removed | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-021 | `annotationType` documented as an int, shown as a string | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-022 | `medianLogRDev` undocumented | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-023 | GAINLOH included in sample-level LOH counts but excluded at chromos… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-024 | reference JSON: numbers as strings, `"."` sentinels, keys such as `… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-025 | a summary JSON example shows a `Sex` key where the CSV has `MetaData1` | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-026 | SULT1A1 CN-record ID coordinates differ from POS/END; the region de… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-027 | two CYP2A6 regions share the same NR value in an example (per-gene… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-028 | meaning of `OverallPloidy` in the targeted CN VCF | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-029 | "CN summary" file content undocumented beyond three columns | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-030 | het-excess: plan's k-regularised formula vs `(f_ab−2pq)/(2pq+ε)` | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-031 | locus-summary `#No_Calls` described as loci rather than samples | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-032 | exact definition of the assay-control "Norm R" | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-033 | the vendor `.msk` and `.dat` model formats are undocumented | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-034 | ISCN notation for mosaic microarray results | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-035 | germline vs somatic wording for LOH types on arrays | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-036 | methylation detection-p method, beta offset, dye-bias method and no… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-037 | "proportion of probes passing" listed as adjustable yet "not used f… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-038 | the cloud GenCall cutoff default is not stated | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-039 | whether "median probe spacing" for effective size is per chromosome… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-040 | how segment QUAL aggregates probe posteriors | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-041 | whether "min GS" for merged records includes no-call probes | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-042 | exact semantics of "Infinium I no-call as missing signal" | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-043 | PAR loci coded `XY` in manifests vs "reported on X" | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-044 | the reference claims VCF 4.1 while using 4.3 FILTER syntax | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-045 | cyto model contents beyond GC ("primarily GC") | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-046 | EX-chemistry threshold template values not archived | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-047 | haploid handling of chrY for female samples in the SNV VCF | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-048 | the cluster-mismatch policy flipped between reference versions | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-049 | CNV minimum-size defaults changed across versions (0 → 20 kb → 25/5… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-050 | a known issue (manifest names > 80 characters) vanished from the no… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-051 | coverage page groups APOE and HLA with the star-allele genes (19);… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-052 | the whole-chromosome-only rule at LRR SD > 0.3 has no stated covera… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-053 | the documented batch-composition advice (same reagent lots; a 2-wee… | §20 | tracked in `docs/known_unknowns.md` | – |
| OQ-054 | IDAT field that holds the scanner ID/version | §20 | tracked in `docs/known_unknowns.md` | – |
### 13.2 Gap-analysis coverage check
Each of the 207 numbered gap items in the archived analyses (A: G1–G65, B: 1–35, C: G1–G28, D: 1–31, E: 1–48) is listed with the spec items that absorb it. Uncovered items: none.
| Gap item | Spec items |
|---|---|
| A-G1 | SCOPE-002, CLI-001, CLI-003 |
| A-G2 | CLI-004, CLI-007 |
| A-G3 | CLI-004, CLI-007 |
| A-G4 | CLI-010, EDGE-030 |
| A-G5 | CLI-006 |
| A-G6 | IN-020, IN-021, IN-022, IN-030 |
| A-G7 | CLI-004, CLI-013, CLI-021 |
| A-G8 | CLI-022, IN-064, EDGE-010, PERF-004 |
| A-G9 | CLI-022, CLI-039, OUT-010 |
| A-G10 | CLI-012, CLI-014 |
| A-G11 | CLI-002 |
| A-G12 | SCOPE-008, PERF-005 |
| A-G13 | SCOPE-004, SCOPE-008, PGX-033 |
| A-G14 | CLI-021, OUT-060, OUT-065, OUT-066, OUT-070, QC-013 |
| A-G15 | IN-013, OUT-060 |
| A-G16 | IN-010, IN-011, OUT-002 |
| A-G17 | IN-045 |
| A-G18 | CLI-022, OUT-035, EDGE-015 |
| A-G19 | CLI-022, OUT-036 |
| A-G20 | CLI-022, OUT-031, OUT-037 |
| A-G21 | OUT-034, EDGE-014 |
| A-G22 | IN-031, EDGE-032 |
| A-G23 | IN-041, IN-046 |
| A-G24 | IN-060, IN-061, EDGE-009 |
| A-G25 | OUT-010, CYTO-028, PCN-011 |
| A-G26 | CLI-023, OUT-050, OUT-051 |
| A-G27 | CLI-024, QC-001, QC-012 |
| A-G28 | CLI-026, QC-040, QC-041, QC-042, QC-058 |
| A-G29 | CLI-026, QC-045 |
| A-G30 | CLI-005, CLI-026, QC-020, QC-022, QC-023, QC-024 |
| A-G31 | CLI-026, QC-046, EDGE-040 |
| A-G32 | OUT-062, QC-005, QC-034 |
| A-G33 | CLI-031, PCN-004, PCN-009, EDGE-026 |
| A-G34 | CLI-032, QC-035, PCN-005, EDGE-026 |
| A-G35 | CLI-032, IN-052, PCN-008 |
| A-G36 | CLI-032, IN-041, IN-048 |
| A-G37 | IN-050, PCN-003, PCN-007 |
| A-G38 | PCN-006 |
| A-G39 | CLI-033, CLI-039, OUT-011, PCN-014, PGX-011 |
| A-G40 | CLI-035, IN-053, PGX-001, PGX-002 |
| A-G41 | CLI-033, CLI-034, PGX-025, PGX-026 |
| A-G42 | SCOPE-010, PGX-030 |
| A-G43 | SCOPE-008, PCN-016, PERF-004 |
| A-G44 | CLI-001, CLI-018, CYTO-032 |
| A-G45 | CLI-028, CYTO-005, CYTO-006, CYTO-007 |
| A-G46 | CYTO-008, EDGE-024 |
| A-G47 | CLI-028, CYTO-018, CYTO-019, CYTO-020, CYTO-030 |
| A-G48 | CLI-028, CYTO-004 |
| A-G49 | CLI-029, CYTO-013 |
| A-G50 | CYTO-013, CYTO-014, CYTO-030 |
| A-G51 | CYTO-016, EDGE-024 |
| A-G52 | CLI-030, IN-051, CYTO-002, CYTO-003, CYTO-038 |
| A-G53 | CLI-029, IN-054, CYTO-033, CYTO-035, CYTO-036 |
| A-G54 | CLI-029, CYTO-032 |
| A-G55 | CYTO-022 |
| A-G56 | CLI-016, IN-003, IN-005, IN-006, IN-007, EDGE-007 |
| A-G57 | IN-008, EDGE-007 |
| A-G58 | IN-001, EDGE-007 |
| A-G59 | IN-002, IN-014, IN-015, MET-004 |
| A-G60 | IN-015, QC-060 |
| A-G61 | IN-025, MET-018 |
| A-G62 | SCOPE-005, PCN-015 |
| A-G63 | SCOPE-007, IN-049 |
| A-G64 | CLI-040 |
| A-G65 | CLI-040, QC-035 |
| B-1 | OUT-012 |
| B-2 | OUT-012 |
| B-3 | OUT-017, OUT-031, OUT-032, OUT-037, OUT-074, EDGE-011 |
| B-4 | OUT-020, EDGE-018 |
| B-5 | CLI-008, CLI-016, OUT-015, OUT-019, OUT-074, EDGE-019 |
| B-6 | IN-063, OUT-011, OUT-014 |
| B-7 | OUT-013 |
| B-8 | OUT-043 |
| B-9 | IN-013, OUT-060, OUT-061, OUT-062, OUT-064, EDGE-037 |
| B-10 | OUT-071, EDGE-036, PERF-010 |
| B-11 | OUT-072 |
| B-12 | IN-024, QC-009, QC-011 |
| B-13 | QC-003, QC-004 |
| B-14 | QC-024, QC-025, MET-007 |
| B-15 | QC-044, QC-045 |
| B-16 | CYTO-006, CYTO-012, CYTO-028, CYTO-029, CYTO-030, EDGE-025 |
| B-17 | CYTO-007, CYTO-013 |
| B-18 | OUT-075, CYTO-033, CYTO-035 |
| B-19 | CYTO-034 |
| B-20 | IN-049, IN-050, PCN-007 |
| B-21 | CYTO-037, PCN-013 |
| B-22 | CYTO-025 |
| B-23 | PCN-001, PCN-002, PCN-011 |
| B-24 | PCN-012 |
| B-25 | PGX-020, PGX-026, PGX-027 |
| B-26 | PGX-013, PGX-014, PGX-018, PGX-028, PGX-034 |
| B-27 | PGX-015, PGX-016, PGX-017 |
| B-28 | CLI-033, CLI-034, PGX-026 |
| B-29 | PGX-021 |
| B-30 | PCN-010, PGX-012 |
| B-31 | IN-001, IN-009, QC-007, MET-001, MET-002, MET-003, MET-004, MET-005, MET-010, MET-012, MET-013, MET-015 |
| B-32 | OUT-050 |
| B-33 | CLI-016, IN-043, IN-044, IN-060, EDGE-009 |
| B-34 | IN-031, EDGE-032 |
| B-35 | OUT-075, EDGE-039 |
| C-G1 | SCOPE-002, OUT-066, OUT-070, QC-009, QC-011, QC-040 |
| C-G2 | CLI-024, QC-012 |
| C-G3 | IN-015, IN-024, QC-011 |
| C-G4 | QC-002, QC-010 |
| C-G5 | QC-004, QC-020 |
| C-G6 | CLI-026, IN-055, QC-021, QC-023 |
| C-G7 | QC-024, QC-025 |
| C-G8 | CLI-027, QC-026 |
| C-G9 | QC-030, QC-031 |
| C-G10 | OUT-062, QC-034 |
| C-G11 | OUT-060, QC-030 |
| C-G12 | QC-047, QC-048, QC-049, QC-050, QC-052, QC-054, QC-055 |
| C-G13 | QC-050, QC-051, QC-056 |
| C-G14 | QC-052 |
| C-G15 | QC-053, QC-054 |
| C-G16 | QC-057, PERF-008 |
| C-G17 | CLI-026, QC-041, QC-042, QC-058 |
| C-G18 | QC-043, MET-014 |
| C-G19 | QC-044, QC-045 |
| C-G20 | CLI-026, QC-046, EDGE-040 |
| C-G21 | CLI-016, QC-021, QC-041, QC-063, EDGE-028 |
| C-G22 | SCOPE-005, IN-065, EDGE-034 |
| C-G23 | OUT-071, OUT-072 |
| C-G24 | CLI-028, IN-025, CYTO-037 |
| C-G25 | PCN-001, PCN-004, PCN-017, PGX-008, PGX-032, PERF-009 |
| C-G26 | CLI-036, QC-006, QC-062, MET-001, MET-006, MET-013 |
| C-G27 | PCN-017, PGX-032, PERF-001, PERF-002, PERF-009 |
| C-G28 | SCOPE-006 |
| D-1 | CYTO-005, CYTO-006 |
| D-2 | CLI-028, CLI-029, CYTO-007, CYTO-011 |
| D-3 | CLI-028, OUT-052, CYTO-004 |
| D-4 | CYTO-018, CYTO-020, CYTO-021, CYTO-022 |
| D-5 | CYTO-012, CYTO-013, CYTO-030, CYTO-031 |
| D-6 | CYTO-017, CYTO-023, CYTO-024 |
| D-7 | CYTO-001, CYTO-010, CYTO-033 |
| D-8 | CYTO-034 |
| D-9 | SCOPE-009, CYTO-039, PERF-003 |
| D-10 | IN-062, PCN-001, PCN-015 |
| D-11 | CLI-031, IN-050, PCN-004, PCN-007 |
| D-12 | PCN-011, PCN-012 |
| D-13 | PGX-014, PGX-016, PGX-017, PGX-019, PGX-020, PGX-029, PGX-035 |
| D-14 | PGX-002, PGX-003, PGX-004, PGX-005, PGX-006, PGX-007, PGX-009, PGX-010, PGX-035 |
| D-15 | IN-053, PGX-001 |
| D-16 | CLI-034, PGX-025 |
| D-17 | PGX-008, PGX-021 |
| D-18 | PGX-026, PGX-027 |
| D-19 | PGX-011 |
| D-20 | SCOPE-010, PGX-030 |
| D-21 | CLI-036, QC-006, QC-024, QC-032, MET-006, MET-007, MET-008, MET-017 |
| D-22 | CLI-036, QC-032, MET-008, MET-010, MET-011 |
| D-23 | MET-003, MET-005, MET-013 |
| D-24 | IN-009, IN-041, QC-007, MET-002, MET-009, MET-015, EDGE-029 |
| D-25 | QC-004, QC-005 |
| D-26 | SCOPE-004, IN-026 |
| D-27 | SCOPE-004, CLI-037 |
| D-28 | IN-022, EDGE-005 |
| D-29 | IN-027 |
| D-30 | CLI-019, CLI-037, EDGE-036 |
| D-31 | SCOPE-009, MET-016, PERF-006 |
| E-1 | IN-033, OUT-001, EDGE-001, EDGE-002 |
| E-2 | IN-032, OUT-061, EDGE-003 |
| E-3 | IN-027, IN-042, IN-043, IN-049, EDGE-008 |
| E-4 | CLI-017, IN-040, IN-045, EDGE-006 |
| E-5 | IN-020, IN-022, OUT-076, EDGE-005 |
| E-6 | CLI-010, EDGE-030 |
| E-7 | IN-017, EDGE-007, PERF-007 |
| E-8 | IN-001, IN-002, IN-008 |
| E-9 | IN-003, IN-004, IN-005, IN-006, IN-007, EDGE-007 |
| E-10 | IN-010, IN-011 |
| E-11 | IN-013, OUT-060 |
| E-12 | OUT-030, OUT-031, EDGE-011 |
| E-13 | OUT-031, OUT-037, EDGE-011 |
| E-14 | OUT-020, EDGE-012 |
| E-15 | OUT-021, EDGE-013 |
| E-16 | OUT-035, OUT-071, EDGE-015, EDGE-016 |
| E-17 | OUT-032, OUT-033, EDGE-011 |
| E-18 | CLI-009, OUT-019, EDGE-019 |
| E-19 | IN-046, OUT-020, EDGE-018 |
| E-20 | OUT-018, EDGE-038 |
| E-21 | OUT-016, EDGE-017 |
| E-22 | IN-061, EDGE-009 |
| E-23 | OUT-038, EDGE-020 |
| E-24 | OUT-039, OUT-041, EDGE-021 |
| E-25 | OUT-034, EDGE-014 |
| E-26 | OUT-062, EDGE-037 |
| E-27 | OUT-014, OUT-076, PGX-029, EDGE-022 |
| E-28 | OUT-052, CYTO-004, EDGE-023 |
| E-29 | CYTO-007, CYTO-008, CYTO-009, CYTO-013, EDGE-024 |
| E-30 | CYTO-011, EDGE-024 |
| E-31 | CYTO-005, CYTO-006, EDGE-024 |
| E-32 | CYTO-019, CYTO-020, CYTO-021, CYTO-022, EDGE-024 |
| E-33 | CYTO-014, CYTO-015, CYTO-016, EDGE-024 |
| E-34 | CYTO-023, CYTO-024, EDGE-024 |
| E-35 | CYTO-001, CYTO-017, CYTO-039, CYTO-040, EDGE-024 |
| E-36 | CYTO-029, CYTO-031, EDGE-025 |
| E-37 | CYTO-026, CYTO-033, CYTO-034 |
| E-38 | CYTO-007 |
| E-39 | CLI-031, IN-050, OUT-002, PCN-004, PCN-005, PCN-007, EDGE-026 |
| E-40 | CLI-033, PGX-004, PGX-011, PGX-018, PGX-021, PGX-023, PGX-024, EDGE-027 |
| E-41 | PGX-012, PGX-019, PGX-028, EDGE-027 |
| E-42 | PGX-014, PGX-015, EDGE-027 |
| E-43 | PGX-013, PGX-029, EDGE-027 |
| E-44 | QC-003, QC-004, QC-020 |
| E-45 | QC-051, QC-052, QC-056, QC-057, EDGE-028, PERF-008 |
| E-46 | MET-006, MET-010, MET-016, MET-017, EDGE-029, PERF-011 |
| E-47 | CLI-005, OUT-073, OUT-077 |
| E-48 | OUT-075, PGX-028, EDGE-039 |
### 13.3 Item counts
| Prefix | Items |
|---|---|
| SCOPE | 10 |
| CLI | 40 |
| IN | 52 |
| OUT | 47 |
| QC | 52 |
| CYTO | 40 |
| PCN | 17 |
| PGX | 35 |
| MET | 18 |
| EDGE | 40 |
| PERF | 11 |
| OQ | 54 |
| **Total** | 416 |
Reference in new issue View Git Blame Copy Permalink