Files
SimurgArray/SIMURGARRAY_FUNCTIONAL_SPEC.md
T
kemalandClaude Opus 5.5 210dbaf964
ci / test (3.12) (push) Failing after 9s
ci / test (3.13) (push) Failing after 12s
CN: per-probe LRR noise from the batch reference (CYTO-042), --probe-lrr-sd/--no-probe-lrr-sd
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01814GKVzZVL3FuF3DeiZXhv
2026-09-29 12:00:04 +01:00

239 KiB
Raw Blame History

SimurgArray — Functional Specification

Companion to SIMURGARRAY_MASTER_PLAN.md (the "plan"). The plan defines algorithms, architecture, data sources and the phased roadmap. This document defines externally visible behaviour: commands, options, validation, file schemas, QC semantics, messages, edge cases and acceptance tests. Version 1.0 · 2026-09-27.

It consolidates a documentation-level study of a commercial Infinium array-analysis product (the "reference product"), whose public documentation was read in full. That study is archived under _phase0_archive/site_analysis/ as files A–E. About 200 suggested additions from their gap analyses were deduplicated and merged here. No vendor binary was inspected (plan §2). Documented behaviour is emulated only where it helps users. Documentation errors are listed in §12 and are never turned into requirements.


0. Conventions

0.1 Item format

Every requirement has a stable ID, a trace tag and optional source tags:

- **CLI-021** \§14 · P4` Title. Requirement text … (src: A-G14, C-G1)`

  • ID prefixes: SCOPE scope, CLI command line, IN inputs, OUT genotyping outputs, QC control QC and the QC report, CYTO genome-wide CNV/LOH, PCN targeted PGx copy number, PGX star alleles and phenotypes, MET methylation control QC, EDGE edge cases and regression tests, PERF performance, OQ open questions.
  • Trace tag: the master-plan section(s), then the roadmap phase (P0–P10, or v2 for the plan's v2 modules). The table in §13 is generated from these tags.
  • Source tags (src:) point at gap-analysis items in the archived study. A-G5 means file A, item G5; B-12 means file B, item 12; and so on for C, D and E. plan means the item restates or sharpens the plan. §13.2 checks that every gap item is covered.
  • Keywords: MUST, SHOULD and MAY as in RFC 2119.
  • Behaviour markers:
    • [EMU]: the reference product's documented behaviour, emulated on purpose.
    • [DIFF]: we deliberately do it differently. The reason is always given.
    • [OURS]: the reference documents nothing here, so this is SimurgArray's own definition.
    • [INF] and [HYP]: as in plan §0 (our inference; a hypothesis still to be tested).

0.2 Naming conventions (SimurgArray-wide)

  • CLI flags: lowercase kebab-case only. No mixed case and no case-sensitive look-alikes.
  • File names: lowercase snake_case. Per-sample files are named <sample>.<kind>.<ext>, where <sample> is the output stem defined in IN-012.
  • TSV/CSV column names and JSON keys: snake_case. Reference-style column names are available only through the explicit compatibility options (OUT-066, QC-009).
  • Numbers are always formatted locale-independently: . as the decimal separator and no thousands separators (EDGE-001).
  • Timestamps in our outputs use ISO-8601 UTC (2026-09-27T10:15:02Z). Legacy formats are written only inside GTC fields, for interoperability (OUT-001).
  • Genomic coordinates are 1-based and fully closed in every TSV and JSON output. VCF follows the VCF 4.3 conventions exactly (OUT-041, CYTO-029).

0.3 Relationship to the plan

Where this spec and the plan disagree about I/O, naming, options or messages, this spec wins. Where they disagree about algorithms or numerics, the plan wins. Each such conflict found during implementation MUST be logged in docs/known_unknowns.md, and both documents updated.


1. Scope & compatibility targets

  • SCOPE-001 §1 · P0 Purpose. SimurgArray reproduces the functional envelope of a modern Infinium secondary-analysis pipeline, locally and offline:

    • IDAT → GTC genotyping with control-probe QC and a sample summary;
    • GTC → SNV VCF and bedGraph;
    • a multi-dataset interactive QC report;
    • genome-wide CNV/LOH/mosaic calling with ISCN annotation (v2);
    • targeted PGx copy number and star-allele diplotypes with CPIC/DPWG phenotypes (v2);
    • methylation-array control QC (v2). (src: plan)
  • SCOPE-002 §1 · P0 Emulation targets [EMU]. SimurgArray emulates these reference behaviours, because users and downstream tools depend on them:

    1. Workflow shape:
      • genotype → VCF;
      • genotype (or control-only QC) → QC report;
      • genotype → VCF → targeted CN → star alleles → phenotype;
      • genotype → CNV/LOH → annotation.
    2. Content of the files that move between steps:
      • control raw and QC metric tables;
      • the sample summary, with sheet columns passed through;
      • the SNV VCF (FORMAT GT:GS:BAF:LRR, merged multi-probe records);
      • the CNV VCF information content;
      • the PGx tables.
    3. Numeric defaults:
      • GenCall cutoff 0.15;
      • QC 0.98 / 0.20;
      • per-type CNV size and probe minimums;
      • QUAL cap 60 and filters Q10 / Q7;
      • PGx batch 24/22 and training 96/90;
      • alternative solutions at ≥ 50 % of the top score;
      • low-GS review at 0.15.
    4. Control-probe metric formulas and QC status semantics (two status axes, null disables a check).
    5. Sample-sheet semantics: GenomeStudio-compatible sections, case-insensitive columns, name precedence, RootFolder. (src: A-G1, C-G1)
  • SCOPE-003 §1 · P0 Deliberate differences [DIFF]. SimurgArray MUST differ from the reference product in each of the following. Each has its own item; this table is the index.

# Reference behaviour SimurgArray behaviour Item
1 silently overwrites the output folder refuses unless --force/--resume CLI-010
2 duplicate IDAT pairs: one picked, OS-dependent error E_DUP_IDAT (or a deterministic opt-in rule) IN-022
3 CN probe thresholds < 10 silently reset reset with a warning CYTO-008
4 CNV VCF: REF=N, positive SVLEN for DEL, GT=1/1 placeholder, ID/POS disagreements VCF-4.3-correct padding-base POS, FASTA REF, signed SVLEN, meaningful GT, consistent IDs; --compat modes available CYTO-029–CYTO-031, PCN-011
5 JSON numbers as strings, "." sentinels, stray key punctuation typed JSON, null, JSON-Schema validated OUT-075
6 output order and PGx scores differ by OS byte-identical outputs across OS, CPU and thread count OUT-076, PGX-029
7 licence server; results discarded when over quota offline; computed results are never discarded CLI-012, PGX-033
8 mixed-case, case-sensitive flags lowercase kebab-case CLI-002
9 unknown sex → silently male in CN calling explicit policy + flag CYTO-024
10 no ISCN for LOH/mosaic/≤ 1 kb; first event sometimes unannotated ISCN for every reported event CYTO-034
11 bedGraph smoothing corrupted LRR raw tracks are never smoothed OUT-052
12 own tabix writer without count metadata htslib index with counts OUT-038
13 cluster file names restricted to [A-Za-z0-9_-] any name accepted IN-045
14 BPM/CSV/EGT/IDAT mismatches silent or per-sample pre-flight validation errors IN-042–IN-044, IN-027
15 semi-custom arrays: PGx must run twice single run; PGx uses whatever mapped content exists PGX-030
16 per-step inconsistencies (raw vs effective size; GAINLOH counted in some places only) one consistent rule, documented CYTO-011, CYTO-033
  • SCOPE-004 §1.3 · P0 Out of scope. The following are not provided and MUST NOT be emulated:

    • cloud platform features: workgroups, credits, sharing, notifications, scanner-to-cloud upload, case creation on interpretation platforms;
    • licence servers or quota;
    • polyploid genotyping ("diploid only", as in the reference);
    • oncology CNV interpretation (annotation_type is always constitutional);
    • novel star-allele discovery.

    The local equivalents of two cloud features are in scope as optional v2 conveniences: the scan-registration gate (IN-026) and the planned-run watcher (CLI-037). (src: A-G13, D-26, D-27)

  • SCOPE-005 §1 · P0 Species and builds.

    • Human GRCh37 and GRCh38 for genotyping, VCF, QC and CNV/LOH.
    • PGx is GRCh38-only in v2.0; see PCN-015.
    • Non-human diploid organisms are supported for genotyping, GTC, QC and bedGraph, given a user-supplied FASTA and a species profile (IN-065). The bovine UMD3 build (BovineSNP50) is the non-human acceptance case.
    • A VCF for a non-human species needs a FASTA whose contig names match the manifest. (src: C-G22, A-G62)
  • SCOPE-006 §16 · P8 Product test matrix. Validation (plan §16.5) MUST include at least one dataset per manifest family:

    • genotyping: ASA, CGA, GCRA, GDA, GSA v3, GSA v4 (EX chemistry), JSA, PRS-type (GRCh37 only), BovineSNP50;
    • PGx: GDA-ePGx, GSAv4-ePGx, GCRA-ePGx (all GRCh38);
    • cyto: GDA-Cyto, GSA-Cyto, CytoSNP-850K (both iScan and NextSeq 550 IDATs);
    • methylation: EPIC v1, EPIC v2, MSA.

    Each is run on GRCh37 and GRCh38 where the product supports both. Families with no public data are recorded as gaps in the validation report. (src: C-G28)

  • SCOPE-007 §1 · P10 Product-file compatibility matrix. The user guide MUST publish a matrix linking manifest revision ↔ cluster file ↔ PGx CN model ↔ cyto model ↔ PGx DB ↔ annotation bundle. It lists, for each combination, which checks are errors and which are warnings (IN-049). simurg inspect --compat <files…> prints the same matrix for the files the user supplies. (src: A-G63)

  • SCOPE-008 §3 · P10 User runtime requirements.

    • Platforms: Linux x86-64 and aarch64, macOS arm64. Windows is best-effort (via WSL) and not a release gate.
    • Resources:
      • Genotyping and QC: 4 cores and 8 GB RAM minimum; 8 cores and 32 GB recommended.
      • Disk: ≥ 30 GB free for GSA-size batches of 1 000 samples, plus the annotation bundle (CYTO-036).
      • PGx CN batches: memory scales with the batch, as documented in PCN-016.
    • Operation:
      • No elevated privileges needed; nothing is written outside the output, cache and tmp directories.
      • No network access at run time (offline-first). Downloads happen only in explicit fetch/build helper scripts. (src: A-G12, A-G13, A-G43)
  • SCOPE-009 §19 · P9 Batch limits. There is no hard sample cap. Every command MUST be tested at:

    • ≥ 1 152 samples for genotyping, cyto and methylation;
    • ≥ 384 samples for PGx;
    • 10 000 synthetic samples for genotyping and QC.

    Memory must stay flat in batch size wherever the computation is per-sample. (src: D-31, D-9)

  • SCOPE-010 §11 · v2 Semi-custom and custom arrays.

    • Genotyping, QC, VCF and CNV work on any Infinium genotyping manifest.
    • PGx uses every probe that maps to a PGx-DB variant, whichever part of the design it comes from. A second "commercial-files" run is never required [DIFF].
    • Warnings are issued when a gene's commercial backbone probes are missing (PGX-030). (src: A-G42, D-20)

2. Command-line interface (simurg)

2.1 Structure and global behaviour

  • CLI-001 §14 · P1 Command tree. Plan §14's flat verbs are canonical. Grouped verbs are subcommands.

    simurg
    ├── version                      # also: simurg --version
    ├── inspect <file>... [--compat]
    ├── ingest | normalize           # plan §14 (developer/debug steps)
    ├── genotype                     # IDAT → GTC + controls + sample summary        (P4)
    ├── gtc-to-vcf                   # GTC → <sample>.snv.vcf.gz                      (P6)
    ├── export  bedgraph | plink | finalreport | locus-summary                         (P6)
    ├── qc
    │   ├── controls                 # IDAT + manifest → control tables, no cluster file (P5)
    │   ├── samples | loci | contamination | relatedness                                (P5)
    │   ├── report                   # multi-dataset HTML + table                      (P5)
    │   └── suggest-thresholds                                                         (P5)
    ├── cn                           # alias: cyto                                      (v2)
    │   ├── build-model
    │   ├── call
    │   └── annotate
    ├── pgx                                                                              (v2)
    │   ├── copy-number call | train
    │   ├── star-allele call
    │   ├── phenotype                # alias: annotate
    │   ├── coverage
    │   └── db build
    ├── methyl                                                                           (v2)
    │   ├── qc
    │   └── convert-manifest
    ├── train                        # ArrayTrain (plan §12)                            (v2)
    ├── pipeline run | plan | watch                                                     (P6; watch v2)
    ├── compare                                                                          (P7)
    └── synth                                                                            (P1+)
    

    Every group and leaf accepts --help. simurg <group> with no leaf prints the group help and exits with 2. (src: A-G1, A-G44, plan)

  • CLI-002 §14 · P1 Syntax rules.

    • Options are lowercase kebab-case. Boolean options come in --x/--no-x pairs where a default-on switch exists.
    • List-valued options accept either a comma list with no spaces (a,b; surrounding spaces are trimmed with a warning) or a repeated flag.
    • Paths may contain spaces and non-ASCII characters.
    • Unknown options are errors (exit 2) and come with a "did you mean" suggestion. (src: A-G11)
  • CLI-003 §14 · P10 Migration aliases. A documented table maps reference-style verbs to simurg verbs:

    • genotype call → genotype
    • genotype gtc-to-vcf → gtc-to-vcf
    • genotype gtc-to-bedgraph → export bedgraph
    • qc call → qc controls
    • cyto call|annotate → cn call|annotate
    • pgx copy-number call|train → identical
    • pgx star-allele call → identical
    • pgx star-allele annotate → pgx phenotype

    The two-word forms are accepted as hidden aliases, with an informational note. Mixed-case reference flags (for example the threshold flags) are not accepted; the error message names the simurg equivalent. (src: A-G1)

  • CLI-004 §14 · P1 Global options (accepted by every leaf command).

Option Type Default Validation / notes
--threads N int 0 = auto: physical cores available to the process, respecting CPU affinity and cgroup quota 0–1024; results MUST NOT depend on N (plan §15)
--log-level enum debug|info|warning|error info
--debug flag off sets debug level and includes tracebacks in error records
--log-json (alias --json-log) flag off JSON Lines on stderr, schema in CLI-007
--log-file PATH path <out>/logs/<command>-<run_id>.log always JSON Lines, regardless of --log-json
--quiet / --no-progress flag progress bars only when stderr is a TTY
--seed N int 42 recorded in the run manifest
--config FILE YAML none see CLI-019
--force flag off allows replacing existing outputs (CLI-010)
--resume flag off reuses completed per-sample outputs whose input hashes match (plan §14 content-addressed cache)
--tmp-dir DIR dir <out>/.simurg_tmp removed on success
--help, --version flag --version prints SimurgArray <semver>+<git sha>

(src: A-G2, A-G3, A-G7, plan)

  • CLI-005 §14, §15 · P1 Option precedence.

    1. Explicit CLI option.
    2. The command's section in the --config YAML.
    3. The named profile (QC-024), if any.
    4. The built-in default.

    The only environment variables read are SIMURG_CACHE_DIR (IN-064) and NO_COLOR. The effective value of every option and threshold, with its source (cli|yaml:<path>|profile:<name>|default), MUST be recorded:

    • in run_manifest.json;
    • in VCF headers (##simurg_options=);
    • in JSON outputs (a parameters object);
    • in the QC table's Thresholds sheet (QC-045).

    (src: A-G30, E-47)

  • CLI-006 §14 · P1 Input-set rule. Commands that take a sample set accept a folder (--idats DIR or --gtcs DIR), a sheet (--sheet FILE), or both.

    • Sheet only: the folder is the sheet's RootFolder (IN-008). If it has none → E_NO_INPUT_FOLDER.
    • Folder only: every discovered sample is processed.
    • Both: the sheet selects and annotates samples; the CLI folder overrides RootFolder, with an info message if they differ.
    • Neither: E_NO_INPUT (exit 2). (src: A-G5, plan)
  • CLI-007 §14 · P1 Logging.

    • Human-readable logs go to stderr. With --log-json, each record is one JSON object per line:
      {"schema":"simurg.log/1","ts":"2026-09-27T10:15:02.123Z","level":"warning",
       "code":"W_MERGE_AMBIGUOUS","msg":"Failed to combine genotypes (ambiguous) - rs1 (InfiniumII): AA, rs1_dup (InfiniumI): TT",
       "command":"gtc-to-vcf","run_id":"01J9…","step":"merge","sample_id":"204753010023_R01C01",
       "locus":"rs1","file":null,"count":1,"elapsed_ms":8123,"exc":null}
      
    • exc holds the formatted traceback only with --debug.
    • code is a stable catalogue ID (CLI-016); msg is free text that may change between versions.
    • Unknown fields MUST be ignored by consumers.

    (src: A-G2, A-G3)

  • CLI-008 §14 · P1 Warning rate limiting.

    • Each warning code is printed at most 20 times per sample, then once as a summary: W_… repeated N more times.
    • Every occurrence is still written to the audit tables (plan §8.2), and the run summary counts them per code.

    (src: B-5)

  • CLI-009 §14 · P1 Run summary. Each command ends with a summary:

    • samples succeeded / failed / skipped;
    • counts per warning code;
    • output paths;
    • wall time.

    It goes to stderr and into run_manifest.json (summary object). (src: E-18, plan)

  • CLI-010 §14 · P1 Overwrite policy [DIFF].

    • If any file the command would write already exists, the command stops before doing any work: E_OUTPUT_EXISTS lists up to 10 paths, exit 2.
    • --force replaces them.
    • --resume keeps outputs whose recorded input hashes and parameters match, and recomputes the rest.
    • Files not produced by the current command are never deleted.
    • Exception: qc controls MAY add its control tables to an existing genotype output folder that lacks them (QC-014). (src: A-G4, E-6)
  • CLI-011 §14 · P1 Atomic writes. Every output is written to <name>.part in the same directory, then fsynced and renamed. An interrupted run MUST NOT leave a truncated file under its final name. (src: plan §14)

  • CLI-012 §14 · P1 Never discard results [DIFF]. Per-sample outputs are finalized as soon as each sample completes. A later failure (another sample, batch step or report) does not remove them. Batch files (summary, QC tables) are rewritten at the end, and are also written if the run fails part-way. (src: A-G10)

  • CLI-013 §14, §15 · P1 Threading.

    • Parallelism is per sample (a process pool) plus, inside a sample, vectorized/Rust threads (P9). The total never exceeds --threads.
    • bgzip compression uses htslib worker threads taken from the same budget.
    • Changing --threads MUST NOT change any output byte (EDGE-022). (src: A-G7)
  • CLI-014 §14 · P1 Exit codes.

Code Meaning
0 success; warnings are allowed
1 internal error (a bug). The message asks for a --debug rerun
2 usage or pre-flight validation error; no per-sample work started. Examples: bad option, missing file, schema or mismatch errors, output exists, batch minimum not met
3 partial failure: ≥ 1 sample failed and ≥ 1 succeeded. Outputs for the successful samples are complete, and the failed samples are listed in the summary
4 the run completed but no sample succeeded (extension of plan §14)
130 / 143 interrupted by SIGINT / SIGTERM (CLI-015)

(src: A-G10, plan)

  • CLI-015 §14 · P1 Interruption. On SIGINT/SIGTERM:

    1. Stop scheduling new work.
    2. Let in-flight atomic writes finish or abort them; either way no partial final files.
    3. Write run_manifest.json with status: "interrupted".
    4. Exit with 130/143.

    A later --resume continues the run. (src: plan)

  • CLI-016 §14 · P1 Message catalogue. Every error and warning has a stable code. The catalogue (docs/messages.md, generated from simurg/core/errors.py) is part of the acceptance tests (EDGE-*). Minimum set:

Code Level Trigger
E_NO_INPUT, E_NO_INPUT_FOLDER error CLI-006
E_OUTPUT_EXISTS error CLI-010
E_FILE_NOT_FOUND, E_FILE_UNREADABLE error any missing or unreadable input, e.g. a manifest path
E_SHEET_* (DUP_COLUMN, RESERVED, TOO_LONG, BARCODE_FORMAT, NAME_EMPTY, NAME_COLLISION, DUP_SAMPLE, NO_KEY_COLUMNS, EMPTY, DELIMITER) error IN-001 – IN-016
W_SHEET_EMPTY_COLUMN, W_SHEET_BARCODE_REPAIRED, W_SHEET_ROOTFOLDER_OVERRIDDEN warning IN-004, IN-007, IN-008
E_DUP_IDAT, W_IDAT_UNPAIRED, E_IDAT_FORMAT, E_IDAT_ENCRYPTED, W_IDAT_NAME_MISMATCH error / warning IN-020 – IN-023
E_IDAT_MANIFEST_MISMATCH, W_IDAT_LOW_ADDRESS_COVERAGE error / warning IN-027
E_GTC_CORRUPT (sample skipped), W_GTC_FOREIGN, E_GTC_UNSUITABLE warning / error IN-031 – IN-033
E_BPM_CSV_MISMATCH, E_EGT_MANIFEST_MISMATCH, E_EGT_LOAD, W_EGT_NAME error / warning IN-042 – IN-045
E_FAI_MISMATCH, E_CONTIG_MISMATCH, E_BUILD_MISMATCH, E_PGX_ALT_CONTIG_MISSING error IN-060 – IN-063
W_REF_NOT_QUERIED, W_SKIP_UNMAPPED, W_SKIP_INTENSITY_ONLY, W_SKIP_INDEL_CONTEXT, W_SKIP_INDEL_REF, W_INDEL_PARTIAL_MATCH, W_SKIP_EXCLUDED warning OUT-019
W_MERGE_INCONSISTENT, W_MERGE_AMBIGUOUS, W_MERGE_ALL_NC, W_INF1_NC_CONFLICT warning OUT-031
E_AUX_INVALID error OUT-035
W_CTRL_MISSING, W_CTRL_ZERO_DIVISOR warning QC-007, QC-008
E_QC_* (NO_DATASETS, DATASET_INCOMPLETE, LABEL_COUNT, LABEL_DUPLICATE, MIXED_TYPES, FORMAT, CONFIG_MISSING, CONFIG_EXT, CONFIG_PARSE, CONFIG_STRUCTURE, THRESHOLD_KEY, THRESHOLD_VALUE, THRESHOLD_RANGE) and W_QC_LABEL_SANITIZED, W_QC_UNMATCHED_SAMPLES error / warning QC-020 – QC-043
E_MODEL_MANIFEST_MISMATCH, E_MODEL_CLUSTER_MISMATCH / W_MODEL_CLUSTER_MISMATCH, E_MODEL_VERSION error / warning IN-050, IN-051
E_PGX_BATCH_TOO_SMALL, W_PGX_BATCH_BELOW_RECOMMENDED, E_PGX_TRAIN_TOO_SMALL error / warning PCN-004, PCN-005
E_PGX_VCF_UNSUITABLE, E_PGX_VCF_UNINDEXED, E_PGX_CN_MISSING error PGX-011
W_CN_PROBE_FLOOR, W_CN_THRESHOLD_EXCEEDS_GENOME, I_CN_SEX_RESOLVED, W_CN_SEX_UNRESOLVED, E_CN_SIDECAR_MISSING, E_CN_VCF_UNINDEXED mixed CYTO-*
E_METH_* and W_METH_* mixed MET-015

(src: B-33, C-G21, A-G56, B-5)

  • CLI-017 §14 · P1 Paths.

    • Relative paths on the CLI resolve against the CWD. Relative paths inside a sheet or config resolve against that file's directory.
    • Symlinks are followed, with loop detection.
    • Output paths are printed as absolute paths. (src: E-4)
  • CLI-018 §13 · P1 Output layout.

    • Default --layout nested (plan §13): per-sample files go under <out>/samples/<sample>/, batch files under <out>/.
    • --layout flat writes everything into <out>/, as the reference does.
    • Every downstream command discovers inputs recursively, so both layouts work interchangeably. This includes the star-allele step, which needs SNV and CN VCFs "in one folder" in the reference. (src: A-G44, plan)
  • CLI-019 §14 · P1 Configuration file.

    • --config takes YAML with an optional version: 1. It has one top-level section per command (genotype:, gtc_to_vcf:, qc:, cn:, pgx:, methyl:) plus runtime:.
    • Keys are the snake_case option names. Unknown keys → E_CONFIG_KEY (exit 2).
    • pipeline run uses the same file, extended with inputs:, reference: and outputs: (plan §14 example). In outputs:, gtc, vcf, bedgraph, plink, final_report, locus_summary, qc_report: true/false.
    • pipeline plan warns when final_report: true and loci × samples > 1e8, and suggests turning it off (about 400 GSA-size samples ≈ 50 GB). (src: D-30, plan)

2.2 Commands

  • CLI-020 §14 · P1 simurg inspect <file>...

    • Prints the header/TOC of IDAT, BPM, CSV manifest, EGT, GTC, sheet, VCF, CN model, PGx DB or annotation bundle.
    • --json gives machine-readable output.
    • --compat prints the compatibility matrix for the given files (SCOPE-007). (src: plan)
  • CLI-021 §14 · P4 simurg genotype (IDAT → GTC + control tables + sample summary).

Option Type Default Validation
--idats DIR dir – CLI-006; recursive discovery, IN-020
--sheet FILE CSV – IN-001 …
--bpm FILE BPM required IN-040
--egt FILE EGT required IN-043/IN-044
--csv FILE CSV manifest optional if given, checked against the BPM (IN-042) and used for control names (QC-001)
--out DIR dir required CLI-010
--cutoff X float 0.15 0 ≤ X ≤ 1; GenCall score < X → no-call
--norm-version enum 1.2.0|1.1.2 1.2.0 plan §7.2
--exact/--clean flag --exact plan §0
--gtc/--no-gtc flag on off keeps calls only in the Parquet output
--controls/--no-controls flag on writes controls_raw.tsv, controls_qc.tsv (QC-009, QC-011)
--summary-format enum tsv|csv|both tsv plus .json always
--compat-column-names flag off OUT-066
--on-duplicate-idat enum error|first|newest error IN-022
--assay-control-probes list TGA_Ctrl_5716 OUT-062
--species-profile name/file human IN-065

Outputs: GTC (OUT-001), sample_summary.{tsv,json} (OUT-060), control tables (QC-009/011), run_manifest.json. (src: A-G14, A-G7, plan)

  • CLI-022 §14, §8 · P6 simurg gtc-to-vcf.
Option Type Default Validation / notes
--gtcs DIR / --sheet FILE dir / CSV – CLI-006
--bpm FILE BPM required
--csv FILE CSV manifest required for indels must contain SourceSeq. If absent, indels are skipped with W_SKIP_INDEL_CONTEXT and a one-time error-level notice
--fasta FILE FASTA required IN-060
--out DIR dir required
--no-bgzip flag off OUT-010
--genome-cache/--no-genome-cache flag cache on IN-064
--auxiliary-loci FILE VCF – OUT-035
--exclude-loci FILE (alias --filter-loci) / --include-loci FILE text – OUT-036
--unsquash-duplicates flag off OUT-034
--inf1-nocall-informative flag off OUT-037
--merge flag off multi-sample BCF (OUT-040)
--drop-ambiguous flag off plan §8.2
--info-fields list none OUT-012
--extra-format list X,Y,NORMX,NORMY,THETA,R none plan §8.3
--filter-gs X float off adds FILTER LowGS when GS < X
--contig-alias enum strict|auto|FILE strict IN-061
--ploidy-policy enum sex-aware|diploid sex-aware OUT-018
--merge-snv-indel flag off OUT-021

(src: A-G18, A-G19, A-G20, A-G9, A-G8, plan)

  • CLI-023 §14 · P6 simurg export …

    • export bedgraph --gtcs/--sheet --bpm [--no-gzip]: OUT-050.
    • export plink --vcfs|--gtcs … [--multiallelic split|drop]: OUT-043.
    • export finalreport --gtcs --bpm --csv [--matrix]: OUT-071.
    • export locus-summary --gtcs --bpm --egt: OUT-072.

    None of them re-genotypes: they read GTCs plus the manifest. (src: A-G26)

  • CLI-024 §14, §9 · P5 simurg qc controls (control-only QC, no cluster file).

Option Type Default Validation
--idats DIR / --sheet FILE CLI-006
--csv FILE or --bpm FILE manifest one required control definitions (QC-001)
--array-type enum genotyping|methylation required selects the metric set (QC-005/QC-006)
--out DIR dir required QC-014 exception to CLI-010
--bg-offset X float 3000 ≥ 0

(src: A-G27, C-G2)

  • CLI-025 §14, §9 · P5 simurg qc samples | loci | contamination | relatedness. These follow plan §9.2–§9.4 and §7.9 and write sample_qc.tsv, locus_summary.tsv, contamination.tsv and relatedness.tsv. They take --gtcs/--sheet, the manifest and --out. relatedness also takes --pedigree-from-sheet (plan §9.4). (src: plan)

  • CLI-026 §14, §9.5 · P5 simurg qc report.

Option Type Default Validation
--data list of dirs required one folder, a comma list, or one parent folder (QC-041)
--label list folder names count must equal the number of datasets (QC-042)
--config FILE YAML none QC-020/QC-021
--profile NAME enum (QC-024) none
--threshold-call-rate X float 0.98 0 ≤ X ≤ 1
--threshold-lrr-sd X float 0.20 X ≥ 0
--threshold KEY=VALUE repeatable – any QC-020 key; VALUE may be null
--output-format enum xlsx|csv|both xlsx anything else → E_QC_FORMAT
--report-name STEM str qc_report_YYYY_MM_DD / qc_table_YYYY_MM_DD QC-044
--drop-metadata COLS / --keep-metadata COLS / --no-metadata list / flag keep all QC-046
--split-by enum none|dataset|plate|run none QC-057
--out DIR dir CWD

(src: A-G28, A-G29, A-G30, A-G31, C-G6, C-G17, C-G20)

  • CLI-027 §9 · P5 simurg qc suggest-thresholds --data … [--k 4] [--min-samples 30] --out FILE.yaml: QC-026. (src: C-G8)

  • CLI-028 §10 · v2 simurg cn call (alias cyto call).

Option Type Default Range / validation
--gtcs DIR / --sheet FILE CLI-006; GTCs must qualify (IN-031)
--model FILE cyto model required IN-051
--summary FILE sample_summary.tsv auto-discovered next to the GTCs source of sex and LRR SD
--fasta FILE FASTA optional; recommended REF padding base (CYTO-029)
--out DIR dir required
--min-del-probes / --min-del-size int / kb 10 / 25 probes 0–250000 (floored to 10, CYTO-008); size 0–250000 kb
--min-dup-probes / --min-dup-size int / kb 10 / 50 same
--min-loh-probes / --min-loh-size int / kb 500 / 3000 same
--min-gainloh-probes / --min-gainloh-size int / kb 300 / 500 same
--min-mdel-probes / --min-mdel-size int / kb 10 / 25 same
--min-mdup-probes / --min-mdup-size int / kb 10 / 50 same
--min-mloh-probes / --min-mloh-size int / kb 500 / 3000 same
--smoothing K int 5 0–1000 probes each side (CYTO-004)
--max-mosaic-fraction F float 1.0 0 < F ≤ 1 (CYTO-020)
--min-mosaic-fraction F float 0.10 0 ≤ F < 1 (LowFraction, CYTO-021)
--filter-qual Q int 10 0–60 (FILTER Q10)
--cn-cap N int 4 3–8
--gc-correct/--no-gc-correct flag on
--unknown-sex-policy enum skip-sex-chromosomes|female|male|error skip-sex-chromosomes CYTO-024
--gt-mode enum vcf|compat vcf CYTO-031
--svlen-mode enum signed|positive signed CYTO-029
--mask-bed FILE BED built-in centromere/gap mask CYTO-017
--no-bgzip flag off warns that cn annotate needs bgzip + index (CLI-039)

(src: A-G45, A-G47, A-G48, D-2, D-3, C-G24)

  • CLI-029 §10 · v2 simurg cn annotate (alias cyto annotate).

    • Required: --vcfs DIR (the cn call output) and --annotation-bundle FILE (CYTO-036).
    • Optional:
      • the same 14 --min-* options, defaulting to the values recorded by cn call; they can only tighten, never loosen;
      • --min-qual Q (default 20, 0–60);
      • --size-basis effective|raw (default effective, CYTO-011);
      • --iscn-min-size bp (default 0);
      • --out DIR. (src: A-G49, A-G53, A-G54, D-2)
  • CLI-030 §10 · v2 simurg cn build-model --bpm --fasta [--csv] --out model.simurg-cyto: CYTO-038. (src: A-G52)

  • CLI-031 §11 · v2 simurg pgx copy-number call.

Option Type Default Validation
--gtcs DIR / --sheet FILE GTCs must qualify (IN-031)
--model FILE PGx CN model required IN-050
--fasta FILE GRCh38 FASTA with alt contigs required IN-062
--out DIR dir required recommended: the SNV VCF folder (PCN-014)
--min-batch N int 24 ≥ 1
--min-passing N int 22 ≤ --min-batch
--recommended-passing N int 96 warning only
--lrr-sd-max X float 0.20 passing means LRR SD < X (PCN-004)
--cluster-mismatch enum error|warn error IN-050
--no-bgzip flag off

(src: A-G33, D-11, E-39)

  • CLI-032 §11 · v2 simurg pgx copy-number train.
Option Type Default Validation
--gtcs / --sheet
--bpm FILE BPM required
--csv FILE CSV manifest optional source of Assay Format
--fasta FILE FASTA required
--mask FILE TSV <bpm-stem>.mask.tsv next to the BPM if present IN-052
--platform enum auto|LCG|EX auto = from the CSV Assay Format heading error if auto and no CSV
--min-samples N / --min-passing N int 96 / 90
--max-samples N int 3000 seeded random subset above this, with a note
--train-lrr-sd-max, --train-call-rate-min, --train-assay-control-min float 0.20 / 0.99 / 1.0 exclusion rules (PCN-005)
--genome-cache/--no-genome-cache flag on
--out DIR dir CWD

(src: A-G34, A-G35, A-G36)

  • CLI-033 §11 · v2 simurg pgx star-allele call.

    • Required: --vcfs DIR[,DIR] (SNV + PGx CN VCFs; recursive; symlinks allowed), --db FILE and --out DIR.
    • Optional:
      • --population NAME (prior; default global);
      • --max-candidates N (10);
      • --feasible-ratio R (0.5);
      • --low-gs X (0.15);
      • --no-cn (run without CN VCFs; CN-dependent genes are reported as not_assessed);
      • --allow-foreign-vcf.
    • Writes <sample>.pgx_calls.parquet plus preliminary TSVs. (src: A-G39, A-G41, B-28, E-40)
  • CLI-034 §11 · v2 simurg pgx phenotype (alias annotate).

    • Required: --calls DIR.
    • Optional: --guideline cpic|dpwg (default cpic) and --out DIR.
    • Writes the TSV set (PGX-026) and <sample>.pgx.json. It can be re-run with another guideline without re-calling. (src: A-G41, B-28, D-16)
  • CLI-035 §11 · v2 simurg pgx db build --sources DIR --manifests BPM[,…] --out FILE: PGX-002. simurg pgx coverage --bpm --db: PGX-035. (src: A-G40)

  • CLI-036 §1.2 · v2 simurg methyl qc.

    • Required: --idats/--sheet, --manifest CSV and --out.
    • Optional:
      • --profile epic|msa|custom;
      • --threshold KEY=VALUE, --config;
      • --pval-threshold (0.05);
      • --bg-offset (3000);
      • --beta-offset (0);
      • --min-prop-passing (off);
      • --pca-probes (10000);
      • --pca-max-samples-full (100);
      • --ffpe (sets restoration to 1).
    • simurg methyl convert-manifest: MET-003. (src: C-G26, D-21, D-22)
  • CLI-037 §14 · P6 (watch: v2) simurg pipeline run|plan|watch config.yaml.

    • plan = dry run: it resolves inputs, validates everything (IN-049) and prints the step DAG, output sizes and warnings.
    • run executes, with --resume support.
    • watch (planned-run mode): polls inputs.idat_dir until every sheet sample has an IDAT pair; with --scan-metrics, a sample whose scan registration fails counts as failed (IN-026).
      • It aborts if failed samples > max_failed_scans (default 0).
      • Otherwise it runs with the passing samples.
      • It writes sample_status.tsv (sample_id, status ∈ {ready, failed, pending}, reason). (src: D-27, D-30)
  • CLI-038 §14 · P7, P1 compare, synth, train follow plan §14, §16.4 and §12. (src: plan)

  • CLI-039 §14 · P6 Cross-step guards.

    • Using --unsquash-duplicates or an exclude/include list in gtc-to-vcf records this in the VCF header; pgx star-allele call refuses such VCFs (PGX-011).
    • pipeline plan also warns before running when the config combines these options with a PGx step.
    • cn call --no-bgzip warns if the configured pipeline contains cn annotate. (src: A 1.14, A-G39, A-G9)
  • CLI-040 §14 · P10 Workflow recipes in the user guide, each as copy-pasteable commands plus a pipeline YAML:

    1. genotyping (IDAT → GTC → VCF);
    2. QC report (genotype or control-only, then the report; multi-run trends);
    3. PGx (genotype → VCF → CN → star alleles → phenotype; optional CN training);
    4. cytogenetics (genotype → cn call → cn annotate → bedGraph);
    5. semi-custom arrays (single run, SCOPE-010);
    6. methylation QC.

    Each recipe states the recommended sample gates (QC-035). (src: A-G64, A-G65)


3. Inputs

3.1 Sample sheet

  • IN-001 §6.6 · P1 Format.

    • The sheet is a CSV file: UTF-8 (a BOM is tolerated), comma-delimited, RFC 4180 quoting, LF or CRLF line endings.
    • Optional sections: [Header] (key,value rows), [Manifests] (ignored) and [Data] (the table).
    • A sheet with no section markers is valid: its first non-empty line is the header row.
    • Section names are matched case-insensitively.
    • The file is rejected with E_SHEET_DELIMITER and a hint in two cases:
      • semicolon- or tab-delimited files (e.g. from European Excel);
      • JSON sheets.
    • An empty [Data] section, or a sheet with no rows, → E_SHEET_EMPTY. (src: A-G58, E-8, B-31)
  • IN-002 §6.6 · P1 Key columns.

    • Required: SentrixBarcode_A and SentrixPosition_A.
    • Also accepted, as aliases for the same pair: beadChipName and sampleSectionName.
    • Missing both forms → E_SHEET_NO_KEY_COLUMNS.
    • Optional recognised columns:
      • Sample_ID, Sample_Name, Sample_Plate, Sample_Well, Sample_Group;
      • reported sex: Gender, Sex or demographicSex;
      • Parent1, Parent2, Replicate. (src: A-G59, E-8, plan §6.6)
  • IN-003 §6.6 · P1 Case-insensitive columns.

    • Header names are matched after Unicode case-folding and trimming.
    • Two columns that become equal after folding are an error: E_SHEET_DUP_COLUMN, with the message "Duplicate column '' (column names are case-insensitive). Rename or remove one of: , ". (src: A-G56, E-9)
  • IN-004 §6.6 · P1 Empty header cells. Columns whose header is empty are ignored, with one W_SHEET_EMPTY_COLUMN warning. Examples: SentrixBarcode_A,SentrixPosition_A,, and a single trailing comma. Such columns MUST NOT count as duplicates of each other, and MUST NOT produce empty-key fields in any output. (src: E-9)

  • IN-005 §6.6 · P1 Reserved column names. A sheet column must not collide (case-insensitively) with any column that SimurgArray itself writes into the sample summary. That covers:

    • every OUT-060 column name;
    • their reference-style spellings under OUT-066, such as Sex Estimate, Call Rate, Log R Ratio Std Dev and Sample Folder.

    A collision → E_SHEET_RESERVED. With --namespace-sheet-columns, all pass-through columns are prefixed sheet_ instead. (src: A-G56, E-9)

  • IN-006 §6.6 · P1 Length limit. Column names and cell values are limited to 500 characters (Unicode code points). A longer one → E_SHEET_TOO_LONG, naming the row, the column and its length. (src: A-G56, E-9)

  • IN-007 §6.6 · P1 Barcode and position validation.

    • Barcode: after trimming it must match ^[0-9]{10,14}$.
      • Values like 204753010023.0 are repaired to 204753010023, with W_SHEET_BARCODE_REPAIRED.
      • Scientific notation (2.04753E+11, 2,04753E+11) cannot be repaired, because precision is lost. It is rejected with E_SHEET_BARCODE_FORMAT and the hint "format the SentrixBarcode_A column as a number with 0 decimals (or as text) before saving".
    • Position: must match ^[Rr][0-9]{2}[Cc][0-9]{2}$ and is normalised to upper case. (src: A-G56, E-9)
  • IN-008 §6.6 · P1 RootFolder.

    • [Header] RootFolder,<path> names the IDAT (or GTC) root. A relative path resolves against the sheet's directory.
    • A CLI folder overrides it (W_SHEET_ROOTFOLDER_OVERRIDDEN, logged at info level when the two paths are equal).
    • Other [Header] keys are kept verbatim in run_manifest.json under sheet_header. (src: A-G57, E-8)
  • IN-009 §6.6 · P1 Sample identity. sample_id is always <barcode>_<position>. A duplicate sample_id among sheet rows → E_SHEET_DUP_SAMPLE. --on-duplicate-sample drop MAY instead keep the first row and warn, as the methylation pipeline of the reference does. (src: B-31, D-24)

  • IN-010 §6.6 · P1 Sample-name precedence [EMU].

    1. The sheet's Sample_Name, if the column exists.
    2. The sample name stored in the IDAT (field 406), or in the GTC (field 10).
    3. sample_id.

    Rules for Sample_Name when present:

    • It must be non-empty in every row (E_SHEET_NAME_EMPTY).
    • It must not equal a different sample's sample_id (E_SHEET_NAME_COLLISION).
    • It must be unique across the sheet (E_SHEET_NAME_COLLISION) [OURS: required so that output file names are unique]. (src: A-G16, E-10)
  • IN-011 §6.6 · P1 Per-command resolution.

    • Names are resolved independently by each command, and the sheet given to that command is the source of truth.
    • The name inside a GTC is never rewritten: genotype stores the IDAT name, or empty.
    • A later command given a sheet uses the sheet's names in its outputs: VCF sample column, summaries, JSON. (src: A-G16, E-10)
  • IN-012 §13 · P1 Output stem. <sample> in file names is the resolved sample name after sanitisation:

    • characters outside [A-Za-z0-9._-] become _;
    • leading dots are stripped;
    • length is capped at 200.

    If two samples sanitise to the same stem → E_SHEET_NAME_COLLISION. The unsanitised name is kept inside the files. (src: plan §13)

  • IN-013 §6.6 · P1 Metadata pass-through.

    • Every sheet column that is not a key column is copied verbatim into sample_summary.{tsv,json}: the header as written in the sheet, the values as strings, sheet column order preserved.
    • They are placed after the fixed columns (OUT-060).
    • The QC report and QC table receive them from the summary (QC-040). (src: A-G15, E-11, B-9)
  • IN-014 §6.6 · P1 Reported sex.

    • Values from Gender, Sex or demographicSex are normalised: M/Male/1 → M, F/Female/2 → F, everything else (including empty) → U.
    • If more than one of these columns is present and they disagree → warning, and the first in the order Sex, Gender, demographicSex wins.
    • The result goes into sex_reported, and is compared with sex_est in sex_check (OUT-060). (src: A-G59)
  • IN-015 §6.6 · P1 Plate, well and group.

    • Sample_Plate and Sample_Well from the sheet override the IDAT fields 408/409. They drive plate heatmaps (QC-052) and the plate/well columns of controls_qc.tsv.
    • Sample_Group is the default colour field in plots: control PCA (QC-060) and methylation PCA (MET-013). (src: C-G3, A-G59, A-G60)
  • IN-016 §6.6 · P1 Sheet vs files.

    • A sheet row with no matching IDAT/GTC gets a summary row with status MISSING_INPUT and counts toward exit 3/4.
    • Files not listed in the sheet are ignored; an info message gives their count. (src: plan)
  • IN-017 §6.6, §19 · P1 Scale. Parsing and matching are O(n) in rows plus files. A 10 000-row sheet against 10 000 IDAT pairs must parse and match in < 2 s (PERF-007). (src: E-7)

  • IN-018 §6.6 · P5 Parent/replicate columns. Parent1, Parent2 and Replicate feed relatedness and Mendelian checks (plan §9.4). Values refer to Sample_ID, Sample_Name or sample_id, resolved in that order. (src: plan)

3.2 IDAT and GTC discovery

  • IN-020 §6.1 · P1 Recursive IDAT discovery.

    • --idats DIR, or the RootFolder, is searched recursively for <barcode>_<position>_{Grn,Red}.idat and .idat.gz.
    • Matching of the channel token is case-insensitive.
    • Symlinks are followed, with cycle detection.
    • Directory listing order MUST NOT influence any result: paths are sorted bytewise. (src: A-G6, E-5)
  • IN-021 §6.1 · P1 Pairing. A sample needs exactly one Grn and one Red file. A lone channel → W_IDAT_UNPAIRED, and the sample fails with MISSING_INPUT. (src: A-G6)

  • IN-022 §6.1 · P1 Duplicate pairs [DIFF].

    • Two or more pairs that resolve to the same sample_id are not allowed, for example rescans in different subfolders.
    • Default --on-duplicate-idat error → E_DUP_IDAT before any processing. It lists every path of every duplicated sample.
    • first picks the bytewise-smallest path, and newest the most recent IDAT scan date (field 300), with ties broken by path. Both are deterministic and OS-independent, and log a warning. (src: A-G6, D-28, E-5)
  • IN-023 §6.1 · P1 IDAT validation.

    • Checked: magic, version 3, TOC offsets inside the file, and equal N and address sets in Red and Grn.
    • Encrypted (version 1) IDATs → E_IDAT_ENCRYPTED.
    • A barcode/position inside the IDAT (402/404) that disagrees with the file name → W_IDAT_NAME_MISMATCH, and the file name wins. (src: plan §6.1)
  • IN-024 §6.1, §9 · P1 IDAT metadata extraction. For the control table and the report, extract:

    • imaging date (the scan entry of RunInfo 300), converted to ISO-8601;
    • scanner ID and scanner software version (RunInfo/scanner fields);
    • plate (408), well (409), sample ID (406), chip type (403), manifest (401).

    Missing fields become empty. An IDAT with no scan time → W_IDAT_NO_SCAN_TIME (also MET-015). (src: C-G3, B-12)

  • IN-025 §6.1 · P1 NextSeq 550 IDATs.

    • IDATs from NextSeq 550 scans must parse. Different RunInfo layouts and missing fields are tolerated.
    • Tests use CytoSNP-850K NS550 data when it is available.
    • QC thresholds carry a caveat for this scanner (MET-018). (src: A-G61, C-G24)
  • IN-026 §14 · v2 Scan-registration gate [OURS].

    • --scan-metrics FILE reads the scanner's per-section metrics file (metrics.txt). A section with stripe registration < 0.75 is marked failed, and its sample is excluded, with reason scan_registration.
    • The status goes into sample_status.tsv (CLI-037) and sample_summary.tsv. (src: D-26)
  • IN-027 §6, §7.1 · P2 IDAT ↔ manifest check [DIFF]. Before processing, for each chip type:

    • compute the fraction of BPM probe addresses present in the IDAT;
    • coverage < 0.99 → E_IDAT_MANIFEST_MISMATCH (exit 2), with the chip type and manifest name;
    • 0.99 ≤ coverage < 1 → W_IDAT_LOW_ADDRESS_COVERAGE;
    • also compare IDAT field 401/403 with the BPM name, warning only.

    The reference silently produces bad calls in this case. (src: E-3, D-29)

  • IN-030 §6.5 · P6 GTC discovery.

    • --gtcs DIR is searched recursively for *.gtc. The sample key is the GTC Sentrix ID (field 1016); for v3/v4 files, the file-name stem.
    • Duplicate keys → E_DUP_GTC.
    • Sheet matching follows IN-016. (src: A-G6)
  • IN-031 §6.5 · P6 GTC acceptance.

    • gtc-to-vcf, export bedgraph and QC accept GTC v3–v5 from any producer. A GTC whose autocall_version string is not SimurgArray's, or that lacks v5 fields, gets W_GTC_FOREIGN once per batch.
    • cn call and pgx copy-number call|train need BAF/LRR (v ≥ 4), normalization transforms, and the cluster-file name. Anything else → E_GTC_UNSUITABLE. (src: A-G22, B-34)
  • IN-032 §6.5 · P4 Corrupt GTC [DIFF]. A GTC that fails validation is skipped: bad magic, truncation, out-of-range TOC, or an unreadable string. That sample gets E_GTC_CORRUPT at warning level and a FAILED summary row; the others continue, and the exit code is 3. The reference aborts the whole run instead. (src: E-2)

  • IN-033 §6.5 · P1 Tolerant string decoding.

    • All .NET 7-bit-length strings are decoded as UTF-8 with replacement characters.
    • A length prefix that runs past EOF is corruption (IN-032), never an uncaught exception.
    • Locale-corrupted date strings are preserved as bytes, and decoded for display only. (src: E-1)

3.3 Manifest, cluster file and model compatibility

  • IN-040 §6.2 · P1 BPM.

    • Versions 3–5 with locus-entry versions 4/6/7/8 (plan §6.2) are supported.
    • Manifest names of any length are supported (tested at > 80 chars), as are paths with spaces and non-ASCII characters.
    • The manifest name and a SHA-256 go into provenance. (src: E-4)
  • IN-041 §6.3 · P1 CSV manifest.

    • [Heading], [Assay] and [Controls] sections are parsed.
    • Assay Format (e.g. LCG, EX) is read from [Heading]. It supplies the platform default (CLI-032) and the QC profile selection (QC-025).
    • SourceSeq is required for indel REF/ALT resolution.
    • A missing [Controls] or [Assay] section, or missing required columns, gives the errors of MET-002 when the CSV is used for QC. (src: A-G23, A-G36, D-24)
  • IN-042 §6.3 · P1 BPM ↔ CSV consistency [DIFF]. When both are supplied:

    • locus-name sets must be equal;
    • counts must be equal;
    • Chr/MapInfo/SNP/AddressA/AddressB must agree for every locus.

    Any difference → E_BPM_CSV_MISMATCH with counts and the first 10 examples. --allow-manifest-mismatch downgrades this to a warning and uses the BPM values. (src: E-3)

  • IN-043 §6.4 · P4 BPM ↔ EGT consistency [DIFF].

    • Every non-control BPM locus name must be present in the EGT. Missing names → E_EGT_MANIFEST_MISMATCH (pre-flight, exit 2), with the count and the first 10 names. The reference instead skips each sample with a dictionary-key error.
    • EGT loci not in the BPM are ignored, with an info count.
    • A difference between the EGT's manifest_name and the BPM name → warning. (src: E-3, B-33)
  • IN-044 §6.4 · P1 EGT load errors. Any of the following → E_EGT_LOAD, naming the cause:

    • a corrupt file;
    • an unsupported version or data_block_version;
    • a record-count disagreement.

    (src: B-33)

  • IN-045 §6.4, §12 · P1 Cluster-file names.

    • Any file name is accepted as input [DIFF].
    • ArrayTrain outputs whose names contain characters outside [A-Za-z0-9_-] get W_EGT_NAME, as a portability notice for other tools. (src: A-G17, E-4)
  • IN-046 §6.3, §8.1 · P6 Manifest mapping conventions are validated and converted to VCF representation:

    • positions are 1-based on the plus strand;
    • PAR loci are reported on X coordinates (chr XY → X PAR);
    • an insertion is anchored at the base immediately 5′ of the insertion;
    • a deletion is at its most 5′ deleted base;
    • Chr=0 or MapInfo=0 means unmapped.

    Indels are left-normalised against the FASTA, and the anchor base is added as VCF requires. (src: A-G23, E-19)

  • IN-047 §15 · P1 Product-file provenance. Every product file (BPM, CSV, EGT, models, DB, bundle) is recorded in run_manifest.json by name, size and SHA-256. Vendor files are never copied into outputs. (src: plan)

  • IN-048 §6 · v2 Mask file. See IN-052. Assay-format platform codes (LCG, EX) are stored in model headers (IN-050). (src: A-G36)

  • IN-049 §6, §14 · P4 Pre-flight check table. pipeline plan and every command run these checks before starting per-sample work.

Check Severity Code Override
sheet structure, keys, reserved, length, barcode format, names error E_SHEET_* –
duplicate IDAT pairs / GTCs error E_DUP_IDAT, E_DUP_GTC --on-duplicate-idat
IDAT address coverage vs BPM < 0.99 error E_IDAT_MANIFEST_MISMATCH –
IDAT address coverage < 1.0 warning W_IDAT_LOW_ADDRESS_COVERAGE –
BPM vs CSV mismatch error E_BPM_CSV_MISMATCH --allow-manifest-mismatch
BPM loci missing from EGT error E_EGT_MANIFEST_MISMATCH –
EGT manifest name ≠ BPM name warning W_EGT_MANIFEST_NAME –
FASTA/.fai inconsistent error E_FAI_MISMATCH –
FASTA contigs vs manifest chromosomes error (strict) E_CONTIG_MISMATCH --contig-alias auto
genome build (manifest vs FASTA lengths) error E_BUILD_MISMATCH –
PGx: alt contig for GSTT1 missing error E_PGX_ALT_CONTIG_MISSING –
CN/cyto model vs manifest probe set error E_MODEL_MANIFEST_MISMATCH –
PGx CN model vs GTC cluster file error (default) / warning E_/W_MODEL_CLUSTER_MISMATCH --cluster-mismatch warn
model schema version unsupported error E_MODEL_VERSION –
PGx batch size / passing count error / warning E_PGX_BATCH_TOO_SMALL, W_PGX_BATCH_BELOW_RECOMMENDED --min-batch, --min-passing
star-allele VCFs unsquashed / filtered / unindexed error E_PGX_VCF_UNSUITABLE, E_PGX_VCF_UNINDEXED --allow-foreign-vcf (foreign only)
cn annotate: sidecar or index missing error E_CN_SIDECAR_MISSING, E_CN_VCF_UNINDEXED –

(src: A-G63, B-20, E-3)

3.4 Models, masks and databases

  • IN-050 §11 · v2 PGx CN model file (our own format, .simurg-pgxcn). It is a zip that contains:

    • header.json:
      • schema_version, software_version;
      • manifest_name, manifest_revision, probe_set_sha256;
      • cluster_file_name, cluster_file_sha256;
      • genome_build, fasta_contigs_sha256, platform (LCG|EX), mask_sha256;
      • regions_bed_version, n_training_samples, training_exclusions, created.
    • probes.parquet: per-probe reference statistics.
    • regions.bed.
    • mixture.json: the per-region CN mixture parameters.

    Compatibility rules:

    • The manifest probe set must match → otherwise E_MODEL_MANIFEST_MISMATCH.
    • The cluster file must match the one recorded in the input GTCs:
      • by default the mismatch is an error, E_MODEL_CLUSTER_MISMATCH, whose message tells the user to retrain the model (PCN-005);
      • --cluster-mismatch warn downgrades it to W_MODEL_CLUSTER_MISMATCH, logging "Cluster file () differs from the CN model's cluster file ()".

    The reference changed this rule across versions: error, then warning, then error again. Both modes are tested (EDGE-026). (src: A-G37, B-20, D-11, E-39)

  • IN-051 §10 · v2 Cyto model file (.simurg-cyto).

    • Contents: header.json (schema version, manifest name/revision, probe-set hash, build, FASTA hash, GC window sizes) plus probes.parquet (per-probe GC content at 1 kb / 100 kb / 1 Mb, and optional reference LRR statistics).
    • It is independent of the cluster file, and it is built by us (CYTO-038).
    • A probe-set mismatch → E_MODEL_MANIFEST_MISMATCH.
    • An older schema_version → E_MODEL_VERSION, with a rebuild hint. (src: A-G52)
  • IN-052 §11 · v2 Probe mask (open format).

    • A TSV with probe_name<TAB>reason; # starts a comment.
    • It is discovered as <bpm-stem>.mask.tsv next to the BPM, or given with --mask.
    • It is used by PGx CN training and calling, and its SHA-256 is recorded in the model.
    • A vendor .msk file is not parsed, because its format is undocumented (OQ-033). (src: A-G35)
  • IN-053 §11 · v2 PGx DB bundle. A SQLite file named pgx_db-<schema>-<sha12>.sqlite, where sha12 is the first 12 hex digits of the content SHA-256. Its meta table holds the source versions (PGX-001), the builder software version and the full SHA-256. See PGX-002. (src: A-G40, D-15)

  • IN-054 §10.4 · v2 Annotation bundle.

    • A versioned zip simurg-cyto-annot-<build>-<date>-<sha12>.zip, built by scripts/build_annotation_bundle.sh from public sources (CYTO-036).
    • cn annotate checks free disk space before extraction, needing ≥ 2 × the bundle's uncompressed size. (src: A-G53)
  • IN-055 §9 · P5 YAML inputs.

    • The extension must be .yaml or .yml (E_QC_CONFIG_EXT).
    • A file named *.yaml.txt or *.yml.txt gets a targeted hint about editors appending .txt.
    • Parse errors report the line and column (E_QC_CONFIG_PARSE). (src: C-G6)

3.5 Reference FASTA and genome cache

  • IN-060 §8 · P6 FASTA + index.

    • The FASTA may be plain or bgzip-compressed; .fai (and .gzi) are expected next to it.
    • A missing .fai is built into the cache directory (IN-064), never next to a read-only input, with an info message.
    • A .fai whose contig lengths or offsets disagree with the FASTA → E_FAI_MISMATCH. The check samples 3 contigs and verifies line lengths. (src: B-33, A-G24)
  • IN-061 §8.2 · P6 Contig naming.

    • Default --contig-alias strict: manifest chromosome names (1…22, X, Y, XY, MT) must exist verbatim as FASTA contigs, except that XY maps to X.
    • auto applies the alias table 1↔chr1, MT↔chrM, 22_KI270879v1_alt↔chr22_KI270879v1_alt.
    • FILE loads a two-column alias TSV.
    • The VCF uses FASTA contig names. (src: A-G24, E-22)
  • IN-062 §11 · v2 Alt contigs for PGx.

    • PGx commands require a GRCh38 FASTA that contains the GSTT1-carrying alt contig (chr22_KI270879v1_alt or 22_KI270879v1_alt). If it is absent → E_PGX_ALT_CONTIG_MISSING, with a hint to use an "analysis set with alts".
    • Genome-wide steps are unaffected by alt contigs: loci on alt contigs are emitted only when the manifest maps probes there. (src: D-10)
  • IN-063 §8 · P6 Build detection.

    • The build is inferred from the chr1 length in the .fai (249 250 621 → GRCh37; 248 956 422 → GRCh38) and compared with the manifest GenomeBuild. A disagreement → E_BUILD_MISMATCH.
    • Non-human builds are taken from the species profile (IN-065).
    • Written as ##simurg_genome_build=GRCh38. (src: B-6)
  • IN-064 §8, §19 · P6 Genome cache.

    • Default: the needed contigs are loaded once into memory (a NumPy uint8 array per contig, about 3.1 GB for human), shared read-only with worker processes through mmap of a cache file at $SIMURG_CACHE_DIR/genomes/<fasta-sha256>/.
    • --no-genome-cache uses indexed random access through htslib, for low-memory machines or small manifests.
    • Results MUST be identical in both modes. (src: A-G8)
  • IN-065 §1, §8 · P6 Species profile. A YAML file that lists:

    • build name;
    • autosome names;
    • sex-chromosome names;
    • haploid contigs (e.g. MT);
    • PAR intervals;
    • the sex system (XY/ZW/none).

    human (GRCh37, GRCh38) and bovine_umd3 are built in. It controls ploidy (OUT-018), autosomal call rate, the LRR SD locus set and the sex estimate. (src: C-G22)


4. Genotyping outputs

4.1 GTC

  • OUT-001 §6.5 · P4 GTC v5 writer.

    • All TOC entries of plan §6.5 are written, in ascending ID order.
    • Strings use .NET 7-bit varint length prefixes and are ASCII-only for every field SimurgArray generates.
    • Date fields (200, 201) are formatted with an invariant culture: M/d/yyyy h:mm:ss tt, with ASCII digits and AM/PM.
    • The output must be byte-identical under LC_ALL = C, tr_TR.UTF-8, ar_SA.UTF-8, ja_JP.UTF-8 and de_DE.UTF-8 (EDGE-001).
    • Non-ASCII sample names from an IDAT are written as UTF-8 with a correct byte-length prefix. (src: E-1)
  • OUT-002 §6.5 · P4 GTC content rules.

    • Field 10 (sample name) is the IDAT name, never the sheet name (IN-011).
    • Field 100 is the cluster file name. Its SHA-256 is recorded in run_manifest.json and in the per-sample Parquet, so that CN models can compare it (IN-050).
    • Field 300 is SimurgArray <version>.
    • Fields 1006–1015 (call rate, gender, LogR dev, p10GC, extended data, BAF, LRR, percentiles) are always filled. (src: A-G16, E-39)
  • OUT-003 §6.5, §16 · P4 Interoperability. Written GTCs must parse without warnings in BeadArrayFiles, gtc2vcf and Picard, and must round-trip byte-identically through SimurgArray's own reader and writer. (src: plan)

4.2 SNV VCF

  • OUT-010 §8.3, §13 · P6 File naming.

    • The default output is <sample>.snv.vcf.gz with a <sample>.snv.vcf.gz.tbi index. The .snv infix keeps SNV VCFs apart from .cyto.cnv.vcf.gz and .pgx_cn.vcf.gz files in the same folder.
    • With --no-bgzip, the output is a plain <sample>.snv.vcf and no index is written. A warning is logged, because pgx star-allele call and cn annotate need compressed, indexed input.
    • --merge writes a single cohort.snv.bcf with a .csi index. (src: A-G25, A-G9)
  • OUT-011 §8.3 · P6 Exact header (default options). Lines appear in exactly this order:

    ##fileformat=VCFv4.3
    ##source=SimurgArray <semver>
    ##reference=file://<absolute FASTA path>
    ##simurg_genome_build=<GRCh37|GRCh38|species build>
    ##simurg_manifest=<bpm name>;sha256=<hex>
    ##simurg_csv_manifest=<csv name>;sha256=<hex>
    ##simurg_cluster_file=<egt name from GTC>
    ##simurg_options=unsquash_duplicates=false;exclude_loci=none;include_loci=none;inf1_nocall_informative=false;auxiliary_loci=none;ploidy_policy=sex-aware;contig_alias=strict
    ##simurg_run=<run_id>
    ##contig=<ID=1,length=248956422>
    …   (one line per FASTA contig that carries ≥ 1 manifest locus, in .fai order; identical for all samples of a run)
    ##FILTER=<ID=PASS,Description="All filters passed">
    ##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
    ##FORMAT=<ID=GS,Number=1,Type=Float,Description="GenCall score. For records merged from several assays, the minimum GenCall score over the contributing assays">
    ##FORMAT=<ID=BAF,Number=1,Type=Float,Description="B allele frequency, oriented so that 1 means homozygous ALT (REF=A, ALT=B relative to the reference genome)">
    ##FORMAT=<ID=LRR,Number=1,Type=Float,Description="Log R ratio">
    ##SAMPLE=<ID=<sample name>,SentrixID=<barcode_position>>
    #CHROM  POS  ID  REF  ALT  QUAL  FILTER  INFO  FORMAT  <sample name>
    
    • The header contains no fileDate and no timestamps, so reruns stay byte-identical (plan §15).
    • The ##simurg_* lines record the effective options (CLI-005).
    • The ##simurg_csv_manifest line is omitted when no CSV manifest was supplied.
    • Optional INFO/FORMAT header lines appear only when the matching options are used. (src: B-6, A-G39, plan §8.3)
  • OUT-012 §8.3 · P6 Record defaults [EMU].

    • By default QUAL is ., FILTER is PASS and INFO is .. Call quality is carried only in FORMAT/GS.
    • --info-fields ASSAY_TYPE,NORM_BIN,GENTRAIN,CLUSTER_SEP,ILMN_STRAND,PROBE_IDS adds those annotations. This changes plan §8.3, where INFO was always on; it is now opt-in.
    • --filter-gs X adds ##FILTER=<ID=LowGS,…> and sets FILTER to LowGS when GS < X. (src: B-1, B-2)
  • OUT-013 §8.3 · P6 Number formatting. GS uses 4 decimal places (%.4f); BAF and LRR use 8 significant digits (%.8g). NaN is written as .. Formatting is locale-independent and deterministic, using the shortest-round-trip-safe routine from simurg.core.fmt. (src: B-7)

  • OUT-014 §8.3 · P6 Record order.

    • Records are sorted by contig in .fai order, then POS, then REF, then ALT, then ID.
    • Output order is identical for every sample of a batch.
    • The sample column holds the resolved sample name (IN-010). The ##SAMPLE line maps it to the Sentrix ID. (src: B-6, E-27)
  • OUT-015 §8.1, §8.3 · P6 REF/ALT assignment.

    • REF is always the FASTA base (or sequence) at POS. ALT alleles come a priori from the manifest: the probe's alleles are mapped to the plus strand (plan §8.1), with REF removed. They never depend on observed data.
    • Multiple ALT alleles are listed in order of first appearance, following BPM order of the contributing probes and then allele order within each probe.
    • When REF is not among a probe's queried alleles, the record is still written and W_REF_NOT_QUERIED is raised. ALT then lists all queried alleles, so no sample can be 0/0 for this probe alone. This is typical for MNVs; see OUT-035. (src: B-5, plan §8.3)
  • OUT-016 §8.3 · P6 BAF/LRR orientation.

    • For single-probe biallelic records, BAF is re-oriented so that it refers to ALT. Let A and B be the probe's plus-strand alleles:
      • if A = REF and B = ALT, write BAF unchanged;
      • if A = ALT and B = REF, write 1 − BAF.
    • LRR is never changed.
    • Test fixtures MUST include a locus where A = ALT (EDGE-017). This intentionally differs from legacy A/B-oriented reports, where BAF may be 1 − BAF. (src: E-21, B §3.3)
  • OUT-017 §8.3 · P6 Unmerged probes carry full FORMAT. Single-probe records carry GT:GS:BAF:LRR (and extra fields if requested). Merged records carry GT:GS only; see OUT-032. (src: B-3)

  • OUT-018 §8.3 · P6 Ploidy. With --ploidy-policy sex-aware (the default):

    • Haploid genotypes are written:
      • on MT for everyone;
      • on Y and non-PAR X for samples with sex_est = M.
    • Diploid genotypes are written:
      • on PAR loci (X coordinates) for everyone;
      • on autosomes.
    • A heterozygous call in a haploid context is written as . (missing) and counted in the audit table as het_in_haploid.
    • Y loci of F samples are written as haploid . (missing).
    • U samples are treated as diploid on X and haploid on Y/MT, and the header records unknown_sex_policy=diploid_x.
    • --ploidy-policy diploid writes diploid genotypes everywhere except MT. (src: E-20, B §3.2)
  • OUT-019 §8.3 · P6 Skip taxonomy. Each skipped locus produces one warning (rate-limited, CLI-008) and one audit row. Totals per code appear in the run summary.

Code Condition
W_SKIP_UNMAPPED Chr = 0, or position 0 or missing
W_SKIP_INTENSITY_ONLY Intensity_Only = 1 (CN probe; kept in bedGraph LRR)
W_SKIP_INDEL_CONTEXT indel whose inserted or deleted sequence cannot be determined (no CSV, no SourceSeq, malformed brackets)
W_SKIP_INDEL_REF indel whose reference allele cannot be determined from the FASTA
W_INDEL_PARTIAL_MATCH source sequence flanks match the genome only partially (the record is still written if the unambiguous part suffices, otherwise skipped)
W_SKIP_EXCLUDED probe named in --exclude-loci or absent from --include-loci
W_SKIP_CONTIG manifest chromosome absent from the FASTA under the active alias policy

(src: B-5, E-18)

  • OUT-020 §8.1 · P6 Indels.

    • REF and ALT are resolved from the CSV SourceSeq/TopGenomicSeq against the FASTA, left-normalised and anchored (IN-046).
    • When the manifest's deletion normalises to an insertion against the reference, or the other way round, the reference-consistent representation is written. The genotype is remapped, and flipped_indel=true goes into the audit table.
    • Duplicate indel probes designed on opposite strands ([D/I] and [I/D]) whose calls are both heterozygous MUST merge to 0/1, never ./.. The reference had this bug until its v1.4; see the regression test in EDGE-012. (src: B-4, E-14, E-19)
  • OUT-021 §8.3 · P6 SNV and indel at the same position.

    • These are written as separate records by default, which is the reference behaviour.
    • --merge-snv-indel merges them into one multi-allelic record using the OUT-031 algorithm. Probes that query different variant types never constrain each other's alleles, except through the allele sets they query. (src: E-15)

4.3 Multi-probe merging ("squashing")

  • OUT-030 §8.3 · P6 Site grouping. After plus-strand mapping and normalisation, probes whose variant representations share CHROM, POS and REF form one site. For indels the normalised span must also be identical. The site's ALT set is the union of all queried non-REF alleles (OUT-015). Duplicate designs of the same variant, and tri-allelic designs, both become one site. (src: B §3.1, E-12)

  • OUT-031 §8.3 · P6 Genotype combination algorithm (exact).

    1. Allele classes. For a probe p with queried plus-strand alleles {a_p, b_p}, each allele z at the site maps to a probe class c_p(z) ∈ {A, B, ∅}:

      • Infinium II (SNV): the extended base determines the dye: A/T give red, C/G give green. So c_p(z) = A if z has the same dye group as a_p, and B if it has the same dye group as b_p. Every base falls into one of the two classes, because Infinium II never queries an A/T or C/G pair.
      • Infinium I (SNV), and every indel probe [INF]: c_p(z) = A if z = a_p, B if z = b_p, and ∅ otherwise (no extension, so no signal).
    2. Predicted call. For a candidate genotype g, take the multiset of classes of its alleles (1 allele if haploid, 2 if diploid) and drop the ∅ entries:

      • {A} or {A,A} → AA;
      • {B} or {B,B} → BB;
      • {A,B} → AB;
      • empty → NC (no signal).
    3. Candidates. Enumerate every unordered genotype over {REF} ∪ ALT with the site's ploidy (OUT-018).

    4. Constrain. Keep g only if, for every probe whose call is not NC, the predicted call equals the observed call. A predicted NC never equals an observed call. An NC probe constrains nothing by default; it is presumed to be a probe failure.

    5. Optional Infinium-I no-call rule (--inf1-nocall-informative). Let S be the survivors of step 4. For each Infinium I probe with an NC call, S' = {g ∈ S : c_p(z) = ∅ for all z ∈ g}. The NC is read as "missing signal", so g carries none of the probe's queried alleles.

      • If S' is non-empty, S := S'.
      • Otherwise keep S, and raise W_INF1_NC_CONFLICT, because other assays show the allele.
      • The exact semantics are [INF]; see OQ-042.
    6. Outcome.

      • |S| = 1 → call it.
      • |S| = 0 → ./. plus W_MERGE_INCONSISTENT.
      • |S| > 1 → ./. plus W_MERGE_AMBIGUOUS.
      • All probes NC → ./. plus W_MERGE_ALL_NC.

      The message text is Failed to combine genotypes (<reason>) - <probe> (<InfiniumI|InfiniumII>): <AA|AB|BB|NC>, …, with the probes in BPM order. It is also written to the audit table.

    Required tests (EDGE-011):

    • Infinium II [A/G] = AA with Infinium I [T/G] = (T,T) → ./., ambiguous between AT and TT.
    • Infinium I [T/A] giving the T-homozygous call, with Infinium II [A/G] = NC → ./., ambiguous; not a T/T call.
    • A pair that yields 0 survivors → inconsistent.
    • All probes NC.
    • The Infinium-I NC option both on and off.

    (src: B-3, E-12, E-13, A-G20, plan §8.3)

  • OUT-032 §8.3 · P6 Merged record content.

    • FORMAT is GT:GS only. BAF and LRR are omitted from the FORMAT string, not written as ..
    • GS is the minimum GS over all contributing probes, including NC probes [INF, OQ-041].
    • QUAL is ., FILTER is PASS and INFO is ., unless --info-fields is set. (src: B-3, E-17)
  • OUT-033 §8.3 · P6 Merged record ID. The names of all contributing probes, in BPM order, joined by , for SNV sites and by ; for indel sites. (src: B §3.1, E-17)

  • OUT-034 §8.3 · P6 --unsquash-duplicates.

    • Each probe gets its own record with full FORMAT, so the site is not merged.
    • Exception: a site whose union allele set is multi-allelic (≥ 2 ALT) is always merged, even with this flag.
    • The header records unsquash_duplicates=true.
    • For assay validation only; PGx refuses such files (PGX-011). (src: A-G21, E-25)
  • OUT-035 §8.3 · P6 Auxiliary loci (MNV restoration). --auxiliary-loci FILE.vcf is a VCF whose records have ID equal to a manifest probe name, and CHROM/POS/REF/ALT equal to the true variant.

    Validation (each failure → E_AUX_INVALID naming the record; exit 2):

    • (a) exactly one ALT (not multi-allelic);
    • (b) it is an MNV: len(REF) = len(ALT) ≥ 2 and REF ≠ ALT;
    • (c) the probe is the only assay at the site (no duplicates);
    • (d) the bases the probe interrogates match the probe's designed alleles, one each, which fixes the allele mapping: the first base of REF and ALT on the plus strand for plus-strand designs (for example, ATGC[AT/CG]GTAA designed as ATGC[A/C]NNNN: A ↔ AT, C ↔ CG), the last base for minus-strand designs, because the probe extends from the other side (as GTCtoVCF);
    • (e) REF equals the FASTA.

    The aux record replaces the probe's REF/ALT, and its genotype is mapped through (d). The final report's Top alleles for MNV probes are never reverse-complemented by this mapping (EDGE-016). (src: A-G18, E-16)

  • OUT-036 §8.3 · P6 Locus filter lists.

    • --exclude-loci FILE (alias --filter-loci; the reference's wording "generates" is a documentation error, OQ-002) and --include-loci FILE take plain text: one probe name per line, UTF-8, # comments, blank lines ignored.
    • Unknown names produce one warning with a count.
    • The list's SHA-256 is recorded in the header and in provenance. (src: A-G19)
  • OUT-037 §8.3 · P6 --inf1-nocall-informative (default off) implements OUT-031 step 5. The user guide MUST state three things:

    • it raises call rate and false positives;
    • it assumes a cluster file that clusters the "zeroed-out" multi-allelic assays;
    • it is recorded in the header.

    (src: A-G20, B-3, E-13)

  • OUT-038 §8.3 · P6 Index.

    • Indexes are written with htslib: tabix .tbi, or .csi when any contig exceeds 2²⁹ bp.
    • The index includes the count metadata (the pseudo-bin), so bcftools index --nrecords and --stats work.
    • Indexes written by third-party tools are accepted downstream. (src: E-23)
  • OUT-039 §16 · P6 Validation. CI runs bcftools view -H, bcftools index --stats and a VCF-4.3 validator on golden outputs. Any error fails the build. (src: E-24)

  • OUT-040 §8.3 · P6 Multi-sample BCF (--merge). The same records and FORMAT rules as the per-sample VCFs. Sample order is sheet order, or sorted sample_id when there is no sheet. (src: plan)

  • OUT-041 §8.3 · P6 VCF version.

    • SimurgArray writes VCFv4.3 consistently, with ;-separated FILTER lists.
    • It never claims 4.1 while using 4.3 syntax, which is the inconsistency noted in the reference (OQ-044). (src: E-24)
  • OUT-042 §8.2 · P6 Ambiguous-strand loci. Plan §8.2 applies unchanged: an ambiguous A/T or C/G locus without RefStrand gets confidence=low in the audit. --drop-ambiguous removes those loci. (src: plan)

  • OUT-043 §8.4 · P6 PLINK export.

    • simurg export plink splits multi-allelic sites into biallelic records by default (--multiallelic split, equivalent to bcftools norm -m-), or drops them (--multiallelic drop).
    • Merged records with no BAF/LRR are fine for PLINK.
    • The user guide shows the equivalent manual pipeline: bcftools merge → norm -m- → plink2 --vcf. (src: B-8)

4.4 bedGraph

  • OUT-050 §13 · P6 bedGraph tracks. Two files per sample: <sample>.baf.bedgraph.gz and <sample>.lrr.bedgraph.gz (--no-gzip gives plain .bedgraph).

    • Line 1: track type=bedGraph name="<sample> BAF" description="SimurgArray <ver>" visibility=full autoScale=off viewLimits=0:1. For the LRR track, use viewLimits=-2:2.
    • Rows: chrom<TAB>start0<TAB>end<TAB>value, with start0 = POS − 1 and end = POS. Values use %.6g.
    • Order: manifest chromosomes in .fai order, or manifest order when there is no FASTA; then position.
    • Excluded:
      • from both tracks: NaN values, unmapped loci and zero-intensity loci;
      • from BAF only: intensity-only loci (they stay in LRR).
    • Probes at the same position: the mean of their finite values is written, so bedGraph never overlaps [OURS].
    • Orientation: BAF is written in the manifest A/B orientation. bedGraph has no REF/ALT, and this matches other bedGraph producers.

    (src: B-32, A-G26)

  • OUT-051 §13 · P6 bedGraph without re-genotyping. export bedgraph reads GTC + BPM (+ sheet), and does not need an EGT or FASTA. The chromosome naming option --chrom-style manifest|ucsc defaults to manifest. (src: A-G26)

  • OUT-052 §13 · P6 No smoothing of raw tracks [DIFF].

    • Raw BAF/LRR tracks are never smoothed.
    • --smoothed-lrr K MAY add a separate <sample>.lrr_smoothed.bedgraph.gz: a moving median over 2K+1 probes within each chromosome.
    • A golden test asserts that the raw track equals the GTC values bit-for-bit after formatting. The reference once corrupted LRR through its smoothing option. (src: E-28, D-3)

4.5 Sample summary

  • OUT-060 §13, §9.2 · P4 (QC columns P5) sample_summary.tsv and .json. One row per sample in the input set: processed, failed and missing samples all get rows. Columns come in this fixed order:
# Column Type Definition
1 sample_id str barcode_position
2 sample_name str resolved name (IN-010)
3 sample_folder str IDAT directory relative to the input root
4 status enum OK, FAILED, MISSING_INPUT, EXCLUDED_SCAN
5 fail_reason str message code + short text; empty if OK
6 autosomal_call_rate float plan §7.7
7 call_rate float plan §7.5 step 10
8 logr_dev float plan §7.6
9 sex_est enum M/F/U plan §7.8
10 sex_reported enum M/F/U IN-014
11 sex_check enum match, mismatch, unknown
12 gc10, gc50 float plan §7.7
13 n_calls, n_nocalls, n_intensity_only int
14 p05_x, p50_x, p95_x, p05_y, p50_y, p95_y int raw intensity percentiles
15 het_rate float autosomal heterozygosity over calls (P5)
16 contamination float plan §9.3 (P5; empty if not run)
17 <probe>_norm_r float one column per assay-control probe present (OUT-062)
18 functional_status, control_status, control_flag_codes, flags str QC-030/031/033 (P5; empty before P5)
19… sheet columns str verbatim, sheet order (IN-013)

(src: B-9, A-G14, A-G15, E-11, C-G11, plan §13)

  • OUT-061 §13 · P4 Failed samples [EMU]. A sample that fails IDAT → GTC still gets a row:

    • status=FAILED and fail_reason set;
    • autosomal_call_rate = call_rate = 0;
    • logr_dev, <probe>_norm_r, gc10 and gc50 are NaN (NaN in TSV, null in JSON);
    • sex is U;
    • its sheet columns are still passed through. (src: E-2, B-9)
  • OUT-062 §9.1, §13 · P4 Assay-control locus metric.

    • For each probe named in --assay-control-probes (default TGA_Ctrl_5716, the targeted-amplification control of ePGx arrays) that exists in the manifest, <probe>_norm_r is the normalized R xn + yn of that locus (plan §7.3–§7.4) [INF, OQ-032].
    • The column is omitted when no listed probe exists.
    • Values < 1.0 add the flag assay_control_fail (QC-033). (src: A-G32, B-9, C-G10, E-26)
  • OUT-063 §13 · P4 Precision. Rates and LRR SD use 8 decimal places (%.8f); intensities are integers. NaN is written as NaN in TSV. (src: B §2.8)

  • OUT-064 §13 · P4 JSON form. sample_summary.json is an array of flat objects: same keys and order as the TSV, numbers as JSON numbers, NaN as null, sheet values as strings. (src: B-9)

  • OUT-065 §13 · P4 CSV form. --summary-format csv|both writes sample_summary.csv with the same columns, for spreadsheet users. (src: A-G14)

  • OUT-066 §13 · P4 Compatibility column names. --compat-column-names renames the first columns to reference-style headers:

    • Sample ID, Sample Name, Sample Folder;
    • Autosomal Call Rate, Call Rate, Log R Ratio Std Dev;
    • Sex Estimate;
    • <probe> Norm R.

    It also writes gt_sample_summary.csv as an alias file. Our own tools read either form. (src: A-G14, C-G1)

4.6 Other genotyping outputs

  • OUT-070 §9.1, §13 · P5 Control tables. simurg genotype writes controls_raw.tsv and controls_qc.tsv (QC-009, QC-011) into the batch folder by default. The genotype output folder is therefore a complete QC-report dataset (QC-040). (src: A-G14, C-G1)

  • OUT-071 §6.7 · P6 Final report export. simurg export finalreport writes a long-format TSV with one row per locus × sample. Columns:

    • SNP Name, SNP, Sample ID;
    • Allele1 - Top, Allele2 - Top, Allele1 - Forward, Allele2 - Forward, Allele1 - Plus, Allele2 - Plus;
    • GC Score, GT Score, Log R Ratio, B Allele Freq, Chr, Position.

    Allele rules:

    • Top alleles come from the TOP-strand base calls.
    • MNV probes are never reverse-complemented (EDGE-016).
    • For intensity-only loci, BAF is NaN and LRR uses the cluster-mean expected R (plan §7.6).

    --matrix gives a matrix layout. It warns (W_LARGE_OUTPUT) when loci × samples > 1e8, with a size estimate. (src: B-10, C-G23, E-16)

  • OUT-072 §7.9 · P6 Locus summary. locus_summary.tsv has these columns, in order:

    • locus_name, locus_id;
    • n_nocalls, n_calls (both counted over samples), call_freq;
    • aa_freq, ab_freq, bb_freq, minor_freq;
    • gentrain_score, gc50, gc10;
    • het_excess, chi_test_p100, cluster_sep;
    • aa_t_mean, aa_t_sd, ab_t_mean, ab_t_sd, bb_t_mean, bb_t_sd;
    • aa_r_mean, aa_r_sd, ab_r_mean, ab_r_sd, bb_r_mean, bb_r_sd;
    • plus_minus_strand (+, -, U).

    het_excess follows plan §7.9, the k-regularised legacy form. The simpler form (f_ab − 2pq)/(2pq + ε) is written as het_excess_simple, and OQ-030 records the discrepancy between the two. (src: B-11, C-G23)

  • OUT-073 §15 · P1 Run manifest. run_manifest.json follows plan §15, plus:

    • summary (CLI-009);
    • effective_options with sources (CLI-005);
    • sheet_header (IN-008);
    • skipped_loci_by_code;
    • failed_samples[], each with a code;
    • schema_versions of every output.

    (src: E-47, plan)

  • OUT-074 §8.2, §15 · P6 Audit tables. audit/*.parquet holds:

    • one row per harmonisation decision (plan §8.2);
    • one row per merge decision, with the probes, calls, candidate count and outcome code;
    • one row per skip (OUT-019);
    • one row per indel flip (OUT-020).

    (src: B-3, B-5)

  • OUT-075 §15 · P6 Schema discipline [DIFF].

    • Every JSON and TSV output has a versioned schema (schema field in JSON; #simurg_schema= comment line at the top of TSVs, which --no-comment-lines suppresses). JSON-Schema files are kept under schemas/.
    • CI validates the golden outputs against them.
    • Numbers are numbers; missing values are null in JSON and empty or NaN in TSV, as documented per column.
    • Keys contain no punctuation other than _.
    • A schema change bumps its version and is logged in CHANGELOG.md under "Output schema changes". (src: E-48, B-18, B-35)
  • OUT-076 §15 · P6 Deterministic ordering. Every output is sorted by documented keys:

    • samples by input order (sheet order, or sorted sample_id);
    • loci by genome order;
    • genes by name;
    • solutions by rank, then lexical tie-breakers.

    No output depends on hash-map iteration order, directory listing order, the OS or the thread count. Golden tests run on x86-64 and aarch64 (plan U11). (src: E-27, E-5)

  • OUT-077 §15 · P6 Default-change changelog. Any change to a default value of any option or threshold is listed in CHANGELOG.md under "Default changes", with old → new. The reference changed CNV minimum sizes across minor versions without a machine-readable record. (src: E-47)


5. QC

5.1 Control-probe metrics

  • QC-001 §9.1 · P5 Control definitions.

    • Controls are read from the BPM control config (addr[:addr…],Category,Color,Name) or from the CSV [Controls] section (Address,Type,Color_Channel,Name).
    • When both manifests are given, the BPM wins, and differences produce one warning.
    • Each control address gives a raw green = Grn.Mean[addr] and red = Red.Mean[addr], using the raw IDAT means with no normalization. (src: A-G27, plan §9.1)
  • QC-002 §9.1 · P5 Canonical control names.

    • Names are normalised before matching: case-fold, collapse whitespace, remove spaces inside parentheses, and treat Bgnd and Bkg as the same token.
    • They are then mapped to canonical IDs through simurg/qc/control_names.yaml. Examples: Extension (A) → ext_A; Biotin (High) → biotin_high; DNP (Bgnd) → dnp_bkg; NP (A) → np_A; String (PM) → string_pm; NSB (Bgnd) Red → nsb_bkg; Hyb (High) → hyb_high.
    • Methylation control names (BS Conversion I C1, GT Mismatch 1 (PM), …) are mapped the same way.
    • When several beads share one canonical ID and a formula uses that probe as a single value, the mean is used [OURS]. Formulas that say min or max take the min or max over all matching beads. (src: C-G4)
  • QC-003 §9.1 · P5 Backgrounds and offset.

    • bkg_grn = the maximum green intensity over Extension (A) and Extension (T).
    • bkg_red = the maximum red intensity over Extension (C) and Extension (G).
    • Offset x = 3000, configurable with --bg-offset or YAML bgOffset, value ≥ 0.

    (src: B-13, E-44)

  • QC-004 §9.1 · P5, v2 Metric catalogue. There are 24 metrics. Table columns:

    • G = computed for genotyping arrays; M = computed for methylation arrays.
    • YAML key is the threshold key used in QC-020.
    • Every metric uses direction below: a value < threshold is flagged.
# Column (controls_qc.tsv) YAML key Formula G M
1 restoration restoration Restoration green / (bkg_grn + x) ✓ ✓
2 staining_green stainingGreen Biotin (High) green / Biotin (Bkg) green ✓ ✓
3 staining_red stainingRed DNP (High) red / DNP (Bkg) red ✓ ✓
4 extension_green extensionGreen min(Ext C, Ext G) green / max(Ext A, Ext T) green ✓ ✓
5 extension_red extensionRed min(Ext A, Ext T) red / max(Ext C, Ext G) red ✓ ✓
6 hyb_high_med hybGreenHighMed Hyb (High) green / Hyb (Medium) green ✓ ✓
7 hyb_med_low hybGreenMediumLow Hyb (Medium) green / Hyb (Low) green ✓ ✓
8 target_removal_1 targetRemovalI (bkg_grn + x) / Target Removal (1) green ✓ ✓
9 target_removal_2 targetRemovalII (bkg_grn + x) / Target Removal 2 green – ✓
10 stringency stringency String (PM) red / String (MM) red (channel configurable, OQ-009) ✓ –
11 nsb_green nonSpecificBindingGreen (bkg_grn + x) / max NSB (Bkg) green ✓ –
12 nsb_red nonSpecificBindingRed (bkg_red + x) / max NSB (Bkg) red ✓ –
13 np_green npGreen min(NP C, NP G) green / max(NP A, NP T) green ✓ ✓
14 np_red npRed min(NP A, NP T) red / max(NP C, NP G) red ✓ ✓
15 specificity_1_green specificityIGreen min(GT Mismatch 1–3 PM) green / max(GT Mismatch 1–3 MM) green – ✓
16 specificity_1_red specificityIRed min(GT Mismatch 4–6 PM) red / max(GT Mismatch 4–6 MM) red – ✓
17 specificity_2 specificityII min(Specificity 1–3 red) / max(Specificity 1–3 green) – ✓
18 specificity_2_bg specificityIIBg (bkg_grn + x) / max(Specificity 1–3 green) (OQ-012) – ✓
19 bs_conv_1_green bisulfiteConversionIGreen min(BS Conv I C1, C2) green / max(BS Conv I U1, U2) green – ✓
20 bs_conv_1_green_bg bisulfiteConversionIGreenBg (bkg_grn + x) / max(BS Conv I U1, U2) green – ✓
21 bs_conv_1_red bisulfiteConversionIRed min(BS Conv I C3–C5) red / max(BS Conv I U3–U5) red – ✓
22 bs_conv_1_red_bg bisulfiteConversionIRedBg (bkg_red + x) / max(BS Conv I U3–U5) red (OQ-012) – ✓
23 bs_conv_2 bisulfiteConversionII min(BS Conv II 1–4 red) / max(BS Conv II 1–4 green) – ✓
24 bs_conv_2_bg bisulfiteConversionIIBg (bkg_grn + x) / max(BS Conv II 1–4 green) – ✓

Notes:

  • This corrects plan §9.1, where NSB red must use bkg_red.

  • Restoration and Target Removal II are new relative to the plan.

  • Genotyping has 13 control metrics plus the assay-control metric (assayControl, OUT-062). Methylation has 21 (MET-006). (src: B-13, D-25, C-G5, E-44)

  • QC-005 §9.1 · P5 Genotyping metric set = rows 1–8 and 10–14 of QC-004, plus assay_control_norm_r, which is copied from the sample summary when present. (src: A-G32, D-25)

  • QC-006 §9.1 · v2 Methylation metric set = the 21 rows with M ✓. The same computation code is shared with MET-006. (src: C-G26, D-21)

  • QC-007 §9.1 · P5 Missing controls.

    • Columns that do not apply to the array type are omitted from controls_qc.tsv.
    • When a control a metric needs is absent from the manifest, the metric is NaN and W_CTRL_MISSING is raised once per batch. The warning lists the missing control types or names, for example on custom chips.
    • These metrics are marked not evaluable. They are excluded from pass/fail and listed in na_metrics (QC-031). (src: B-31, D-24)
  • QC-008 §9.1 · P5 Degenerate values.

    • A zero denominator raises W_CTRL_ZERO_DIVISOR. With a positive numerator the value is +Inf (written Inf), which passes every below check: signal over a perfectly clean background is not a failure (old scanners write background-subtracted intensities of exactly 0; K-021). 0/0 gives NaN.
    • Unlike QC-007, a NaN caused by the signal (the probe exists on the chip) counts as out of range, so the sample is flagged with the metric's code.
    • Bead addresses listed by the manifest but absent from the IDAT are skipped; a control none of whose addresses is on the chip is missing (QC-007), not a signal failure.
    • Negative intensities cannot occur (uint16 input). (src: plan) [OURS]
  • QC-009 §9.1 · P5 controls_raw.tsv is long format, with one row per control bead per sample:

    • columns: sample_id, category, bead_address, control_name, color_label, green_intensity, red_intensity;
    • row order: sample order, then manifest control order.

    --compat-column-names writes controls.raw_metrics.csv instead, with columns SampleId, Category, Beadtype, Control, Color, GreenIntensity, RedIntensity. All controls are written, including NEGATIVE and NORM. The exclusions of QC-010 apply only to the report. (src: B-12, C-G1)

  • QC-010 §9.5 · P5 Report exclusion list. Raw-probe views and the QC table's raw-intensity columns leave out:

    • the categories NEGATIVE and NORM;
    • the methylation-only NP (G) 1–NP (G) 5;
    • Biotin (5K) and DNP (20K).

    Derived metrics are still computed from every probe they need. The list is configurable in control_names.yaml. (src: C-G4)

  • QC-011 §9.1 · P5 controls_qc.tsv has one row per sample. Columns, in order:

    • sample_id, array_type;
    • imaging_date (ISO-8601), sample_plate, sample_well, scanner_id, scanner_version: taken from the IDAT (IN-024), with plate and well overridden by the sheet;
    • bg_offset;
    • the applicable metric columns, in QC-004 order.

    --compat-column-names writes controls.qc_metrics.csv with SampleId, ImagingDate, SamplePlate, SampleWell, ScannerId, ScannerVersion, RestorationQC, StainingGreenQC, …. Its date format is M/D/YYYY h:mm:ss AM/PM. (src: B-12, C-G1, C-G3)

  • QC-012 §9, §14 · P5 Control-only QC path. simurg qc controls produces QC-009 and QC-011 from IDATs plus a manifest. It needs no cluster file and does no genotyping, and it works for both genotyping and methylation arrays. The resulting folder is a valid QC-report dataset with functional QC = NA. (src: A-G27, C-G2)

  • QC-013 §9, §13 · P5 Genotype emits controls. See OUT-070. (src: A-G14)

  • QC-014 §14 · P5 Adding controls to an existing genotype folder. qc controls --out <folder> MAY write into a folder that already contains a sample_summary, for example from an older SimurgArray version or an external converter, without --force, as long as the control tables themselves do not exist yet. The sample IDs must match the summary; unmatched IDs produce a warning. This mirrors the reference's legacy path. (src: C §1.2)

5.2 Threshold configuration

  • QC-020 §9 · P5 Threshold YAML schema. Two top-level spellings are accepted:

    • qc: → report:
    • QC: → Report:, for files exported by other tools.
    qc:
      profile: genotyping            # optional; see QC-024
      report:
        callRate: 0.98               # number enables; null (or empty) disables
        logRDev: 0.20
        stainingGreen: 5
        stringency: {threshold: 2, direction: below}   # mapping form allows direction
        restoration: null
        assayControl: 1              # alias: tgaControl
        bgOffset: 3000
    
    • Keys: callRate, logRDev, assayControl, bgOffset, and every YAML key in QC-004.
    • Aliases: tgaControl = assayControl; the plan's snake_case call_rate_min = callRate and logr_dev_max = logRDev.
    • direction is below (flag if value < threshold) or above (flag if value > threshold). Defaults are above for logRDev and below for every other key.
    • A value exactly equal to the threshold passes. (src: A-G30, C-G5, E-44)
  • QC-021 §9 · P5 Validation. Each of these errors exits with code 2:

    • the file must exist (E_QC_CONFIG_MISSING: "QC config file was provided but does not exist: ");
    • the extension must be .yaml or .yml (E_QC_CONFIG_EXT);
    • it must be valid YAML (E_QC_CONFIG_PARSE);
    • the wrapper qc/QC → report/Report must be present (E_QC_CONFIG_STRUCTURE);
    • only known keys are allowed (E_QC_THRESHOLD_KEY, which lists the valid keys);
    • values must be numeric or null (E_QC_THRESHOLD_VALUE);
    • callRate must be in [0, 1] and every other threshold ≥ 0 (E_QC_THRESHOLD_RANGE).

    The same checks apply to --threshold KEY=VALUE and to YAML uploaded into the HTML report (QC-055). (src: C-G6, C-G21)

  • QC-022 §9 · P5 Disabled checks. A null or empty value disables a check. Disabled checks are excluded from both status axes, from threshold lines, from heatmap binary colouring and from the table's conditional formatting. (src: A-G30, C 1.8)

  • QC-023 §9 · P5 Precedence and built-ins.

    • Order, highest first: CLI (--threshold-call-rate, --threshold-lrr-sd, --threshold KEY=VALUE), then the --config YAML, then --profile (or the YAML's profile:), then built-in.
    • Built-in (no profile) [EMU]: callRate = 0.98 and logRDev = 0.20 are enabled. Every other threshold is unset.
    • Example: the YAML sets callRate: 0.90 and the CLI sets 0.95 → 0.95 is used, with source cli.
    • Each effective threshold is recorded with its source (CLI-005, QC-045). (src: A-G30, C-G6)
  • QC-024 §9 · P5, v2 Shipped profiles (simurg/qc/profiles/*.yaml) are starting points for review, not truths. The documentation says they came from the centre and spread of background distributions in a small set of good and failed datasets, and that they should be recalibrated with QC-026. Values where the reference's own documents disagree are listed in OQ-007/OQ-008.

Key genotyping (LCG/HTS) genotyping_ex (EX; provisional, OQ-046) genotyping_example (alternative values) methylation_epic methylation_msa
callRate / logRDev 0.98 / 0.20 0.98 / 0.20 0.98 / 0.15 – –
stainingGreen / Red 5 / 5 5 / 5 5 / 5 5 / 5 5 / 5
extensionGreen / Red 5 / 5 5 / 5 5 / 5 5 / 5 5 / 5
hybGreenHighMed / MediumLow 1 / 1 1 / 1 1.25 / 1.25 1 / 1 1 / 1
targetRemovalI / II 1 / – 1 / – 5 / – 1 / 1 1 / 1
restoration null (1 with +ffpe) null null 0 (1 with +ffpe) 0 (1 with +ffpe)
stringency 2 2 1.5 – –
nonSpecificBindingGreen / Red 1 / 1 1 / 1 5 / 5 – –
npGreen / npRed 5 / 5 5 / 5 5 / 5 5 / 5 2.5 / 3
assayControl 1 1 1 – –
bisulfiteConversionIGreen / Red – – – 1 / 1 1 / 1
bisulfiteConversionIGreenBg / RedBg – – – 1 / 1 0.5 / 0.5
bisulfiteConversionII / IIBg – – – 1 / 1 0.5 / 0.5
specificityIGreen / Red / II / IIBg – – – 1 / 1 / 1 / 1 1 / 1 / 1 / 1
bgOffset 3000 3000 3000 3000 3000
  • The suffix +ffpe (e.g. --profile methylation_epic+ffpe) sets restoration: 1.

  • Methylation profiles also carry pvalThreshold: 0.05 (MET-011). (src: C-G7, B-14, D-21, A-G30)

  • QC-025 §9 · P5 Profile auto-selection. --profile auto picks the profile from the array type (dataset detection, QC-043) and the manifest Assay Format: EX → genotyping_ex, and everything else → genotyping. For methylation it goes by manifest name: EPIC → methylation_epic, MSA → methylation_msa. The report shows which profile was used for each dataset. (src: B-14, C-G7)

  • QC-026 §9 · P5 Data-driven threshold suggestion. simurg qc suggest-thresholds reads one or more datasets. For every enabled or known metric it computes a robust centre m (the median) and a spread s = 1.4826·MAD over samples, on the log scale for ratio metrics.

    • It suggests exp(m − k·s) for below keys and exp(m + k·s) for above keys, with k = 4 by default.
    • It needs ≥ 30 samples per metric; otherwise the metric is left unset, with a note.
    • Output: a YAML file (QC-020 schema, with comments showing m, s, n and the % flagged at both the current and the suggested value) and a TSV of the same numbers. (src: C-G8)

5.3 QC status semantics

  • QC-030 §9.2 · P5 Functional status. Values: PASS, FAIL or NA.

    • PASS iff every enabled functional check passes: autosomal_call_rate ≥ callRate and logr_dev ≤ logRDev.
    • FAIL if any enabled check fails. Failed samples (OUT-061) are FAIL.
    • NA when no genotyping summary exists: control-only datasets and methylation.
    • Overall call rate is reported but never drives the status. (src: C-G9, C-G11)
  • QC-031 §9.2 · P5 Control status. Values: CLEAR or FLAGGED.

    • FLAGGED iff any enabled control metric is out of range, including assayControl.
    • control_flag_codes lists the failing YAML keys, ;-joined in QC-004 order (e.g. npRed;stringency).
    • na_metrics lists metrics that could not be evaluated (QC-007).
    • The two status axes are independent: a sample can be PASS and FLAGGED. The documentation explains how to read the combination: biological or sample failures versus technical assay steps. (src: C-G9, C 1.8)
  • QC-032 §9.2 · P5 Methylation pass. pass_qc = 1 iff every enabled methylation control metric passes; fail_codes lists the failures (MET-008). Probe detection rate never affects it unless --min-prop-passing is set. (src: D-21, D-22)

  • QC-033 §9.2 · P5 Sample QC flags (flags column, ;-joined, informational):

    • low_call_rate, high_lrr_sd: mirror the functional checks;
    • sex_mismatch;
    • het_outlier (|z| > 3 within the batch);
    • contaminated (plan §9.3 estimate > 0.02 [OURS]);
    • assay_control_fail (OUT-062);
    • duplicate_of:<id>, unexpected_relatedness (plan §9.4).

    (src: plan §9.2)

  • QC-034 §9.2 · P5 Assay-control metric in QC. assayControl (default threshold 1 in the genotyping profiles) takes part in control status, in the trend statistics (QC-054) and in the PGx review gate (QC-035). (src: C-G10, A-G32)

  • QC-035 §9.2, §11 · P5 Recommended downstream gates. These are documented in the recipes (CLI-040) and applied as pgx_review and cn_training_exclude flags:

    • PGx review: logr_dev > 0.2, call_rate < 0.99, or assay_control_norm_r < 1.0.
    • CN training: the same three conditions (PCN-005).
    • Cyto: logr_dev > 0.2 → SampleQuality; > 0.3 → whole-chromosome-only mode (CYTO-014). (src: A-G65, A-G34)

5.4 QC report — inputs, outputs, datasets

  • QC-040 §9.5 · P5 Input contract. The report reads only dataset folders that contain:

    • controls_raw.tsv and controls_qc.tsv (or their compat CSV names): required;
    • sample_summary.tsv (or .csv, or the compat gt_sample_summary.csv): optional but recommended, because it enables functional QC, call-rate and LRR SD views, heatmaps and metadata.

    It never touches raw data, so reports can be regenerated or merged later. Joins use sample_id. IDs present in one file but not the other → W_QC_UNMATCHED_SAMPLES, listing up to 20 of them. Those samples appear with the fields that are available. (src: C-G1, A-G28, C 1.13)

  • QC-041 §9.5 · P5 --data forms.

    1. A folder that contains the required files.
    2. A comma list of such folders.
    3. A parent folder, searched recursively for every subfolder containing both control files. Discovered folders are sorted by path.

    Error cases:

    • A missing path → E_QC_NO_DATASETS: "Input data folder does not exist: ".
    • No valid folder → "No valid dataset folders detected. Each folder must include at least: controls_raw.tsv, controls_qc.tsv".
    • A listed folder lacking files → E_QC_DATASET_INCOMPLETE: "Dataset is missing required file(s): …". (src: A-G28, C-G17, C-G21)
  • QC-042 §9.5 · P5 Labels.

    • --label assigns labels in list or discovery order.
    • Count mismatch → E_QC_LABEL_COUNT: "Number of labels (2) does not match number of datasets (3)."
    • With no --label, each dataset is labelled with its folder name. Repeated folder names get _2, _3, … suffixes.
    • Labels are sanitised to [A-Za-z0-9_.-], with other characters replaced by _. Each changed label raises W_QC_LABEL_SANITIZED ("Label sanitized: My Label<1> → My_Label_1").
    • Labels that are duplicates after sanitisation → E_QC_LABEL_DUPLICATE.
    • The label identifies the dataset in every view, table and export. (src: C-G17, A-G28)
  • QC-043 §9.5 · P5 Dataset types.

    • A dataset is methylation if its controls_qc.tsv has array_type=methylation, or if its controls include bisulfite-conversion categories; otherwise it is genotyping.
    • Mixing the two types → E_QC_MIXED_TYPES, which lists both groups.
    • Allowed mixes: PGx and non-PGx datasets (assay-control columns are empty for non-PGx), legacy and EX chips, and different versions of one family. (src: C-G18)
  • QC-044 §9.5 · P5 Output files.

    • qc_report_YYYY_MM_DD.html and qc_table_YYYY_MM_DD.{xlsx|csv}, dated from the run date in UTC.
    • --report-name STEM sets a fixed stem, for reproducible file names.
    • HTML content is byte-deterministic given identical inputs and options, apart from one embedded generation timestamp. (src: C-G19, B-15)
  • QC-045 §9.5 · P5 QC table layout.

    • xlsx (default) has four sheets:
      1. Samples: one row per sample. Column order:

        • dataset, sample_id, sample_name;
        • functional_status, control_status, control_flag_codes, na_metrics;
        • the functional metrics (autosomal call rate, call rate, LRR SD, sex estimate, assay control);
        • every derived control metric;
        • selected raw control intensities in wide form, one column per <control_name>_<channel> (excluding QC-010);
        • metadata columns, alphabetically ordered.

        Formatting: conditional fills (PASS blue, FAIL orange, out-of-range functional metric orange, out-of-range control metric light amber, flag-code cell amber), a frozen header, autofilter.

      2. Thresholds: key, description, threshold, direction, enabled, source (default|profile:<n>|yaml:<path>|cli).

      3. Datasets: label, path, type, n_samples, profile.

      4. About: software version, generation time and input file hashes.

    • csv: the Samples sheet only, values only.
    • --output-format both writes both formats. (src: A-G29, C-G19, B-15)
  • QC-046 §9.5 · P5 Metadata privacy.

    • The report and the table carry every sheet column found in sample_summary. The HTML shows a notice when metadata is present, and the docs warn about sharing.
    • --drop-metadata COLS, --keep-metadata COLS (an allowlist) or --no-metadata remove columns before anything is embedded.
    • --redact-paths blanks sample_folder.
    • Editing sample_summary.tsv and re-running the report also works, because the report reads only files (QC-040). (src: A-G31, C-G20)

5.5 QC report — HTML specification

  • QC-047 §9.5 · P5 Technology.

    • The report is one self-contained HTML file: all JS and CSS inlined (bundled Plotly.js, MIT) and no network requests.
    • Data is embedded as compressed JSON and decoded in the page.
    • It must work offline in current Firefox and Chromium.
    • Colours are colour-blind-safe. Every colour pairs with a text label or legend. (src: C-G12)
  • QC-048 §9.5 · P5 Chrome.

    • Header: "SimurgArray QC Report", software version, generation date, dataset count, sample count, and the metadata notice (QC-046).
    • Left navigation: Control Dashboard · Automated QC · Sample Heatmaps · Trend Analysis · ⚙ QC Thresholds.
    • Plot toolbar: PNG export, zoom, pan, box select, lasso select, zoom in/out, autoscale, reset axes. (src: C-G12)
  • QC-049 §9.5 · P5 View A: Control Dashboard.

    • Controls: Dataset (one at a time); Sort plots by (default Sentrix barcode, then any metric or metadata field); Colour palette; Facet by (None by default, or an eligible metadata field, QC-056).
    • Top card: "Autosomal Call Rate", a horizontal box plot of the dataset with individual outlier points. Shown only when functional metrics exist.
    • Cards: one per control category × channel, two per row. Categories: Staining, Extension, Target Removal, Hybridization, Restoration, Stringency, Non-Specific Binding, Non-Polymorphic, Assay control; for methylation also Bisulfite Conversion I/II and Specificity I/II.
    • Card header: the title, plus an expected behaviour sentence written by us. Example: "Staining – Green: Biotin (High) should be strong; DNP (High/Bkg) and Biotin (Bkg) should be near background."
    • Plot: x = samples (categorical, in sort order, tick labels hidden); y = raw intensity from 0. One trace per control probe, each probe with a fixed colour and a legend entry.
    • Hover: sample ID, barcode, position, dataset, autosomal call rate (if available) and the value. (src: C-G12)
  • QC-050 §9.5 · P5 View B: Automated QC.

    • Dataset selector.
    • Two status histograms, horizontal bars labelled with count and percentage (e.g. "76 (40%)"):
      • Functional: FAIL (orange), PASS (blue). Hidden when the status is NA.
      • Control: FLAGGED (amber), CLEAR (blue).
    • Sample table.
      • Columns: Dataset, Sample ID, Functional QC Status, Control QC Flag Codes, Autosomal Call Rate, Overall Call Rate, LRR SD, Sex Estimate (shown as Male/Female/Unknown), one column per derived metric, then metadata.
      • Features:
        • per-column sort;
        • a column show/hide menu;
        • per-column footer filters (text for strings, min–max range for numbers);
        • global search;
        • pagination (10 per page by default; 10/25/50/100/All);
        • "Showing a to b of N entries";
        • Export ▾ (xlsx, csv);
        • "Select none".
      • Cell colouring as in QC-045.
      • Export writes the selected rows if any are selected, otherwise the currently filtered rows.
    • Per-metric scatter cards.
      • Header: the metric name and its formula as help text (QC-004 text, e.g. "(Bkg Red + offset) / highest NSB Red").
      • Axes: x = samples in the current view ("Sample ID"); y = the value.
      • Points are coloured by functional status by default (PASS blue, FAIL orange, NA grey), and can be recoloured by an eligible metadata field.
      • Sort by autosomal call rate, LRR SD, any metadata field or any metric.
      • A dashed threshold line, annotated "Flag if below X" or "Fail if above X"; none when disabled. (src: C-G12, C-G13)
  • QC-051 §9.5 · P5 Linked selection.

    • A box or lasso selection in any scatter (Views A and B) highlights the same samples in every scatter and filters the table.
    • Double-clicking a plot clears the selection.
    • Selecting table rows highlights the matching points; "Select none" clears it.
    • Plot selection, table search and column filters combine with AND.
    • Moving or resizing a selection over 12 000 points must not freeze the page for more than 200 ms (EDGE-028). The reference has a known freeze here. (src: C-G13, E-45)
  • QC-052 §9.5 · P5 View C: Sample Heatmaps.

    • Controls: Dataset; Colour scale (Continuous | Binary); Group by (Chip | Plate); Max chips per page (default 2, chip mode only); Sort by (barcode, plate, or metric mean); Plate selector (plate mode); paging with Prev/Next and "1 / N".
    • Panels: two metrics side by side for every chip or plate: "Autosomal Call Rate (higher is better)" and "LRR SD (lower is better)". Each has the subtitle "Cut-off = X", or "Cut-off disabled".
    • Chip layout:
      • rows R01…Rnn on the y-axis and columns C01…Cmm on the x-axis; the panel title is the chip barcode;
      • the grid dimensions come from a built-in chip-format table keyed by IDAT chip type or manifest (e.g. 24-sample = 12×2, 48-sample = 12×4, 8-sample = 8×1);
      • if the chip type is unknown, the grid uses the maximum observed row and column;
      • empty positions are drawn hatched, labelled "missing".
    • Plate layout: 8×12 (rows A–H, columns 1–12), keyed by sample_well, with the plate name as title. The sheet wins over the IDAT (IN-015). Samples with no plate are listed below the panels.
    • Cells: annotated with the value (3 decimals for call rate, 2 for LRR SD).
      • Continuous colouring: call rate light→dark green over the data min–max; LRR SD dark→light blue.
      • Binary colouring: call rate orange if below the cut-off, dark green otherwise; LRR SD blue "Below cut-off", yellow "Above cut-off", with a two-swatch legend.
    • Hover: sample ID, name, position or well, and the value.
    • Heatmap controls must stay aligned for ≥ 50 plates or barcodes (EDGE-028). (src: C-G14, C-G12, E-45)
  • QC-053 §9.5 · P5 Chip completeness. Per dataset:

    • n_chips;
    • n_complete_chips: every position of the chip format filled;
    • n_missing_positions = Σ (expected positions − observed).

    These use the chip-format table of QC-052. (src: C-G15)

  • QC-054 §9.5 · P5 View D: Trend Analysis.

    • Summary table, one row per dataset, sortable, with grouped headers:
      • Scan: date range (YYYY-MM-DD to YYYY-MM-DD, or a single date), # samples, # complete / # chips, # missing positions.
      • Autosomal Call Rate: min, median, mean (SD), # < threshold.
      • LRR SD: mean (SD), median, # > threshold.
      • Assay control: mean (SD), # < threshold. Blank for non-PGx datasets.
      • Sex estimate: # M, # F, # U.
    • Plots, with Group by (dataset (default) | chip) and Sort by:
      1. Autosomal call rate: horizontal box plot per group, with a dashed red threshold line.
      2. LRR SD: box plot with a threshold line.
      3. Assay control: box plot with a threshold line (PGx datasets only).
      4. Sample count: bar chart.
      5. Sex estimates: grouped bars (Male blue, Female orange, Unknown grey).
    • Box plots: Q1–Q3 box, median line, whiskers to the most extreme points within 1.5 × IQR, and outliers drawn. This rule is stated explicitly, not left to library defaults.
    • A single dataset still gets this view. (src: C-G15, C-G12)
  • QC-055 §9.5 · P5 View E: QC Thresholds (modal).

    • Layout: two sections, "Sample QC metrics" (Threshold, Fail if below/above) and "Control QC metrics" (Threshold, Flag if below/above). Columns: threshold input, direction dropdown, metric, description (formula). A blank input disables the check.
    • Buttons:
      • Apply recomputes both status axes client-side and refreshes every visual (histograms, table colours, threshold lines, heatmap cut-offs, trend counts).
      • Download YAML exports the current thresholds in the QC-020 schema, so the file can be reused with --config.
      • Upload YAML loads a file into the fields, validated as in QC-021, with inline errors.
    • Changes last for the session only and are never written back into the HTML. (src: C-G12)
  • QC-056 §9.5 · P5 Metadata eligibility.

    • A metadata field is offered for Colour by or Facet by when it has 2–20 unique non-empty values.
    • Faceting shows up to 12 facet panels. Extra levels are grouped as "Other", with a note [DIFF]; the reference collapses anything above 6.
    • Hover text shows up to 200 characters per field, then "…" [DIFF]; the reference truncates at 40. The field list in hover is capped at 12 fields. (src: C-G13, E-45)
  • QC-057 §9.5 · P5 Scale behaviour.

    • > 1 000 samples in a dataset: control and metric plots are split into tabs, one per category.
    • > 12 000: scatters switch to WebGL rendering (scattergl) and also offer a density (2-D histogram) mode. They are not disabled [DIFF]. A notice explains the mode.
    • > 50 000: density mode by default.
    • > 20 000 in total across datasets: a warning in the CLI and in the page.
    • --split-by dataset|plate|run writes one report per group, plus an index page.
    • Switching from a large dataset to a small one must not need a reload; the reference needs one. (src: C-G16, E-45)
  • QC-058 §9.5 · P5 Multi-dataset behaviour. Each view has a dataset selector: one dataset at a time, except Trend Analysis, which shows all of them. Dataset labels (QC-042) prefix sample IDs in hover and in exports. A report built from one dataset still shows every view. (src: A-G28, C-G17)

  • QC-059 §9.5 · P5 Sex view. The plan's sex plot is kept as a card inside Trend Analysis: X vs Y median R per sample, coloured by sex_est, with the symbol showing sex_check. (src: plan §9.5)

  • QC-060 §9.5 · P5 Control-metric PCA (optional card). A PCA of the log-transformed, standardised derived control metrics across the samples of a dataset. It is coloured by Sample_Group, or by any eligible field, and is available for genotyping and methylation alike. With fewer than 3 metrics or fewer than 10 samples, the card shows "insufficient data". (src: A-G60)

  • QC-061 §9.5 · P5 Robust CSV/TSV reading. Input files are UTF-8 (a BOM is allowed), comma- or tab-delimited as the extension says, with RFC-4180 quoting. Parse errors name the file, the line and the column. (src: C 1.13)

  • QC-062 §9.5 · P5 Methylation datasets in the report.

    • Views A, B and D work with the methylation metric set. There is no functional status; pass_qc is shown as the control axis.
    • Heatmaps colour by the fraction of probes passing detection, when methyl_qc_summary.tsv is present. (src: C-G26)
  • QC-063 §16 · P5 Acceptance tests. Every error and warning in QC-021/041/042/043, plus E_QC_FORMAT ("Invalid --output-format value. Only csv, xlsx or both are accepted."), has a CLI test that asserts the code and the exit status (EDGE-028). (src: C-G21)

  • QC-064 §9.3, §9.4 · P5 Contamination and relatedness stay as in plan §9.3–§9.4. Their outputs join sample_summary (contamination) and the flags column (QC-033), and appear as optional table columns in the report. (src: plan)


6. Cytogenetics: genome-wide CNV / LOH / mosaicism (v2)

6.1 Scope, inputs, pre-processing

  • CYTO-001 §10 · v2 Scope.

    • Constitutional analysis only (annotation_type = "constitutional").
    • The genome is assumed to be mostly diploid. Triploid or near-haploid genomes are a documented limitation: events may be missed.
    • Any human genotyping array is accepted. Sensitivity depends on probe density, and the report states the median probe spacing. (src: D-7, E-35)
  • CYTO-002 §10 · v2 Inputs.

    • GTCs that satisfy IN-031.
    • A cyto model (IN-051).
    • The batch sample_summary.tsv, which supplies sex and LRR SD.
    • Optionally a FASTA, for REF padding bases.
    • Nothing depends on the cluster file beyond the GTC contents. (src: A-G52)
  • CYTO-003 §10.1 · v2 GC-wave correction.

    • LRR is regressed on the model's GC covariates (plan §10.1), and the residuals are used.
    • The on/off state is written as ##GCCorrect=true|false and as JSON gc_correct.
    • The correction factor (GCWF) goes into the JSON. (src: A-G52)
  • CYTO-004 §10.2 · v2 LRR smoothing.

    • --smoothing K (default 5, range 0–1000) applies a moving median over 2K+1 consecutive probes of the same chromosome. At chromosome ends the window is truncated, never wrapped. K = 0 disables smoothing.
    • Smoothed LRR is used only as segmentation input and for the visual track (OUT-052).
    • Stored LRR, NR, LRD and all statistics use unsmoothed values. (src: A-G48, D-3, E-28)
  • CYTO-005 §10.2 · v2 State space.

    • Constitutional states (plan §10.2): CN0, CN1, CN2, CN2-LOH, CN3, CN3-LOH (gain-LOH), CN≥4.
    • Mosaic states: loss, gain and CN-LOH, each with cell fraction f.
    • Haploid-region states (male X/Y outside PAR): CN0, CN1, CN2, plus mosaic states between them.
    • CN is capped by --cn-cap (default 4, meaning "4 or more"). (src: A-G45, D-1, E-31)

6.2 Event taxonomy and reporting thresholds

  • CYTO-006 §10.2 · v2 Event types (emulating the reference header definitions; see OQ-016/OQ-035).
Call type ALT Subtype (LOHTYPE) Rule Min probes / size (default)
DEL <DEL> – CN < baseline (2 for autosomes/PAR/female X; 1 in haploid regions), non-mosaic 10 / 25 kb
DUP <DUP> – CN > baseline, non-mosaic 10 / 50 kb
LOH <LOH> AOH non-mosaic absence of heterozygosity with CN = 2 500 / 3000 kb
LOH <LOH> AOH (constitutional gain-LOH) or GAINLOH (mosaic gain-LOH) absence of heterozygosity with CN ≥ 3 300 / 500 kb (the gainloh thresholds)
mosaic DEL <DEL> + MOSAIC – mosaic loss 10 / 25 kb
mosaic DUP <DUP> + MOSAIC – mosaic gain, including male chrY duplication 10 / 50 kb
mosaic LOH <LOH> + MOSAIC CNLOH mosaic copy-neutral LOH 500 / 3000 kb

LOH at CN = 1 (hemizygous) is part of the DEL call and is not reported separately. (src: A-G45, B-16, D-1, E-31)

  • CYTO-007 §10.2 · v2 Per-type thresholds.

    • The 14 options --min-{del,dup,loh,gainloh,mdel,mdup,mloh}-{probes,size} of CLI-028 are applied independently.
    • Sizes are given in kb on the CLI and stored in bp internally [DIFF]. The JSON shows *_bp integers, so units can never be wrong.
    • Ranges are 0–250 000 (probes and kb alike). (src: A-G45, B-17, D-2, E-29, E-38)
  • CYTO-008 §10.2 · v2 Probe floor [DIFF]. Any --min-*-probes value < 10 is raised to 10, and W_CN_PROBE_FLOOR is logged once for each option. The reference resets such values silently. (src: A-G46, E-29)

  • CYTO-009 §10.2 · v2 Thresholds that exceed the genome. When a size threshold exceeds the sample's effective genome size, every event of that type is filtered out. This is not an error: W_CN_THRESHOLD_EXCEEDS_GENOME is logged and an empty result is written. (src: E-29)

  • CYTO-010 §10.2 · v2 Effective size.

    • Let the probes inside an event be sorted by position, with spacings s_i.
    • Let m be the median inter-probe spacing of that chromosome over all manifest probes that are used [INF, OQ-039].
    • A spacing with s_i > 150·m is a gap.
    • Effective size = (end − start + 1) − Σ_gaps (s_i − m).
    • The effective genome size is the sum of the chromosome effective spans. (src: B §3.11, D-7)
  • CYTO-011 §10.2 · v2 Size basis [DIFF].

    • cn call filters by effective size.
    • cn annotate --size-basis defaults to effective too, which keeps the two steps consistent. raw is available. The reference filtered by effective size at call and raw size at annotate.
    • The chosen basis is written to the sidecar and the JSON.
    • A regression test uses an event that spans a centromere gap (EDGE-024). (src: D-2, E-30)
  • CYTO-012 §10.2 · v2 QUAL.

    • QUAL = min(60, max(0, round(−10·log10(1 − P)))), where P is the mean over the event's probes of the posterior marginal probability of the called state class, from forward–backward [OURS, OQ-040].
    • If P = 1, QUAL is 60.
    • The VCF writes an integer. The JSON quality is a float with 1 decimal. (src: B-16, D-5)
  • CYTO-013 §10.2 · v2 Filters.

    • VCF FILTER values: Q10 (QUAL < --filter-qual, default 10), SampleQuality (sample logr_dev > 0.2), LowFraction (mosaic MF < --min-mosaic-fraction), or PASS. They are ;-joined when several apply.
    • Reporting gate: cn annotate --min-qual (default 20) removes events with QUAL < min-qual from the JSON and ISCN. They stay in the VCF.
    • Events with QUAL 0 are kept in the VCF, and reach the JSON whenever min-qual ≤ 0 (EDGE-024). (src: A-G49, A-G50, B-17, D-5, E-29)
  • CYTO-014 §10.2 · v2 Sample-level noise gating.

    • logr_dev > 0.2: every record gets FILTER SampleQuality, and the JSON gets qc_flags: ["high_noise"].
    • logr_dev > 0.3: whole-chromosome-only mode. Only events that cover ≥ 90 % of a chromosome's effective span are called and reported [OURS, OQ-052]. The JSON gets qc_flags: ["whole_chromosome_only"]. (src: A-G50, E-33)
  • CYTO-015 §10.2 · v2 Post-filters, applied in this order:

    1. LRR direction consistency. A gain needs ≥ 60 % of its probes with LRR > 0; a loss needs ≥ 60 % with LRR < 0 (--lrr-direction-min-frac, default 0.6 [HYP]). LOH has no requirement. Failing events are dropped and counted.
    2. Short LOH. LOH < 500 kb whose immediate neighbours on both sides are copy-neutral (CN2, non-LOH) is removed.
    3. Merging. Adjacent events of the same type, CN and mosaic state, separated by ≤ 3 probes and by a gap ≤ 10 % of their combined length, are merged [OURS].
    4. Per-type thresholds (CYTO-007). (src: E-33, plan §10.2)
  • CYTO-016 §10.2 · v2 Excess events. More than 10 DEL+DUP+LOH events on one chromosome sets sample flag excess_events in the JSON and in cn_sample_summary.tsv, with the chromosome list, for visual review. (src: A-G51, E-33)

  • CYTO-017 §10.2 · v2 Centromeres and gaps.

    • A built-in mask BED per build covers centromeres, the acrocentric p-arms and large assembly gaps. It is replaceable with --mask-bed.
    • Events are trimmed to the unmasked sequence.
    • Events that are ≥ 50 % masked are dropped.
    • Trimmed events carry the JSON flag centromere_trimmed. (src: D-6, E-35)

6.3 Mosaicism

  • CYTO-018 §10.3 · v2 Mosaic fraction (MF) [OURS, based on LIT (MoChA, PennCNV-style models)]. With BAF split d = |BAF_het-band − 0.5| over heterozygous-informative probes, and allelic imbalance measured on phased or unphased BAF:

    • mosaic loss: f = 2d/(0.5 + d);
    • mosaic gain: f = 2d/(0.5 − d);
    • mosaic CN-LOH: f = 2d.

    In haploid regions, or with too few heterozygous probes, MF is estimated from LRR as f = (2^{(LRR_event − LRR_base)/γ} − 1)·sign, with the per-sample compression γ fitted from the constitutional state means. MF is clipped to [0, 1]. (src: A-G47, D-4)

  • CYTO-019 §10.3 · v2 MF not estimable. With fewer than 20 informative probes for the method in use, MF is . in the VCF and null in the JSON [EMU: the reference writes NaN]. (src: A-G47, E-32)

  • CYTO-020 §10.3 · v2 Promotion. A mosaic event with MF ≥ --max-mosaic-fraction (default 1.0, so off) is promoted:

    • it becomes a constitutional call, without the MOSAIC flag;
    • it gains the HIGHFRACTION flag;
    • MF is kept;
    • the ID prefix loses Mosaic;
    • the constitutional thresholds of its type apply.

    (src: A-G47, D-4, E-32)

  • CYTO-021 §10.3 · v2 Low fraction [DIFF]. A mosaic event with MF < --min-mosaic-fraction (default 0.10 [HYP, OQ-020]) is kept with FILTER LowFraction. It is excluded from ISCN, but listed in the JSON with filters: ["LowFraction"]. There is no hard cut-off; the reference's old 5 % cut-off was removed. (src: D-4, E-32)

  • CYTO-022 §10.3, §16 · v2 Sensitivity target. Mosaic MF ≥ 15 % is detectable for one-copy-difference states:

    • diploid CN2↔CN1 and CN2↔CN3;
    • haploid CN1↔CN0 and CN1↔CN2, including male chrY duplication.

    The validation (plan §16.4 synthetic) simulates MF = 5, 10, 15, 20 and 30 % at 1, 5 and 20 Mb, and reports sensitivity. The gate is ≥ 90 % sensitivity at 15 % / 5 Mb on GSA-density data. (src: A-G55, D-4, E-32)

6.4 Sex, ploidy, summary statistics

  • CYTO-023 §10.2 · v2 PAR. PAR1/PAR2 probes are called as diploid for both sexes. Manifest XY loci are mapped to X PAR positions (IN-046). A missing PAR mapping in a manifest must not make PAR uncallable: PAR intervals come from the species profile. (src: D-6, E-34)

  • CYTO-024 §10.2 · v2 Sex resolution [DIFF].

    • The sex used for baselines is: sex_est from the summary, else the sheet's reported sex.
    • If it is U, the caller estimates sex from the median LRR of chrX and chrY and the X heterozygosity rate, and logs I_CN_SEX_RESOLVED ("Sample sex updated from U to <M|F>").
    • If sex is still unresolved, --unknown-sex-policy decides:
      • skip-sex-chromosomes (default): no X/Y events; flag sex_unresolved;
      • female or male: assume that sex; flag sex_assumed;
      • error: the sample fails.
    • The sample is never assumed male silently. (src: D-6, E-34)
  • CYTO-025 §10.2 · v2 Overall ploidy.

    • overall_ploidy = the length-weighted mean CN over all called contigs (autosomes, X and Y, using effective lengths), with baseline CN per the sex model. Males therefore sit at ≈ 1.95 without events.
    • Header: ##OverallPloidy=<%.3f>. (src: B-22)
  • CYTO-026 §10.2 · v2 Copy-number median. copy_number_median = the length-weighted median CN over all segments, with no minimum segment length. The per-chromosome copy_number_mean is length-weighted too. (src: B §3.2, E-37)

  • CYTO-027 §10.2 · v2 Event statistics.

    • NR = median over the event's probes of 2^{LRR} (≈ CN/2) [INF].
    • LRD = SD of the unsmoothed LRR in the event.
    • bafDev = SD of BAF over the event's probes, excluding homozygous calls.
    • percent_het = the percentage of called probes in the event that are heterozygous. (src: B §2.2)

6.5 CNV VCF

  • CYTO-028 §10, §13 · v2 File. <sample>.cyto.cnv.vcf.gz with its .tbi (bgzip by default; CLI-028). Records sorted by .fai or manifest contig order, then POS. (src: B-16, A-G25)

  • CYTO-029 §10 · v2 Coordinates [DIFF, VCF-correct 1-based].

    • For an event whose first affected base is S and last affected base is E (1-based, inclusive):
      • POS = S − 1: the padding base, VCF 4.3 §5.3 (if S = 1, POS = 1 and the padding base is the base after the event, per the spec);
      • REF = the FASTA base at POS, or N only when no FASTA was supplied (a warning is logged);
      • INFO/END = E;
      • INFO/SVLEN:
        • signed mode (default): −(E − S + 1) for DEL and +(E − S + 1) for DUP and LOH;
        • --svlen-mode positive: always positive.
    • ID = <Prefix>:<CHROM>:<S>-<E>, with CHROM exactly as the VCF CHROM field.
    • Prefixes: DEL, DUP, AOH, GAINLOH, CNLOH, MosaicDEL, MosaicDUP.
    • Tests assert that S = POS + 1 and that the ID coordinates equal POS + 1 and END for every record (EDGE-025).
    • Every TSV and JSON uses start = S (1-based), and never a 0-based start. (src: E-36, B-16)
  • CYTO-030 §10 · v2 Exact header.

    ##fileformat=VCFv4.3
    ##source=SimurgArray <semver> cn
    ##reference=file://<fasta or "none">
    ##simurg_genome_build=GRCh38
    ##simurg_manifest=<name>;sha256=<hex>
    ##simurg_cyto_model=<file>;sha256=<hex>;schema=<n>
    ##simurg_thresholds=min_del_probes=10;min_del_size_bp=25000;min_dup_probes=10;min_dup_size_bp=50000;min_loh_probes=500;min_loh_size_bp=3000000;min_gainloh_probes=300;min_gainloh_size_bp=500000;min_mdel_probes=10;min_mdel_size_bp=25000;min_mdup_probes=10;min_mdup_size_bp=50000;min_mloh_probes=500;min_mloh_size_bp=3000000;smoothing=5;max_mosaic_fraction=1.0;min_mosaic_fraction=0.10;filter_qual=10;size_basis=effective;cn_cap=4
    ##simurg_sex_used=<M|F|U>;source=<summary|sheet|resolved|assumed|unresolved>
    ##OverallPloidy=1.953
    ##GCCorrect=true
    ##contig=<ID=1,length=248956422>        (all contigs of the called set, .fai order)
    ##ALT=<ID=DEL,Description="Copy number loss">
    ##ALT=<ID=DUP,Description="Copy number gain">
    ##ALT=<ID=LOH,Description="Absence/loss of heterozygosity (AOH/LOH/ROH)">
    ##FILTER=<ID=PASS,Description="All filters passed">
    ##FILTER=<ID=Q10,Description="Quality below 10">
    ##FILTER=<ID=SampleQuality,Description="Sample flagged as potentially low quality: LRR SD above 0.2">
    ##FILTER=<ID=LowFraction,Description="Mosaic event with estimated mosaic fraction below 0.10">
    ##INFO=<ID=END,Number=1,Type=Integer,Description="Last base of the event (1-based, inclusive)">
    ##INFO=<ID=SVTYPE,Number=1,Type=String,Description="DEL, DUP or LOH">
    ##INFO=<ID=SVLEN,Number=1,Type=Integer,Description="Event length in bp; negative for deletions">
    ##INFO=<ID=EFFLEN,Number=1,Type=Integer,Description="Effective length in bp (gaps above 150x median probe spacing replaced by the median spacing)">
    ##INFO=<ID=PROBE,Number=1,Type=Integer,Description="Number of probes in the event">
    ##INFO=<ID=MOSAIC,Number=0,Type=Flag,Description="Mosaic event">
    ##INFO=<ID=HIGHFRACTION,Number=0,Type=Flag,Description="Mosaic event promoted to constitutional because its mosaic fraction reached the maximum">
    ##INFO=<ID=LOHTYPE,Number=1,Type=String,Description="AOH (constitutional, copy-neutral or gain), CNLOH (mosaic copy-neutral), GAINLOH (mosaic copy-gain)">
    ##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype (see SimurgArray docs, CYTO-031)">
    ##FORMAT=<ID=CN,Number=1,Type=Integer,Description="Copy number; the maximum value 4 means 4 or more">
    ##FORMAT=<ID=NR,Number=1,Type=Float,Description="Aggregated normalized intensity (median 2^LRR; about CN/2)">
    ##FORMAT=<ID=LRD,Number=1,Type=Float,Description="Standard deviation of LRR within the event">
    ##FORMAT=<ID=MF,Number=1,Type=Float,Description="Estimated mosaic fraction; missing if not mosaic or not estimable">
    #CHROM POS ID REF ALT QUAL FILTER INFO FORMAT <sample name>
    
    • The numbers in the FILTER and CN descriptions follow the effective options.
    • FORMAT is always GT:CN:NR:LRD:MF; MF is . when not applicable.
    • NR has 4 decimals, LRD 4 and MF 3. (src: B-16, A-G47, A-G50, D-5)
  • CYTO-031 §10 · v2 GT semantics [DIFF]. With --gt-mode vcf (the default):

Event Diploid region Haploid region
DEL CN1 0/1 –
DEL CN0 1/1 1
DUP CN3 0/1 –
DUP CN ≥ 4 ./1 1 (CN ≥ 2)
LOH (any subtype) 1/1 –
mosaic events 0/1 1

--gt-mode compat writes 1/1 for every event, as the reference does; the header records the mode. (src: E-36, D-5)

  • CYTO-032 §10 · v2 Sidecar and segments.
    • cn call also writes cn_call_manifest.json in the output folder, which cn annotate requires. It holds:
      • the list of samples;
      • effective thresholds;
      • model and summary hashes;
      • per-sample QC: logr_dev, sex_used, flags.
    • It also writes <sample>.cn_segments.parquet: every segment, including those below threshold, with posteriors. annotate can then re-filter without re-calling.
    • Annotate errors:
      • missing sidecar → E_CN_SIDECAR_MISSING;
      • VCFs not bgzipped and indexed → E_CN_VCF_UNINDEXED. (src: A-G44, A-G54)

6.6 Per-sample JSON report (cn annotate)

  • CYTO-033 §10.4 · v2 <sample>.cyto.json (schema simurg.cyto/1; typed JSON; null for missing values; percentages 0–100 as numbers; sizes in bp).

    {
      "schema": "simurg.cyto/1",
      "software_version": "0.9.0",
      "annotation_bundle": {"file": "…zip", "sha256": "…", "sources": [
          {"name": "RefSeq", "version": "GCF_000001405.40-RS_2023_10", "release_date": "2023-10-07"},
          {"name": "Ensembl", "version": "112", "release_date": "2024-05-14"},
          {"name": "UCSC cytoBand", "version": "hg38", "release_date": null}]},
      "reference_genome": "file://…", "genome_build": "GRCh38",
      "annotation_type": "constitutional", "iscn_version": "ISCN 2020",
      "sample_id": "204753010023_R01C01", "sample_name": "NA12878",
      "gc_correct": true, "gcwf": 0.012,
      "parameters": {"min_del_probes": 10, "min_del_size_bp": 25000, "…": "…", "min_qual": 20, "size_basis": "effective"},
      "overall_ploidy": 1.953, "call_rate": 0.9941, "logr_dev": 0.159, "logr_dev_robust": 0.046,
      "baf_dev": {"aa": 0.0161, "ab": 0.0441, "bb": 0.0168},
      "num_loh_over_1mb": 2, "num_loh_over_8mb": 1, "total_size_loh_over_1mb": 16992771,
      "percent_loh": 0.55, "copy_number_median": 2.0,
      "sex_estimate": "M", "sex_source": "summary",
      "qc_flags": [],
      "iscn_traditional": "del(1)(p36.33p36.33),dup(9)(p24.3q21.11)",
      "iscn_microarray": "arr[GRCh38] 1p36.33(1567924_1632825)x1,9p24.3q21.11(14326_69444970)x3,16p11.2q11.2(31382412_46974423)x2 hmz",
      "chromosomes": [ { "id": "1", "size": 248956422, "effective_size": 230000000,
          "percent_het": 11.57, "has_mosaicism": false,
          "lrr_median": 0.001, "lrr_mean": 0.002, "lrr_dev": 0.15,
          "num_loh_over_1mb": 0, "num_loh_over_8mb": 0, "total_size_loh_over_1mb": 0, "percent_loh": 0.0,
          "copy_number_median": 2.0, "copy_number_mean": 1.9924,
          "min_lrr": -1.2, "max_lrr": 0.9, "median_mosaic_fraction": null,
          "number_del": 1, "number_dup": 0, "number_loh": 0, "number_gainloh": 0, "number_mosaic": 0 } ],
      "events": [ { "id": "DEL:1:1567924-1632825", "chrom": "1", "start": 1567924, "end": 1632825,
          "call_type": "DEL", "loh_type": null, "mosaic": false, "mosaic_fraction": null, "high_fraction": false,
          "copy_number": 1, "quality": 60.0, "filters": ["PASS"],
          "size": 64902, "effective_size": 64902, "probe_count": 99,
          "percent_het": 0.0, "lrr_median": -0.45, "lrr_dev": 0.21, "baf_dev": 0.03,
          "start_cytoband": "p36.33", "end_cytoband": "p36.33",
          "iscn_traditional": "del(1)(p36.33p36.33)",
          "iscn_microarray": "arr[GRCh38] 1p36.33(1567924_1632825)x1",
          "flags": [], "gene_count": 3, "genes": ["CDK11A", "SLC35E2B", "ENSG00000…"],
          "dgv_overlap": [] } ]
    }
    

    Field rules:

    • logr_dev = population SD of autosomal LRR (plan §7.6).
    • logr_dev_robust = 1.4826·MAD of autosomal LRR. It takes the place of the reference's undocumented medianLogRDev (OQ-022).
    • baf_dev.{aa,ab,bb} = SD of BAF over autosomal probes called AA, AB or BB outside events.
    • LOH counts and sizes, at both sample and chromosome level, include gain-LOH [DIFF]. number_gainloh is reported separately.
    • percent_loh = total LOH length / effective genome length × 100.
    • size = end − start + 1.
    • Sex is written as M/F/U, the same codes as in the summary.
    • --compat-json MAY write reference-style key names (camelCase) and string sentinels, for comparison tests only. (src: B-18, D-7, E-37, A-G53)
  • CYTO-034 §10.4 · v2 ISCN 2020 generator.

    • Microarray form: arr[GRCh37|GRCh38] followed by comma-joined events, ordered by chromosome (1–22, X, Y) and then position. Each event is written as <chr><band_start>[<band_end>](<S>_<E>)x<CN>, where band_end is omitted when it equals band_start.
      • LOH: x2 hmz; gain-LOH: x3 hmz.
      • Mosaic events: x<CN_base>~<CN_event> (e.g. x1~2) [HYP, OQ-034].
      • Coordinates are 1-based. Thousands separators are never used.
    • Normal results:
      • arr[GRCh38](1-22,X)x2 for a female;
      • arr[GRCh38](1-22)x2,(XY)x1 for a male;
      • arr[GRCh38](1-22)x2 when sex is unresolved.
    • Traditional form: del(<chr>)(<bands>) and dup(<chr>)(<bands>) for CNVs only. LOH has no karyotype notation here and is omitted.
    • Coverage: ISCN is produced for every reported event, including the first, LOH and mosaic events (--iscn-min-size, default 0). The reference misses these cases [DIFF].
    • The per-sample string contains only events that pass FILTER and min-qual.
    • Golden tests compare against hand-checked ISCN strings. (src: B-19, D-8, E-37)
  • CYTO-035 §10.4 · v2 Genes and bands.

    • Genes = every transcript whose span overlaps the event by ≥ 1 bp. Each is named by its HGNC symbol (via RefSeq), or by its Ensembl gene ID when it has no symbol.
    • The list is deduplicated and sorted by transcript start, then name. gene_count is the length of the list.
    • Cytobands come from UCSC cytoBand for the build.
    • DGV overlap (plan §10.4) is optional, and uses reciprocal overlap ≥ 50 %. (src: A-G53, B-18)
  • CYTO-036 §10.4 · v2 Annotation bundle build. scripts/build_annotation_bundle.sh <build> downloads the following public files and packages them with a manifest of versions and hashes (IN-054):

    • RefSeq GFF (pinned GCF_000001405.40 release);
    • Ensembl GTF (pinned release, e.g. 112);
    • HGNC;
    • UCSC cytoBand and the centromere/gap tables;
    • DGV (optional).

    The bundle is independent of any array product. (src: A-G53, plan §10.4)

  • CYTO-037 §10 · v2 Batch outputs.

    • cn_sample_summary.tsv, one row per sample. Columns: sample_id, sample_name, sample_folder, status, logr_dev, overall_ploidy, gc_correct, sex_used, sex_source, n_del, n_dup, n_loh, n_gainloh, n_mosaic, n_filtered, qc_flags.
    • The LRR/BAF bedGraph tracks of OUT-050 when export bedgraph or pipeline requests them. (src: B-21, C-G24)
  • CYTO-038 §10 · v2 Model building.

    • simurg cn build-model --bpm --fasta computes, for every non-control probe, its GC fraction in windows of 1 kb, 100 kb and 1 Mb centred on the probe, plus probe-sequence GC when the CSV SourceSeq is given.
    • It writes IN-051.
    • It needs no cluster file. The model is rebuilt only when the manifest or the model schema changes. (src: A-G52)
  • CYTO-039 §10, §19 · v2 Batch scale. Samples are processed independently and streamed, so memory is O(1) in batch size. Tests run 1 152 samples; the reference failed at about 200. There is no hidden batch normalisation. (src: D-9, E-35)

  • CYTO-040 §10 · v2 Degenerate samples. None of these may crash or overflow:

    • zero events;
    • an all-NaN chromosome;
    • a single-probe chromosome;
    • a chromosome with no heterozygous probes;
    • LRR SD = 0.

    Each yields valid, possibly empty, VCF and JSON output (EDGE-024). (src: E-35)

  • CYTO-041 §10.3 · v2 MF refinements [OURS]. Two refinements of CYTO-018, from real-data validation (docs/validation/V2_cn.md, K-027):

    • noise-corrected split: per-probe expected noise σ_i = the reference's heterozygous BAF spread of the probe × the sample's scale (median |BAF − 0.5| / spread over autosomal heterozygous calls, ÷ 0.6745; the sample's heterozygous BAF SD without a reference spread). Over the event's heterozygous calls, m = median(|BAF − 0.5| / σ_i); the split in σ units δ solves Φ(m − δ) − Φ(−m − δ) = ½ (δ = 0 when m ≤ 0.6745) and d = median(σ_i)·δ. Without this, noise alone gave MF ≈ 0.1–0.3 and genomic waves were called as mosaic losses.
    • complete loss: a mosaic loss whose called genotypes hold fewer than a quarter of the heterozygotes the PFB predicts (Σ 2p(1−p) ≥ 3) has MF = 1. With the default --max-mosaic-fraction 1.0 it is therefore promoted (CYTO-020) to a constitutional DEL with HIGHFRACTION. This catches constitutional deletions whose LRR is compressed towards the mosaic states.
  • CYTO-042 §10.2 · v2 Per-probe LRR noise [OURS]. A model built with --reference-gtcs stores each probe's robust LRR SD across the reference (1.4826·MAD after removing each sample's median and the probe's reference median). cn call scales the sample's LRR emission SD per probe by that spread relative to the autosomal median spread, clipped to [0.5, 3]. Without the column, or with --no-probe-lrr-sd, every probe uses the sample SD.


7. Targeted PGx copy number (v2)

  • PCN-001 §11 · v2 Region catalogue.
    • Seven genes, ten target regions. They are defined in a versioned BED shipped with the PGx DB (pgx_cn_regions.GRCh38.v<N>.bed: chrom, start0, end, region, gene). The BED is derived from gene annotation plus probe coverage and validated against the public region descriptions.
Gene Regions Notes
CYP2D6 CYP2D6.p5 (5′ flank/promoter), CYP2D6.intron.2, CYP2D6.exon.9 Sub-regions separate deletions (*5), duplications (xN), CYP2D7-derived exon-9 conversions (*36-like) and 2D7-2D6 hybrids (*13-like)
CYP2A6 CYP2A6.exon.1, CYP2A6.intron.7 *4-type deletion and conversion
GSTM1 whole gene *0 null deletion
GSTT1 whole gene lies on the GRCh38 alt contig chr22_KI270879v1_alt (IN-062)
UGT2B17 whole gene common deletion
SULT1A1 whole gene copy gain
CYP2E1 whole gene copy gain
  • The catalogue is extensible: a new BED version plus a retrained model is enough; no code change is needed. (src: B-23, D-10, C-G25)

  • PCN-002 §11 · v2 Genotype every region [EMU]. Each sample gets exactly one record per region in every output, whether its CN is 2 or not. A region that cannot be assessed (for example all its probes are masked, or it has too few finite values) gets CN ., QUAL 0 and FILTER NoCall [OURS]. (src: B-23)

  • PCN-003 §11 · v2 Inputs.

    • GTCs that satisfy IN-031.
    • The model (IN-050).
    • A GRCh38 FASTA with the alt contigs: used for REF padding bases and contig headers, and checked against the model's FASTA hash.
    • The sample summary, which supplies LRR SD and the assay control. (src: A-G37)
  • PCN-004 §11 · v2 Batch rules. Batch statistics are only valid when enough samples are good.

    • Passing means logr_dev < 0.2 (strict), set by --lrr-sd-max.
    • Fewer than --min-batch samples in total (24), or fewer than --min-passing passing samples (22), → E_PGX_BATCH_TOO_SMALL: " sample(s) passed QC. Requires at least samples to proceed." Exit 2.
    • Fewer than --recommended-passing passing samples (96) → W_PGX_BATCH_BELOW_RECOMMENDED.
    • Only passing samples build the batch reference.
    • Non-passing samples are still genotyped, with FILTER SampleQuality.
    • Boundary semantics are listed in OQ-004. (src: A-G33, D-11, C-G25, E-39)
  • PCN-005 §11 · v2 Model training.

    • Needs ≥ 96 samples, of which ≥ 90 pass (logr_dev ≤ 0.2); otherwise E_PGX_TRAIN_TOO_SMALL. The documentation recommends ≥ 150 samples.
    • Above --max-samples (3 000) a seeded random subset is used, because returns diminish.
    • Samples are excluded automatically if any of these hold:
      • logr_dev > 0.2;
      • call_rate < 0.99;
      • assay_control_norm_r < 1.0.
    • Exclusions go to train_exclusions.tsv (sample_id, reason).
    • The model assumes the training cohort's CN distribution looks like a general population's. The documentation requires the user to validate a retrained model against truth data before relying on it.
    • The model must be retrained whenever the cluster file changes (IN-050). (src: A-G34, E-39)
  • PCN-006 §11 · v2 Batch-composition warnings [OURS]. Warnings are raised in two cases:

    • the sheet contains a reagent-lot, plate-batch or processing-batch column with more than one value (column names configurable; defaults Reagent_Lot, Batch, AMP_Plate);
    • IDAT scan dates span more than 14 days.

    Warnings only, never errors. (src: A-G38)

  • PCN-007 §11 · v2 Model compatibility. See IN-050: a manifest mismatch is an error; a cluster-file mismatch is an error by default, or a warning with --cluster-mismatch warn. The recorded platform (LCG/EX) must match the manifest's Assay Format, otherwise a warning is issued. (src: A-G37, B-20, D-11, E-39)

  • PCN-008 §11 · v2 Mask. Masked probes (IN-052) are excluded from training and calling. A region left with fewer than 10 unmasked probes is flagged in the model header. (src: A-G35)

  • PCN-009 §11 · v2 Calling algorithm [HYP, plan §11; U10].

    1. Per probe, compute normalized R (plan §7.3–7.4) and GC-correct it using the probe GC stored in the model.
    2. Divide by the probe's batch median over passing samples, and take a robust per-sample scale (the median over non-target reference probes stored in the model).
    3. For each region, NR = the median over its unmasked probes of the normalized ratio. NR ≈ CN/2.
    4. Compute the CN posterior from the region's trained 1-D Gaussian mixture over NR, with components CN 0…5 and frequency priors from training.
    5. CN = argmax of the posterior, capped at 5, meaning "5 or more".
    6. QUAL = min(60, round(−10·log10(1 − P_max))).

    The procedure is robust to outlier samples: batch medians rather than means, and outlier samples (|z| > 5 on genome-wide NR) are excluded from the batch reference. NR is computed per region and never shared across regions [DIFF, OQ-027]. (src: A-G33, plan §11)

  • PCN-010 §11 · v2 CN-consistent BAF evidence.

    • For SNP probes inside a region, the expected BAF for k alternate copies out of n is k/n.
    • The per-region BAF likelihood of each CN state is computed and stored in pgx_calls for the star-allele caller (PGX-012).
    • It is not used to change CN by default; --bafs-in-cn is experimental. (src: B-30, plan §11)
  • PCN-011 §11, §13 · v2 PGx CN VCF. One file per sample, <sample>.pgx_cn.vcf.gz with its .tbi. The header:

    ##fileformat=VCFv4.3
    ##source=SimurgArray <semver> pgx-cn
    ##reference=file://<fasta>
    ##simurg_genome_build=GRCh38
    ##simurg_manifest=<name>;sha256=<hex>
    ##simurg_pgx_cn_model=<file>;sha256=<hex>;cluster_file=<name>;regions=<bed version>
    ##simurg_batch=n_samples=<N>;n_passing=<P>;lrr_sd_max=0.2
    ##OverallPloidy=<%.3f>                      (length-weighted mean CN over target regions [INF, OQ-028])
    ##GCCorrect=true
    ##contig=<ID=chr1,length=248956422> …      (contigs that carry target regions, including chr22_KI270879v1_alt)
    ##ALT=<ID=CNV,Description="Copy number variable region (targeted genotype)">
    ##FILTER=<ID=PASS,Description="All filters passed">
    ##FILTER=<ID=Q7,Description="Quality below 7">
    ##FILTER=<ID=SampleQuality,Description="Sample LRR SD above 0.2">
    ##FILTER=<ID=NoCall,Description="Region could not be assessed">
    ##INFO=<ID=END,Number=1,Type=Integer,Description="Last base of the region (1-based, inclusive)">
    ##INFO=<ID=SVTYPE,Number=1,Type=String,Description="CNV">
    ##INFO=<ID=CNVLEN,Number=1,Type=Integer,Description="Region length in bp">
    ##INFO=<ID=PROBE,Number=1,Type=Integer,Description="Number of probes used for the region">
    ##INFO=<ID=REGION,Number=1,Type=String,Description="Target region name">
    ##INFO=<ID=GENE,Number=1,Type=String,Description="Gene symbol">
    ##FORMAT=<ID=CN,Number=1,Type=Integer,Description="Copy number; 5 means 5 or more">
    ##FORMAT=<ID=NR,Number=1,Type=Float,Description="Aggregated normalized intensity (about CN/2)">
    #CHROM POS ID REF ALT QUAL FILTER INFO FORMAT <sample name>
    

    Record rules:

    • POS is the padding base, S − 1; REF is the FASTA base; END is E; CNVLEN = E − S + 1.
    • ID = CNV:<region>:<CHROM>:<S>-<E>. The coordinates in the ID are always the same as POS + 1 and END [DIFF: the reference's IDs sometimes disagree with POS/END, OQ-026].
    • QUAL is an integer in 0–60. FILTER is Q7 if QUAL < 7; SampleQuality and NoCall are added as they apply.
    • FORMAT is CN:NR, with no GT [EMU], and NR has 6 decimals.
    • Records are sorted by contig order, then POS. (src: B-23, D-12, A-G25)
  • PCN-012 §11, §13 · v2 Batch summary. pgx_cn_batch_summary.tsv has columns region, gene, n_gain, n_loss, n_neutral, n_nocall, n_filtered:

    • gain means CN > 2, loss CN < 2, neutral CN = 2;
    • n_filtered counts samples within the other columns that carry any non-PASS FILTER;
    • one row per region, sorted alphabetically, then a final ALL totals row;
    • n_gain + n_loss + n_neutral + n_nocall equals N for every row. (src: B-24, D-12)
  • PCN-013 §11, §13 · v2 Per-sample summary. pgx_cn_sample_summary.tsv has columns:

    • sample_id, sample_name, sample_folder, logr_dev;
    • passed_batch_qc, used_in_reference;
    • one cn_<region> column per region;
    • flags.

    It is the "CN summary" file. (src: B-21, C §2.2)

  • PCN-014 §11 · v2 Placement. The docs recommend writing CN VCFs into the SNV VCF folder. pgx star-allele call --vcfs also accepts several folders and pairs files by sample_id, which is read from the VCF headers (##SAMPLE), not from file names. (src: A-G39)

  • PCN-015 §11 · v2 Genome build. PGx CN and star-allele calling support GRCh38 only in v2.0. A GRCh37 input → E_BUILD_MISMATCH, with a note. GRCh37 support is a future item: region liftover, and GSTT1 lies on the primary chr22 in GRCh37. (src: A-G62, D-10)

  • PCN-016 §11, §19 · v2 Memory model.

    • Calling streams per-sample normalized region data. The batch keeps only per-probe batch medians (O(probes in regions)).
    • Training memory is O(n_samples × n_region_probes).
    • Documented target: 500 GDA-ePGx samples within 8 GB, and 3 000 within 32 GB. (src: A-G43)
  • PCN-017 §11, §16 · v2 Validation. On 1000 Genomes array data, with WGS-derived CN truth:

    • per-region CN concordance ≥ 95 %;
    • CYP2D6 region call rate ≥ 97 %;
    • the outlier-robustness test of EDGE-026 (10 % injected outlier samples) must leave the other samples' calls unchanged in ≥ 99 % of cases. (src: C-G25, C-G27)

8. Star alleles and phenotypes (v2)

8.1 Database

  • PGX-001 §11 · v2 Allele-definition sources, with pinned versions. Every version is recorded in the DB meta table and in every output.
Source Minimum version (parity target) Use
PharmVar 6.1 star-allele definitions (CYP*, NUDT15, SLCO1B1, …)
PharmGKB allele definition tables snapshot 2024-05-16 or later non-PharmVar genes, variant-based genes
CPIC 1.38.0 or later allele function, activity scores, diplotype → phenotype tables
DPWG (via PharmGKB) June 2023 or later DPWG phenotype tables
UGT allele nomenclature (legacy) 2010-12-21 UGT1A1/UGT1A4/UGT2B15 alleles
Human CYP allele nomenclature (legacy archive) July 2024 lettered alleles (CYP1A2 *1A…, CYP2E1)
Population frequencies the CPIC/PharmGKB frequency tables of the same release priors (PGX-014)

(src: D-15, A-G40, plan §11)

  • PGX-002 §11 · v2 DB build. simurg pgx db build produces a SQLite file (IN-053) with these tables:

    • genes(gene, result_basis ∈ {star_allele, haplotype_named, variant_based, tagging_variant});
    • alleles(gene, allele, kind ∈ {star_allele, reference, rsid_variant_allele, hgvs_variant, named_haplotype}, function, activity);
    • allele_variants(gene, allele, build, chrom, pos, ref, alt, hgvs, rsid);
    • probe_map(manifest, probe_name, chrom, pos, ref, alt) (many probes → one variant);
    • masks(manifest, probe_name, reason);
    • limitations(manifest, gene, allele, status, reported_as);
    • frequencies(gene, allele, population, freq);
    • phenotypes(guideline, gene, diplotype_or_activity, phenotype);
    • meta(key, value).

    The build is deterministic: identical inputs give an identical file hash. (src: A-G40, D-14)

  • PGX-003 §11 · v2 Probe-to-variant mapping.

    • SNV VCF records are linked to DB variants through their ID probe names (split on , and ;) via probe_map.
    • The link is then verified on CHROM, POS, REF and the ALT set [DIFF].
    • A record whose ID matches but whose coordinates disagree → a warning, and the variant counts as not covered.
    • Records with no probe ID fall back to position and allele matching. (src: A 3.8, D-14)
  • PGX-004 §11 · v2 Multi-allelic extension. When a definition names one alternate allele at a site that the array's merged record shows as multi-allelic, the definition is extended to the site's full allele set. Other alleles at the site count as "not this allele", not as missing. This is derived automatically from the manifest; the reference hand-listed 52 extensions in CACNA1S, CFTR, COMT, DPYD and RYR1, and those serve as a test oracle. (src: D-14, E-40)

  • PGX-005 §11 · v2 Build-specific definitions. Definitions are stored per build. Where the build's reference base differs from a definition's reference base, the definition is re-expressed. G6PD is the canonical test:

    • GRCh37 "Mediterranean Haplotype" = rs2230037 G>A + rs5030868 G>A.
    • GRCh38 = rs5030868 G>A with rs2230037 reference A.
    • The multi-name allele "Mediterranean, Dallas, Panama, Sassari, Cagliari, Birmingham" is handled the same way.
    • rs2230037 is ignored in all other G6PD alleles. (src: D-14)
  • PGX-006 §11 · v2 *0 whole-gene deletion. For genes with a whole-gene CN region and no named deletion allele (e.g. GSTM1, GSTT1, UGT2B17), CN 0 or 1 adds *0 haplotypes. CYP2D6 uses its named allele *5. (src: D-14)

  • PGX-007 §11 · v2 Tagging-variant genes.

Gene Allele Tag
HLA-A *31:01 rs1061235 A>T
HLA-B *15:02 rs144012689 T>A
HLA-B *57:01 rs2395029 T>G
HLA-B *58:01 rs9263726 G>A
IFNL3/4 – rs12979860

These are reported as call_type = tagging_variant, with the fixed caveat "tag reliability varies by population". (src: D-14)

  • PGX-008 §11 · v2 Coverage parity target (gene list and counts). The archived coverage table, 2 405 allele/variant entries across 58 genes, gives the minimum content our DB must represent. Entry kinds:
    • 1 420 rsID variant alleles;
    • 674 star alleles;
    • 186 named haplotypes;
    • 85 HGVS-only variants;
    • 40 reference entries.
Result basis Genes (entries)
star allele (17) CYP2D6 184 (incl. 14 duplication and 7 hybrid/tandem forms; sub-alleles such as *4M), CYP2C9 85, CYP2A6 51, CYP2B6 46, TPMT 46, CYP3A4 45, SLCO1B1 42, CYP1A2 39 (legacy lettered), CYP2C19 34, NUDT15 20, CYP2C8 18, CYP4F2 16, CYP2E1 11, UGT1A1 9 (incl. *80+*28, *80+*37), UGT1A4 8, UGT2B15 7, CYP3A5 6
named haplotype (2) G6PD 187, APOE 3 (E2/E3/E4)
tagging variant (3) HLA-B 4, HLA-A 2, IFNL3/4 2
variant based (36) DPYD 475, RYR1 355, OPRM1 175, CFTR 59, DRD2 56, COMT 46, OPRD1 46, CACNA1C 37, HTR2A 32, OPRK1 24, MT-RNR1 24, TNF 23, BDNF 23, IL6 21, HTR2C 17, GRIN2B 15, MTHFR 13, ITGB3 12, FKBP5 12, ADH1B 10, ANKK1 10, HMGCR 9, ATM 7, KIF6 6, F2 5, CACNA1S 4, LPA 3, F13A1 3, GRIK1 3, GRIK4 3, YEATS4 2, ABCG2 2, F5 2, ALDH2 2, ANK3 2, VKORC1 2
  • The counts are parity checks, not data to copy. Our DB is built only from public sources (PGX-001).

  • The mitochondrial gene MT-RNR1 needs NC_012920.1:m. HGVS and haploid genotype handling.

  • Suffixed identifiers for several definitions at one rsID (e.g. rs114096998.2.G>C) must be supported. (src: D-17, C-G25)

  • PGX-009 §11 · v2 Array limitations are derived, not hard-coded.

    • For every manifest, simurg pgx coverage computes which alleles are indistinguishable from each other, or not callable, given the probes present and not masked.
    • Results go to the limitations table and are reported as array-equivalents (PGX-017).
    • The reference's known limitations are regression oracles:
      • APOE E2/E4 not callable on GSAv4/GCRA-ePGx;
      • CYP2A6 *5 unsupported on GDA-ePGx;
      • CYP4F2 *2 reported as *1, and *4 as *3;
      • UGT1A1 *28/*37 TA-repeat not covered;
      • UGT2B15 *4/*5 unsupported on GSAv4/GCRA-ePGx;
      • CYP2D6 *40/*58 reported as *40;
      • NUDT15 *6/*9 reported as *6;
      • HLA-B*15:02 absent on GDA-ePGx. (src: D-14)
  • PGX-010 §11 · v2 Masked variants.

    • Per manifest, probes known to perform poorly are masked: their calls are treated as no-calls.
    • Alleles defined only by masked variants are never called, and are listed as "not callable (masked)".
    • Our masks come from concordance against WGS truth (plan §16.5): a probe with < 95 % genotype concordance over ≥ 100 samples is masked.
    • The reference's 24 masked-variant rows (8 genes: CYP1A2, CYP2C19, CYP2C9, CYP2D6, CYP4F2, UGT1A1, OPRM1, COMT; 4 on GDA-ePGx, 10 each on GSAv4/GCRA-ePGx) are a comparison oracle only. (src: D-14)

8.2 Calling

  • PGX-011 §11 · v2 Input guard. pgx star-allele call requires all of the following:

    1. SNV VCFs made with unsquash_duplicates=false and no exclude/include list, as read from ##simurg_options. Otherwise E_PGX_VCF_UNSUITABLE.
    2. Files bgzipped and indexed. Otherwise E_PGX_VCF_UNINDEXED.
    3. A pgx_cn VCF for every sample, unless --no-cn is given. Otherwise E_PGX_CN_MISSING.
    4. The same build and manifest across all inputs.

    Notes:

    • Symlinked inputs are accepted.
    • VCFs from other converters (no ##simurg_options) are refused unless --allow-foreign-vcf is given, in which case a warning is issued and squashing is assumed.
    • Any valid tabix or CSI index is accepted, including one rebuilt with htslib. (src: A-G39, D-19, E-40)
  • PGX-012 §11 · v2 Variant evidence model. Each DB variant gets one of these states:

    • called (GT, GS, BAF);
    • no_call (./.);
    • masked;
    • not_covered (no probe on the array).

    In a region whose CN is n ≠ 2, the variant copy number k ∈ {0..n} is the maximum-likelihood value under the observed BAF ~ N(k/n, σ²), with σ estimated per batch from AB clusters. It is reported as variant_copy_number. GS enters as a per-call error rate: ε(GS) = 0.5·(1 − GS)², clipped to [1e-4, 0.25] [OURS]. (src: B-30, E-41)

  • PGX-013 §11 · v2 Candidate enumeration [DIFF: solver-free].

    • For each gene, SimurgArray enumerates diplotypes from the gene's haplotypes: star alleles, *0, duplications xN, and the hybrid and tandem structures in the DB, such as *36+*10, *13+*4, *68+*4.

    • The enumeration is constrained by the region CN signature. CYP2D6 rules:

      • total CN(p5) = number of gene copies;
      • CN(exon.9) < CN(p5) implies CYP2D7-exon-9 conversion copies, i.e. *36-type (or *83, *4.013-like) units;
      • CN(p5) < CN(intron.2) or CN(exon.9) implies 2D7-2D6 *13-type hybrids.

      The full table is in docs/pgx_cyp2d6_cn_rules.md, derived from PharmVar structural-variant documentation.

    • Branch-and-bound pruning keeps the top 200 candidates.

    • The enumeration is deterministic: fixed order, and no LP/MIP solver. The reference's intermittent solver error and its OS-dependent scores cannot occur here. (src: E-43, B-26)

  • PGX-014 §11 · v2 Scoring. For each candidate solution s:

    • LL(s) = Σ_v log P(obs_v | s) + Σ_r log P(NR_r | CN_r(s)).
      • Variant term: a genotype likelihood with error ε(GS) (PGX-012), plus a BAF term when the variant CN ≠ 2.
      • Region term: from the CN model.
      • no_call, masked and not_covered variants contribute 0.
    • raw_score(s) = exp(LL(s)) / Σ_s' exp(LL(s')): no prior.
    • score(s) = exp(LL(s) + log π(s)) / Σ_s' exp(LL(s') + log π(s')), where π(s) is the product of the two haplotype frequencies in --population (default global, with a floor frequency of 1e-5 for unknown ones).
    • Both scores lie in [0, 1] and are rounded to 4 decimals in outputs.
    • Ranking: score descending, then raw_score, then the solution string in natural order. (src: B-26, D-13, E-42, plan §11)
  • PGX-015 §11 · v2 Feasible alternatives.

    • Every solution with score ≥ --feasible-ratio × top score (default 0.5) is feasible.
    • The gene's genotype field lists all feasible solutions ;-joined, in rank order.
    • candidate_solutions lists up to --max-candidates (default 10) solutions, feasible or not, with their rank. (src: B-27, E-42)
  • PGX-016 §11 · v2 Low-GS review. A supporting variant with GS < 0.15 (--low-gs) gets low_gs: true. The gene gets the flag review_alternatives, and the docs advise checking those clusters for shift. (src: B-27, D-13)

  • PGX-017 §11 · v2 Array-equivalent alleles.

    • Two alleles are array-equivalent in a sample when no covered, unmasked, called variant separates them.
    • For each allele of a solution, the reported allele is the equivalence-class member with the highest population frequency. Ties are broken by lowest star number, then natural order.
    • The other members are listed in array_equivalent_alleles.
    • Sub-alleles such as *4.013 collapse onto their core allele (*4) and are listed as equivalents [OURS]. (src: B-27, D-13)
  • PGX-018 §11 · v2 Background alleles and variant change.

    • For structural or hybrid solutions, background_allele names the allele the SV sits on (e.g. *10 under *36).
    • variant_change describes the observed change: an SV note, a reference call, or "unresolved due to no-call at ".
    • Exon-9 conversion alleles other than *36 (e.g. *83) are reported as themselves, not folded into *36 (EDGE-027). (src: B-26, E-40)
  • PGX-019 §11 · v2 No-call handling.

    • Two lists are kept apart:
      • (a) variant_no_calls: variants on the array but no-called or masked in this sample;
      • (b) variants_not_covered: variants absent from the array. This list is the same for every sample of a manifest.
    • Each carries alleles_impacted.
    • When no-calls leave several candidates, all of them are ranked with a variant_change delta: the variants that differ from the top solution.
    • Candidates whose edit distance from the top exceeds --max-edit-distance (default 3) are flagged distant_alternative. (src: E-41, D-13)
  • PGX-020 §11 · v2 Unresolved variants. An unresolved variant is a defining variant of a solution's allele that is not called. Reasons: not_on_array, no_call, masked. (src: B-25, D-13)

  • PGX-021 §11 · v2 Variant-type calls.

    • Variant-based and tagging genes are reported with call_type = variant. The genotype uses explicit notation: rs9923231 C/T, or Reference when homozygous reference.
    • REF>ALT is always written with >, never _ (EDGE-027).
    • Multi-definition rsIDs keep their suffix identifiers. (src: B-29, D-17, E-40)
  • PGX-022 §11 · v2 Nomenclature.

    • Duplications: *2x2. When the copy count exceeds the CN cap: *1xN (N = unknown, ≥ 3).
    • Tandems and hybrids: *36+*10, *13+*68x2+*4.
    • Diplotypes: A/B, with alleles in natural order unless structure requires otherwise. (src: B §3.12)
  • PGX-023 §11 · v2 Definition sanity tests. Definition regressions are checked in CI:

    • the CYP2A6 *1 definition must contain no spurious variants (the reference once included an extra delins);
    • ABCG2 reference calls get the phenotype "Normal" by default.

    (src: E-40)

  • PGX-024 §11 · v2 Unknown allele in lookup. A called allele or diplotype with no guideline entry gives phenotype Indeterminate and activity null, and flags phenotype_not_in_guideline. It never crashes. The reference crashed when cluster drift produced a CYP2E1 allele with no entry. (src: E-40)

8.3 Phenotype annotation

  • PGX-025 §11 · v2 Guidelines.
    • --guideline cpic (default) or dpwg.
    • Phenotype is a direct lookup of the diplotype, or of the activity score, in the chosen guideline's table.
    • Coverage:
      • CPIC, 17 genes: CACNA1S, CYP2B6, CYP2C19, CYP2C9, CYP2D6, CYP3A5, DPYD, G6PD, MT-RNR1, NUDT15, RYR1, SLCO1B1, TPMT, UGT1A1, CFTR, IFNL3/4, VKORC1. Activity scores: CYP2C9, CYP2D6, DPYD.
      • DPWG, 14 genes: CYP1A2, CYP2B6, CYP2C19, CYP2C9, CYP2D6, CYP3A4, CYP3A5, DPYD, NUDT15, SLCO1B1, TPMT, UGT1A1, VKORC1, F5. Activity scores: CYP2D6, DPYD.
    • Genes outside the chosen guideline get phenotype null.
    • With several feasible solutions, a phenotype is reported only when all of them agree. Otherwise the gene gets Indeterminate plus the list of per-solution phenotypes. (src: D-16, A-G41)

8.4 Outputs

  • PGX-026 §11, §13 · v2 Files.

    • Per sample: <sample>.pgx_calls.parquet (written by call) and <sample>.pgx.json (written by phenotype).
    • Per batch (under pgx/):
      • diplotype_summary.tsv;
      • diplotype_annotations.tsv;
      • supporting_variants.tsv;
      • unresolved_variants.tsv;
      • variants_not_covered.tsv.
    • Every TSV starts with # metadata lines: #simurg_schema=, #software_version=, #genome_build=GRCh38, #pgx_db=<file>;sha256=, #allele_sources=, #phenotype_sources=, #guideline=. (src: B-25, D-18, A-G41, B-28)
  • PGX-027 §11 · v2 TSV columns (in order).

    • diplotype_summary.tsv: sample_id, sample_name, solution_id, gene, call_type, solution, array_equivalent_alleles_1, array_equivalent_alleles_2, no_call_variants, alleles_impacted_by_no_call, background_alleles, variant_change, score, raw_score, copy_number_solution, flags
      • One row per feasible or candidate solution.
      • copy_number_solution is written as CYP2D6.p5=5;CYP2D6.intron.2=5;CYP2D6.exon.9=2.
    • diplotype_annotations.tsv: sample_id, sample_name, gene, guideline, genotype, activity_score, phenotype, flags
      • One row per sample × gene.
    • supporting_variants.tsv: sample_id, solution_id, gene, allele, hgvs, rsid, chrom, pos, ref, alts, genotype, variant_copy_number, gs, baf, probe_ids, low_gs
    • unresolved_variants.tsv: sample_id, gene, solution_id, allele, hgvs, rsid, reason, probe_ids
    • variants_not_covered.tsv: gene, hgvs, rsid, alleles_impacted
      • Depends on the manifest and DB only.

    Formatting rules:

    • Lists inside a cell are ;-joined.
    • Sorting: sample, gene, solution_id, then allele. (src: B-25, D-18)
  • PGX-028 §11 · v2 JSON schema (simurg.pgx/1).

    {"schema":"simurg.pgx/1","software_version":"…","genome_build":"GRCh38",
     "pgx_db":{"file":"pgx_db-1-3fa2c0d9e1b4.sqlite","sha256":"…"},
     "allele_sources":[{"name":"PharmVar","version":"6.1"}],
     "phenotype_sources":[{"name":"CPIC","version":"1.38.0"}],
     "guideline":"cpic","sample_id":"…","sample_name":"…",
     "genes":[{"gene":"CYP2D6","call_type":"star_allele",
       "genotype":"*27/*36x2+*83+*10","feasible_solutions":["*27/*36x2+*83+*10"],
       "activity_score":null,"phenotype":"Indeterminate","score":0.5963,"raw_score":0.9981,"flags":[],
       "array_equivalent_alleles":[{"allele":"*27","equivalents":["*27"]}],
       "supporting_variants":[{"chrom":"chr22","pos":42126611,"ref":"C","alts":["G"],"gt":"0/1","gs":0.8123,
          "baf":0.3185,"variant_copy_number":1,"hgvs":"NC_000022.11:g.42126611C>G","rsid":"rs1135840",
          "probe_ids":["…"],"allele_ids":["*2","*10"],"low_gs":false}],
       "variant_no_calls":[{"hgvs":"…","rsid":"…","probe_ids":["…"],"alleles_impacted":["*4"]}],
       "variants_not_covered":[{"hgvs":"…","rsid":null,"alleles_impacted":["*15"]}],
       "candidate_solutions":[{"rank":1,"genotype":"*27/*36x2+*83+*10","score":0.5963,"raw_score":0.9981,
          "activity_score":null,"phenotype":"Indeterminate",
          "alleles":[{"allele":"*27","supporting_variants":[],"unresolved_variant_sites":[],
                      "array_equivalent_alleles":["*27"],"background_allele":null},
                     {"allele":"*36x2+*83+*10","supporting_variants":[],"unresolved_variant_sites":[],
                      "array_equivalent_alleles":[],"background_allele":"*10"}],
          "variant_no_calls":[],"alleles_impacted_by_no_calls":[],"variant_change":null,
          "copy_number":[{"region":"CYP2D6.p5","cn":5},{"region":"CYP2D6.intron.2","cn":5},{"region":"CYP2D6.exon.9","cn":2}]}],
       "alleles_tested":["*1","*2","…"]}]}
    

    Rules:

    • Numbers are numbers, and missing values are null.
    • Both alleles of every candidate carry full supporting and unresolved details. The reference filled only the first allele.
    • alleles_tested is in natural sort order.
    • The same information appears in the TSVs, and a CI test checks JSON ↔ TSV equivalence. (src: B-26, E-41, E-48)
  • PGX-029 §11, §15 · v2 Determinism. Given identical inputs, outputs are byte-identical across OS, CPU architecture and thread count. Enumeration and summation run in a fixed order, and float formatting is fixed. Golden tests run on x86-64 and aarch64. The reference showed Linux/Windows differences in scores, unresolved variants and collapsed alleles. (src: E-27, E-43, D-13)

  • PGX-030 §11 · v2 Semi-custom arrays [DIFF].

    • Star-allele calling works on any manifest with probe_map entries, so a semi-custom array is handled in one run.
    • pgx coverage reports, for each gene, any definitions whose commercial-backbone probes are missing. A gene losing more than 10 % of its covering probes → a warning.
    • Add-on content that maps to DB variants is used; other add-on content is ignored. (src: A-G42, D-20)
  • PGX-031 §11 · v2 Novel alleles. Out of scope. When the best solution still leaves covered, called variants unexplained (observed ALT not in any allele of the solution), the gene gets the flag unexplained_variants, listing them. It is not reported as a novel allele. (src: D §5.5)

  • PGX-032 §11, §16 · v2 Validation targets (plan §16.5), measured against WGS-derived calls for 1000 Genomes samples:

    • star-allele call rate ≥ 99 % on in-silico data;
    • diplotype concordance ≥ 90 % on in-silico data and ≥ 90 % against WGS truth for CYP2D6/CYP2C19/CYP2C9;
    • CYP2D6 call rate ≥ 97 %. (src: C-G25, C-G27)
  • PGX-033 §11 · v2 No licence, no network [DIFF]. PGx runs fully offline, with no per-sample quota. Results are written per sample as they complete (CLI-012). (src: A-G13, A 2.3)

  • PGX-034 §11 · v2 alleles_tested is the per-gene list of alleles callable with the manifest, from PGX-009, in natural order. It is identical for every sample of a manifest. (src: B-26)

  • PGX-035 §11 · v2 simurg pgx coverage --bpm --db [--csv] writes pgx_coverage.tsv: gene, allele, allele_kind, result_basis, status (callable|not_covered|masked|equivalent_to:<allele>|reported_as:<allele>), covering_probes. The same information feeds variants_not_covered.tsv. The output mirrors the structure of the archived coverage CSV, so the two can be compared directly. (src: D-14, D-13)


9. Methylation control QC (v2)

  • MET-001 §1.2, §9 · v2 Scope.

    • Control-probe QC and a basic signal summary for Infinium methylation arrays: EPIC v1, EPIC v2, EPIC-Xtra-style custom arrays, and MSA.
    • Runs locally through simurg methyl qc; the reference offers this only in its cloud.
    • No differential-methylation analysis.
    • Genome build is irrelevant: no mapping is used. (src: C-G26, B-31)
  • MET-002 §6.3 · v2 Manifest validation. A methylation CSV manifest must contain:

    • an [Assay] section with IlmnID, AddressA_ID, AddressB_ID and Color_Channel;
    • a [Controls] section with exactly 4 columns: Address, Type, Color_Channel, Name.

    Errors (exit 2):

    • E_METH_NO_ASSAY ("Probe section not found");
    • E_METH_NO_CONTROLS ("Missing control probes in manifest");
    • E_METH_MISSING_COLUMNS ("Missing required columns: …");
    • E_METH_CONTROLS_FORMAT ("Controls not formatted correctly. Must have 4 columns (Address,Type,Color_Channel,Name)").

    (src: D-24, B-31)

  • MET-003 §6.3 · v2 Manifest conversion. simurg methyl convert-manifest (and every methyl qc run) writes two files:

    • probes.tsv: probe_id, u, m, col.
      • u = AddressA_ID and m = AddressB_ID.
      • col is R or G for Infinium I and NA for Infinium II. This is the correct reading of a misprint in the reference documentation, OQ-014.
      • Controls get probe_id = ctl_<address>.
    • controls.tsv: address, type, color_channel, name, probe_id.

    The layout mirrors the manifest format of the open-source methylation toolkits, so they can be used as oracles. (src: B-31, D-23)

  • MET-004 §6.6 · v2 Sample sheet.

    • IN-001 … IN-016 apply.
    • Duplicate sample_id → error by default; --on-duplicate-sample drop keeps the first row with a warning (IN-009).
    • Sample_Group colours the PCA. Other columns are passed through to the QC summary.
    • Errors: E_METH_NO_IDATS ("No IDATs found"), E_SHEET_EMPTY, and E_SHEET_NO_KEY_COLUMNS. For the last, the beadChipName/sampleSectionName aliases are named in the message. (src: B-31, A-G59)
  • MET-005 §7.1 · v2 Raw signal assembly.

    • Infinium I: M = channel[AddressB] and U = channel[AddressA], where the channel is the probe's Color_Channel. Out-of-band signal is the other channel.
    • Infinium II: M = Grn[AddressA] and U = Red[AddressA].
    • Beads with NBeads < 3 are masked (detection p = 1) [OURS].
    • Per-sample control table <sample>_ctrl.tsv.gz: address, type, color_channel, name, probe_id, green, red. The reference labels these MG/MR/UG/UR; their per-probe meaning is documented [OURS]. (src: B-31, D-23)
  • MET-006 §9.1 · v2 21 control metrics.

    • The QC-004 rows marked M: restoration; staining ×2; extension ×2; hybridization ×2; target removal I/II; bisulfite conversion I green/green-bg/red/red-bg; bisulfite conversion II and II-bg; specificity I green/red; specificity II and II-bg; non-polymorphic ×2.
    • Defined as in QC-004, with bkg_G/bkg_R as in QC-003 and offset x = 3000 (--bg-offset).
    • The same code path as genotyping control QC.
    • The two documented background definitions disagree (OQ-012). We use the newer ones, and the alternative is selectable (--bs-conv-red-bg-set U3-U5|U4-U6). (src: D-21, C-G26, E-46)
  • MET-007 §9.1 · v2 Thresholds.

    • The profiles methylation_epic and methylation_msa come from QC-024. The +ffpe variant sets restoration to 1.
    • Pass rule: value ≥ threshold, the same inclusive rule as genotyping QC. The reference documents > for methylation (OQ-013).
    • The documentation reproduces the guidance:
      • hybridization, target removal, specificity and the background ratios should not normally be changed;
      • staining, extension, BS-conversion C/U ratios and non-polymorphic may need raising for some scanners;
      • custom or discontinued chips, and NextSeq 550 IDATs, need user thresholds. (src: D-21, B-14)
  • MET-008 §9.2 · v2 Pass/fail.

    • pass_qc = 1 iff every enabled, evaluable metric passes.
    • fail_codes lists the failing keys, ;-joined.
    • na_metrics lists metrics that are NA because controls are missing (MET-009).
    • The fraction of probes passing detection does not affect pass_qc unless --min-prop-passing X is given. (src: D-21, D-22)
  • MET-009 §9.1 · v2 Missing controls. Missing control types or names produce W_METH_MISSING_CONTROL_TYPES or W_METH_MISSING_CONTROL_NAMES, with the list. The affected metric becomes NA, and is excluded from pass_qc (QC-007). (src: D-24)

  • MET-010 §7 · v2 Signal processing [OURS]: open reimplementations of published methods. The reference's formulas are undocumented (OQ-036). Each step is published or an open source, and is validated against an open-source methylation toolkit run as a subprocess oracle.

    1. Detection p-value: pOOBAH-style [LIT: Zhou et al. 2018 NAR]. It uses the empirical CDF of out-of-band Infinium I signal per channel, and gives p = 1 − ECDF(M + U).
    2. Background correction: noob [LIT: Triche et al. 2013 NAR]. A normal-exponential convolution is fitted per channel on the out-of-band signal and applied to the in-band signal.
    3. Dye-bias correction: by default, linear scaling so that the red and green means of the normalization controls (NORM_A/T red vs NORM_C/G green) match their average. --dye-bias nonlinear applies quantile matching of Infinium I red vs green (as in open-source toolkits).
    4. Beta: β = M / (M + U + α), with α = --beta-offset, default 0 and floored signals ≥ 1. α = 100 gives the legacy-desktop convention.
    5. M-value: log2(β' / (1 − β')), where β' is β clipped to [1e-6, 1 − 1e-6].

    A failure in any step for a sample → E_METH_PREP_FAILED for that sample only. The other samples continue, and the exit code is 3. (src: D-22, E-46, B-31)

  • MET-011 §9 · v2 Detection threshold.

    • --pval-threshold defaults to 0.05. A probe passes if p < threshold.
    • Reported per sample: n_probes (all analytical CG and SNP probes), n_pass_detection, prop_probes_passing.
    • The docs recommend excluding probes with p > 0.05 downstream, and warn that detection rates depend on the method used (this method is stricter than legacy ones). (src: D-22)
  • MET-012 §9 · v2 Average signals. Four per-sample means over all analytical probes: avg_green_raw, avg_red_raw, avg_green_norm and avg_red_norm. The _norm values are taken after dye-bias correction and noob. (src: B-31)

  • MET-013 §13 · v2 Outputs.

    • Per sample:
      • <sample>_ctrl.tsv.gz (MET-005);
      • <sample>_cgs.tsv.gz: probe_id, beta, m_value, detection_p, with values at %.6g.
    • Per batch:
      • methyl_qc_summary.tsv and .xlsx. Columns: sentrix_id (12 digits), sentrix_position, sample_id, the pass-through metadata, the 21 metrics, the 4 average signals, scan_time (ISO-8601), n_probes, n_pass_detection, prop_probes_passing, pass_qc, fail_codes, na_metrics. The xlsx highlights failing cells.
      • methyl_qc_summary.html, self-contained, as in QC-047, with two plots:
        • a histogram of prop_probes_passing, coloured by pass status;
        • a PCA of betas over all analytical probes. The PCA uses probes finite in every sample. With more than --pca-max-samples-full (100) samples, it uses a seeded random subset of --pca-probes (10 000) probes. It is coloured by Sample_Group.
      • methyl_qc_summary.pdf: optional, --pdf.
      • pcs.tsv.gz: sample_id, pc1, pc2, sample_group; NA when the sheet has no group.
      • probes.tsv, controls.tsv (MET-003).
      • controls_raw.tsv and controls_qc.tsv (QC-009/011), so that qc report accepts the folder.
    • No common probes → W_METH_NO_FEATURES ("No features, skipping PCA plot"), and the PCA is omitted. (src: B-31, C-G26, D-23)
  • MET-014 §9.5 · v2 Report integration. Methylation datasets can be loaded into qc report (QC-062). They cannot be mixed with genotyping datasets (QC-043). (src: C-G18)

  • MET-015 §14 · v2 Message catalogue.

Code Level Message
E_METH_NO_IDATS error "No IDATs found"
E_SHEET_EMPTY error "No samples in sample sheet"
E_SHEET_DELIMITER / structural error "Sample sheet not correctly formatted"
E_SHEET_NO_KEY_COLUMNS error key columns missing, naming both key forms
E_SHEET_DUP_SAMPLE / W_… error / warning duplicate samples (dropped under --on-duplicate-sample drop)
E_METH_NO_CONTROLS, E_METH_NO_ASSAY, E_METH_MISSING_COLUMNS, E_METH_CONTROLS_FORMAT error MET-002
W_METH_MISSING_SAMPLE warning "Missing sample: " (IDATs absent; the sample fails, the run continues)
W_IDAT_NO_SCAN_TIME warning "No scan time available"
E_METH_PREP_FAILED error (sample) "Prep failed" plus the step name
W_METH_MISSING_CONTROL_TYPES / _NAMES warning metric set to NA
W_METH_NO_FEATURES warning PCA skipped

(src: B-31, D-24)

  • MET-016 §19 · v2 Scale.

    • Per-sample processing is streamed.
    • The PCA uses randomised SVD over at most 10 000 probes when n > 100, so memory is bounded.
    • Regression tests run 80, 90, 100 and 110 samples on a > 900 k-probe manifest (EPIC v2 or MSA), the reference's crash zone, and 1 152 samples. (src: E-46, D-31)
  • MET-017 §9 · v2 Custom manifests. A manifest whose controls are missing or partial still runs: the affected metrics are NA (MET-009). Thresholds for custom or discontinued chips must be user-supplied. suggest-thresholds (QC-026) works on methylation datasets. (src: E-46, D-21)

  • MET-018 §9 · v2 NextSeq 550 IDATs. These are processed normally. The report adds a caveat that the shipped thresholds were not calibrated for this scanner. (src: A-G61)


10. Edge cases & regression-test catalogue

Every item below is an automated test (plan §16.1). "Ref-bug" marks a case where the reference product had a defect; SimurgArray must behave correctly.

  • EDGE-001 §16 · P1 Locale independence. The test suite runs under LC_ALL = C.UTF-8, tr_TR.UTF-8 (dotless-i case folding), de_DE.UTF-8 (decimal comma), ar_SA.UTF-8 (Arabic-Indic digits) and ja_JP.UTF-8.

    • GTC, VCF, TSV, JSON and HTML outputs are byte-identical across all locales.
    • Sheet column matching is not affected by Turkish case rules.
    • Ref-bug: non-ASCII date strings in GTCs broke later reads. (src: E-1)
  • EDGE-002 §16 · P1 GTCs from other tools with bad strings. GTCs written by other tools whose strings have a wrong length or invalid UTF-8 (synthesised) → a clean E_GTC_CORRUPT, or a decode with replacement characters, never an uncaught exception. (src: E-1)

  • EDGE-003 §16 · P4 Corrupt or truncated GTC in a batch of 3 → the other 2 are processed, the summary shows the failed sample, and the exit code is 3. Ref-bug: the whole run aborted. (src: E-2)

  • EDGE-004 §16 · P1 Malformed IDATs. Truncated file, wrong magic, bad TOC offset, encrypted v1, mismatched Red/Grn N → the specific error, per sample. (src: plan §16.1)

  • EDGE-005 §16 · P1 Duplicate IDAT pairs. The same sample in two subfolders gives E_DUP_IDAT with both paths. The result must be identical under a directory listing shuffled by a test hook, and on Linux and macOS. --on-duplicate-idat first|newest must be deterministic. (src: E-5, D-28)

  • EDGE-006 §16 · P1 Long and odd names. Manifest name > 80 characters (ref-bug: failure); cluster file names with spaces and (); output paths with spaces and non-ASCII characters; paths > 260 characters. (src: E-4)

  • EDGE-007 §16 · P1 Sample-sheet matrix. Each of these is a separate test:

    • trailing empty column(s) (ref-bug: a crash, or an empty-key field);
    • Sample_Name and sample_name both present;
    • a reserved column (Sex Estimate, call_rate);
    • a 501-character value;
    • barcode 2.04753E+11 and barcode 204753010023.0;
    • a BOM, CRLF line endings, quoted commas;
    • semicolon delimiters;
    • a [Manifests] section present;
    • a sectionless sheet;
    • RootFolder absolute, relative, and conflicting with the CLI;
    • beadChipName/sampleSectionName keys;
    • an empty Sample_Name in one row;
    • Sample_Name equal to another sample's ID;
    • duplicate names;
    • a 10 000-row sheet (performance, PERF-007). (src: E-9, A-G56, A-G57, A-G58, E-7)
  • EDGE-008 §16 · P4 Manifest mismatch matrix. Each of these must be caught before processing starts:

    • BPM vs CSV differing by one locus (ref-bug: not detected);
    • an EGT built for another manifest (ref-bug: every sample skipped with a dictionary error);
    • IDATs from another chip type (ref-bug: silently inaccurate results). (src: E-3)
  • EDGE-009 §16 · P6 FASTA problems. A .fai from another FASTA → E_FAI_MISMATCH. A missing .fai is built into the cache. chr-prefixed vs bare contig names give an error in strict mode and succeed with --contig-alias auto. The alt contig is named either 22_KI270879v1_alt or chr22_KI270879v1_alt. A GRCh37 FASTA with a GRCh38 manifest → E_BUILD_MISMATCH. (src: E-22, A-G24, B-33)

  • EDGE-010 §16 · P6 Genome cache on and off give identical VCFs. (src: A-G8)

  • EDGE-011 §16 · P6 Multi-probe merge scenarios (OUT-031):

    • the two published ambiguity scenarios;
    • an inconsistent pair;
    • all probes NC;
    • --inf1-nocall-informative on and off, including the conflict fallback;
    • the message format and codes;
    • GS equal to the minimum;
    • no BAF/LRR in the FORMAT;
    • ID joining with , and ;. (src: E-12, E-13, E-17, B-3)
  • EDGE-012 §16 · P6 Opposite-strand duplicate indels ([D/I] and [I/D]), both heterozygous → 0/1. Ref-bug: ./.. (src: E-14)

  • EDGE-013 §16 · P6 SNV and indel at the same position. Written as separate records by default, and merged with --merge-snv-indel. (src: E-15)

  • EDGE-014 §16 · P6 --unsquash-duplicates on a multi-allelic duplicated design → the site stays merged. (src: E-25, A-G21)

  • EDGE-015 §16 · P6 Auxiliary loci. One test per violated restriction (multi-allelic, not an MNV, duplicated assay, allele mapping mismatch, REF ≠ FASTA) → E_AUX_INVALID. A valid MNV restores its true alleles. (src: E-16, A-G18)

  • EDGE-016 §16 · P6 MNV designs in the final-report export. Top alleles are not reverse-complemented. Ref-bug: open issue. (src: E-16)

  • EDGE-017 §16 · P6 BAF orientation. At a locus where the probe's A allele is ALT, the VCF BAF equals 1 − GTC BAF. LRR is unchanged. (src: E-21)

  • EDGE-018 §16 · P6 Indel flips. A manifest deletion that normalises to an insertion is written in the reference-consistent form with the genotype remapped, and the audit carries flipped_indel. Checked against dbSNP-normalised truth. (src: E-19, B-4)

  • EDGE-019 §16 · P6 Skip taxonomy. Unmapped (Chr=0), intensity-only and unresolvable indels each produce exactly one warning code, and the run-summary counts are correct. (src: E-18, B-5)

  • EDGE-020 §16 · P6 Index metadata. bcftools index --nrecords on our .tbi equals the record count. Ref-bug: the index lacked count metadata. The PGx caller accepts an index rebuilt by htslib. (src: E-23)

  • EDGE-021 §16 · P6 VCF validity. Every golden VCF passes bcftools view and a VCF-4.3 validator. The header's declared version matches the syntax used. (src: E-24)

  • EDGE-022 §15, §16 · P6 Cross-platform determinism. The same inputs on Linux x86-64, Linux aarch64 and macOS arm64, with --threads 1, 4 and 16 → byte-identical outputs, excluding run_manifest.json timestamps. Ref-bug: output order and PGx scores differed by OS. (src: E-27, plan §15)

  • EDGE-023 §16 · P6 bedGraph values. The raw tracks equal the GTC values after formatting. A smoothed track, if requested, is a separate file. Ref-bug: smoothing corrupted LRR. (src: E-28)

  • EDGE-024 §16 · v2 Cyto degenerate and regression cases. Each is a separate test:

    • a noisy sample with zero events (ref-bug: overflow);
    • an all-NaN chromosome;
    • a single-probe chromosome;
    • a QUAL-0 event reaching the JSON (ref-bug);
    • the first event getting ISCN (ref-bug);
    • thresholds shown in the right units (ref-bug);
    • a threshold larger than the genome → an empty result and a warning;
    • a probe threshold of 5 → floored to 10 with a warning;
    • an event spanning a centromere gap: effective vs raw size, consistent between call and annotate;
    • 200 and 1 152-sample batches with flat memory (ref-bug);
    • unknown sex (ref-bug: silently treated as male);
    • female with Y probes;
    • PAR events on a manifest without XY loci (ref-bug);
    • mosaic chrY duplication in a male;
    • LOH < 500 kb between copy-neutral neighbours → removed;
    • a gain with mostly negative LRR → removed;
    • logr_dev 0.25 → SampleQuality; 0.35 → whole-chromosome only;
    • more than 10 events on one chromosome → excess_events. (src: E-29, E-30, E-31, E-32, E-33, E-34, E-35, A-G46, A-G51)
  • EDGE-025 §16 · v2 CNV VCF coordinates. For every record, ID coordinates = (POS + 1, END), REF = the FASTA base at POS, and the SVLEN sign follows the mode. The same checks apply to pgx_cn VCFs. Ref-bug: ID and POS disagreed; REF was N. (src: E-36, B-16)

  • EDGE-026 §16 · v2 PGx CN batch and model rules.

    • Batch of 23 → error.
    • 24 samples with 21 passing → error.
    • 24 samples with 22 passing → OK, with a below-recommended warning.
    • A sample with logr_dev exactly 0.20 does not pass at calling (strict <) but passes in training (≤).
    • Cluster mismatch under error and under warn (both reference behaviours).
    • Manifest mismatch → error.
    • Training with 95 samples → error.
    • 10 % injected outlier samples → the other samples' calls are unchanged. (src: E-39, A-G33, A-G34)
  • EDGE-027 §16 · v2 Star-allele regressions. Each is a separate test:

    • an allele absent from the phenotype table (ref-bug: crash);
    • ABCG2 reference → Normal;
    • CYP2D6 *83 not folded into *36;
    • CYP2A6 *1 definition;
    • HGVS and REF>ALT written with >, never _;
    • symlinked VCFs accepted;
    • unsquashed, filtered or unindexed VCFs refused;
    • JSON: both alleles carry supporting details;
    • JSON ↔ TSV equivalence;
    • multi-allelic extension;
    • G6PD Mediterranean on GRCh37 vs GRCh38;
    • many-probes-to-one-variant mapping;
    • no LP/MIP solver dependence. (src: E-40, E-41, E-42, E-43)
  • EDGE-028 §16 · P5 QC report error and behaviour tests.

    • Every E_QC_* and W_QC_* code, including .yaml.txt, a JSON config, a missing wrapper, an unknown key, a non-numeric value, callRate 1.2 and a negative threshold.
    • Browser-level tests (Playwright, headless):
      • 1 001 samples → tabs;
      • 12 001 samples → WebGL scatter, not disabled;
      • switching from a 12 000-sample dataset to a 500-sample dataset without a reload (ref-bug);
      • 15 facet levels;
      • 150-character hover text;
      • dragging and resizing a selection on 12 000 points without freezing (ref-bug);
      • heatmap control alignment with 60 plates (ref-bug);
      • "Download YAML" → re-upload round trip;
      • export of selected rows only. (src: C-G21, E-45)
  • EDGE-029 §16 · v2 Methylation. A manifest missing [Controls]; a control type missing (the metric is NA and excluded); 80–110 samples on a > 900 k-probe array (ref-bug: crash); a missing scan time; a prep failure in one sample, isolated; zero common probes → PCA skipped. (src: E-46, D-24)

  • EDGE-030 §16 · P1 Overwrite and resume. A rerun into the same folder → E_OUTPUT_EXISTS. --force replaces the outputs. --resume after deleting one sample's outputs recomputes only that sample. (src: E-6, A-G4)

  • EDGE-031 §16 · P1 Interruption. SIGINT during a batch → no *.part file left under a final name, a run manifest with status=interrupted, and exit 130. --resume completes the batch. (src: plan §14)

  • EDGE-032 §16 · P6 GTCs from other producers. Accepted by gtc-to-vcf with W_GTC_FOREIGN. Refused by cn call and pgx copy-number with E_GTC_UNSUITABLE. (src: A-G22, B-34)

  • EDGE-033 §16 · P1 IDAT naming. An IDAT whose internal barcode disagrees with its file name → W_IDAT_NAME_MISMATCH, and the file name is used. (src: plan §6.1)

  • EDGE-034 §16 · P8 Non-human diploid. A BovineSNP50 run on UMD3 with a user FASTA produces GTC, summary and QC; ploidy follows the bovine profile. The VCF is produced only when the FASTA contigs match. (src: C-G22)

  • EDGE-035 §16 · P4 Missing inputs. A sheet row with no IDATs → a summary row with MISSING_INPUT, and exit 3 (or 4 if every row is missing). (src: plan)

  • EDGE-036 §16 · P6 Final-report size guard. loci × samples > 1e8 → W_LARGE_OUTPUT, plus a pipeline plan suggestion to turn the output off. (src: B-10, D-30)

  • EDGE-037 §16 · P4 Assay-control column. A PGx manifest gives a TGA_Ctrl_5716_norm_r column. A non-PGx manifest has no such column. A failed sample has NaN. (src: E-26, B-9)

  • EDGE-038 §16 · P6 Unknown sex in the VCF. A sample with sex_est = U gets diploid X and haploid Y/MT, and the header records the policy. (src: E-20)

  • EDGE-039 §16 · P1 Schema conformance. Every JSON output validates against its JSON Schema. No key contains : or other punctuation. Numbers are never strings. (src: E-48, B-35)

  • EDGE-040 §16 · P5 Metadata redaction. --drop-metadata removes the columns from the HTML (checked by searching the embedded JSON for them) and from the xlsx/csv. (src: A-G31, C-G20)


11. Performance & benchmark targets

  • PERF-001 §19 · P6 (Python), P9 (Rust) Genotyping throughput for GSA-size data (~650 k loci), IDAT → GTC

    • SNV VCF:
    • Python reference implementation: ≤ 35 s per sample on one core. This matches the reference product's stated "as little as 35 s per sample".
    • After P9: ≥ 20 samples/min/core (plan §18).
    • Omni2.5-size data: ≤ 4× the GSA time. (src: C-G27, plan §18)
  • PERF-002 §19 · v2 PGx throughput. CN plus star alleles plus phenotype: ≤ 60 s per sample amortised on 8 cores, for a 96-sample batch. (src: C-G27)

  • PERF-003 §19 · v2 Cyto throughput. cn call plus annotate: ≤ 60 s per sample on one core for an 850 k array, with memory ≤ 2 GB per worker, independent of batch size. (src: D-9)

  • PERF-004 §19 · P6 Memory.

    • Genotyping worker: ≤ 1.5 GB for GSA and ≤ 4 GB for Omni5.
    • Genome cache: ≤ 3.5 GB, shared (IN-064).
    • --no-genome-cache mode: ≤ 1 GB.
    • PGx CN: see PCN-016.
    • QC report generation: ≤ 4 GB for 12 000 samples. (src: A-G43, A-G8)
  • PERF-005 §3, §19 · P10 Reference hardware. 8 cores, 32 GB RAM, ≥ 30 GB free disk, no GPU. All targets are measured on this configuration, and on the plan's 16–32-core machine for scaling. (src: A-G12)

  • PERF-006 §19 · P9 Scaling.

    • Near-linear throughput up to 32 threads for per-sample steps.
    • Batch sizes 10, 100, 1 000 and 10 000 (synthetic), with no super-linear growth in time or memory.
    • Tested batch minimums: 1 152 for genotyping, cyto and methylation; 384 for PGx. (src: D-31, SCOPE-009)
  • PERF-007 §19 · P1 Sample sheet. A 10 000-row sheet parses and matches against 10 000 IDAT pairs in < 2 s. The reference slowed down badly above 1 000 rows. (src: E-7)

  • PERF-008 §19 · P5 QC report.

    • The HTML for 12 000 samples is ≤ 60 MB and becomes interactive in < 10 s on the reference hardware.
    • 50 000 samples work in density mode.
    • Selection updates take < 200 ms on 12 000 points. (src: C-G16, E-45)
  • PERF-009 §16.5 · P8, v2 Accuracy targets.

    • Genotype concordance vs WGS truth > 99.5 % (a floor). Against vendor GTCs, plan §16.3's ≥ 99.999 % applies.
    • PGx CN concordance > 95 %.
    • CYP2D6 call rate > 97 %.
    • Star-allele in-silico call rate ≥ 99 %, and diplotype accuracy ≥ 90 %.
    • Mosaic sensitivity ≥ 90 % at 15 % MF / 5 Mb. (src: C-G27, C-G25)
  • PERF-010 §19 · P6 Large outputs. The final report and the multi-sample BCF are streamed, and never held in memory. A 2 000-sample × 2.4 M-locus final report stays within 2 GB RSS. (src: B-10)

  • PERF-011 §19 · v2 Methylation. 1 152 EPIC v2 samples complete within 2 h on the reference hardware, with PCA memory ≤ 8 GB. (src: E-46)


12. Open questions and documentation inconsistencies

These are inconsistencies or gaps in the reference documentation, or open choices of ours. None of them is a requirement. Where a decision was needed, the item names it; everything else goes into docs/known_unknowns.md (plan §20).

ID Issue Decision / action
OQ-001 The reference marks the IDAT folder "required" and also says "folder, sheet, or both" we follow "folder, sheet or both" (CLI-006)
OQ-002 --filter-loci is described as "generates a text file" treated as an input exclude list (OUT-036)
OQ-003 bedGraph described as LRR-only in one place, LRR + BAF in another both tracks (OUT-050)
OQ-004 PGx pass boundary: < 0.2 (calling) vs ≤ 0.2 (training, recommendations) both kept as documented and configurable (PCN-004/005, EDGE-026)
OQ-005 an example command omits a module name; another uses a relative path doc typos; ignored
OQ-006 an older doc lists the cyto DB option under a different name; a typo "star-alle" ignored
OQ-007 LRR SD default: 0.15 (YAML example) vs 0.20 (CLI/UI) 0.20 built in; 0.15 only in genotyping_example (QC-024)
OQ-008 control threshold conflicts: target removal 5 vs 1; hyb 1.25 vs 1; stringency 1.5 vs 2; NSB 5 vs 1 UI values → genotyping; example values → genotyping_example; calibrate with QC-026 on public data
OQ-009 stringency channel: red (formula text) vs green (UI) red by default; stringencyChannel: green option; decide from real control data (P5 experiment)
OQ-010 UI formula for NSB green uses "Bkg Red" treated as a typo; bkg_grn (QC-004)
OQ-011 the example controls.qc_metrics row has NSB green = red and NP ≈ 0.1 example values judged synthetic; no formulas inferred from them
OQ-012 methylation backgrounds: BS-conv I red bg U3–U5 vs U4–U6; specificity II bg S1–S4 vs S1–S3 newer definition by default; alternatives selectable (MET-006)
OQ-013 methylation pass is "> threshold"; genotyping flags "below" uniform inclusive rule value ≥ threshold passes (QC-020, MET-007)
OQ-014 converted-manifest doc says col is NA for Infinium I misprint; NA is for Infinium II (MET-003)
OQ-015 cyto LOH example: CHROM 21, ID on chr22 doc error; our IDs always equal CHROM (CYTO-029)
OQ-016 "GAINLOH" example carries LOHTYPE=AOH; LOHTYPE declared Number=A taxonomy per CYTO-006; Number=1
OQ-017 cyto VCF POS convention changed between doc versions (start vs start−1); chromosome token in IDs varies (chr1 vs 19) VCF-correct padding base; IDs use the CHROM string (CYTO-029)
OQ-018 positive SVLEN for deletions signed by default; --svlen-mode positive (CYTO-029)
OQ-019 interplay of the minQual 20 reporting gate and the Q10 FILTER both kept: FILTER in the VCF, gate in JSON/ISCN (CYTO-013)
OQ-020 LowFraction threshold undocumented; old 5 % cut-off removed default 0.10 [HYP]; calibrate by simulation (CYTO-021)
OQ-021 annotationType documented as an int, shown as a string string constitutional (CYTO-033)
OQ-022 medianLogRDev undocumented replaced by the defined logr_dev_robust (CYTO-033)
OQ-023 GAINLOH included in sample-level LOH counts but excluded at chromosome level included at both levels, plus separate counts (CYTO-033)
OQ-024 reference JSON: numbers as strings, "." sentinels, keys such as "CN:", inconsistent casing, a table/example mismatch (id vs hgvs) typed JSON, clean snake_case keys (OUT-075, PGX-028)
OQ-025 a summary JSON example shows a Sex key where the CSV has MetaData1 doc error; pass-through keys are verbatim sheet headers (IN-013)
OQ-026 SULT1A1 CN-record ID coordinates differ from POS/END; the region definition changed between versions ID and POS/END always consistent; region from our BED (PCN-001/011)
OQ-027 two CYP2A6 regions share the same NR value in an example (per-gene aggregation?) NR computed per region (PCN-009)
OQ-028 meaning of OverallPloidy in the targeted CN VCF length-weighted mean over target regions [INF]; revisit with data
OQ-029 "CN summary" file content undocumented beyond three columns defined by us (PCN-013)
OQ-030 het-excess: plan's k-regularised formula vs (f_ab−2pq)/(2pq+ε) both written (OUT-072); test which matches the reference locus summaries
OQ-031 locus-summary #No_Calls described as loci rather than samples counted over samples (OUT-072)
OQ-032 exact definition of the assay-control "Norm R" xn + yn of the control locus [INF]; verify against summaries from PGx demo data (P5 experiment)
OQ-033 the vendor .msk and .dat model formats are undocumented not parsed; open formats defined (IN-050/051/052)
OQ-034 ISCN notation for mosaic microarray results x<a>~<b> [HYP]; check against the ISCN 2020 text before v2 release
OQ-035 germline vs somatic wording for LOH types on arrays mosaic ≡ somatic, non-mosaic ≡ constitutional (CYTO-006)
OQ-036 methylation detection-p method, beta offset, dye-bias method and noob parameters undocumented open published methods (MET-010); compare against an open-source toolkit
OQ-037 "proportion of probes passing" listed as adjustable yet "not used for pass/fail" not used by default; opt-in (MET-008)
OQ-038 the cloud GenCall cutoff default is not stated 0.15 everywhere
OQ-039 whether "median probe spacing" for effective size is per chromosome or genome-wide per chromosome [INF] (CYTO-010)
OQ-040 how segment QUAL aggregates probe posteriors mean marginal posterior [OURS] (CYTO-012)
OQ-041 whether "min GS" for merged records includes no-call probes includes them [INF] (OUT-032); verify on reference VCFs if any become available
OQ-042 exact semantics of "Infinium I no-call as missing signal" OUT-031 step 5 [INF]
OQ-043 PAR loci coded XY in manifests vs "reported on X" mapped to X PAR coordinates (IN-046)
OQ-044 the reference claims VCF 4.1 while using 4.3 FILTER syntax we write VCF 4.3 consistently (OUT-041)
OQ-045 cyto model contents beyond GC ("primarily GC") our model: GC plus optional reference LRR statistics (IN-051)
OQ-046 EX-chemistry threshold template values not archived genotyping_ex = genotyping until calibrated on GSA v4 data
OQ-047 haploid handling of chrY for female samples in the SNV VCF haploid missing . (OUT-018)
OQ-048 the cluster-mismatch policy flipped between reference versions configurable, error by default (IN-050)
OQ-049 CNV minimum-size defaults changed across versions (0 → 20 kb → 25/50 kb) latest values adopted; changelog rule (OUT-077)
OQ-050 a known issue (manifest names > 80 characters) vanished from the notes without being marked fixed covered by EDGE-006
OQ-051 coverage page groups APOE and HLA with the star-allele genes (19); the parsed table classifies them as named-haplotype and tagging the parsed classification is used (PGX-008)
OQ-052 the whole-chromosome-only rule at LRR SD > 0.3 has no stated coverage criterion ≥ 90 % of the effective span [OURS] (CYTO-014)
OQ-053 the documented batch-composition advice (same reagent lots; a 2-week window) is not enforced by the reference warnings only (PCN-006)
OQ-054 IDAT field that holds the scanner ID/version to be confirmed on real IDATs (P1 experiment, plan U2 family)

13. Traceability

13.1 Spec item → master plan § → phase

Generated from the item tags (build.py). Phases: P0–P10 as in plan §18; v2 = the plan's v2 modules.

ID Title Plan § Phase Sources
SCOPE-001 Purpose §1 P0 plan
SCOPE-002 Emulation targets [EMU] §1 P0 A-G1, C-G1
SCOPE-003 Deliberate differences [DIFF] §1 P0 plan
SCOPE-004 Out of scope §1.3 P0 A-G13, D-26, D-27
SCOPE-005 Species and builds §1 P0 A-G62, C-G22
SCOPE-006 Product test matrix §16 P8 C-G28
SCOPE-007 Product-file compatibility matrix §1 P10 A-G63
SCOPE-008 User runtime requirements §3 P10 A-G12, A-G13, A-G43
SCOPE-009 Batch limits §19 P9 D-9, D-31
SCOPE-010 Semi-custom and custom arrays §11 v2 A-G42, D-20
CLI-001 Command tree §14 P1 A-G1, A-G44
CLI-002 Syntax rules §14 P1 A-G11
CLI-003 Migration aliases §14 P10 A-G1
CLI-004 Global options §14 P1 A-G2, A-G3, A-G7
CLI-005 Option precedence §14, §15 P1 A-G30, E-47
CLI-006 Input-set rule §14 P1 A-G5
CLI-007 Logging §14 P1 A-G2, A-G3
CLI-008 Warning rate limiting §14 P1 B-5
CLI-009 Run summary §14 P1 E-18
CLI-010 Overwrite policy [DIFF] §14 P1 A-G4, E-6
CLI-011 Atomic writes §14 P1 plan
CLI-012 Never discard results [DIFF] §14 P1 A-G10
CLI-013 Threading §14, §15 P1 A-G7
CLI-014 Exit codes §14 P1 A-G10
CLI-015 Interruption §14 P1 plan
CLI-016 Message catalogue §14 P1 A-G56, B-5, B-33, C-G21
CLI-017 Paths §14 P1 E-4
CLI-018 Output layout §13 P1 A-G44
CLI-019 Configuration file §14 P1 D-30
CLI-020 simurg inspect <file>... §14 P1 plan
CLI-021 simurg genotype §14 P4 A-G7, A-G14
CLI-022 simurg gtc-to-vcf §14, §8 P6 A-G8, A-G9, A-G18, A-G19, A-G20
CLI-023 simurg export … §14 P6 A-G26
CLI-024 simurg qc controls §14, §9 P5 A-G27, C-G2
CLI-025 simurg qc samples | loci | contamination | relatedness §14, §9 P5 plan
CLI-026 simurg qc report §14, §9.5 P5 A-G28, A-G29, A-G30, A-G31, C-G6, C-G17, C-G20
CLI-027 `simurg qc suggest-thresholds --data … [--k 4] [--min-samples 30] -… §9 P5 C-G8
CLI-028 simurg cn call §10 v2 A-G45, A-G47, A-G48, C-G24, D-2, D-3
CLI-029 simurg cn annotate §10 v2 A-G49, A-G53, A-G54, D-2
CLI-030 simurg cn build-model --bpm --fasta [--csv] --out model.simurg-cyto §10 v2 A-G52
CLI-031 simurg pgx copy-number call §11 v2 A-G33, D-11, E-39
CLI-032 simurg pgx copy-number train §11 v2 A-G34, A-G35, A-G36
CLI-033 simurg pgx star-allele call §11 v2 A-G39, A-G41, B-28, E-40
CLI-034 simurg pgx phenotype §11 v2 A-G41, B-28, D-16
CLI-035 simurg pgx db build --sources DIR --manifests BPM[,…] --out FILE §11 v2 A-G40
CLI-036 simurg methyl qc §1.2 v2 C-G26, D-21, D-22
CLI-037 simurg pipeline run|plan|watch config.yaml §14 P6 (watch: v2) D-27, D-30
CLI-038 compare, synth, train §14 P7, P1 plan
CLI-039 Cross-step guards §14 P6 A-G9, A-G39
CLI-040 Workflow recipes §14 P10 A-G64, A-G65
IN-001 Format §6.6 P1 A-G58, B-31, E-8
IN-002 Key columns §6.6 P1 A-G59, E-8
IN-003 Case-insensitive columns §6.6 P1 A-G56, E-9
IN-004 Empty header cells §6.6 P1 E-9
IN-005 Reserved column names §6.6 P1 A-G56, E-9
IN-006 Length limit §6.6 P1 A-G56, E-9
IN-007 Barcode and position validation §6.6 P1 A-G56, E-9
IN-008 RootFolder §6.6 P1 A-G57, E-8
IN-009 Sample identity §6.6 P1 B-31, D-24
IN-010 Sample-name precedence [EMU] §6.6 P1 A-G16, E-10
IN-011 Per-command resolution §6.6 P1 A-G16, E-10
IN-012 Output stem §13 P1 plan
IN-013 Metadata pass-through §6.6 P1 A-G15, B-9, E-11
IN-014 Reported sex §6.6 P1 A-G59
IN-015 Plate, well and group §6.6 P1 A-G59, A-G60, C-G3
IN-016 Sheet vs files §6.6 P1 plan
IN-017 Scale §6.6, §19 P1 E-7
IN-018 Parent/replicate columns §6.6 P5 plan
IN-020 Recursive IDAT discovery §6.1 P1 A-G6, E-5
IN-021 Pairing §6.1 P1 A-G6
IN-022 Duplicate pairs [DIFF] §6.1 P1 A-G6, D-28, E-5
IN-023 IDAT validation §6.1 P1 plan
IN-024 IDAT metadata extraction §6.1, §9 P1 B-12, C-G3
IN-025 NextSeq 550 IDATs §6.1 P1 A-G61, C-G24
IN-026 Scan-registration gate [OURS] §14 v2 D-26
IN-027 IDAT ↔ manifest check [DIFF] §6, §7.1 P2 D-29, E-3
IN-030 GTC discovery §6.5 P6 A-G6
IN-031 GTC acceptance §6.5 P6 A-G22, B-34
IN-032 Corrupt GTC [DIFF] §6.5 P4 E-2
IN-033 Tolerant string decoding §6.5 P1 E-1
IN-040 BPM §6.2 P1 E-4
IN-041 CSV manifest §6.3 P1 A-G23, A-G36, D-24
IN-042 BPM ↔ CSV consistency [DIFF] §6.3 P1 E-3
IN-043 BPM ↔ EGT consistency [DIFF] §6.4 P4 B-33, E-3
IN-044 EGT load errors §6.4 P1 B-33
IN-045 Cluster-file names §6.4, §12 P1 A-G17, E-4
IN-046 Manifest mapping conventions §6.3, §8.1 P6 A-G23, E-19
IN-047 Product-file provenance §15 P1 plan
IN-048 Mask file §6 v2 A-G36
IN-049 Pre-flight check table §6, §14 P4 A-G63, B-20, E-3
IN-050 PGx CN model file §11 v2 A-G37, B-20, D-11, E-39
IN-051 Cyto model file §10 v2 A-G52
IN-052 Probe mask (open format) §11 v2 A-G35
IN-053 PGx DB bundle §11 v2 A-G40, D-15
IN-054 Annotation bundle §10.4 v2 A-G53
IN-055 YAML inputs §9 P5 C-G6
IN-060 FASTA + index §8 P6 A-G24, B-33
IN-061 Contig naming §8.2 P6 A-G24, E-22
IN-062 Alt contigs for PGx §11 v2 D-10
IN-063 Build detection §8 P6 B-6
IN-064 Genome cache §8, §19 P6 A-G8
IN-065 Species profile §1, §8 P6 C-G22
OUT-001 GTC v5 writer §6.5 P4 E-1
OUT-002 GTC content rules §6.5 P4 A-G16, E-39
OUT-003 Interoperability §6.5, §16 P4 plan
OUT-010 File naming §8.3, §13 P6 A-G9, A-G25
OUT-011 Exact header §8.3 P6 A-G39, B-6
OUT-012 Record defaults [EMU] §8.3 P6 B-1, B-2
OUT-013 Number formatting §8.3 P6 B-7
OUT-014 Record order §8.3 P6 B-6, E-27
OUT-015 REF/ALT assignment §8.1, §8.3 P6 B-5
OUT-016 BAF/LRR orientation §8.3 P6 E-21
OUT-017 Unmerged probes carry full FORMAT §8.3 P6 B-3
OUT-018 Ploidy §8.3 P6 E-20
OUT-019 Skip taxonomy §8.3 P6 B-5, E-18
OUT-020 Indels §8.1 P6 B-4, E-14, E-19
OUT-021 SNV and indel at the same position §8.3 P6 E-15
OUT-030 Site grouping §8.3 P6 E-12
OUT-031 Genotype combination algorithm §8.3 P6 A-G20, B-3, E-12, E-13
OUT-032 Merged record content §8.3 P6 B-3, E-17
OUT-033 Merged record ID §8.3 P6 E-17
OUT-034 --unsquash-duplicates §8.3 P6 A-G21, E-25
OUT-035 Auxiliary loci (MNV restoration) §8.3 P6 A-G18, E-16
OUT-036 Locus filter lists §8.3 P6 A-G19
OUT-037 --inf1-nocall-informative §8.3 P6 A-G20, B-3, E-13
OUT-038 Index §8.3 P6 E-23
OUT-039 Validation §16 P6 E-24
OUT-040 Multi-sample BCF §8.3 P6 plan
OUT-041 VCF version §8.3 P6 E-24
OUT-042 Ambiguous-strand loci §8.2 P6 plan
OUT-043 PLINK export §8.4 P6 B-8
OUT-050 bedGraph tracks §13 P6 A-G26, B-32
OUT-051 bedGraph without re-genotyping §13 P6 A-G26
OUT-052 No smoothing of raw tracks [DIFF] §13 P6 D-3, E-28
OUT-060 sample_summary.tsv and .json §13, §9.2 P4 (QC columns P5) A-G14, A-G15, B-9, C-G11, E-11
OUT-061 Failed samples [EMU] §13 P4 B-9, E-2
OUT-062 Assay-control locus metric §9.1, §13 P4 A-G32, B-9, C-G10, E-26
OUT-063 Precision §13 P4 plan
OUT-064 JSON form §13 P4 B-9
OUT-065 CSV form §13 P4 A-G14
OUT-066 Compatibility column names §13 P4 A-G14, C-G1
OUT-070 Control tables §9.1, §13 P5 A-G14, C-G1
OUT-071 Final report export §6.7 P6 B-10, C-G23, E-16
OUT-072 Locus summary §7.9 P6 B-11, C-G23
OUT-073 Run manifest §15 P1 E-47
OUT-074 Audit tables §8.2, §15 P6 B-3, B-5
OUT-075 Schema discipline [DIFF] §15 P6 B-18, B-35, E-48
OUT-076 Deterministic ordering §15 P6 E-5, E-27
OUT-077 Default-change changelog §15 P6 E-47
QC-001 Control definitions §9.1 P5 A-G27
QC-002 Canonical control names §9.1 P5 C-G4
QC-003 Backgrounds and offset §9.1 P5 B-13, E-44
QC-004 Metric catalogue §9.1 P5, v2 B-13, C-G5, D-25, E-44
QC-005 Genotyping metric set §9.1 P5 A-G32, D-25
QC-006 Methylation metric set §9.1 v2 C-G26, D-21
QC-007 Missing controls §9.1 P5 B-31, D-24
QC-008 Degenerate values §9.1 P5 plan
QC-009 controls_raw.tsv §9.1 P5 B-12, C-G1
QC-010 Report exclusion list §9.5 P5 C-G4
QC-011 controls_qc.tsv §9.1 P5 B-12, C-G1, C-G3
QC-012 Control-only QC path §9, §14 P5 A-G27, C-G2
QC-013 Genotype emits controls §9, §13 P5 A-G14
QC-014 Adding controls to an existing genotype folder §14 P5 plan
QC-020 Threshold YAML schema §9 P5 A-G30, C-G5, E-44
QC-021 Validation §9 P5 C-G6, C-G21
QC-022 Disabled checks §9 P5 A-G30
QC-023 Precedence and built-ins §9 P5 A-G30, C-G6
QC-024 Shipped profiles §9 P5, v2 A-G30, B-14, C-G7, D-21
QC-025 Profile auto-selection §9 P5 B-14, C-G7
QC-026 Data-driven threshold suggestion §9 P5 C-G8
QC-030 Functional status §9.2 P5 C-G9, C-G11
QC-031 Control status §9.2 P5 C-G9
QC-032 Methylation pass §9.2 P5 D-21, D-22
QC-033 Sample QC flags §9.2 P5 plan
QC-034 Assay-control metric in QC §9.2 P5 A-G32, C-G10
QC-035 Recommended downstream gates §9.2, §11 P5 A-G34, A-G65
QC-040 Input contract §9.5 P5 A-G28, C-G1
QC-041 --data forms §9.5 P5 A-G28, C-G17, C-G21
QC-042 Labels §9.5 P5 A-G28, C-G17
QC-043 Dataset types §9.5 P5 C-G18
QC-044 Output files §9.5 P5 B-15, C-G19
QC-045 QC table layout §9.5 P5 A-G29, B-15, C-G19
QC-046 Metadata privacy §9.5 P5 A-G31, C-G20
QC-047 Technology §9.5 P5 C-G12
QC-048 Chrome §9.5 P5 C-G12
QC-049 View A: Control Dashboard §9.5 P5 C-G12
QC-050 View B: Automated QC §9.5 P5 C-G12, C-G13
QC-051 Linked selection §9.5 P5 C-G13, E-45
QC-052 View C: Sample Heatmaps §9.5 P5 C-G12, C-G14, E-45
QC-053 Chip completeness §9.5 P5 C-G15
QC-054 View D: Trend Analysis §9.5 P5 C-G12, C-G15
QC-055 View E: QC Thresholds (modal) §9.5 P5 C-G12
QC-056 Metadata eligibility §9.5 P5 C-G13, E-45
QC-057 Scale behaviour §9.5 P5 C-G16, E-45
QC-058 Multi-dataset behaviour §9.5 P5 A-G28, C-G17
QC-059 Sex view §9.5 P5 plan
QC-060 Control-metric PCA (optional card) §9.5 P5 A-G60
QC-061 Robust CSV/TSV reading §9.5 P5 plan
QC-062 Methylation datasets in the report §9.5 P5 C-G26
QC-063 Acceptance tests §16 P5 C-G21
QC-064 Contamination and relatedness §9.3, §9.4 P5 plan
CYTO-001 Scope §10 v2 D-7, E-35
CYTO-002 Inputs §10 v2 A-G52
CYTO-003 GC-wave correction §10.1 v2 A-G52
CYTO-004 LRR smoothing §10.2 v2 A-G48, D-3, E-28
CYTO-005 State space §10.2 v2 A-G45, D-1, E-31
CYTO-006 Event types §10.2 v2 A-G45, B-16, D-1, E-31
CYTO-007 Per-type thresholds §10.2 v2 A-G45, B-17, D-2, E-29, E-38
CYTO-008 Probe floor [DIFF] §10.2 v2 A-G46, E-29
CYTO-009 Thresholds that exceed the genome §10.2 v2 E-29
CYTO-010 Effective size §10.2 v2 D-7
CYTO-011 Size basis [DIFF] §10.2 v2 D-2, E-30
CYTO-012 QUAL §10.2 v2 B-16, D-5
CYTO-013 Filters §10.2 v2 A-G49, A-G50, B-17, D-5, E-29
CYTO-014 Sample-level noise gating §10.2 v2 A-G50, E-33
CYTO-015 Post-filters §10.2 v2 E-33
CYTO-016 Excess events §10.2 v2 A-G51, E-33
CYTO-017 Centromeres and gaps §10.2 v2 D-6, E-35
CYTO-018 Mosaic fraction (MF) §10.3 v2 A-G47, D-4
CYTO-019 MF not estimable §10.3 v2 A-G47, E-32
CYTO-020 Promotion §10.3 v2 A-G47, D-4, E-32
CYTO-021 Low fraction [DIFF] §10.3 v2 D-4, E-32
CYTO-022 Sensitivity target §10.3, §16 v2 A-G55, D-4, E-32
CYTO-023 PAR §10.2 v2 D-6, E-34
CYTO-024 Sex resolution [DIFF] §10.2 v2 D-6, E-34
CYTO-025 Overall ploidy §10.2 v2 B-22
CYTO-026 Copy-number median §10.2 v2 E-37
CYTO-027 Event statistics §10.2 v2 plan
CYTO-028 File §10, §13 v2 A-G25, B-16
CYTO-029 Coordinates [DIFF, VCF-correct 1-based] §10 v2 B-16, E-36
CYTO-030 Exact header §10 v2 A-G47, A-G50, B-16, D-5
CYTO-031 GT semantics [DIFF] §10 v2 D-5, E-36
CYTO-032 Sidecar and segments §10 v2 A-G44, A-G54
CYTO-033 <sample>.cyto.json §10.4 v2 A-G53, B-18, D-7, E-37
CYTO-034 ISCN 2020 generator §10.4 v2 B-19, D-8, E-37
CYTO-035 Genes and bands §10.4 v2 A-G53, B-18
CYTO-036 Annotation bundle build §10.4 v2 A-G53
CYTO-037 Batch outputs §10 v2 B-21, C-G24
CYTO-038 Model building §10 v2 A-G52
CYTO-039 Batch scale §10, §19 v2 D-9, E-35
CYTO-040 Degenerate samples §10 v2 E-35
CYTO-041 MF refinements §10.3 v2 K-027
CYTO-042 Per-probe LRR noise §10.2 v2 K-027
PCN-001 Region catalogue §11 v2 B-23, C-G25, D-10
PCN-002 Genotype every region [EMU] §11 v2 B-23
PCN-003 Inputs §11 v2 A-G37
PCN-004 Batch rules §11 v2 A-G33, C-G25, D-11, E-39
PCN-005 Model training §11 v2 A-G34, E-39
PCN-006 Batch-composition warnings [OURS] §11 v2 A-G38
PCN-007 Model compatibility §11 v2 A-G37, B-20, D-11, E-39
PCN-008 Mask §11 v2 A-G35
PCN-009 Calling algorithm [HYP, plan §11; U10] §11 v2 A-G33
PCN-010 CN-consistent BAF evidence §11 v2 B-30
PCN-011 PGx CN VCF §11, §13 v2 A-G25, B-23, D-12
PCN-012 Batch summary §11, §13 v2 B-24, D-12
PCN-013 Per-sample summary §11, §13 v2 B-21
PCN-014 Placement §11 v2 A-G39
PCN-015 Genome build §11 v2 A-G62, D-10
PCN-016 Memory model §11, §19 v2 A-G43
PCN-017 Validation §11, §16 v2 C-G25, C-G27
PGX-001 Allele-definition sources, with pinned versions §11 v2 A-G40, D-15
PGX-002 DB build §11 v2 A-G40, D-14
PGX-003 Probe-to-variant mapping §11 v2 D-14
PGX-004 Multi-allelic extension §11 v2 D-14, E-40
PGX-005 Build-specific definitions §11 v2 D-14
PGX-006 *0 whole-gene deletion §11 v2 D-14
PGX-007 Tagging-variant genes §11 v2 D-14
PGX-008 Coverage parity target (gene list and counts) §11 v2 C-G25, D-17
PGX-009 Array limitations are derived, not hard-coded §11 v2 D-14
PGX-010 Masked variants §11 v2 D-14
PGX-011 Input guard §11 v2 A-G39, D-19, E-40
PGX-012 Variant evidence model §11 v2 B-30, E-41
PGX-013 Candidate enumeration [DIFF: solver-free] §11 v2 B-26, E-43
PGX-014 Scoring §11 v2 B-26, D-13, E-42
PGX-015 Feasible alternatives §11 v2 B-27, E-42
PGX-016 Low-GS review §11 v2 B-27, D-13
PGX-017 Array-equivalent alleles §11 v2 B-27, D-13
PGX-018 Background alleles and variant change §11 v2 B-26, E-40
PGX-019 No-call handling §11 v2 D-13, E-41
PGX-020 Unresolved variants §11 v2 B-25, D-13
PGX-021 Variant-type calls §11 v2 B-29, D-17, E-40
PGX-022 Nomenclature §11 v2 plan
PGX-023 Definition sanity tests §11 v2 E-40
PGX-024 Unknown allele in lookup §11 v2 E-40
PGX-025 Guidelines §11 v2 A-G41, D-16
PGX-026 Files §11, §13 v2 A-G41, B-25, B-28, D-18
PGX-027 TSV columns §11 v2 B-25, D-18
PGX-028 JSON schema (simurg.pgx/1) §11 v2 B-26, E-41, E-48
PGX-029 Determinism §11, §15 v2 D-13, E-27, E-43
PGX-030 Semi-custom arrays [DIFF] §11 v2 A-G42, D-20
PGX-031 Novel alleles §11 v2 plan
PGX-032 Validation targets §11, §16 v2 C-G25, C-G27
PGX-033 No licence, no network [DIFF] §11 v2 A-G13
PGX-034 alleles_tested §11 v2 B-26
PGX-035 simurg pgx coverage --bpm --db [--csv] §11 v2 D-13, D-14
MET-001 Scope §1.2, §9 v2 B-31, C-G26
MET-002 Manifest validation §6.3 v2 B-31, D-24
MET-003 Manifest conversion §6.3 v2 B-31, D-23
MET-004 Sample sheet §6.6 v2 A-G59, B-31
MET-005 Raw signal assembly §7.1 v2 B-31, D-23
MET-006 21 control metrics §9.1 v2 C-G26, D-21, E-46
MET-007 Thresholds §9.1 v2 B-14, D-21
MET-008 Pass/fail §9.2 v2 D-21, D-22
MET-009 Missing controls §9.1 v2 D-24
MET-010 Signal processing [OURS]: open reimplementations of published methods §7 v2 B-31, D-22, E-46
MET-011 Detection threshold §9 v2 D-22
MET-012 Average signals §9 v2 B-31
MET-013 Outputs §13 v2 B-31, C-G26, D-23
MET-014 Report integration §9.5 v2 C-G18
MET-015 Message catalogue §14 v2 B-31, D-24
MET-016 Scale §19 v2 D-31, E-46
MET-017 Custom manifests §9 v2 D-21, E-46
MET-018 NextSeq 550 IDATs §9 v2 A-G61
EDGE-001 Locale independence §16 P1 E-1
EDGE-002 GTCs from other tools with bad strings §16 P1 E-1
EDGE-003 Corrupt or truncated GTC §16 P4 E-2
EDGE-004 Malformed IDATs §16 P1 plan
EDGE-005 Duplicate IDAT pairs §16 P1 D-28, E-5
EDGE-006 Long and odd names §16 P1 E-4
EDGE-007 Sample-sheet matrix §16 P1 A-G56, A-G57, A-G58, E-7, E-9
EDGE-008 Manifest mismatch matrix §16 P4 E-3
EDGE-009 FASTA problems §16 P6 A-G24, B-33, E-22
EDGE-010 Genome cache on and off §16 P6 A-G8
EDGE-011 Multi-probe merge scenarios §16 P6 B-3, E-12, E-13, E-17
EDGE-012 Opposite-strand duplicate indels §16 P6 E-14
EDGE-013 SNV and indel at the same position §16 P6 E-15
EDGE-014 --unsquash-duplicates on a multi-allelic duplicated design §16 P6 A-G21, E-25
EDGE-015 Auxiliary loci §16 P6 A-G18, E-16
EDGE-016 MNV designs in the final-report export §16 P6 E-16
EDGE-017 BAF orientation §16 P6 E-21
EDGE-018 Indel flips §16 P6 B-4, E-19
EDGE-019 Skip taxonomy §16 P6 B-5, E-18
EDGE-020 Index metadata §16 P6 E-23
EDGE-021 VCF validity §16 P6 E-24
EDGE-022 Cross-platform determinism §15, §16 P6 E-27
EDGE-023 bedGraph values §16 P6 E-28
EDGE-024 Cyto degenerate and regression cases §16 v2 A-G46, A-G51, E-29, E-30, E-31, E-32, E-33, E-34, E-35
EDGE-025 CNV VCF coordinates §16 v2 B-16, E-36
EDGE-026 PGx CN batch and model rules §16 v2 A-G33, A-G34, E-39
EDGE-027 Star-allele regressions §16 v2 E-40, E-41, E-42, E-43
EDGE-028 QC report error and behaviour tests §16 P5 C-G21, E-45
EDGE-029 Methylation §16 v2 D-24, E-46
EDGE-030 Overwrite and resume §16 P1 A-G4, E-6
EDGE-031 Interruption §16 P1 plan
EDGE-032 GTCs from other producers §16 P6 A-G22, B-34
EDGE-033 IDAT naming §16 P1 plan
EDGE-034 Non-human diploid §16 P8 C-G22
EDGE-035 Missing inputs §16 P4 plan
EDGE-036 Final-report size guard §16 P6 B-10, D-30
EDGE-037 Assay-control column §16 P4 B-9, E-26
EDGE-038 Unknown sex in the VCF §16 P6 E-20
EDGE-039 Schema conformance §16 P1 B-35, E-48
EDGE-040 Metadata redaction §16 P5 A-G31, C-G20
PERF-001 Genotyping throughput §19 P6 (Python), P9 (Rust) C-G27
PERF-002 PGx throughput §19 v2 C-G27
PERF-003 Cyto throughput §19 v2 D-9
PERF-004 Memory §19 P6 A-G8, A-G43
PERF-005 Reference hardware §3, §19 P10 A-G12
PERF-006 Scaling §19 P9 D-31
PERF-007 Sample sheet §19 P1 E-7
PERF-008 QC report §19 P5 C-G16, E-45
PERF-009 Accuracy targets §16.5 P8, v2 C-G25, C-G27
PERF-010 Large outputs §19 P6 B-10
PERF-011 Methylation §19 v2 E-46
OQ-001 The reference marks the IDAT folder "required" and also says "folde… §20 tracked in docs/known_unknowns.md –
OQ-002 --filter-loci is described as "generates a text file" §20 tracked in docs/known_unknowns.md –
OQ-003 bedGraph described as LRR-only in one place, LRR + BAF in another §20 tracked in docs/known_unknowns.md –
OQ-004 PGx pass boundary: < 0.2 (calling) vs ≤ 0.2 (training, recommen… §20 tracked in docs/known_unknowns.md –
OQ-005 an example command omits a module name; another uses a relative path §20 tracked in docs/known_unknowns.md –
OQ-006 an older doc lists the cyto DB option under a different name; a typ… §20 tracked in docs/known_unknowns.md –
OQ-007 LRR SD default: 0.15 (YAML example) vs 0.20 (CLI/UI) §20 tracked in docs/known_unknowns.md –
OQ-008 control threshold conflicts: target removal 5 vs 1; hyb 1.25 vs 1;… §20 tracked in docs/known_unknowns.md –
OQ-009 stringency channel: red (formula text) vs green (UI) §20 tracked in docs/known_unknowns.md –
OQ-010 UI formula for NSB green uses "Bkg Red" §20 tracked in docs/known_unknowns.md –
OQ-011 the example controls.qc_metrics row has NSB green = red and NP ≈ 0.1 §20 tracked in docs/known_unknowns.md –
OQ-012 methylation backgrounds: BS-conv I red bg U3–U5 vs U4–U6; specifici… §20 tracked in docs/known_unknowns.md –
OQ-013 methylation pass is "> threshold"; genotyping flags "below" §20 tracked in docs/known_unknowns.md –
OQ-014 converted-manifest doc says col is NA for Infinium I §20 tracked in docs/known_unknowns.md –
OQ-015 cyto LOH example: CHROM 21, ID on chr22 §20 tracked in docs/known_unknowns.md –
OQ-016 "GAINLOH" example carries LOHTYPE=AOH; LOHTYPE declared Number=A §20 tracked in docs/known_unknowns.md –
OQ-017 cyto VCF POS convention changed between doc versions (start vs star… §20 tracked in docs/known_unknowns.md –
OQ-018 positive SVLEN for deletions §20 tracked in docs/known_unknowns.md –
OQ-019 interplay of the minQual 20 reporting gate and the Q10 FILTER §20 tracked in docs/known_unknowns.md –
OQ-020 LowFraction threshold undocumented; old 5 % cut-off removed §20 tracked in docs/known_unknowns.md –
OQ-021 annotationType documented as an int, shown as a string §20 tracked in docs/known_unknowns.md –
OQ-022 medianLogRDev undocumented §20 tracked in docs/known_unknowns.md –
OQ-023 GAINLOH included in sample-level LOH counts but excluded at chromos… §20 tracked in docs/known_unknowns.md –
OQ-024 reference JSON: numbers as strings, "." sentinels, keys such as `… §20 tracked in docs/known_unknowns.md –
OQ-025 a summary JSON example shows a Sex key where the CSV has MetaData1 §20 tracked in docs/known_unknowns.md –
OQ-026 SULT1A1 CN-record ID coordinates differ from POS/END; the region de… §20 tracked in docs/known_unknowns.md –
OQ-027 two CYP2A6 regions share the same NR value in an example (per-gene… §20 tracked in docs/known_unknowns.md –
OQ-028 meaning of OverallPloidy in the targeted CN VCF §20 tracked in docs/known_unknowns.md –
OQ-029 "CN summary" file content undocumented beyond three columns §20 tracked in docs/known_unknowns.md –
OQ-030 het-excess: plan's k-regularised formula vs (f_ab−2pq)/(2pq+ε) §20 tracked in docs/known_unknowns.md –
OQ-031 locus-summary #No_Calls described as loci rather than samples §20 tracked in docs/known_unknowns.md –
OQ-032 exact definition of the assay-control "Norm R" §20 tracked in docs/known_unknowns.md –
OQ-033 the vendor .msk and .dat model formats are undocumented §20 tracked in docs/known_unknowns.md –
OQ-034 ISCN notation for mosaic microarray results §20 tracked in docs/known_unknowns.md –
OQ-035 germline vs somatic wording for LOH types on arrays §20 tracked in docs/known_unknowns.md –
OQ-036 methylation detection-p method, beta offset, dye-bias method and no… §20 tracked in docs/known_unknowns.md –
OQ-037 "proportion of probes passing" listed as adjustable yet "not used f… §20 tracked in docs/known_unknowns.md –
OQ-038 the cloud GenCall cutoff default is not stated §20 tracked in docs/known_unknowns.md –
OQ-039 whether "median probe spacing" for effective size is per chromosome… §20 tracked in docs/known_unknowns.md –
OQ-040 how segment QUAL aggregates probe posteriors §20 tracked in docs/known_unknowns.md –
OQ-041 whether "min GS" for merged records includes no-call probes §20 tracked in docs/known_unknowns.md –
OQ-042 exact semantics of "Infinium I no-call as missing signal" §20 tracked in docs/known_unknowns.md –
OQ-043 PAR loci coded XY in manifests vs "reported on X" §20 tracked in docs/known_unknowns.md –
OQ-044 the reference claims VCF 4.1 while using 4.3 FILTER syntax §20 tracked in docs/known_unknowns.md –
OQ-045 cyto model contents beyond GC ("primarily GC") §20 tracked in docs/known_unknowns.md –
OQ-046 EX-chemistry threshold template values not archived §20 tracked in docs/known_unknowns.md –
OQ-047 haploid handling of chrY for female samples in the SNV VCF §20 tracked in docs/known_unknowns.md –
OQ-048 the cluster-mismatch policy flipped between reference versions §20 tracked in docs/known_unknowns.md –
OQ-049 CNV minimum-size defaults changed across versions (0 → 20 kb → 25/5… §20 tracked in docs/known_unknowns.md –
OQ-050 a known issue (manifest names > 80 characters) vanished from the no… §20 tracked in docs/known_unknowns.md –
OQ-051 coverage page groups APOE and HLA with the star-allele genes (19);… §20 tracked in docs/known_unknowns.md –
OQ-052 the whole-chromosome-only rule at LRR SD > 0.3 has no stated covera… §20 tracked in docs/known_unknowns.md –
OQ-053 the documented batch-composition advice (same reagent lots; a 2-wee… §20 tracked in docs/known_unknowns.md –
OQ-054 IDAT field that holds the scanner ID/version §20 tracked in docs/known_unknowns.md –

13.2 Gap-analysis coverage check

Each of the 207 numbered gap items in the archived analyses (A: G1–G65, B: 1–35, C: G1–G28, D: 1–31, E: 1–48) is listed with the spec items that absorb it. Uncovered items: none.

Gap item Spec items
A-G1 SCOPE-002, CLI-001, CLI-003
A-G2 CLI-004, CLI-007
A-G3 CLI-004, CLI-007
A-G4 CLI-010, EDGE-030
A-G5 CLI-006
A-G6 IN-020, IN-021, IN-022, IN-030
A-G7 CLI-004, CLI-013, CLI-021
A-G8 CLI-022, IN-064, EDGE-010, PERF-004
A-G9 CLI-022, CLI-039, OUT-010
A-G10 CLI-012, CLI-014
A-G11 CLI-002
A-G12 SCOPE-008, PERF-005
A-G13 SCOPE-004, SCOPE-008, PGX-033
A-G14 CLI-021, OUT-060, OUT-065, OUT-066, OUT-070, QC-013
A-G15 IN-013, OUT-060
A-G16 IN-010, IN-011, OUT-002
A-G17 IN-045
A-G18 CLI-022, OUT-035, EDGE-015
A-G19 CLI-022, OUT-036
A-G20 CLI-022, OUT-031, OUT-037
A-G21 OUT-034, EDGE-014
A-G22 IN-031, EDGE-032
A-G23 IN-041, IN-046
A-G24 IN-060, IN-061, EDGE-009
A-G25 OUT-010, CYTO-028, PCN-011
A-G26 CLI-023, OUT-050, OUT-051
A-G27 CLI-024, QC-001, QC-012
A-G28 CLI-026, QC-040, QC-041, QC-042, QC-058
A-G29 CLI-026, QC-045
A-G30 CLI-005, CLI-026, QC-020, QC-022, QC-023, QC-024
A-G31 CLI-026, QC-046, EDGE-040
A-G32 OUT-062, QC-005, QC-034
A-G33 CLI-031, PCN-004, PCN-009, EDGE-026
A-G34 CLI-032, QC-035, PCN-005, EDGE-026
A-G35 CLI-032, IN-052, PCN-008
A-G36 CLI-032, IN-041, IN-048
A-G37 IN-050, PCN-003, PCN-007
A-G38 PCN-006
A-G39 CLI-033, CLI-039, OUT-011, PCN-014, PGX-011
A-G40 CLI-035, IN-053, PGX-001, PGX-002
A-G41 CLI-033, CLI-034, PGX-025, PGX-026
A-G42 SCOPE-010, PGX-030
A-G43 SCOPE-008, PCN-016, PERF-004
A-G44 CLI-001, CLI-018, CYTO-032
A-G45 CLI-028, CYTO-005, CYTO-006, CYTO-007
A-G46 CYTO-008, EDGE-024
A-G47 CLI-028, CYTO-018, CYTO-019, CYTO-020, CYTO-030
A-G48 CLI-028, CYTO-004
A-G49 CLI-029, CYTO-013
A-G50 CYTO-013, CYTO-014, CYTO-030
A-G51 CYTO-016, EDGE-024
A-G52 CLI-030, IN-051, CYTO-002, CYTO-003, CYTO-038
A-G53 CLI-029, IN-054, CYTO-033, CYTO-035, CYTO-036
A-G54 CLI-029, CYTO-032
A-G55 CYTO-022
A-G56 CLI-016, IN-003, IN-005, IN-006, IN-007, EDGE-007
A-G57 IN-008, EDGE-007
A-G58 IN-001, EDGE-007
A-G59 IN-002, IN-014, IN-015, MET-004
A-G60 IN-015, QC-060
A-G61 IN-025, MET-018
A-G62 SCOPE-005, PCN-015
A-G63 SCOPE-007, IN-049
A-G64 CLI-040
A-G65 CLI-040, QC-035
B-1 OUT-012
B-2 OUT-012
B-3 OUT-017, OUT-031, OUT-032, OUT-037, OUT-074, EDGE-011
B-4 OUT-020, EDGE-018
B-5 CLI-008, CLI-016, OUT-015, OUT-019, OUT-074, EDGE-019
B-6 IN-063, OUT-011, OUT-014
B-7 OUT-013
B-8 OUT-043
B-9 IN-013, OUT-060, OUT-061, OUT-062, OUT-064, EDGE-037
B-10 OUT-071, EDGE-036, PERF-010
B-11 OUT-072
B-12 IN-024, QC-009, QC-011
B-13 QC-003, QC-004
B-14 QC-024, QC-025, MET-007
B-15 QC-044, QC-045
B-16 CYTO-006, CYTO-012, CYTO-028, CYTO-029, CYTO-030, EDGE-025
B-17 CYTO-007, CYTO-013
B-18 OUT-075, CYTO-033, CYTO-035
B-19 CYTO-034
B-20 IN-049, IN-050, PCN-007
B-21 CYTO-037, PCN-013
B-22 CYTO-025
B-23 PCN-001, PCN-002, PCN-011
B-24 PCN-012
B-25 PGX-020, PGX-026, PGX-027
B-26 PGX-013, PGX-014, PGX-018, PGX-028, PGX-034
B-27 PGX-015, PGX-016, PGX-017
B-28 CLI-033, CLI-034, PGX-026
B-29 PGX-021
B-30 PCN-010, PGX-012
B-31 IN-001, IN-009, QC-007, MET-001, MET-002, MET-003, MET-004, MET-005, MET-010, MET-012, MET-013, MET-015
B-32 OUT-050
B-33 CLI-016, IN-043, IN-044, IN-060, EDGE-009
B-34 IN-031, EDGE-032
B-35 OUT-075, EDGE-039
C-G1 SCOPE-002, OUT-066, OUT-070, QC-009, QC-011, QC-040
C-G2 CLI-024, QC-012
C-G3 IN-015, IN-024, QC-011
C-G4 QC-002, QC-010
C-G5 QC-004, QC-020
C-G6 CLI-026, IN-055, QC-021, QC-023
C-G7 QC-024, QC-025
C-G8 CLI-027, QC-026
C-G9 QC-030, QC-031
C-G10 OUT-062, QC-034
C-G11 OUT-060, QC-030
C-G12 QC-047, QC-048, QC-049, QC-050, QC-052, QC-054, QC-055
C-G13 QC-050, QC-051, QC-056
C-G14 QC-052
C-G15 QC-053, QC-054
C-G16 QC-057, PERF-008
C-G17 CLI-026, QC-041, QC-042, QC-058
C-G18 QC-043, MET-014
C-G19 QC-044, QC-045
C-G20 CLI-026, QC-046, EDGE-040
C-G21 CLI-016, QC-021, QC-041, QC-063, EDGE-028
C-G22 SCOPE-005, IN-065, EDGE-034
C-G23 OUT-071, OUT-072
C-G24 CLI-028, IN-025, CYTO-037
C-G25 PCN-001, PCN-004, PCN-017, PGX-008, PGX-032, PERF-009
C-G26 CLI-036, QC-006, QC-062, MET-001, MET-006, MET-013
C-G27 PCN-017, PGX-032, PERF-001, PERF-002, PERF-009
C-G28 SCOPE-006
D-1 CYTO-005, CYTO-006
D-2 CLI-028, CLI-029, CYTO-007, CYTO-011
D-3 CLI-028, OUT-052, CYTO-004
D-4 CYTO-018, CYTO-020, CYTO-021, CYTO-022
D-5 CYTO-012, CYTO-013, CYTO-030, CYTO-031
D-6 CYTO-017, CYTO-023, CYTO-024
D-7 CYTO-001, CYTO-010, CYTO-033
D-8 CYTO-034
D-9 SCOPE-009, CYTO-039, PERF-003
D-10 IN-062, PCN-001, PCN-015
D-11 CLI-031, IN-050, PCN-004, PCN-007
D-12 PCN-011, PCN-012
D-13 PGX-014, PGX-016, PGX-017, PGX-019, PGX-020, PGX-029, PGX-035
D-14 PGX-002, PGX-003, PGX-004, PGX-005, PGX-006, PGX-007, PGX-009, PGX-010, PGX-035
D-15 IN-053, PGX-001
D-16 CLI-034, PGX-025
D-17 PGX-008, PGX-021
D-18 PGX-026, PGX-027
D-19 PGX-011
D-20 SCOPE-010, PGX-030
D-21 CLI-036, QC-006, QC-024, QC-032, MET-006, MET-007, MET-008, MET-017
D-22 CLI-036, QC-032, MET-008, MET-010, MET-011
D-23 MET-003, MET-005, MET-013
D-24 IN-009, IN-041, QC-007, MET-002, MET-009, MET-015, EDGE-029
D-25 QC-004, QC-005
D-26 SCOPE-004, IN-026
D-27 SCOPE-004, CLI-037
D-28 IN-022, EDGE-005
D-29 IN-027
D-30 CLI-019, CLI-037, EDGE-036
D-31 SCOPE-009, MET-016, PERF-006
E-1 IN-033, OUT-001, EDGE-001, EDGE-002
E-2 IN-032, OUT-061, EDGE-003
E-3 IN-027, IN-042, IN-043, IN-049, EDGE-008
E-4 CLI-017, IN-040, IN-045, EDGE-006
E-5 IN-020, IN-022, OUT-076, EDGE-005
E-6 CLI-010, EDGE-030
E-7 IN-017, EDGE-007, PERF-007
E-8 IN-001, IN-002, IN-008
E-9 IN-003, IN-004, IN-005, IN-006, IN-007, EDGE-007
E-10 IN-010, IN-011
E-11 IN-013, OUT-060
E-12 OUT-030, OUT-031, EDGE-011
E-13 OUT-031, OUT-037, EDGE-011
E-14 OUT-020, EDGE-012
E-15 OUT-021, EDGE-013
E-16 OUT-035, OUT-071, EDGE-015, EDGE-016
E-17 OUT-032, OUT-033, EDGE-011
E-18 CLI-009, OUT-019, EDGE-019
E-19 IN-046, OUT-020, EDGE-018
E-20 OUT-018, EDGE-038
E-21 OUT-016, EDGE-017
E-22 IN-061, EDGE-009
E-23 OUT-038, EDGE-020
E-24 OUT-039, OUT-041, EDGE-021
E-25 OUT-034, EDGE-014
E-26 OUT-062, EDGE-037
E-27 OUT-014, OUT-076, PGX-029, EDGE-022
E-28 OUT-052, CYTO-004, EDGE-023
E-29 CYTO-007, CYTO-008, CYTO-009, CYTO-013, EDGE-024
E-30 CYTO-011, EDGE-024
E-31 CYTO-005, CYTO-006, EDGE-024
E-32 CYTO-019, CYTO-020, CYTO-021, CYTO-022, EDGE-024
E-33 CYTO-014, CYTO-015, CYTO-016, EDGE-024
E-34 CYTO-023, CYTO-024, EDGE-024
E-35 CYTO-001, CYTO-017, CYTO-039, CYTO-040, EDGE-024
E-36 CYTO-029, CYTO-031, EDGE-025
E-37 CYTO-026, CYTO-033, CYTO-034
E-38 CYTO-007
E-39 CLI-031, IN-050, OUT-002, PCN-004, PCN-005, PCN-007, EDGE-026
E-40 CLI-033, PGX-004, PGX-011, PGX-018, PGX-021, PGX-023, PGX-024, EDGE-027
E-41 PGX-012, PGX-019, PGX-028, EDGE-027
E-42 PGX-014, PGX-015, EDGE-027
E-43 PGX-013, PGX-029, EDGE-027
E-44 QC-003, QC-004, QC-020
E-45 QC-051, QC-052, QC-056, QC-057, EDGE-028, PERF-008
E-46 MET-006, MET-010, MET-016, MET-017, EDGE-029, PERF-011
E-47 CLI-005, OUT-073, OUT-077
E-48 OUT-075, PGX-028, EDGE-039

13.3 Item counts

Prefix Items
SCOPE 10
CLI 40
IN 52
OUT 47
QC 52
CYTO 40
PCN 17
PGX 35
MET 18
EDGE 40
PERF 11
OQ 54
Total 416