What is exercised before every release. Three independent pillars — code tests (do the units behave?), verification (does the solver solve the equations we claim?), and validation (does the solution agree with experiment?). Verification and Validation together are commonly called V&V; we keep them as separate categories below because they catch different classes of regression. Status snapshot as of 2026-05-22.
Pytest unit suites under solvers/**/tests/. Per-function checks, deterministic, run in CI on every commit. The first line of defence — fastest signal, cheapest fix.
Method-of-manufactured-solutions, conservation tracking, grid convergence, regression on analytic limit cases. Catches discretisation / patch errors. Independent of experimental data.
Comparison against field trials, lab experiments, and certified reference codes. Hanna FB / NMSE / FAC2 acceptance bands. Catches modelling errors that no amount of unit-test or verification effort can find.
Per-module unit tests for integral consequence models. Coverage 65% by line, weighted toward the consequence engines themselves; auxiliary helpers (i18n, scenario IO) are the remaining 35%.
| Module | Tests | Lines cov. | Status | Reference |
|---|---|---|---|---|
| gaussian_plume.py | 22 | 92% | PASS | Briggs (1973), Pasquill A–F |
| dense_gas.py (Britter-McQuaid) | 14 | 84% | PASS | HSE CRR 17/1988 nomogram |
| hegadas/hegadas_s.py | 18 | 78% | PASS | Witlox 1994 / Colenbrander 1980 |
| pool_fire.py | 19 | 81% | PASS | Mudan; Shokri-Beyler / API 521 |
| jet_fire.py | 17 | 80% | PASS | Chamberlain 1987 + API 521 |
| flash_fire.py | 8 | 62% | PARTIAL | ½·LFL contour |
| vce.py (TNO MEM) | 12 | 78% | PASS | CPR 14E classes 3–10 |
| bst.py (Baker-Strehlow-Tang) | 11 | 76% | PASS | Baker et al. 1994 curves |
| cam.py (Combustion Assessment) | 9 | 71% | PASS | → TNO class / BST Mach |
| bleve.py | 15 | 79% | PASS | TNO Yellow Book; Roberts 1982 |
| bleve_fragmentation.py | 7 | 58% | PARTIAL | Holden & Reeves drag |
| toxic_cloud.py / toxic_mixture.py | 21 | 83% | PASS | AEGL + ten-Berge n |
| source_terms.py | 26 | 88% | PASS | Bernoulli · choked · Leung HEM |
| two_phase.py | 10 | 74% | PASS | Leung omega method |
| qra.py (probit + IR + F-N) | 28 | 86% | PASS | TNO Green Book + HSE 2010 |
| lopa.py | 9 | 72% | PASS | IPL PFD aggregation |
| qra_engine/fault_tree.py | 14 | 80% | PASS | MCS · Fussell-Vesely |
| pdr_calculator.py | 11 | 75% | PASS | Tier-1 → Tier-2 PDR helper |
| stl_reader.py | 8 | 55% | PARTIAL | Binary + ASCII; AABB-tree SDF |
| montecarlo.py · sobol.py | 12 | 77% | PASS | UQ + Saltelli sampling |
| aermod_shim.py | 5 | 48% | PARTIAL | EPA reference comparator |
Targets the AgniKawach problem layer (pelelmex_prob.H),
the in-tree patches, and the radiation engine. 87% line coverage on AgniKawach-owned code;
the upstream Pele/AMReX path is exercised through V&V (next sections), not unit tests.
| Subsystem | Tests | Lines cov. | Status | Reference |
|---|---|---|---|---|
| Patch 0001 — aux-LES contribution | 14 | 90% | PASS | μt/Sct wired into aux-var diff |
| Patch 0002 — D(x) ramp + wind scaling | 18 | 91% | PASS | Calibrated u_cal=4 m/s |
| Patch 0003 — TurbInflow vel clamp | 9 | 86% | PASS | urms ≥ 0.6 stable through 1500 s |
| Patch 0004 — PDR L-scale fix | 7 | 81% | PASS | L = min(dx, Av⁻¹) |
| Patch 0005 — TurbInflow LP filter | 8 | 72% | PASS | EB+AMR centerline 1.11× at x=200 m (FAC2) |
| Patch 0006 — VOS porosity field | 40 | 93% | PASS | Sub-grid drag + turb injection |
| Brinkman Items 1–5 (IBM, substep, …) | 26 | 88% | PASS | Items 1–5 dispersion bundle |
| Boussinesq + MW coupling | 21 | 89% | PASS | Light- + dense-gas formula path |
| Source-term injection | 15 | 85% | PASS | Volumetric Gaussian release |
| EB cut-cell wrappers (AKMeshAdapter) | 11 | 78% | PASS | AMReX EB redistribution |
| Radiation MC (4 modes + samplers) | 36 | 90% | PASS | + analytical-limit kernel test (periodic-axis wrap) |
| Radiation P1 + Marshak BC | 12 | 80% | PASS | Marshak (Lambert was the bug) |
| WSGGM (RADCAL) + soot Rayleigh | 19 | 83% | PASS | Accumulation bug fixed |
| Sobol sampler + hot-ray adaptive | 13 | 81% | PASS | Default on; auto-fire for narrow sources |
| Fire / explosion / cryogenic stubs | 6 | 18% | DRAFT | Phase 3–5 build-out |
The known coverage gap. Postprocess transforms plotfiles into Hanna metrics + branded PDFs; regressions here silently bias V&V acceptance. Priority for next test sprint.
| Module | Tests | Lines cov. | Status | Reference |
|---|---|---|---|---|
| postprocess.py (Hanna metrics + pure fns) | 39 | 43% | PARTIAL | FB · NMSE · FAC2 · MG · VG · pg σ · virt-src · wrap · resolve · meta |
| extract_centerline.py (Cell_H parser) | 14 | 29% | PARTIAL | Multi-fab parser regression-tested with synthetic Cell_H |
| virtual-source σ shift | 5 | 56% | PARTIAL | σ_eff = √(σ²P + r²) |
| agnikawach_report.py (PDF render) | 1 | 8% | GAP | Smoke only |
| tier1_to_tier2.py (case generator) | 37 | 83% | PASS | Tier-1 → Tier-2 sizing |
| stl_voxelizer.py (radiation) | 8 | 70% | PASS | Shared SDF helper |
Pre-flight tooling that gates every Tier-2 launch. Tested independently of the solver because they are reused from Tier-1 source-term IO and from V&V regression scripts.
| Subsystem | Tests | Format support | Status | Page |
|---|---|---|---|---|
| Geometry pipeline classifier | 30 | STL · OBJ · PLY | PASS | geometry_viewer → |
| STEP / IGES via OCP backend | 12 | STEP · IGES · GLB | PASS | cadquery / OCP optional dep |
| Grid-convergence runner | 20 | Richardson ratio · per-arc PASS/FAIL | PASS | grid_convergence → |
| VOS porosity field generator | 26 | Patch 0006 + G8 | PASS | vos_inspector → |
| EB ↔ Brinkman mask consistency | 9 | Same SDF source | PASS | Cross-check on every site |
Independent of experimental data. Method-of-manufactured-solutions, conservation tracking, analytic limit cases. Catches discretization / patch regressions without relying on field-trial accuracy bands.
| Study | Method | Order observed | Status | Reference |
|---|---|---|---|---|
| Advection MMS (uniform) | Manufactured solution; tracer transport | ~2.0 | PASS | Godunov PPM 2nd order on smooth fields |
| Diffusion MMS (Gaussian decay) | Pure-diffusion analytic; no advection | ~2.0 | PASS | CN + MLMG |
| Mass conservation (closed box) | Tracer integral over time | < 1e-9 | PASS | Matches AMReX Hydro reflux precision |
| Pasquill-D analytic plume | Centerline match against closed-form | FAC2 | PASS | Tier-1 ↔ Tier-2 cross-check |
| Brinkman → EB consistency | Same cube, two paths, ≤ 20% deviation | ≤ 20 % | PASS | After Items 1–4 (IBM wall drag) |
| Marshak BC limit (P1) | Slab radiation against analytic | ±2 % | PASS | VCR-001 / VCR-002 baseline |
| WSGGM accumulation | Single-band → grey limit | ±1 % | PASS | Accumulation bug fixed 2026-04-28 |
| MC periodic-axis wall reflection | Analytical κ-slab; ray wrap on reflect | < 1 % | PASS | Regression for 2026-05-06 kernel fix |
| EB ↔ Brinkman + AMR centerline | Same cube; AMR wake-resolution path | 1.11× | PASS | F-up 3 (2026-05-05); FAC2 at x=200 m |
Required step before any production run. Three runs at coarse / medium / fine grid; if the Richardson ratio falls inside the asymptotic band on every output arc, the grid is signed off. Live tool at grid_convergence.html.
| Configuration | Grids tested | Arcs PASS | Verdict | Notes |
|---|---|---|---|---|
| PG-turb-8m (uniform) | 16 / 8 / 4 m | 7 / 7 | SIGNED-OFF | VC-005 production config |
| AMR-3level (turb inflow) | L=0,1,2 | 2 / 7 | FAIL | Known AMR + TurbInflow CFL collapse |
| Brinkman cube (8 m) | 16 / 8 / 4 m | 6 / 7 | CONDITIONAL | Far-field arc warns; centerline OK |
| CODASC canyon (EB) | 2 / 1 / 0.5 m | 5 / 5 | SIGNED-OFF | Cut-cell + state redistribution |
| VCR-007 L-shape (radiation) | N rays × 4 | 3 / 3 | SIGNED-OFF | MC variance halves with 4× rays |
Field-trial and experimental campaigns. Acceptance: Hanna FB ∈ [−0.3, +0.3], NMSE ≤ 4, FAC2 ≥ 0.5 on the centerline arc set.
| Case | Dataset | Substance | FAC2 | Status | Tier-2 anchor |
|---|---|---|---|---|---|
| VC-002 | Pasquill-D analytic | tracer | 0.84 | PASS | sc-dispersion → |
| VC-003 | PG-no-AMR-8m | tracer | 0.81 | PASS | amr.max_dt=0.12 enforced |
| VC-004 | PG-turbinflow | tracer | 0.79 | PASS | bare turbinflows= namespace |
| VC-005 | PG-38 (Barad 1958, u=4.28) | tracer | 0.84 | PASS | Production config |
| VC-005-multi | PG-23, 24, 38 | tracer | 0.62 | PASS | D(x) wind scaling required |
| VC-006 | AMR + TurbInflow stress | tracer | — | CFL collapse | Documented limitation |
| VC-008 | High urms (0.6 m/s) | tracer | 0.71 | PASS | Patch 0003 vel clamp |
| VC-009 / VC-010 | Class F / A stable | tracer | 0.58 | WORKAROUND | gravity=0; reduced Cs/D |
| VC-014 | Anisotropic dz=2 m | tracer | — | BLOCKED | Setup.cpp:48 isotropic assert |
| EB-001 → EB-004 | CODASC canyon | tracer | 0.74 | PASS | EB cut-cell path |
| DG-CL2-dense | Maplin Sands · Cl₂ | Cl₂ | 0.66 | PASS | |Δρ/ρ| ≤ 0.03 budget |
| DG-LNG-heavy | Burro / Coyote LNG | CH₄ vapour | 0.69 | PASS | HEGADAS-S cross-check |
Standalone radiation V&V (28 cases). MC is the engine of record; P1 reported alongside as a fast comparator. Acceptance: ±15% wall flux versus published reference.
| Case | Reference | τ regime | Wall flux Δ | Status |
|---|---|---|---|---|
| VCR-001 | Thynell & Lin (slab) — periodic-axis fix 2026-05-06 | τ ≈ 1 | +1 % | PASS |
| VCR-002 | Hsu & Farmer (case A) | τ ≈ 0.5 | +11 % | PASS |
| VCR-003 | Selçuk (planar) | τ ≈ 2 | −6 % | PASS |
| VCR-004 | Crosbie (3-D box) | τ ≈ 3 | −8 % | PASS |
| VCR-005 | Hsu & Farmer C2 | τ ≈ 0.3 | +22 % | FAIL (P1 only) |
| VCR-006 | Chui (cylinder) | τ ≈ 1 | +9 % | PASS |
| VCR-007 (k=0.5 / k=1.0) | Henson L-shape | τ ≈ 0.5–1 | ±13 % | PASS |
| VCR-008 | Anisotropic equil. | τ ≈ 1 | −7 % | PASS |
| VCR-009 (k=2 / 5 / 10) | Soot-laden gas | τ ≈ 2–10 | ±9 % | PASS |
| VCR-010 (1800 / 2200 / 2500 K) | High-T gas | τ ≈ 1 | ±18 % | PARTIAL |
| VCR-011 (k=0.5 ε=1.0) | Flux benchmark | τ ≈ 0.5 | +24 % | FAIL (P1) |
| AR Confluence H₂O+CO₂ | Bressloff thesis | τ ≈ 1–3 | ±12 % | PASS |
| VCR-006 cylinder (Henson §6.1.2 B) | κ ∈ {0.1, 1.0, 10} | τ ≈ 0.1–10 | 4.6 / 7.3 / 0.5 % | PASS |
| T1R-001..006 (Altaç-Tekkalmaz 2008) | BMP-A..F · Tables 3–8 digitised | τ varies | RULES v2 | BUILD-OUT |
| 15 new T-series cases (Altaç Phase 3) | Pending postprocess | τ varies | — | PENDING |
tMax);
fixed in MCRadKernels.H and protected by
test_radiation_kernel_analytical.py.
Combustion-side validation pending Phase 3 build-out. Tier-1 fire models validated; Tier-2 radiation engine validated (above); coupled CFD path is the gap.
| Case | Reference | Tier-1 | Tier-2 | Notes |
|---|---|---|---|---|
| Pool-fire 10 m propane | Mudan / Rew-Hulbert | PASS | PENDING | Tier-1 solid-flame V&V |
| Pool-fire LNG | Maplin Sands LNG | PASS | PENDING | PeleMP cryogenic spray pending |
| Jet-fire 50 mm propane | Chamberlain 1987 | PASS | PENDING | EDC closure draft |
| Multi-burner flare | Cook 1990 | PASS | PENDING | Tier-1 superposition |
| NIST firemodels/exp | NIST datasets | CURATE | PENDING | Phase 3 ingest |
Tier-1 paths validated against TNO Yellow Book and HSE. Tier-2 PeleC blast-wave path is the next major workload (post-Phase 3).
| Case | Reference | Tier-1 | Tier-2 | Notes |
|---|---|---|---|---|
| VCE — propane congested | TNO MEM CPR 14E | PASS | PENDING | PDR required at coarse grid |
| VCE — BST high reactivity | Baker et al. 1994 | PASS | PENDING | Same CAM input |
| BLEVE — propane 10 t | Roberts 1982; Hymes | PASS | PENDING | PeleC + radiation MC |
| Toxic — Cl₂ AEGL-2 | NRC AEGL tables | PASS | DRAFT | AKToxicSolver wrapper |
| Toxic mixture — sour gas | TNO Green Book probit | PASS | PENDING | Multi-species transport |
Output adapters that translate the QRA bundle into regulator-ready format. Tested via snapshot tests against fixed example scenarios; PDFs regenerated and diffed on every commit.
| Pack | Scope | Tests | Status | Reference |
|---|---|---|---|---|
| PNGRB ERDMP (India) | MCA + evac tables, branded PDF | 11 | PASS | Petroleum & NG Regulatory Board |
| OISD (India) | Min-distance lookup by inventory | 7 | PASS | OISD-116 / 118 cross-check |
| COMAH (UK / NI) | Lower- vs upper-tier classification | 9 | PASS | Substance × inventory |
| Seveso-III (EU) | Threshold + additive aggregation + domino | 14 | PASS | Directive 2012/18/EU |
Full-pipeline smoke tests run nightly. Each test exercises one scenario from the example library through the entire dispatch path. Wall time gates flag regressions of more than 30%.
| Pipeline | Stages | Wall time | Status | Notes |
|---|---|---|---|---|
| Tier-1 scenario → PDF | Tier-1 solver · QRA · PDF | ~12 s | PASS | CI · every commit |
| Tier-1 → Tier-2 auto-size | Tier-1 · sizer · Tier-2 input gen | ~5 s | PASS | 37 unit tests |
| Geometry → SDF → solver mask | STL ingest · voxelize · mask write | ~8 s | PASS | EB & Brinkman both |
| Tier-2 dispersion smoke (60 s sim) | Build · run · plotfile · postprocess | ~6 min | PASS | Nightly · OMP=2 |
| Radiation MC smoke (10 cells) | Mode 3 wall-flux · CSV write | ~8 s | PASS | Default Sobol on |
| Mega-sweep (68 cases × 17 categories) | Parametric stress test | ~36 h | PASS | Manual; not CI |
| Frontend → API → Solver | scenario_builder.html · FastAPI · solver | — | PARTIAL | FastAPI mock; live solver pending |
Where coverage is materially below the rest of the tree. Ordered by risk-of-undetected-regression.
| Gap | Why it matters | Priority |
|---|---|---|
| Postprocess line coverage 13 % | Hanna metrics + Cell_H parser bias V&V acceptance silently if regressed. | P0 |
| Frontend → live solver pipe | FastAPI + WebSocket are mocked end-to-end; needs A100 spot runner. | P0 |
| Fire CFD validation (Phase 3) | Combustion + radiation coupled path uncovered — only standalone parts validated. | P1 |
| CUDA build never executed | Dev box P600 cannot run USE_CUDA=TRUE; no validated A100 run yet. | P1 |
| Anisotropic dz blocked | Setup.cpp:48 isotropic assert — VC-014 cannot run; ~2 hr solver patch. | P1 |
| VCE / BLEVE Tier-2 (Phase 4–5) | PeleC compressible scenarios in stub form only. | P2 |
| AERMOD shim PRIME terrain | Comparator currently flat-terrain only. | P2 |