Define manual quality gates and release-readiness checks for DSAMbayes. The
normal cadence is quarterly; these pages deliberately do not prescribe CI/CD.
CLI log and results/quality_gate_validate/00_run_metadata/config.compiled.yaml
QG-7
Runner smoke: run
Rscript scripts/dsambayes.R run --config config/blm_timeseries.yaml --run-dir results/quality_gate_run
Exit code 0 and core artefacts exist
CLI log and selected artefacts under results/quality_gate_run/
QG-8
Docs build check
python3 docs-site/build_content.py && (cd docs-site && hugo --cleanDestinationDir)
Exit code 0
Build log and generated site output under docs-site/public/
Gate Definitions
QG-1 Lint
Command:
Rscript scripts/check.R --lint
Fail conditions:
Non-zero exit code
Any lint issue reported
Any SKIP: output
QG-2 Style
Command:
Rscript scripts/check.R --style
Fail conditions:
Non-zero exit code
Any file reported as requiring reformat
Any SKIP: output
QG-3 Unit Tests
Command:
Rscript scripts/check.R --test
Fail conditions:
Non-zero exit code
Any test failure or error
QG-4 Stan Release Evidence
Command:
Rscript scripts/check.R --stan-release-evidence
Fail conditions:
Non-zero exit code
Any high-budget Stan evidence test failure
Any unresolved diagnostic-threshold failure
QG-5 Package Check
Command:
_R_CHECK_FORCE_SUGGESTS_=false\
R -q -e 'rcmdcheck::rcmdcheck(args = c("--no-manual"), error_on = "warning")'
Fail conditions:
Any ERROR
Any WARNING for release sign-off
Escalation condition:
Any NOTE must be reviewed and explicitly accepted with rationale.
Operational note:
Local package checks do not force Suggests because DSAMdecomp is an
optional decomposition-only dependency and the upstream available package is
telemetry-enabled. Decomposition-specific checks should be run separately
against an approved telemetry-free DSAMdecomp install.
python3 docs-site/build_content.py
(cd docs-site && hugo --cleanDestinationDir)
Fail conditions:
Non-zero exit code
docs-site/build_content.py fails before content mirroring completes
Hugo build aborts before site generation
Missing docs-site/public/index.html
Command Reference
Recommended environment setup before running gates:
# Navigate to your local DSAMbayes checkout and select the host librarycd /path/to/DSAMbayes
source scripts/r-library-path.sh
dsambayes_set_r_library host
mkdir -p "$R_LIBS_USER" .cache
exportXDG_CACHE_HOME="$PWD/.cache"export_R_CHECK_FORCE_SUGGESTS_=false
Optional consolidated local gate (does not replace all release gates):
Do not proceed to sign-off with unresolved ERROR or WARNING.
NOTEs require written rationale and reviewer acceptance.
If a gate fails due to environment setup, fix the environment and re-run the full affected gate.
DSAMdecomp absence alone is not a blocker for QG-4 because the canonical
package check flow treats it as an optional Suggests dependency.
If a gate fails due to product code, raise a remediation change and re-run from QG-1.
Sign-off Criteria
Release sign-off requires all of the following:
QG-1 to QG-8 passed.
No SKIP outcomes across mandatory gates.
Evidence bundle completed and reviewed.
Final decision recorded in sign-off-template.md.
Testing and Validation
Purpose
Define the canonical testing and validation workflow for DSAMbayes v1.3.3, from local pre-merge checks through release-quality gates.
Audience
Engineers running local checks before merge
Maintainers preparing release candidates
Reviewers validating release evidence
Validation layers
Layer
Objective
Primary command(s)
Output proof
Lint
Catch style and static issues early
Rscript scripts/check.R --lint
Exit code 0, no lint failures
Style
Enforce formatting compliance on changed files
Rscript scripts/check.R --style
Exit code 0, no reformat-required files
Unit tests
Catch behavioural regressions in package logic
Rscript scripts/check.R --test
Exit code 0, no test failures
Minimal smoke
Keep a cheap Stan-backed safety check in the routine local loop
Rscript scripts/check.R --smoke
Exit code 0, fast unit tests plus minimal Stan smoke pass
Stan smoke
Exercise the broader compiled Stan suite beyond the minimal smoke tier
Rscript scripts/check.R --stan-smoke
Exit code 0, Stan smoke tests enabled
Stan recovery evidence
Re-run the full coefficient-recovery file under the nightly Stan gate
Rscript scripts/check.R --stan-recovery
Exit code 0, full-Stan DGP recovery tests pass
Stan release evidence
Re-run high-budget pooled and hierarchical evidence checks for release candidates
Rscript scripts/check.R --stan-release-evidence
Exit code 0, targeted high-budget Stan evidence passes diagnostic thresholds
Docs sanity
Catch local docs-link breakage and high-value contract drift
Rscript scripts/check.R --docs
Exit code 0, docs sanity checks pass
Package check
Validate package-level install and check behaviour
R -q -e 'rcmdcheck::rcmdcheck(...)'
No ERROR; no unresolved WARNING
Runner validate
Validate config and data contracts without fitting
Rscript scripts/dsambayes.R validate ...
Exit code 0, metadata artefacts
Runner run
Validate end-to-end runner execution and artefacts
Rscript scripts/dsambayes.R run ...
Exit code 0, core run artefacts
Docs build
Validate docs-site/Hugo buildability
python3 docs-site/build_content.py && (cd docs-site && hugo --cleanDestinationDir)
Exit code 0, successful site build
Environment setup
Run all commands from repository root:
# Navigate to your local DSAMbayes checkoutcd /path/to/DSAMbayes
source scripts/r-library-path.sh
dsambayes_set_r_library host
mkdir -p "$R_LIBS_USER" .cache
exportXDG_CACHE_HOME="$PWD/.cache"
Expected outcome: checks run in a repo-scoped environment with reproducible library and cache paths.
Dependency source portability
Before a release candidate is signed off, verify all non-CRAN dependency
sources in renv.lock and DESCRIPTION.
The preferred release path is a pinned private GitHub remote for optional
decomposition dependencies. A local fallback may restore those dependencies
from sibling checkouts:
If any entries remain as file:// sources in renv.lock, release evidence
must record the required local paths and pinned commit hashes. For an external
release or clone-portable handoff, prefer reachable pinned remotes before
sign-off.
When validating either dependency source path, use telemetry-disabled restores:
exportGITHUB_PAT="<token with access to private tandpds repos>"exportDSAMDECOMP_DISABLE_TELEMETRY=1exportDSAMBAYES_DISABLE_TELEMETRY=1exportUSE_BUNDLED_LIBUV=1XDG_CACHE_HOME="$PWD/.cache"\
Rscript -e 'renv::restore(packages = c("teller", "DSAMdecomp"), prompt = FALSE)'
Expected outcome: teller and DSAMdecomp restore and load from the recorded
sources. If private GitHub authentication is unavailable, or a local fallback
path is missing, hold the release until dependency sources are reproducible in
the release environment.
--smoke runs the unit path plus a minimal Stan-backed subset: one cheap BLM MCMC smoke, one pooled deployment-artifact smoke on a real pooled fit, and tiny run_from_yaml() runner smokes including pooled deployment_model.rds roundtrip coverage.
--docs runs the local docs/link sanity helper against README and the tracked docs surfaces.
--release runs lint, style, unit tests, minimal Stan smoke, docs sanity, and coverage as a broader local code gate.
Implementation note:
scripts/check.R --all remains a legacy convenience gate for lint, style, tests, and coverage only.
scripts/check.R --docs is the explicit docs/link drift check.
scripts/check.R --release is the clearer code-focused local release profile.
Neither profile replaces rcmdcheck, runner smoke checks, or docs build.
Stan-specific note:
use Rscript scripts/check.R --stan-smoke for the broader opt-in compiled Stan suite
use --stan-smoke-full for the fuller nightly variant
use --stan-recovery when you need a targeted rerun of the DGP recovery evidence without invoking the rest of the full Stan suite
use --stan-release-evidence for the high-budget release-candidate lane covering the warning-prone pooled and hierarchical paths
Expected outcome: all gates complete with exit code 0, with no unresolved release blockers.
Local package-check note:
DSAMdecomp remains an optional Suggests dependency. Canonical local
rcmdcheck runs set _R_CHECK_FORCE_SUGGESTS_=false so release gates do not
depend on installing a telemetry-enabled decomposition package.
Runner smoke-test expectations
Minimum release smoke expectations:
scripts/check.R --smoke succeeds, proving at least one cheap Stan MCMC path, pooled deployment-artifact coverage on a real pooled fit, and tiny runner fit paths.
validate command succeeds and writes metadata artefacts.
run command succeeds and writes model, fitted/observed output, and diagnostics artefacts.
Required runner artefact paths exist under results/quality_gate_validate/ and results/quality_gate_run/.
If validate fails, run the same command again with a clean run directory path and inspect CLI error output.
If run fails before fitting, inspect both 00_run_metadata/config.resolved.yaml and 00_run_metadata/config.compiled.yaml to confirm the authored and compiled values.
If run fails during fitting, verify local Stan toolchain and cache path from Install and Setup.
If artefacts are missing after success exit code, inspect outputs.* flags in the resolved config and confirm any compile-time artifacts in config.compiled.yaml.
Evidence capture
For release evidence, capture:
Full terminal logs and exit codes for SMK-VAL-01 to SMK-RUN-01.
repository history size does not materially shrink yet
Phase 2: Shrink existing history
This phase is disruptive and should only happen after coordination with every collaborator and any CI jobs or deployment hooks that clone this repository.
Safety rules
Freeze merges to main during the rewrite window.
Ask every collaborator to stop pushing until the rewrite is complete.
Create a backup mirror before modifying history.
Use a temporary clone or mirror for the rewrite, not an active working copy.
Suggested backup
cd ..
git clone --mirror DSAMbayes-Charles-Dev DSAMbayes-Charles-Dev.git-backup
Suggested rewrite target set
Remove historical content for paths that are local caches or generated outputs:
If branch protections block force-pushes, temporarily adjust them before the rewrite window and restore them immediately after.
Collaborator recovery after rewrite
Every collaborator should re-clone. If someone must salvage local work, they should:
git fetch origin
git switch main
git reset --hard origin/main
Re-cloning is still safer than trying to reuse an old clone after a large rewrite.
Phase 3: Optional future hardening
If large binary artifacts must remain versioned in future, move them to Git LFS. Do not use Git LFS for ephemeral caches, run outputs, or package libraries that should stay untracked.
Recommended order
Merge the .gitignore change.
Remove tracked benchmark cache files from the index with git rm --cached.
Confirm CI and docs are unaffected.
Decide whether the current 1.4G.git size justifies a history rewrite.
If yes, schedule a short maintenance window and perform the rewrite from a mirror clone.
Release Evidence Pack
Purpose
Define the exact evidence bundle required before manual DSAMbayes release
sign-off. Substitute the actual release version and candidate SHA; this page is
not version-specific.
Audience
Release owner preparing sign-off materials
Reviewers validating release readiness
Maintainers reproducing release gate outcomes
Evidence root and naming
Use one evidence root per candidate release.
Recommended path:
release_evidence/vX.Y.Z/<YYYYMMDD>_<short_sha>/
Example:
release_evidence/v1.3.3/20260714_ab12cd3/
Expected outcome: all sign-off evidence is stored in one deterministic location.
Candidate identity rule:
The candidate hash in 00_release_identity/release_identity.txt,
40_signoff/sign_off_record.md, and the evidence-root short SHA must agree.
If administrative docs or sign-off text change after gate execution, either
update the existing bundle without changing candidate identity or rerun the
gates into a new evidence root for a new candidate.
Mandatory evidence bundle
All items below are mandatory.
ID
Evidence item
Required content
Source
Required path in evidence root
EVD-01
Release identity
Candidate commit hash, branch, intended tag, package version
Expected outcome: dependency sources are visible to reviewers. Any file://
source must be accepted as a local-release prerequisite or replaced with a
reachable pinned remote before external release.
Capture gate logs and exit codes:
Run these commands inside a container whose R version matches renv.lock
(currently 4.5.1). Set the container-specific library before collecting logs.
Provide the mandatory go or no-go checklist before manually creating a
DSAMbayes release tag. It supports the normal quarterly release cycle; it does
not require CI/CD or a rapid release cadence.
How to use this checklist
Complete this checklist after running all release-quality gates.
GO only if every checklist item RL-01 to RL-15 passes.
NO-GO if any mandatory item fails or evidence is incomplete.
HOLD if no hard failure exists but final approval is pending.
Tag creation is allowed only after GO decision is recorded.
A NOTE or expected low-budget smoke-test sampler warning is not a blanket
exception: its source and reviewer treatment must be recorded. QG-4
high-budget diagnostic-threshold failures always block release pending an
explicit human decision.
Completion record template
Use this section when running the checklist.
Field
Value
Release version
<fill>
Candidate commit hash
<fill>
Checklist executor
<fill>
Checklist completion date (UTC)
<YYYY-MM-DD>
Checklist result (GO/NO-GO/HOLD)
<fill>
Evidence root path
<fill>
Sign-off record path
<fill>
Audit continuity reference
For programme-level historical traceability, also review:
Define the deliberate, evidence-led release process for DSAMbayes. It supports
the normal quarterly release cycle and material ad-hoc releases. It is not a
CI/CD process and does not imply continuous deployment or a high release
cadence.
The point is to make the infrequent release decision reproducible and
reviewable, not to automate it for its own sake.
Roles and release boundary
Release owner: fixes the candidate commit, runs or coordinates the gates,
and prepares the evidence bundle.
Reviewer: independently checks the evidence and records the GO, NO-GO, or
HOLD decision. The reviewer should be different from the release owner when
practical.
Maintainers: resolve failures or explicitly approve a documented exception.
One release has one candidate commit. Do not make product, dependency, or
documentation changes after gates begin. If a change is required, create a new
candidate and rerun the affected gates.
Preconditions
Before starting:
The intended version is consistent in DESCRIPTION and CHANGELOG.md.
The candidate commit is identified and the working tree is clean apart from
deliberately excluded local evidence and run artefacts.
The local R runtime and package library are recorded. The current tested
baseline is R 4.5.1; DESCRIPTION declares the supported runtime floor.
The floor is a compatibility policy, not a claim that every R version has
been exercised for every release.
Required non-CRAN dependencies, including DSAMdecomp, are reachable at
the recorded revision. Record the source and commit SHA in the evidence
bundle. Do not record credentials.
No Stan template, prior, boundary-default, or fit-semantics change is
released without the required human model review.
Manual release flow
1. Freeze and identify the candidate
git status --short
git rev-parse HEAD
git rev-parse --abbrev-ref HEAD
Record the full SHA, branch, intended tag, package version, R version, and
platform in 00_release_identity/release_identity.txt of the evidence bundle.
Use the current package version rather than editing this playbook for each
release.
Use a local container only when it makes the environment easier to reproduce;
it is evidence capture, not a CI service. Its R version must match renv.lock
(currently 4.5.1); record the image and digest if one is used. The helper
selects .Rlib-container-r<active-R-version>/; never mount a host library as
the container package library.
3. Create the evidence root
Use the layout defined in Release Evidence Pack.
The evidence root is local working material until the release owner decides
how it will be retained. It must not contain .env, credentials, private data,
or unredacted tokens.
4. Run the mandatory gates once, against the frozen candidate
Run QG-1 to QG-8 from Quality Gates, capturing
both complete logs and exit codes. The canonical commands are:
scripts/check.R --release is a useful local consolidation check, but it does
not replace rcmdcheck, the two runner checks, the docs-site build, or the
high-budget --stan-release-evidence gate.
5. Interpret warnings correctly
The release decision is based on the mandatory gates, not on a search for a
silent terminal.
A non-zero exit code, missing required tool, test failure, unresolved package
WARNING, failed diagnostic threshold, or missing required runner artefact
is a blocker.
rcmdcheckNOTEs require written review and acceptance; they are not
silently waived.
Low-budget Stan smoke tests can emit deliberately exercised sampler warnings.
They are not release evidence and must be recorded as expected test-fixture
behaviour if observed. The high-budget QG-4 diagnostics are the release
evidence and may not be waived without an explicit human decision.
Any new, unexplained warning is a HOLD until it is understood or removed.
6. Review and sign off
Complete the Release Readiness Checklist
and copy Release Sign-off Template into the
evidence bundle. A GO requires all mandatory checks to pass, the candidate SHA
to match throughout, and the reviewer to sign the decision.
7. Tag and publish manually
Only after GO:
git tag -a vX.Y.Z -m "DSAMbayes vX.Y.Z"git push origin vX.Y.Z
Create any GitHub release and publish any docs mirror as separate manual acts.
Record what was published, by whom, and when. No automatic deployment follows
from tagging.
8. Retain the decision record
Retain the signed evidence bundle for approved and rejected candidates. Record
post-release defects as a new hotfix candidate; do not retag or rewrite the
published release history.
Go/no-go rules
GO only when QG-1 to QG-8 pass and the sign-off record is complete.
HOLD when evidence is incomplete, a warning is unexplained, or approval is
pending.
NO-GO for any release blocker. Fix the issue, select a new candidate SHA,
and rerun the affected gates.
Hotfixes
For a post-release defect, branch from the affected tag, make the smallest
safe change, update the version and changelog, and run the same manual process.
The smaller scope does not remove the evidence or human-review requirement.
Provide the final approval record for a DSAMbayes release candidate after all mandatory evidence has been reviewed.
Instructions
Copy this template into the candidate evidence bundle as 40_signoff/sign_off_record.md.
Complete every field.
Use GO, NO-GO, or HOLD for the decision.
If any exception is accepted, record explicit rationale and owner.
Copy the candidate commit hash and evidence-root path directly from
00_release_identity/release_identity.txt; do not introduce a second
candidate hash during sign-off.