Run a full-scale benchmark outside package builds

Prepare an external benchmark environment

Run from a GEModelR source checkout on a machine where R and the required package build are available. Supply your own authorized GTAP inputs and an appropriate machine; the package repository does not contain them. You need:

• a local directory with the GTAP data files used by the runner;
• the corresponding TABLO model file;
• an RDS closure file, and optionally a shocks file;
• enough physical memory and time to run each solver process; and
• a compiler and build environment suitable for installing GEModelR with the requested native backend.

Choose a fresh output directory outside the repository. The runner writes per-run CSV, RDS, and log artifacts there; these may contain full solution vectors or machine-specific details and should remain under your control.

Run the matched A/B workflow

This literal shell example is documentation only. It is not evaluated while building the package or site.

Rscript benchmarks/run_gtap12a_ab.R \
  --output-dir="/external/benchmark-output/gtap12a-ab" \
  --data-dir="/authorized/path/gtap12a" \
  --tablo="/authorized/path/gtapv7.tab" \
  --closure-file="/authorized/path/closure.rds" \
  --threads=4 \
  --panel-size=64 \
  --steps="1,3"

The runner launches fresh R processes for its configured reference and candidate backends, records warmup and measured runs, and invokes the gate. The current defaults compare StructuredSchurFGMRES with StructuredSchurFGMRESCpp. Follow the runner’s --help-style argument validation and repository documentation for additional supported options.

The gate can also be run directly against an existing output directory:

Rscript benchmarks/check_benchmark_gate.R \
  --input-dir="/external/benchmark-output/gtap12a-ab" \
  --reference=StructuredSchurFGMRES \
  --candidate=StructuredSchurFGMRESCpp

Preserve the acceptance evidence

The existing checker requires finite, nonnegative solve-time and peak-RSS measurements, a finite full-system relative residual, finite solution and selected-output flags, and evidence that no dense full-system operation was used. It compares paired solution artifacts and checks the configured residual, solution-difference, speed, and memory thresholds.

For a redacted summary, retain the evidence needed to interpret and reproduce the run:

• solver outcomes: maximum full relative residual, solution and selected output finiteness, and dense-fallback status;
• performance: solve time and peak resident memory for each backend;
• hardware and runtime: operating system/platform, CPU model and core count, physical RAM, R version, and relevant dependency versions;
• package identity: GEModelR version and source commit or source-tree record;
• model/configuration identity: model family and dimensions, run settings, and the runner’s input/configuration fingerprints.

The current runner emits MD5-based fingerprints for change detection. Those fingerprints are not signatures and do not authenticate provenance. Do not publish private model files, HAR arrays, closure or shock inputs, full solution artifacts, or logs containing sensitive paths. The checked-in benchmarks/GTAP12A_CPP_RESULTS.md document is a summary example; it does not replace a fresh run on your own authorized inputs and hardware.