---
title: "Run a full-scale benchmark outside package builds"
author: "GEModelR maintainers"
output:
  rmarkdown::html_vignette:
    toc: true
vignette: >
  %\VignetteIndexEntry{Run a full-scale benchmark outside package builds}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

Full-scale GTAP measurements are an external workflow. This article documents
the repository's existing A/B runner and acceptance checker; it does not run
them. Source-package, vignette, and pkgdown builds need no GTAP files or
benchmark hardware.

## Prepare an external benchmark environment

Run from a GEModelR source checkout on a machine where R and the required
package build are available. Supply your own authorized GTAP inputs and an
appropriate machine; the package repository does not contain them. You need:

- a local directory with the GTAP data files used by the runner;
- the corresponding TABLO model file;
- an RDS closure file, and optionally a shocks file;
- enough physical memory and time to run each solver process; and
- a compiler and build environment suitable for installing GEModelR with the
  requested native backend.

Choose a fresh output directory outside the repository. The runner writes
per-run CSV, RDS, and log artifacts there; these may contain full solution
vectors or machine-specific details and should remain under your control.

## Run the matched A/B workflow

This literal shell example is documentation only. It is not evaluated while
building the package or site.

```sh
Rscript benchmarks/run_gtap12a_ab.R \
  --output-dir="/external/benchmark-output/gtap12a-ab" \
  --data-dir="/authorized/path/gtap12a" \
  --tablo="/authorized/path/gtapv7.tab" \
  --closure-file="/authorized/path/closure.rds" \
  --threads=4 \
  --panel-size=64 \
  --steps="1,3"
```

The runner launches fresh R processes for its configured reference and
candidate backends, records warmup and measured runs, and invokes the gate.
The current defaults compare `StructuredSchurFGMRES` with
`StructuredSchurFGMRESCpp`. Follow the runner's `--help`-style argument
validation and repository documentation for additional supported options.

The gate can also be run directly against an existing output directory:

```sh
Rscript benchmarks/check_benchmark_gate.R \
  --input-dir="/external/benchmark-output/gtap12a-ab" \
  --reference=StructuredSchurFGMRES \
  --candidate=StructuredSchurFGMRESCpp
```

## Preserve the acceptance evidence

The existing checker requires finite, nonnegative solve-time and peak-RSS
measurements, a finite full-system relative residual, finite solution and
selected-output flags, and evidence that no dense full-system operation was
used. It compares paired solution artifacts and checks the configured
residual, solution-difference, speed, and memory thresholds.

For a redacted summary, retain the evidence needed to interpret and reproduce
the run:

- solver outcomes: maximum full relative residual, solution and selected
  output finiteness, and dense-fallback status;
- performance: solve time and peak resident memory for each backend;
- hardware and runtime: operating system/platform, CPU model and core count,
  physical RAM, R version, and relevant dependency versions;
- package identity: GEModelR version and source commit or source-tree record;
- model/configuration identity: model family and dimensions, run settings,
  and the runner's input/configuration fingerprints.

The current runner emits MD5-based fingerprints for change detection. Those
fingerprints are not signatures and do not authenticate provenance. Do not
publish private model files, HAR arrays, closure or shock inputs, full solution
artifacts, or logs containing sensitive paths. The checked-in
`benchmarks/GTAP12A_CPP_RESULTS.md` document is a summary example; it does not
replace a fresh run on your own authorized inputs and hardware.