Skip to content
SciProofv0.3.2

Getting started

Install

Section titled “Install”
Terminal window
pipx install sciproof               # or: python -m pip install sciproof   (Python 3.11+)
sciproof --version

From source (for development):

Terminal window
git clone https://github.com/hayagrivai-labs/sciproof.git && cd sciproof
python -m pip install -e ".[dev]"

Quick start

Section titled “Quick start”

The repository contains a small deterministic experiment (a seeded Monte Carlo estimate of pi). Run everything below from the repository root, or run python examples/demo.py to see the whole story in a temporary directory.

1. Capture the experiment. Declare the files that matter; everything after -- is the command (it is not run through a shell):

Terminal window
$ sciproof capture --name baseline \
    --input examples/experiment.py --input examples/params.json \
    --output examples/out/results.json \
    --output-numeric examples/out/convergence.csv --rtol 1e-9 \
    -- python examples/experiment.py
Evidence written to baseline.sciproof.json
  command exit code: 0
  inputs recorded:   2
  outputs recorded:  2
Note: stdout/stderr are stored in the manifest; check them for secrets before sharing it.

2. Inspect the evidence. This never runs the recorded command:

Terminal window
$ sciproof inspect baseline.sciproof.json
SciProof manifest: baseline
  format:      sciproof.manifest 1.0
  captured:    2026-10-17T18:47:12+00:00


Command (recorded, NOT executed by 'inspect'):
  python examples/experiment.py
...
Outputs (2):
  examples/out/results.json  sha256:723d792e1b1ad975...  69 bytes  compare: exact hash
  examples/out/convergence.csv  sha256:131eb2551a942bb9...  89 bytes  compare: numeric (rtol=1e-09, atol=0.0)

3. Verify integrity (nothing is executed):

Terminal window
$ sciproof verify baseline.sciproof.json
  [PASS] input examples/experiment.py: unchanged (SHA-256 matches)
  [PASS] input examples/params.json: unchanged (SHA-256 matches)
  [PASS] output examples/out/results.json: unchanged (SHA-256 matches)
  [PASS] output examples/out/convergence.csv: unchanged (SHA-256 matches)


  note: scope: artifact integrity only. The recorded command was NOT executed, so this does not show that the results can be reproduced.


  basis: PASS because all 4 performed checks succeeded; this covers only the checks listed above, not what is under 'not checked'
  not checked: reproducibility: the recorded command was not executed
  not checked: scientific validity: a matching result is not shown to be correct
  not checked: files that were not declared as inputs or outputs
  not checked: authenticity: manifests are unsigned, so who produced this evidence is not verified


RESULT: PASS

4. Verify by re-execution. Running the command is a deliberate act: it needs --rerun and either an interactive confirmation or --yes.

Terminal window
$ sciproof verify baseline.sciproof.json --rerun --yes
  ...
  [PASS] exit code: 0, same as recorded
  [PASS] rerun output examples/out/results.json: identical to the recorded output (exact SHA-256)
  [PASS] rerun output examples/out/convergence.csv: numerically equivalent (rtol=1e-09, atol=0.0)
RESULT: PASS

5. A modified output is detected:

Terminal window
$ sciproof verify baseline.sciproof.json
  [FAIL] output examples/out/results.json: modified since capture (SHA-256 differs)
  basis: FAIL because 1 (output examples/out/results.json) check(s) contradict the evidence; one failed check is enough
  ...
RESULT: FAIL

Exit codes: 0 PASS, 1 FAIL, 2 INCONCLUSIVE, 3 usage error or invalid manifest. A missing input, a missing numeric snapshot, a command that cannot be started, a timeout or a declined rerun are INCONCLUSIVE, never PASS.

Every report says why the verdict was chosen (basis) and what was not checked. With --json it also carries the evidence behind each check (digests, tolerances, the exact differences). Details: docs/REPORT.md; the exact numeric comparison rules: docs/COMPARISON.md.

A walkthrough with NumPy and pandas, including how to choose tolerances: docs/TUTORIAL_NUMPY_PANDAS.md.

Commands

Section titled “Commands”
sciproof capture --name NAME [--input P]... [--output P]... [--output-numeric P]...
                 [--rtol X] [--atol Y] [--tolerance P 'rtol=X atol=Y']...
                 [--timeout S] [--max-log-bytes N]
                 [--claim OUTPUT 'LOCATION OP VALUE']... [--note TEXT] [--record-hardware]
                 [--force] [--allow-nonzero]
                 -- COMMAND [ARGS...]
sciproof verify  MANIFEST [--rerun] [--yes] [--strict-env] [--clean-env]
                 [--allowed-signers FILE] [--policy FILE]
                 [--format text|json|markdown|junit] [--save-json FILE]
                 [--pass-env NAME]... [--timeout S] [--json]
sciproof inspect MANIFEST
sciproof diff    OLD NEW [--format text|json|markdown]
sciproof badge   MANIFEST [-o FILE.svg] [--rerun --yes]
sciproof update  MANIFEST [--yes] [--dry-run] [--note TEXT]
sciproof calibrate MANIFEST [--runs N] [--yes] [--timeout S]
sciproof check-policy FILE
sciproof sign    MANIFEST --key KEY
sciproof list    [DIR] [-r] [--check] [--format text|json|markdown]
sciproof export  MANIFEST [--format in-toto|openlineage|latex] [--value NAME OUTPUT LOCATION]...
                 [-o FILE]
sciproof suggest-inputs [--input P]... [--output P]... -- COMMAND [ARGS...]
  • Paths must be inside the current directory; the manifest is written there as NAME.sciproof.json. Numeric outputs are also copied to NAME.sciproof.d/ (needed to compare with a tolerance; keep it next to the manifest).
  • Declared inputs must be read-only for the command; capture fails if the command changes one.
  • --input and --output accept a directory: every regular file below it is recorded, and a verification names the files that were added, removed or modified. Symbolic links inside are refused, empty directories are not recorded, and there is a limit of 10,000 files (details). --output-numeric takes single files only.
  • A command that exits with a nonzero status is still recorded (capture then exits with 1 unless --allow-nonzero).
  • --json prints the verification report as a single JSON object (same exit codes), for CI and scripts. Prompts and errors go to stderr, so stdout stays valid JSON.
  • suggest-inputs runs a Python command once (it really runs, with side effects) and lists files it opened for reading that are not declared, so you can add them as --input. It only sees reads made by Python code; C extensions, other programs, network and environment variables are invisible, so an empty answer is not proof. If the hook never starts (the command is not Python, or Python ran with -I or -S), it says so and exits with 2 instead of reporting “no undeclared files”. Delete stale __pycache__ before a rerun: Python reuses a .pyc when the source’s size and modification second are unchanged.
  • --strict-env makes a rerun INCONCLUSIVE (without executing) if Python version, platform or recorded package versions differ.
  • --format markdown prints the verification report as GitHub-flavoured Markdown for CI summaries (sciproof verify ... --format markdown >> "$GITHUB_STEP_SUMMARY"), including a table of numeric differences (recorded, found, absolute and relative difference, allowed).
  • diff compares two manifests and lists what changed between two recorded runs: command, environment and packages, Git state, inputs, outputs (numeric ones with the baseline tolerance) and exit code. It executes nothing. Exit status 0 if nothing differs, 1 otherwise (details).
  • --claim OUTPUT 'LOCATION OP VALUE' records a statement about one value in a .json or .csv output ('$.accuracy >= 0.9', '$.loss ~= 0.25 atol=0.01') that verify checks on the recorded output and on every rerun. It shows that the value satisfies the comparison, nothing more (details).
  • --note TEXT stores free text with the run (why it was made); --record-hardware also records the CPU count, processor and NVIDIA GPUs (no host name, user name or serial numbers), and a rerun on different hardware says so.
  • sign and verify --allowed-signers FILE sign a manifest and check the signature with OpenSSH (ssh-keygen -Y); SciProof implements no cryptography and trusts only the file you name (details).
  • list shows the manifests in a directory, newest first; list --check verifies all of them (integrity only) and exits with the worst result. export writes a manifest as an unsigned in-toto Statement or as OpenLineage run events (details).
  • --policy FILE checks a TOML file of rules about the evidence itself (not older than N days, clean Git tree, signed, at least one claim, no tolerance looser than a ceiling, rerun with --clean-env, …); each rule is a check of the report, and an unmet requirement is INCONCLUSIVE, not a pass (docs/POLICY.md).
  • calibrate runs the recorded command several times and measures how much the outputs vary: exact outputs must be identical, numeric ones get a suggested tolerance; it executes the command, so it asks for confirmation (details).
  • update captures the recorded experiment again with the declarations already in the manifest and shows what changed; it executes the recorded command, so it asks for confirmation (details).
  • In CI: --format junit for test viewers, and a GitHub Action (uses: hayagrivai-labs/sciproof@v0.3.2) that verifies a manifest and writes the report to the job summary (details).
  • --clean-env gives a rerun only a minimal environment (PATH, temp directories and the numerics-related variables SciProof records) instead of all of yours, so a manifest from someone else cannot read your credentials from it. --pass-env NAME adds a variable. It limits environment variables only: the command can still read your files and use the network, so it is not a sandbox.

Python API: docs/PYTHON_API.md.

Manifest format: docs/MANIFEST.md.

Report a problem with this page· Licence: MIT· GitHub