SciProof records a command, its inputs, outputs and environment in one JSON manifest, and later verifies it with an explicit verdict: PASSFAIL or INCONCLUSIVE. Local-first, offline, no runtime dependencies.
pipx install sciproofAlpha software. The idea is unproven with real users; the docs say so openly.
The quick start from the README, replayed. Without JavaScript, or with reduced motion enabled, the whole session is shown at once.
$ sciproof capture --name baseline \
--input examples/experiment.py --input examples/params.json \
--output examples/out/results.json \
--output-numeric examples/out/convergence.csv --rtol 1e-9 \
-- python examples/experiment.py
Evidence written to baseline.sciproof.json
command exit code: 0
inputs recorded: 2
outputs recorded: 2
Note: stdout/stderr are stored in the manifest; check them for secrets before sharing it.
$ sciproof inspect baseline.sciproof.json
SciProof manifest: baseline
format: sciproof.manifest 1.0
captured: 2026-10-17T18:47:12+00:00
Command (recorded, NOT executed by 'inspect'):
python examples/experiment.py
...
Outputs (2):
examples/out/results.json sha256:723d792e1b1ad975... 69 bytes compare: exact hash
examples/out/convergence.csv sha256:131eb2551a942bb9... 89 bytes compare: numeric (rtol=1e-09, atol=0.0)
$ sciproof verify baseline.sciproof.json
[PASS] input examples/experiment.py: unchanged (SHA-256 matches)
[PASS] input examples/params.json: unchanged (SHA-256 matches)
[PASS] output examples/out/results.json: unchanged (SHA-256 matches)
[PASS] output examples/out/convergence.csv: unchanged (SHA-256 matches)
note: scope: artifact integrity only. The recorded command was NOT executed, so this does not show that the results can be reproduced.
basis: PASS because all 4 performed checks succeeded; this covers only the checks listed above, not what is under 'not checked'
not checked: reproducibility: the recorded command was not executed
not checked: scientific validity: a matching result is not shown to be correct
not checked: files that were not declared as inputs or outputs
not checked: authenticity: manifests are unsigned, so who produced this evidence is not verified
RESULT: PASS
$ sciproof verify baseline.sciproof.json --rerun --yes
...
[PASS] exit code: 0, same as recorded
[PASS] rerun output examples/out/results.json: identical to the recorded output (exact SHA-256)
[PASS] rerun output examples/out/convergence.csv: numerically equivalent (rtol=1e-09, atol=0.0)
RESULT: PASS
$ sciproof verify baseline.sciproof.json
[FAIL] output examples/out/results.json: modified since capture (SHA-256 differs)
basis: FAIL because 1 (output examples/out/results.json) check(s) contradict the evidence; one failed check is enough
...
RESULT: FAIL
Source: the “Quick start” section of README.md. Outputs contain “...” where the README shortens them.
One command records the evidence. Two ways of checking it give three possible verdicts.
You have a script that produced a table or a figure. Months later, a reviewer, a collaborator or you yourself ask: which code and data made this, in which environment, and does it still come out the same? SciProof records the answer in a single JSON evidence manifest and can later verify it with an explicit verdict: PASS, FAIL or INCONCLUSIVE.
| Level | Question | SciProof |
|---|---|---|
| Integrity | Are the files what was recorded? | SHA-256 checks |
| Execution reproducibility | Does the command still produce the recorded results? | opt-in rerun; environment differences are reported, not fixed |
| Numerical equivalence | Equal within a declared tolerance? | --output-numeric with rtol/atol for .json, .csv, .tsv and (with NumPy) .npy |
| Scientific validity | Is the result correct? | Not addressed. A reproducible result can still be wrong. |
Bitwise reproducibility across different hardware or platforms is not promised. A PASS never means the science is right. More in Concepts.
Outputs declared with --output-numeric are compared with a relative (rtol) and an absolute (atol) tolerance that you choose at capture time. Try the rule SciProof uses. The defaults are the ones of sciproof capture: rtol=1e-9, atol=0.
| |recorded − found| | |
| rtol · max(|recorded|, |found|) | |
| atol | |
| allowed = max(relative, atol) |
src/sciproof/numeric.pydef _numbers_close(a: float, b: float, rtol: float, atol: float) -> bool:
if math.isnan(a) or math.isnan(b):
return math.isnan(a) and math.isnan(b)
if math.isinf(a) or math.isinf(b):
return a == b
return abs(a - b) <= max(rtol * max(abs(a), abs(b)), atol)SciProof cannot tell you which tolerance is scientifically appropriate. NaN only matches NaN and infinities must match exactly; integers and booleans are compared exactly. See thetutorial for choosing values.
This manifest was produced while building this page by running sciproof capture --name baseline --input examples/experiment.py --input examples/params.json --output examples/out/results.json --output-numeric examples/out/convergence.csv --rtol 1e-9 -- python examples/experiment.py. Select any field to see what it means, as documented in Manifest format.
Each line of the tree is a field of the JSON manifest.
Scenarios, not testimonials: SciProof is new and has no public users to quote yet.
Before submitting, capture the script that produced a figure and keep the manifest with the thesis repository. Months later, sciproof verify tells you whether the data and outputs are still the ones you reported, and --rerun whether the figure still comes out of the same command.
Given a manifest, run sciproof inspect first: it shows the command, environment and hashes without executing anything. Re-run only if you trust the command, ideally in a disposable VM, and read the verdict, including what was not checked.
Declare model metrics as numeric outputs with an explicit rtol/atol, so a re-run on slightly different numerics is judged by a tolerance you wrote down, and an unrelated change shows up as a FAIL instead of a shrug.
A job can run sciproof verify --json and fail on exit code 1 (FAIL) or 2 (INCONCLUSIVE). Integrity checks execute nothing; re-execution belongs only in trusted pipelines.
Recording a command with its hashes is not new. These notes keep the wording of the project's own comparison, including where an existing tool already does the job.
| Tool | Relation to SciProof |
|---|---|
| DVC | Pipelines with hashes of dependencies and outputs; heavier, more powerful. If you already use it you largely have integrity checking. |
| MLflow Tracking | Experiment tracking for ML with parameters and metrics, optional UI. |
The full analysis, including what could not be confirmed, is in thecomparison page. Its differentiation claim is described there as an unproven hypothesis.
sciproof verify --rerun executes the command recorded in the manifest, with your permissions and without a sandbox. A manifest you did not create is as untrusted as a script from the same source.inspect and plain verify never execute it.inspect and plain verify never execute it; --rerun runs it with your permissions, unsandboxed. Only rerun manifests you trust, ideally in a disposable VM or container. See SECURITY.md..json, .csv and .tsv (standard library only) and .npy arrays (pip install 'sciproof[numpy]'; numeric, boolean and complex dtypes, never unpickled).Full threat model: Security & threat model (also available as SECURITY.md in the repository).
SciProof is an open-source project. These numbers are read from GitHub; they were captured when the site was built and refresh in your browser if GitHub's API answers.
Updated .Your browser contacts api.github.com to refresh them; no cookies are set and nothing else is sent.
alok-10892 contributions
kdutt0974-source6 contributions
masyarakatai4 contributions
amarjithkumar2007-wq3 contributions
rahul05ranjan2 contributions
Aaqibhafeezkhan1 contributions
Golden007-prog1 contributions
slopggobblercobbler811 contributions
abhinavrastogi26271 contributions