Make sandbox results reproducible enough to review

Reproducibility needs more than isolation

An isolated execution environment keeps a workload separate, but it does not automatically make two runs identical. Runtime versions, dependencies, input data, network responses and environment settings can all change. Define the level of repeatability the task needs: rerunning a pure file transformation is different from reproducing a live website response. Lupin's hosted container interface provides execution, files and lifecycle controls, while the task must retain its own evidence of inputs and environment. A successful command in one sandbox is a result to review, not a universal claim that the workflow is deterministic.

Record what the environment actually contains

Inspect the language runtime and required tools with small bounded commands before relying on them. Record observed versions rather than copying a provider's default-machine description into the task. The current Lupin image field is passed into provider environment metadata by the adapter; it does not establish a documented arbitrary container-image deployment contract. Likewise, the public API does not expose general machine-size selection. Avoid claiming a specific CPU, memory allocation or custom image unless the configured provider path and actual environment demonstrate it. Store the facts needed to explain differences between runs.

Pin inputs and dependency resolution

Keep the exact input file or its durable location and a content digest. Use lockfiles or explicit versions for dependencies when the ecosystem supports them, and save the package manifest used by the task. A dependency installation that resolves latest at runtime can produce different behavior later. Do not automatically update versions while trying to reproduce a prior failure; that creates a new experiment. If network data is essential, record the source URL, retrieval time and relevant response identity. Prefer a small local fixture when the goal is to verify transformation logic independently of a changing external service.

{"command":"python --version && pwd","timeout_seconds":30}

Capture commands and meaningful checks

Retain the reviewed command or script, explicit working directory, expected output and actual exit result. Keep stdout and stderr bounded and sanitize secrets. A zero exit code does not prove the artifact meets the task, so add a semantic check such as parsing the generated file or comparing required fields. Record which checks actually ran. Source review can establish that an example aligns with an API contract, while a local fixture can establish behavior under its tested inputs. Neither alone proves live provider availability, production throughput or success on every future dataset.

Export a compact reproduction bundle

Save the input manifest, script, dependency lock, environment observations, output artifact and verification summary in the intended artifact location before stopping the sandbox. Large bundles need an approved storage route beyond the small built-in file transfer. Do not rely on indefinite stopped-resource retention or an old desktop-shell URL. A later authorized run can use the saved bundle to recreate the task and compare results. If exact reproduction is impossible because a source was live or a dependency changed, state that limit and identify the smallest stable fixture that still demonstrates the behavior under investigation.

Sources & verification

Reviewed against the current Lupin route, provider adapter and shared input contracts, plus the linked official references. Examples were not run against live services.

• docs.boat.dev · environments ↗Checked 2026-11-05
• docs.python.org · 3 · library · hashlib.html ↗Checked 2026-11-05

Prepared with AI assistance by Lupin editorial. How we research and maintain these guides.