Verify agent code with a small local fixture

Choose the claim the fixture can prove

A fixture is a small controlled input used to exercise a specific behavior. It is useful for parsing, formatting, validation and pure transformations that should not depend on a live provider. Begin with a concrete claim, such as preserving every required column or rejecting an invalid record. Do not describe a fixture test as proof that a production integration is available or that every real dataset will work. Lupin's sandbox can execute the check, but the test's meaning comes from its inputs and assertions. Keep that boundary visible in the task's verification record.

Build a representative minimal input

Include the smallest normal case plus the edge case that could change the outcome: an empty list, a missing field, non-ASCII text or a malformed value. Use synthetic data instead of copying private production records when the behavior does not require them. Retain the exact fixture bytes or a content digest with the script. The built-in file route can upload small fixtures through base64, with a roughly two-megabyte transfer bound and safe-path validation. Keep fixture paths distinct from final artifacts so a failed test cannot overwrite the deliverable being reviewed.

Assert meaning rather than implementation shape

Check the property the user needs, not a string that merely repeats the program's internal steps. A JSON output test should parse the file and inspect required values; a table test should verify the records and columns that matter. Include a failure assertion when the program is supposed to reject bad input. Keep the check independent enough to catch an incorrect implementation. A script that prints success unconditionally or compares an output to itself provides no meaningful evidence. Avoid adding broad test machinery for a reversible one-line task when a focused semantic check suffices.

import json
fixture = {"names": ["Ada", "민수"]}
encoded = json.dumps(fixture, ensure_ascii=False)
assert json.loads(encoded) == fixture
print("Unicode round trip passed")

Run the check without external side effects

Use a bounded command with an explicit operation identity, inspect exit_code, stdout and stderr, and keep network dependencies out of the fixture when they are not part of the claim. No paid provider call, outbound message or purchase is needed to validate a pure transformation. If the command response is lost, inspect history and artifacts before repeating it under another key. The command route's HTTP success does not guarantee the assertion passed. Save the actual result and identify the environment version if it affects interpretation. Report unrun examples as source-reviewed rather than tested.

Describe coverage honestly

A useful verification note names the fixture, behavior checked and observed outcome, plus a practical limitation. It might establish that the parser handles an empty input and preserves Unicode while leaving live service authentication untested. Export the artifact and test evidence before stopping the sandbox. If a check fails, fix the relevant logic and rerun the focused check; do not broaden claims merely because the revised command exits zero. This approach gives the user concrete confidence in the behavior that was exercised while keeping production readiness, provider availability and untested edge cases separate.

Sources & verification

Reviewed against the current Lupin route, provider adapter and shared input contracts, plus the linked official references. Examples were not run against live services.

• docs.python.org · 3 · library · subprocess.html ↗Checked 2026-11-05
• docs.python.org · 3 · library · hashlib.html ↗Checked 2026-11-05

Prepared with AI assistance by Lupin editorial. How we research and maintain these guides.