Wrapper protocol 1

Role: Model author. Prerequisites: A scientific declaration and an implementation to call. The generated wrapper.py and protocol.py provide an editable working boundary; your scientific solver need not import Enumeray.

Use the complete model examples archive to acquire matching code, fixtures and executable tests together. The fixture family map identifies each model, entrypoint and input tree. Individual request/result downloads are envelopes to inspect, rather than complete projects.

Know the process interface

The connector starts your registered entrypoint in an isolated job workspace and appends:

--request REQUEST_JSON --result RESULT_JSON

For the starter’s execution object, entrypoint: ["wrapper.py", "execute"] produces the equivalent of:

/opt/enumeray/models/example/r1/wrapper.py execute \
  --request /path/to/job/request.json \
  --result /path/to/job/result.json

The entrypoint is a release-relative executable, with fixed additional arguments. Paths in this illustration come from the connector; scientific inputs must not choose the executable, hosts, credentials or arbitrary command arguments. Read installed code/data relative to the release, and write outputs in the job workspace. The wrapper receives no API or storage credentials.

Complete scalar request and result

Download the request and expected result. The complete scalar request is:

{
  "wrapper_protocol_version": 1,
  "run_id": "11111111-1111-4111-8111-111111111111",
  "assignment_id": "22222222-2222-4222-8222-222222222222",
  "attempt_id": "33333333-3333-4333-8333-333333333333",
  "launch_intent_id": "44444444-4444-4444-8444-444444444444",
  "model_id": "example",
  "publication_id": "starter-publication",
  "inputs": {"value": 2},
  "input_files": {}
}

inputs is the scientific input object. The other fields identify this execution, independently of the scientific model version. Real requests are created by the connector; these fixture UUIDs are for local tests.

The wrapper must echo all four UUIDs exactly, preserve protocol version 1, and produce this complete successful result:

{
  "wrapper_protocol_version": 1,
  "run_id": "11111111-1111-4111-8111-111111111111",
  "assignment_id": "22222222-2222-4222-8222-222222222222",
  "attempt_id": "33333333-3333-4333-8333-333333333333",
  "launch_intent_id": "44444444-4444-4444-8444-444444444444",
  "outcome": "succeeded",
  "outputs": {"doubled": 4},
  "failure": null,
  "files": {}
}

Results do not echo model_id, publication_id or the inputs. The result identities bind the evidence to the launched execution. A plain {"doubled":4} file is not a wrapper result.

Both envelopes are limited to 64 KiB. Reject malformed envelopes, unknown envelope fields, duplicate JSON keys, non-finite values, unsupported protocol versions and malformed/noncanonical identity UUIDs. The starter also checks the model ID against its installed publication.json and accepts a syntactically valid publication ID rather than hardcoding one; the same immutable installation can serve a new publication. Adapt scientific input validation when changing its supported calculation.

Map a Python calculation

In a default scalar project, model.py provides a function taking a finite Decimal and returning the doubled value. protocol.read_json parses JSON numbers as Decimal and protocol.exact_json writes them as numbers. Keep this behavior if exact decimal input/output matters.

The following is the complete scalar implementation.py mapping for the generated project:

from decimal import Decimal
from model import double
from protocol import ModelError

def evaluate(request, workspace, mode):
    if mode != "json":
        raise ModelError("JSON inputs are required.")
    if (request["input_files"]
            or request["inputs"].keys() != {"value"}
            or not isinstance(request["inputs"]["value"], Decimal)):
        raise ModelError("One finite numeric value is required.")
    return {"doubled": double(request["inputs"]["value"])}, {}

The tuple is (ordinary_outputs, file_outputs). The generated wrapper.py calls this function, constructs the result envelope, publishes it durably, and chooses an exit status. Replace double with your own scientific function, update validation and output names, and update publication.json and tests together. Keep diagnostic text in stderr and put scientific data in the declared result.

For numerical methods with tolerances, assert known values in the test suite explicitly, for example:

self.assertAlmostEqual(float(self.result()["outputs"]["doubled"]), 4.0, places=12)

Choose the tolerance from your scientific method rather than copying it indiscriminately. The doubling starter’s existing tests use exact equality.

Call an existing executable

An executable starter uses a fixed native command; it still uses the same Enumeray request/result envelopes:

enumeray model init solver-example --directory solver-example --kind executable
cd solver-example
python3 -B -m unittest discover -s . -p 'test_*.py'

The generated implementation.py defines:

RELEASE = Path(__file__).resolve().parent
SOLVER_ARGV = [sys.executable, str(RELEASE / "demo_solver.py")]

Replace that line with your installed command, for example:

SOLVER_ARGV = [str(RELEASE / "bin" / "solver")]

The demo’s exact native interface is:

solver --mode json --input INPUT_JSON --output OUTPUT_JSON
solver --mode files --input INPUT_CSV --output OUTPUT_CSV

For JSON mode, the mapping writes {"value":2} to a temporary native input file and expects {"doubled":4} in the native output file. For file mode, it passes the verified staged CSV path and expects a CSV output. Replace the argument names and file-format translation to fit your actual solver.

The native launch is an argument array with shell=False:

completed = subprocess.run(
    [*SOLVER_ARGV, "--mode", mode, "--input", str(source), "--output", str(target)],
    shell=False,
    stdin=subprocess.DEVNULL,
    stdout=subprocess.DEVNULL,
    stderr=subprocess.DEVNULL,
    cwd=workspace,
    check=False,
)
if completed.returncode:
    raise ModelError("Scientific executable failed; inspect its declared native interface.")

The starter suppresses native stdout/stderr. If you change this for debugging, choose diagnostics appropriate for your model’s scientific-data access policy. Validate native output before returning it; a native exit code 0 alone does not establish a valid result. Keep the executable in the foreground until its output is complete.

File input and output example

A file starter processes a CSV whose bytes are:

index,value
0,1
1,2

Its public scientific inputs are an artifact reference. Its complete local wrapper request additionally names the staged path. Download this request.

{
  "wrapper_protocol_version": 1,
  "run_id": "11111111-1111-4111-8111-111111111111",
  "assignment_id": "22222222-2222-4222-8222-222222222222",
  "attempt_id": "33333333-3333-4333-8333-333333333333",
  "launch_intent_id": "44444444-4444-4444-8444-444444444444",
  "model_id": "csv-example",
  "publication_id": "starter-publication",
  "inputs": {
    "dataset": {"artifact_id": "55555555-5555-4555-8555-555555555555"}
  },
  "input_files": {
    "dataset": {
      "artifact_id": "55555555-5555-4555-8555-555555555555",
      "path": "inputs/dataset.csv"
    }
  }
}

The connector verifies and stages the completed artifact before launch. input_files.dataset.path is relative to the job workspace; the original artifact reference remains in inputs for provenance. The generated mapping calls staged_dataset(request, workspace) to check the declared slot, artifact identity and workspace boundary, and transform_csv to calculate outputs.

The output CSV is index,value\n0,2\n1,4\n. The complete wrapper result reports the file path rather than a fabricated public artifact UUID. Download the result.

{
  "wrapper_protocol_version": 1,
  "run_id": "11111111-1111-4111-8111-111111111111",
  "assignment_id": "22222222-2222-4222-8222-222222222222",
  "attempt_id": "33333333-3333-4333-8333-333333333333",
  "launch_intent_id": "44444444-4444-4444-8444-444444444444",
  "outcome": "succeeded",
  "outputs": {"sample_count": 2},
  "failure": null,
  "files": {
    "processed_dataset": {
      "path": "processed.csv",
      "media_type": "text/csv",
      "scientific_metadata": {}
    }
  }
}

The file property must be absent from ordinary outputs. A file report accepts only path, media_type and optional scientific_metadata. Do not report size_bytes, sha256 or artifact_id: the connector measures and checks the exact file, uploads it, and puts its completed artifact reference into the final public scientific outputs.

Paths must be relative to the workspace, without parent traversal or symbolic links. Omit optional absent files; protocol 1 has no wrapper-supplied optional-output failure annotation. Adapt the starter’s fixed slot names and byte bounds to your declaration.

Run file fixtures manually

From a generated file project:

PROJECT_DIR="$(pwd)"
WORK_DIR="$(mktemp -d)"
mkdir "$WORK_DIR/inputs"
cp fixtures/dataset.csv "$WORK_DIR/inputs/dataset.csv"
cd "$WORK_DIR"
python3 "$PROJECT_DIR/wrapper.py" execute \
  --request "$PROJECT_DIR/fixtures/request.json" \
  --result "$WORK_DIR/result.json"
python3 -m json.tool "$WORK_DIR/result.json"
cmp "$WORK_DIR/processed.csv" "$PROJECT_DIR/fixtures/expected.csv"
cd "$PROJECT_DIR"

Exit 0 and a silent successful cmp establish that this fixture produced the expected bytes. The fixture artifact UUID is a local label; a real user must upload a file and supply the completed UUID.

Report failures and preserve evidence

If the envelope is valid but the scientific calculation fails, publish a failed result with the same four identities, empty outputs/files, and a safe failure object. The scalar result above becomes:

{
  "wrapper_protocol_version": 1,
  "run_id": "11111111-1111-4111-8111-111111111111",
  "assignment_id": "22222222-2222-4222-8222-222222222222",
  "attempt_id": "33333333-3333-4333-8333-333333333333",
  "launch_intent_id": "44444444-4444-4444-8444-444444444444",
  "outcome": "failed",
  "outputs": {},
  "failure": {
    "code": "model_failure",
    "message": "Model could not process its declared inputs or publish outputs."
  },
  "files": {}
}

The starter exits 0 after success, 1 after publishing a scientific failure, and 2 when the request/declaration is invalid or result publication fails. Invalid envelopes must not publish a result pretending the model ran. Keep failure diagnostics free of undeclared scientific values or private paths.

protocol.publish writes and syncs a temporary file, publishes it with an atomic same-filesystem operation that refuses an existing destination, and syncs the directory. The wrapper refuses existing result evidence before starting the calculation. Preserve this behavior when adapting the boundary. A nonzero exit is not grounds to discard an already published result or rerun an uncertain execution.

Before installation, run the full local tests and add cases for your own native errors, file formats and numerical behavior. For Slurm, the model-owned script launches exec "$1" local-worker --directory "$2"; see publication and the runner contract.

Collection inputs and outputs

Use x-enumeray-collection on a top-level named object slot whose sole required property is the string collection_id. The annotation declares media_types, max_members, max_file_bytes and max_total_bytes; at most 10,000 members and 16 MiB of manifest evidence are supported. Empty collections are valid. Slots may be optional, but every member listed in a collection is required.

The wrapper request adds input_collections only when collections are staged:

{"dataset":{"collection_id":"COMPLETED_UUID","root":"collections/COMPLETED_UUID/files","manifest":"collections/COMPLETED_UUID/manifest.json"}}

These paths are relative to the workspace. The manifest is a JSON array of pinned members with path, media_type, scientific_metadata, size_bytes, checksum_sha256, artifact_id and version_id. Read member paths relative to root. All required input members are verified before launch; storage capabilities and service credentials never reach the wrapper.

Report outputs with a collections object in the normal protocol-1 result:

{"snapshots":{"root":"snapshots","manifest":"snapshots-members.json"}}

This snippet is the value of collections, not a complete result envelope. Keep collection slots out of ordinary outputs, just as with file slots. The sidecar is an explicit JSON array of {path, media_type, scientific_metadata} objects; paths are relative to the reported root. No recursive output discovery occurs. The runner measures sizes/checksums and retains the frozen manifest before publication. Do not claim artifact identities or integrity fields in this sidecar. Keep both envelopes within 64 KiB; large member lists belong in the sidecar.

The model examples archive contains the complete collection declaration, wrapper, protocol helper, fixtures and tests under examples/collections/. It copies a variable-size verified input set into an output collection and demonstrates nested paths and empty sets. Run from the extracted archive root:

python3 -B -m unittest discover -s examples/collections -p 'test_*.py'
COLLECTION_DIR="$(pwd)/examples/collections"
WORK_DIR="$(mktemp -d)"
cp -R "$COLLECTION_DIR/fixtures/workspace/." "$WORK_DIR/"
python3 -B "$COLLECTION_DIR/wrapper.py" execute \
  --request "$COLLECTION_DIR/fixtures/request.json" \
  --result "$WORK_DIR/result.json"
cmp "$WORK_DIR/inputs/mesh/a.bin" "$WORK_DIR/snapshots/mesh/a.bin"
cmp "$WORK_DIR/inputs/series/snapshot-002.bin" "$WORK_DIR/snapshots/series/snapshot-002.bin"
python3 -m json.tool "$WORK_DIR/result.json"

The local request targets collection-example and stages the two members below inputs/. Successful execution copies their exact bytes into snapshots/, writes snapshots-members.json and reports member_count: 2. An empty collection reports 0. The suite also checks malformed UUIDs/model identity, duplicate/unknown fields, invalid mappings/paths, existing evidence, explicit scientific failure and interrupted/competing result publication. Validation of the complete request and mapping precedes scientific side effects; an existing result is refused before copying. Atomic no-replacement publication preserves another writer’s result. Local wrapper acceptance is separate from validation by the service and runner.

The individually labeled collection wrapper requires the protocol helper alongside it. Save the wrapper as wrapper.py when using the project fixtures and tests; acquire the archive for the complete runnable project. The connector requires explicit collection capabilities and matching API/runner binaries; contract and wrapper protocol versions remain 1. Transfer recovery retains the original execution identity rather than executing the model again.