Limits and retention

These values describe the current implementation. Some are fixed protocol bounds, some are configurable defaults and some are site admission policy. Increasing a general limit does not bypass a narrower route or declared scientific bound.

Area	Bound / default	Practical effect
General API request body	1 MiB default; configurable 1 byte–16 MiB	Routes may enforce smaller limits
Author publication / local JSON configuration reads	1 MiB	Large scientific declarations need reduction
Compact scientific run inputs	64 KiB	Independent of the general HTTP request limit
Wrapper request/result and assignment result	64 KiB	Bulk scientific data belongs in declared files
Connector heartbeat body	8 KiB	Operational reports are bounded
Upload reservation / completion	64 KiB / 8 KiB	Metadata/verification payloads are bounded separately from file bytes
Artifact file	100 GiB default	Central admission policy; new reservations select multipart strictly above 64 MiB
Single PUT	5 GiB	Transport ceiling, independent of the 64 MiB selection threshold; existing reservations retain their original transport
Run input/output aggregate	1 TiB each	Actual pinned input bytes and reserved/completed output bytes
Outstanding uploads	1 TiB per owner; 10 TiB per destination	Includes abandoned reservations until confirmed cleanup; published storage has separate retention
Connector scientific spool	1 TiB default	Persistent input/output-copy reservations; --max-spool-bytes changes admission
Declared file max_bytes	At most 10,000 × 5 GiB	Structural bound; deployment policy and scientific declarations can be much narrower
HTTP multipart run attachments	At most 8; JSON request part at most 1 MiB	Whole HTTP body uses the deployment limit (1 MiB default, configurable up to 16 MiB); compact inputs independently cap at 64 KiB and attachment declarations may be narrower
Scientific schema complexity	128 nodes per schema	Local supported-reference profile only
Collection members	10,000 per collection and per run direction, including standalone files	Model declarations may impose smaller limits
Collection manifests	16 MiB; metadata requests/pages at most 64 KiB	Member listings also contain at most 100 entries
Named scientific outputs	At most 64	Closed object with unconditional required names
Desired connector configuration	1,024 required publications and 2 MiB	Explicit retirement frees unused configuration; rejection preserves existing configuration
Retained local installation cache	4 MiB	Shared or recoverable references retain entries after retirement
Installation protection inventory	1 MiB	Missing capability, unknown metadata or exhausted protection capacity retains payloads
Completed run cleanup	Zero-second retention; 30-second sweep	Reclaims eligible local/compute copies after confirmed release and final delivery
Personal-token lifetime	Up to 365 days; CLI default 30 days	Delegated tokens also respect parent lifetime
Upload session	Single PUT: 1 hour; multipart: 48 hours	Central policy captures each multipart deadline and capability lifetime; hard maxima are 48 hours and 15 minutes

Logs and archives

The API retains at most 1,024 log batches and seven days of payload history. Its ingestion verification window retains 64 receipts. SSE buffers at most 32 entries and admits 128 streams per application; keepalive comments arrive every 15 seconds. Expired history requires the supplied recovery cursor and an explicit gap indication.

Connector rotating capture defaults to sixteen 64 KiB segments per source, with one extra during rotation. Delivery batches use at most 2 KiB of raw source bytes; retrieval read-ahead is bounded separately. Pipes continue draining when retention fills. Enumeray never truncates active slurm.log; run-owned logs become eligible for completed-run cleanup after final delivery. Slurm operators manage upstream/site-owned log retention separately.

Archives are disabled by default. Enabled default spools retain 16 MiB per source (15 MiB prefix plus up to 1 MiB tail) under a 50 MiB compressed policy ceiling. Publication defaults to a 24-hour deadline after finalization. The API independently caps archive uploads at 512 MiB and published retention at 30 days. The smaller compressed-policy ceiling and 512 MiB structural bound both apply; exceeding either rejects reservation with 422 invalid_upload. Metadata request-size overflow is a separate 413. Read-time expiry and dependency pinning remain in force even before cleanup runs.

Capacity and timeouts

Connector defaults are one compute worker, two transfer workers and 64 pending assignments. The pending cap must be at least the compute-worker count. Queued Slurm allocations reserve compute slots; confirmed allocation release frees them even when publication remains pending.

Idle polling defaults to two seconds; the API currently reports a 15-second heartbeat interval. Transfer attempts have independent progress/deadline bounds and retry backoff; these are not scientific runtime deadlines. Unpublished deliverables remain retained indefinitely unless an explicit retention policy is configured, and that policy does not delete local evidence.

Completed-run cleanup is independent of unpublished-delivery deadlines. Configure --completed-retention-seconds and --cleanup-interval-seconds on the connector. Active or uncertain execution, pending delivery and unavailable compute hosts remain retained. Archived evidence and replay guards are preserved; published object versions are outside connector cleanup. See cleanup operations.

Model batch scripts and runtimes own resource requests and scientific deadlines. Global/per-user queue limits and grant/context/connector limits are operational admission controls. See configuration and connector operation for tuning without changing scientific execution semantics.

Transfer selection and independent admission

New file	Selected transport	Independent admission checks
1 MiB	Single PUT	Must still fit per-file policy, model declaration and aggregate quotas
100 MiB	Multipart	Above 64 MiB even though below the 5 GiB single-PUT ceiling; the same admission checks apply
10 GiB	Multipart	Above the single-PUT ceiling; the default 100 GiB admission alone does not override a narrower declaration or exhausted quota

Exactly 64 MiB selects single PUT; one byte more selects multipart. Changing policy affects new reservations. Replay or inspection of an existing reservation preserves its original transport, expiry and upload/artifact identity. A selection threshold never overrides another bound. The downloadable conformance records supply the machine-readable values asserted against the implementation.

Multipart object uploads

New uploads larger than 64 MiB transfer directly to versioned storage in sequential parts. The default part size is 64 MiB and grows when necessary to stay within 10,000 parts. Progress and receipts are durable; an interrupted part retries in full while confirmed parts are retained. Existing reservations keep their original transport. Downloads and direct/SSH staging and collection resume verified prefixes; HTTP requests cover at most 64 MiB and checkpoint payloads every 8 MiB.

The independent storage worker assembles the frozen manifest and streams the exact completed object version through SHA-256 verification. Multipart completion returns HTTP 202 while this is pending. An authenticated completion retry then publishes the immutable receipt. Composite checksums and multipart ETags never substitute for whole-file SHA-256. The worker uses fixed read buffers and defaults to two concurrent jobs.

Bulk I/O uses independent 15-second connection, 60-second stall and 48-hour overall budgets. Transfer authorization permits a request to start for up to 15 minutes; the bulk budget governs an active transfer. Short API request timeouts remain independent. The 100 GiB default is provisional admission policy, pending representative throughput and capacity qualification under TASK-082.

Operators may shorten bulk budgets using ENUMERAY_TRANSFER_CONNECT_SECONDS, ENUMERAY_TRANSFER_STALL_SECONDS and ENUMERAY_TRANSFER_OVERALL_SECONDS on transfer processes. Their maxima are the defaults above, keeping abandoned-byte cleanup bounded. Central upload policy may shorten multipart session and authorization lifetimes for new reservations.