enumeray connector run

Continuously claim and execute assignments while retaining recovery evidence.

Use when: You are running the long-lived production connector service for an enrolled host.

Role and prerequisites: Connector host operator. Install enumeray on the execution control host. Retain its private journal on durable storage; keep one process owner and the matching connector credential. See recovery rules before taking over retained state.

Example

enumeray connector run \
  --journal ./connector-state/journal \
  --api-base-url https://enumeray.example.org \
  --connector-id lab-worker --token-file ./connector-state/token \
  --max-workers 2 --max-transfer-workers 2 --max-pending-assignments 32

The default recovery mode is manual. After an unclosed owner session, establish that the prior process stopped everywhere before using --confirm-prior-process-stopped. exclusive-local is appropriate only for one installation on an unshared, uncloned local volume. The host’s --max-workers must fit API-side connector capacity and site capacity.

Inputs and options

Input	Meaning and use
--journal <JOURNAL>	Private durable journal directory. Reuse the same path after restart and never copy it to create another active connector. It contains ownership and backend execution evidence.
--log-policy-file <LOG_POLICY_FILE>	Connector-owned log capture/archive policy JSON, pinned before launching an assignment. Start from the log policy example and choose bounded capture and retention.
--api-base-url <API_BASE_URL>	Enumeray API origin. Use HTTPS, or loopback HTTP for local testing; enrollment/run can explicitly allow a trusted private HTTP network. A retained identity is bound to its original origin.
--connector-id <CONNECTOR_ID>	Enrolled connector ID created by enumeray connector create. It must match the token and retained journal.
--token-file <TOKEN_FILE>	Private regular file containing only the connector bearer token, typically the token file written by enroll. One-shot commands read a token from stdin if omitted.
--credentials-file <CREDENTIALS_FILE>	Private JSON map from SSH credential references in contexts to private key files on the connector host. OpenSSH runs as the connector OS account: provision the verified remote host key in that account’s ~/.ssh/known_hosts. Install the matching public key in the remote execution account’s authorized_keys. The private map stays on the connector host; see SSH context examples.
--max-workers <MAX_WORKERS>	Compute capacity slots for assignments, including queued Slurm jobs; 1–64, default 1. Choose within the service-side connector capacity and actual site capacity.
--max-transfer-workers <MAX_TRANSFER_WORKERS>	Independent bulk I/O subprocess bound; 1–64, default 2. Increasing it affects transfers, not the compute launch allowance.
--max-spool-bytes <MAX_SPOOL_BYTES>	Persistent scientific input/output-copy budget in bytes; default 1 TiB. Capacity remains reserved through interruption until confirmed cleanup.
--max-pending-assignments <MAX_PENDING_ASSIGNMENTS>	Total retained unfinished-assignment bound, including outputs still awaiting publication; 1–64, default 64. At the bound the connector stops accepting more work.
--unpublished-retention-seconds <UNPUBLISHED_RETENTION_SECONDS>	Publication deadline for finalized outputs that cannot be published; 1–315360000 seconds. Omit to retain indefinitely. Expiry alone does not authorize deleting execution files.
--poll-interval-seconds <POLL_INTERVAL_SECONDS>	Service polling interval, 1–300 seconds, default 2. Heartbeats and assignment progress continue independently of bulk transfer workers.
--shutdown-timeout-seconds <SHUTDOWN_TIMEOUT_SECONDS>	Grace period for service shutdown, 1–3600 seconds, default 30. Retain the journal for unfinished work after the service exits.
--cleanup-interval-seconds <CLEANUP_INTERVAL_SECONDS>	Service cleanup cadence; default 30 seconds. Each pass is bounded and preserves pending or uncertain work.
--completed-retention-seconds <COMPLETED_RETENTION_SECONDS>	Retention before reclaiming eligible completed run data; default 0 seconds. Eligibility still requires confirmed execution release and final delivery, independently of unpublished-output deadlines.
--recovery-mode <RECOVERY_MODE>	manual (default) requires an external stopped-owner confirmation after an unclosed session. exclusive-local is only for a single installation on an unshared, uncloned local volume, where local liveness evidence is trustworthy.
--confirm-prior-process-stopped	Explicit recovery assertion that the journal’s previous process stopped on every host, including out-of-band processes. Establish this externally before using the flag; it is not a backend cancellation command.
--allow-insecure-http	Allow sending credentials over HTTP to a trusted container/private network. Omit for HTTPS; this choice is retained in the run journal.
--allow-development-execution	Explicit host consent for trusted same-user process/direct development contexts. Required on every launch-capable invocation. Managed identity separation remains enforced; model code can access this OS user’s connector credentials and files.

Host commands use explicit connector credentials or the private journal and emit JSON results; compute-runner uses only its stdin protocol and directory. User profile and API override options do not apply.

Result and next step

Runs until stopped, with durable ownership, independent heartbeat/progress work and bounded compute/transfer capacity. The process owns the journal exclusively and preserves unfinished work on shutdown.

Run it under a supervisor with a private durable state volume. Inspect readiness using enumeray connector diagnostics and follow safe restart/recovery.

Exact syntax, defaults and accepted values

Continuously claim and execute assignments over HTTP

Usage: enumeray connector run [OPTIONS] --journal <JOURNAL> --api-base-url <API_BASE_URL> --connector-id <CONNECTOR_ID> --token-file <TOKEN_FILE>

Options:
      --api <API>
          User API origin override; credentials never cross origins. Host/runner commands use explicit configuration

      --journal <JOURNAL>
          Private durable journal directory; reuse after restart and never clone an active journal

      --log-policy-file <LOG_POLICY_FILE>
          Connector-owned capture/archive policy, pinned before launch

      --profile <PROFILE>
          Named local user API/login profile; host/runner commands use explicit configuration

          [default: default]

      --api-base-url <API_BASE_URL>
          API origin bound to the enrolled identity; non-loopback HTTP requires --allow-insecure-http

      --config <CONFIG>
          Private user profiles file; defaults to $XDG_CONFIG_HOME/enumeray/profiles.json or ~/.config/enumeray/profiles.json

      --connector-id <CONNECTOR_ID>
          Enrolled connector ID; must match the token and retained journal

      --json
          Emit user API results as JSON (NDJSON for following logs) and runtime errors on stderr; host/runner output follows its own protocol

      --timeout <TIMEOUT>
          User API HTTP timeout in seconds; live logs use it for read inactivity. Does not limit execution time

          [default: 60]

      --token-file <TOKEN_FILE>
          Private regular file containing only the long-lived connector bearer token from enrollment

      --credentials-file <CREDENTIALS_FILE>
          Private credential-reference map; never sent to Enumeray

      --max-workers <MAX_WORKERS>
          Maximum assignments reserving compute capacity (1-64); queued Slurm allocations count too

          [default: 1]

      --max-transfer-workers <MAX_TRANSFER_WORKERS>
          Maximum simultaneous bulk I/O subprocesses (1-64), independent of compute capacity

          [default: 2]

      --max-spool-bytes <MAX_SPOOL_BYTES>
          Persistent scientific input/output-copy budget in bytes; default 1 TiB, reserved until confirmed cleanup

          [default: 1099511627776]

      --max-pending-assignments <MAX_PENDING_ASSIGNMENTS>
          Maximum retained unfinished assignments (1-64), including unpublished outputs; must be at least --max-workers

          [default: 64]

      --unpublished-retention-seconds <UNPUBLISHED_RETENTION_SECONDS>
          Retention deadline in seconds for unpublished finalized outputs (1-315360000); omit to retain indefinitely

      --poll-interval-seconds <POLL_INTERVAL_SECONDS>
          Service polling interval in seconds (1-300); heartbeat and transfer work remain independent

          [default: 2]

      --shutdown-timeout-seconds <SHUTDOWN_TIMEOUT_SECONDS>
          Shutdown grace period in seconds (1-3600); unfinished work remains in the journal for recovery

          [default: 30]

      --cleanup-interval-seconds <CLEANUP_INTERVAL_SECONDS>
          Cadence in seconds for bounded, low-priority completed payload cleanup (1-3600)

          [default: 30]

      --completed-retention-seconds <COMPLETED_RETENTION_SECONDS>
          Debugging grace in seconds after safe completion; zero adds no delay before eligible cleanup

          [default: 0]

      --recovery-mode <RECOVERY_MODE>
          Journal ownership recovery policy; manual requires external stopped-owner confirmation

          Possible values:
          - manual:          Require external stopped-owner confirmation after an unclosed journal session
          - exclusive-local: Use local liveness evidence only on a single unshared, uncloned local installation

          [default: manual]

      --confirm-prior-process-stopped
          Confirm an unclosed manual-recovery session's previous owner stopped everywhere

      --allow-insecure-http
          Permit bearer credentials over HTTP on a trusted container/private network

      --allow-development-execution
          Permit trusted process/direct development contexts under this regular OS user

  -h, --help
          Print help (see a summary with '-h')