Connected and HTTP worker transports

Support two execution transports from the initial release without creating two execution models. Long-lived processes use outbound persistent connections by default. Serverless deployments explicitly expose an HTTP handler for invocation by Ryusen. Both carry the same RFD 2 execution envelopes and have the same step.run semantics.

connect is every SDK’s default. A worker opens an authenticated outbound bidirectional session to Ryusen, advertises the logical functions it can execute and its capacity, receives work, and reconnects automatically. This ephemeral capability advertisement is distinct from durable application-manifest and trigger publication under RFD 2. It avoids inbound networking and lets the server dispatch only to live capable workers.

http is explicit. The application exposes an SDK handler and registers or configures its URL with Ryusen. The server invokes that URL for discovery, forward execution, and compensation. This mode is intended for AWS Lambda, Google Cloud Functions, and other request-driven environments where persistent connections are unavailable or inappropriate.

SDKs must not silently fall back between modes. A connect failure is visible and retried as a connect failure; it must not unexpectedly expose or select an HTTP endpoint.

Worker transport ends at one server-side dispatch interface. The durable engine creates and leases work before either transport sees it. Both modes receive the same target, history, operation ID, attempt number, and fencing generation, and produce the same protocol response.

Transport sessions, sockets, heartbeats, and endpoint health are ephemeral. Run state, work, leases, attempts, accepted results, and history remain in the transactional store. Losing every gateway process or notification cannot lose committed work.

The connected protocol is a sequence of versioned envelopes over a reliable, ordered, framed bidirectional session. The selected carrier supplies framing; the message meanings and state machine remain identical. Canonical execution payloads remain portable across Rust, Go, Elixir, and TypeScript.

The initial proof-of-concept protocol version is a lockstep server-and-SDK contract: adding a frame such as the pre-start start_rejected decision requires deploying the matching SDK with the gateway. It does not promise mixed-version rolling operation within protocol version 1. Before independently deployable gateways and workers are supported, such additions must either increment the negotiated protocol version or introduce an explicit capability and retain the older interaction for peers that do not advertise it.

The connection lifecycle is:

The gateway tracks capacity per live session and never assigns more than the advertised maximum. Assignments consume capacity until rejection or completion. This is transport backpressure, not user-configured workflow concurrency.

A release ID is opaque and stable across every replica and reconnect belonging to one rollout. The gateway records a server commit position when a release first successfully registers and preserves that position across connection churn. It rejects the same release ID paired with a different manifest hash or execution contract. A build ID may additionally be recorded for diagnostics but has no routing semantics.

The execution contract summarizes mechanically checkable compatibility: protocol range, portable-value codec versions, target function IDs, and required execution features. It does not claim to prove that arbitrary user-code changes are semantically compatible. An SDK without a stable rollout identifier joins an unversioned group derived from this contract; process start time and reconnect time never masquerade as release order.

Routing prefers the greatest compatible release position and balances within that group using live capacity. Older compatible groups may remain fallback during rollout but receive no assignments after declaring drain. This preference is evaluated for every assignment: neither a run nor a logical step is pinned to a release. If no compatible capacity exists, work remains queued rather than being sent to an incompatible worker. Explicit releases outrank unversioned groups; if all candidates are unversioned, routing balances them without pretending process or connection age indicates code age.

This adopts the useful separation in Inngest Connect between a synchronized app, worker-group identity, and live connection capacity. Inngest dynamically routes across healthy groups and biases newer CreatedAt values; Ryusen additionally requires an explicit stable release identity and hard compatibility filtering. The relevant prior art is inngest/inngestgo/connect for SDK registration and inngest/inngest/pkg/connect/state plus pkg/connect/routing for grouping and gateway selection.

For each durable lease, the gateway sends assignment containing a unique request ID and lease token. The worker reserves local capacity and responds with assignment_accepted but does not invoke user code yet. The gateway transactionally marks the attempt started and responds with start_authorized. Only then may the worker execute the callback.

This establishes a useful failure boundary:

For an effectful target, the authorization frame can be lost after its transaction commits. This may conservatively produce Unknown even though the worker never received permission and user code did not run. Avoiding a duplicate effect is more important than avoiding operator resolution in this indistinguishable case.

The worker extends leases while authorized work runs. It sends result until the gateway returns a correlated terminal decision. result_accepted confirms that the durable transition committed. result_rejected tells the worker to discard that assignment and result and return to readiness; it does not prove that an external effect did not occur or that the effect’s durable outcome was settled. Duplicate results for the same request and content are idempotent. Unacknowledged results are retained in memory and resent after reconnect. A worker-process crash can still lose an effectful result and leave an Unknown outcome, which is inherent without a durable participant protocol.

The current Rust PoC reconnects immediately after the first transport loss and uses bounded exponential backoff when connection attempts or newly ready sessions continue to fail. A session must remain ready for a stability interval before a later loss earns another immediate retry. Authentication rejection and malformed protocol handshakes are terminal; connection, handshake, and registration waits are bounded. In-flight result retention and redelivery remain future work as described above. The local PoC accepts only ws:// endpoints; the production authentication and TLS requirements below remain future work.

After cancellation or timeout commits, the gateway sends a best-effort stop_requested frame for authorized work. Repeatable discovery can be fenced without settlement. For an effectful callback, this frame asks the SDK to cooperatively stop work but does not revoke start authorization or prove that an external effect stopped. The durable RFD 4 settlement token, not delivery of the frame, controls whether a late result is accepted.

A worker that can stop cooperatively still returns a structured terminal result. If the transport disappears or no result establishes the callback outcome, lease expiry transitions the settlement to Unknown. Reconnection may redeliver the result while the attempt generation remains unresolved. It races operator resolution on the same durable compare-and-set. A result arriving after operator resolution is retained diagnostically and rejected as stale.

Both peers use heartbeats to detect half-open connections. Workers reconnect with bounded exponential backoff and re-register their current executable function capabilities without changing the durable application manifest. A reconnect includes authorized in-flight request IDs and lease tokens so a valid attempt may continue lease extension and return its result over the new session. It reuses the original release and instance identity, so reconnect cannot make old code appear newer. Invalid or expired attempts are fenced.

When registration establishes a newly preferred compatible release, the gateway notifies the durable engine for each advertised function. That notification wakes DefinitionBlocked runs so current code can be rediscovered immediately; it does not pin those runs to the release or guarantee which compatible worker receives the assignment. Reconnects of the same release do not create a false code-change signal, though an operator may always request an immediate retry.

On graceful worker shutdown, worker_draining stops new assignments while authorized work completes for a bounded period. On gateway shutdown, gateway_draining asks workers to establish a replacement connection where one is available. The single-process local server may simply reconnect workers after restart; durable lease recovery remains authoritative.

HTTP uses one authenticated request-response exchange per protocol invocation. The configured endpoint advertises its executable function capabilities through the registration request and handles the canonical execution body. Trigger configuration comes from the separately published durable application manifest. Non-success transport status, structured application failure, and target_not_found remain distinct.

The server records the attempt before sending the HTTP request. HTTP outcomes are classified by what they prove about callback execution:

The ambiguous class includes timeout, disconnect, truncated or malformed body, invalid response authentication or correlation, and any bare non-success status such as 429, 500, 502, or 503. An intermediary status cannot prove that user code did not run: a callback may commit its external effect before middleware, a proxy, or response serialization fails. Only the semantic protocol response, not the HTTP status, proves pre-start rejection or callback completion.

Repeatable discovery may be invoked again after every transport ambiguity. Rediscovery remains safe from tracked external effects even when ambient state changes its declarations. A valid structured application failure follows the configured callback retry policy; as specified by RFD 1, application code may still have produced a partial effect and should use the stable operation ID for idempotency. HTTP success is acknowledged only after the response’s state transition commits.

Canonical execution requests are limited to 1,000,000 encoded bytes. The durable replay portion is limited to 500,000 encoded bytes so event input and envelope metadata retain bounded headroom. Recursive protocol values are limited to 48 nested array or object values so a result remains decodable after being wrapped in replay and carrier envelopes. Ryusen checks an HTTP request before start authorization; legacy work that exceeds the request limit is deferred without invoking the endpoint so it cannot block unrelated ready work. A result that would grow durable replay beyond its byte or nesting limit is not accepted. Because the callback has already run in that case, normal lease recovery classifies the effect as Unknown rather than retrying it unsafely.

HTTP endpoints may be scaled or replaced freely. They implement logical functions, not pinned builds. Ryusen performs protocol validation but leaves revision selection and rolling-deployment behavior to the configured endpoint. Because HTTP registration and unversioned groups cannot reliably prove that code changed, blocked runs use capped low-frequency probes in addition to explicit operator retry rather than treating every request or process restart as a new release.

Each SDK’s primary startup API runs a connected worker. HTTP serving requires an explicit constructor, option, or adapter. The same function registry and step.run callbacks are reusable in either mode; selecting HTTP does not require rewriting workflow code.

Environment detection may recommend HTTP mode on known serverless platforms but does not silently change the configured mode in the initial release.

Both initial SDKs implement the same connected session for this milestone.

Capacity is one and assignments are processed serially: a worker accepts one assignment, completes it, and only then reads the next. The gateway tracks per-session capacity and the local session loop already serializes assignments; lease extension, heartbeats, graceful drain, and in-flight result redelivery remain deferred, so an SDK must not advertise capacity it cannot execute concurrently.

HTTP mode is explicit. For the local server the endpoint is configured with --http-worker-url. The server owns request signing: with a configured signing key it sends the key as a bearer token, an expiry, and a signature over the exact request body; the local development server has no key and sends invocations unsigned. The SDK owns serving the canonical handler and verifying the credential, expiry, and signature before parsing or executing anything, then returning a canonical protocol response the server classifies. Verification is skipped only in an explicit development mode for a local server, and an HTTP worker with neither a signing key nor development mode refuses to start. Control-plane endpoint registration is deferred; the SDK neither discovers nor selects its own endpoint. The same registry and callbacks serve both modes, and selecting HTTP never requires rewriting workflow code.

For the local SDK milestone, WebSocket is the mandatory connected carrier and the gRPC prototype and four-language comparison below are deferred. The existing Rust PoC and the local gateway already implement hello, registration, one assignment, start authorization, and one result over WebSocket, and the execution protocol does not depend on the carrier’s API. The comparison below remains the record of what a later carrier change must re-evaluate; it is not a gate on this milestone.

A later release that returns to the carrier question re-runs the interoperability spike rather than changing the carrier from opinion. The spike implements hello, registration, one assignment, start authorization, and one result in Rust, Go, Elixir, and TypeScript using both leading candidates:

The decision uses implementation evidence for library maintenance, cancellation and backpressure behavior, reconnect control, common proxy compatibility, diagnostics, binary size, build tooling, and conformance-test ergonomics. One carrier is mandatory across all initial SDKs; Ryusen does not choose a different carrier per language. Supporting both initially is optional only if the shared adapter makes the incremental cost small.

The execution protocol does not depend on either carrier’s APIs. Gateway code adapts a framed bidirectional session to the shared dispatch interface, and transport conformance is reusable if the selected carrier changes later.

Other alternatives considered are:

Manifest publication and both execution modes authenticate before mutating configuration or registering capabilities. Who may call the server is decided by an injected authenticator (RFD 7), so a hosted deployment supplies its own without changing the protocol: connected workers present their identity during the handshake, and HTTP requests use signed or bearer authorization and must reject replay outside the request’s validity window.

Local development has no authentication: ryusen dev admits every caller of its API and gateway, and is protected by binding loopback and refusing cross-site and DNS-rebound browser requests instead. Binding a non-loopback listener requires the explicit --allow-remote option. Signing HTTP invocations so an endpoint can prove they came from the server is a separate concern from admitting callers: a deployment injects a signing key, and the local development server sends invocations unsigned to endpoints running in their explicit development mode.

Authentication establishes a principal. Organization or tenant membership may grant access to workspaces, but a workspace is the event and team-authorization boundary and an application is the deployment and execution boundary. Server- controlled credential state grants capabilities; selectors in a URL, manifest, registration, endpoint configuration, or execution envelope are never authority.

The initial capability model distinguishes:

An application credential may additionally hold workspace events:send, as in the local default, without receiving event-read or cross-application execution authority. Connected registration, HTTP endpoint registration, and every worker invocation require an exact credential-derived (workspace_id, application_id) execution scope. Function IDs, release IDs, manifest hashes, and compatible protocol metadata do not permit routing across that boundary.

Every data-plane and control-plane API authorizes the requested operation and scope before looking up the selected event, application, run, attempt, manifest, worker, or endpoint. Unauthorized requests therefore do not reveal whether a resource exists through status differences, response shape, timing-dependent metadata, SSE subscriptions, or idempotency replay. After authorization, normal not-found and conflict responses may be returned within that scope.

Transport conformance covers handshake order, capacity, duplicate assignments, start authorization, stale fencing, heartbeat loss, reconnect with in-flight work, repeatable discovery redispatch, effectful ambiguity, cancellation delivery, cancellation/result races, result redelivery, worker and gateway drain, HTTP invocation, proven pre-send failure, authenticated pre-start rejection, callback effect followed by HTTP 500, malformed post-execution responses, and the different ambiguity boundaries. Every initial SDK passes both connected and HTTP fixtures.

The initial implementation does not provide multi-region gateway discovery, cross-gateway session replication, transport-level workflow concurrency, durable result spooling inside SDK processes, automatic transport fallback, or a different mandatory carrier for each language.

Gateway, HTTP dispatch, SDK transport loops, and conformance are tracked in RFD 6 implementation checklist.