Language-neutral execution protocol

Define the wire contract between the Ryusen server and independently developed language SDKs. The protocol must support the semantics in RFD 1 without making Rust types or one language’s runtime behavior part of the public contract.

The server owns run state, scheduling, retries, timers, flow control, compensation order, and history. An SDK owns idiomatic APIs, history-aware rediscovery, callback execution, and conversion between language values and protocol values. Callbacks are never serialized into server storage.

Execution targets declare whether work is repeatable orchestration or an effectful callback. Discovery may be invoked again after an ambiguous transport failure even though ambient state or current code can change its declarations; user code must place tracked external effects inside forward or compensation callbacks. The protocol classification controls lease recovery and is not inferred from transport behavior.

Versioned execution envelopes are independent of their transport. RFD 6 maps them onto a framed bidirectional connected session and request-response HTTP. The message schemas and conformance fixtures are canonical; SDK surface syntax, wire framing, and transport loops may differ by language.

Ryusen’s durable routing hierarchy is organization or tenant, workspace, then application. A workspace is the event-routing and team-authorization boundary: events are submitted to a workspace and may trigger functions in any application inside it. An application remains the deployment and execution boundary.

An application manifest is the complete desired set of logical function identifiers and trigger declarations for one application inside one workspace. Publishing a manifest creates an immutable, content-hashed application generation. Activation also creates an immutable workspace routing generation that maps every application in the workspace to its active application-manifest generation. The server fences and atomically activates both heads. The publication protocol is a durable control-plane operation separate from worker registration. Reconnecting, draining, or disconnecting a worker never adds or removes triggers.

Every manifest publication, connected registration, and HTTP endpoint registration is authorized under a credential-derived (workspace_id, application_id) scope. Workspace and application identifiers in a request select a scope but never grant access to it. Event submission and event reads require workspace capabilities; manifest deployment and execution registration require application capabilities; a workspace operator may inspect or control applications according to its role. The authenticated principal must be authorized before request-selected resources are looked up or their existence is disclosed. Stored manifest generations, release groups, endpoint capabilities, and function identities include workspace and application scope in their durable identity.

Manifest publication carries the application’s expected parent generation so concurrent or stale publishers for that app cannot silently replace one another. Identical content is idempotent. The server reads and compare-and-sets the workspace routing head internally. It retries a workspace-head conflict caused by an unrelated application deployment by rebuilding the routing generation over the newer workspace parent; publishers need not synchronize application deployments themselves. A same-application parent conflict remains visible to the publisher. Activating a complete application generation gives omission explicit meaning: a function or trigger absent from the new desired state is removed for events that acknowledge the new workspace routing generation. Old application and workspace routing generations remain available for audit and for fan-out work that already references them. Local development may publish the SDK’s complete manifest automatically; hosted publication requires an explicit deployment control-plane action.

An SDK publishes a complete manifest, never a delta. The payload carries the complete desired function declarations, trigger bindings, and the application’s opaque manifest contents. The server canonicalizes declarations by sorting function identifiers and trigger bindings, and rejects an empty or duplicate function identifier, an empty trigger identifier or event name, and a trigger that names an undeclared function. A function declaration carries an optional whole-run timeout and no SDK default. A generation’s identity hashes the scope, the expected parent, the contents, and the canonical declarations, so two publications with identical content but different parents are distinct generations. The contents bytes must be deterministic for the same registry state, and both SDKs use the same canonical encoding, so either SDK publishing an unchanged application produces the same bytes and does not create a new generation. Repeating an identical publication returns its original activation; a publication whose expected parent is not the active application generation fails without mutation and the conflict is visible to the publisher. RFD 5 owns the local startup sequence that reads the active generation, skips an already-current application, and registers with the generation the publication returned.

A workspace routing generation logically contains the workspace ID, generation ID, expected parent, canonical content hash, server commit order, and the map of application IDs to immutable application-manifest generations. It is a frozen routing/configuration snapshot selected when an event is acknowledged, not a worker build or release pin. Runs created from it continue to execute on current compatible worker code under the optimistic live-code rules.

A function declaration may include an optional whole-run timeout duration. Its absence means no deadline unless a visible deployment policy supplies an inherited default. Manifest activation rejects, rather than clamps, a duration above a configured policy maximum. Event fan-out resolves the effective absolute deadline and records whether it came from the function or policy; SDK clocks do not participate in deadline enforcement.

A connected worker registration advertises its authorized workspace and application scope, an optional opaque release ID, stable instance identity, the manifest content hash, logical function capabilities, execution contract, protocol capabilities, capacity, and SDK/runtime metadata. Every replica and reconnect from one rollout reuses its release ID within that scope. Reusing one scoped release ID with conflicting contract metadata is rejected.

The first successful registration of a release records a durable server commit position. That position, rather than a client timestamp or sortable release ID, orders releases and does not change when an old worker reconnects. The release record contains coordination metadata, not an executable artifact. When no stable release ID is available, an SDK uses an explicitly unversioned group derived from the execution contract and Ryusen makes no claim about relative code age within that group.

Connection and capacity records are ephemeral and affect dispatch availability only. Runs target a logical function, never a historical release. Connected routing first requires an exact credential-derived workspace and application match, then excludes workers that cannot satisfy the current contract and durable history requirements, prefers the newest compatible release, and balances across its healthy instances. Older compatible instances may serve as rollout fallback until they enter drain. Lack of a compatible authorized live worker leaves addressed work durable and queued. An explicitly identified release is preferred over a compatible unversioned group; when every candidate is unversioned, Ryusen balances capacity without recency claims.

The protocol does not provide deployment pinning. Rolling deployments may serve compatible releases concurrently, and every release must interpret durable history using stable function and step identifiers. Protocol, value-codec, and declared feature compatibility can be enforced mechanically; arbitrary semantic compatibility of user code cannot.

An event freezes W12 for routing, not R17 for execution. Worker lifecycle changes registration only; it never moves a manifest or routing head.

An invocation contains the run input, accepted step results, terminal step failures, and an execution target. In forward discovery mode, the SDK runs user code until it finds unresolved work or the function returns. Completed matching steps return their recorded values without executing their callbacks. A matching terminal step failure is reproduced at the step.run call through the SDK’s idiomatic error channel without executing the callback again.

Discovery has four authoritative responses: declare unresolved work, complete the function, report an uncaught function failure, or report definition_blocked. The last response describes a recoverable workflow-definition conflict such as an ID encountered twice in one pass. It includes a stable reason, step ID, and optional first and duplicate source metadata. Catching a reproduced step failure is entirely local language control flow; the SDK continues discovery until it reaches another unresolved step or the function completes. Rust and Go SDKs use result and error return values, Elixir uses tagged tuples, and TypeScript uses promise rejection. The protocol carries the same explicit success-or-failure outcome for every language and does not prescribe exceptions.

An unresolved step.run is declared before its callback is dispatched. The declaration includes its stable step identifier, display name, retry metadata, and whether the current code provides compensation. The server persists the declaration and creates durable execution work atomically. Recorded compensation presence explains the forward contract at that time but does not gate a later unwind target.

A declaration may instead be a durable timer: the same stable step identity with exactly one of a relative delay and an absolute deadline. The delay is an unsigned 64-bit millisecond count and the deadline a signed 64-bit instant; a document outside those domains does not decode in any SDK. The server anchors a relative delay to the instant it accepted the declaration and records the instant the wait may settle at; a deadline that already elapsed settles as soon as it can be dispatched, never before acceptance. The declaration is a step, so it is declared once: a repeated declaration keeps the recorded anchor. The server owns the clock, the scheduling, restart, and cancellation, and the accepted wake is an ordinary accepted step result, so a later invocation replays the wait instead of arming it again. A timer declaration carries no compensator, and a timer is never part of an unwind. The wire carries only the non-negative millisecond count: an SDK converts the author-facing delay — a human-readable duration such as 200ms, 1.5s, or 2h30m, or the language’s typed numeric form — into that count, rounding a positive value up so a wait never wakes early, keeping an exact zero as zero, and treating a negative value as due immediately. Malformed input and a magnitude past the portable range are refused by the SDK rather than converted, and sleep_until remains an absolute instant. A discovery pass stops at its first unresolved boundary, so a declaration made after that boundary — a repeated step identity included — cannot replace the answer the server already has.

A declaration may instead be a durable external event wait: the same stable step identity with the exact event name, the correlation the event must carry, and a required relative timeout. The correlation is a string-keyed, string-valued subset filter over the matched event’s caller-supplied correlation metadata: every declared pair must appear in the matched event with the same value, and the event may carry more pairs than the registration declared. The registration commits in the transaction that accepts the declaration, and that commit is the registration boundary: only an event the server accepts after it can ever match, so an event that arrived before the workflow waited never settles the wait. The server anchors the timeout to the instant it accepted the declaration and records the instant the wait may time out at, so a repeated declaration keeps the registration and the anchor. The server owns matching, the timeout, restart, and cancellation; the accepted match is the matched event’s payload as an ordinary accepted step result, and an elapsed timeout is an accepted step failure with the sdk.timeout category, so a later invocation replays either instead of registering again. A matched payload the durable record cannot carry is an accepted step failure with the sdk.invalid_result category instead, so one unrecordable match never strands the wait or blocks the event’s own fan-out; a registration whose own timeout outcome the run’s replay budget cannot carry is refused before it commits, so no wait is ever registered unfinishable. A wait declaration carries no compensator, and a wait is never part of an unwind. The wire carries only the non-negative millisecond timeout: an SDK converts the author-facing duration — a human-readable duration such as 200ms, 1.5s, or 2h30m, or the language’s typed numeric form — with the same rounding and refusal rules as a timer, and a negative or zero timeout is an immediate durable timeout the server anchors, schedules, and settles rather than a local bypass.

A declaration may instead be a durable child invocation: the same stable step identity with the invoked function of the same application, the exact input the child run receives, and an optional relative timeout. The child’s identity is derived from durable facts — the storage scope, the parent run, and the step — so a repeated declaration of the same step names the same child whatever delivered it, and a declaration that disagrees with the recorded one is refused rather than adopted. The server materializes the child as an ordinary run in the transaction that accepts the declaration and owns its scheduling, its deadline, its restart, and its cancellation; the child’s accepted terminal outcome is an ordinary accepted step result, so a later invocation replays it instead of invoking a second child. The child is bounded by the tightest of the invoked function’s declared run timeout, the invoke timeout the declaration carries, and the parent run’s own remaining deadline. The wire carries only the non-negative millisecond timeout: an SDK converts the author-facing duration — a human-readable duration such as 200ms, 1.5s, or 2h30m, or the language’s typed numeric form — with the same rounding and refusal rules as a timer, and a negative or zero timeout is an immediate durable timeout the server anchors, schedules, and settles rather than a local bypass, so no effect of the child may run. A child invocation declaration carries no compensator, and a settled invocation is never part of an unwind.

A declaration may instead be a durable event emission: the same stable step identity with the exact event name, the exact event payload bytes, and the routing metadata the accepted event carries. The payload is the portable bytes value: an SDK converts the author-facing payload with the same conversion the public event submission uses — a JSON document serializes as given and opaque bytes cross unchanged — and a payload that is not bytes is refused rather than reinterpreted, so a tagged protocol document can never be delivered to a consumer as an application payload. The routing metadata is the string-keyed, string-valued caller-supplied correlation metadata an event submission carries; it is empty when the workflow declares none, and it is never inherited from the run’s own triggering event, so a durable wait whose correlation is a subset of it can be settled by the emitted event. The member is optional in a document: an emission that declares no routing metadata carries an empty map, and the canonical encoding always spells the member. The emitted event’s identity is derived from durable facts — the storage scope, the run, and the step — so a repeated declaration of the same step names the same logical event whatever delivered it, and a declaration that disagrees with the recorded one is refused rather than accepted a second time. The server accepts the event through the same transactional event acceptance every submitted event uses — one event record, one idempotency claim, one workspace list entry, and one fan-out obligation — and settles the step with the acceptance receipt in the same transaction, so a lost acknowledgement, a rediscovery, a replay, or a restart can never accept a second event. The receipt is the event’s identity, and the workflow continues with it without waiting for the event’s fan-out, trigger runs, or matching waits: acceptance is not delivery, and nothing here claims a downstream consumer runs exactly once. An emission declaration carries no compensator, a retry policy that asks for more than one attempt is refused because the acceptance is not retried by its parent, and a settled emission is never part of an unwind.

Forward execution and compensation execution are explicit targeted modes. A target includes run ID, step ID, phase, attempt, stable operation ID, and fencing generation. In compensation mode, the SDK replays accepted history, invokes only the target step’s current compensator with its recorded forward result, and does not discover or schedule new forward work. Every completed step may receive such a target regardless of whether its historical declaration reported a compensator.

If current code no longer emits the target, the SDK returns target_not_found. For forward work the server makes the old target obsolete and rediscovers the current function. For compensation it records a skip and continues unwinding. Transport failure and application failure remain distinct from this response.

A targeted invocation may instead return rejected_before_start only when the SDK did not invoke the callback. The response names the request, attempt, and fencing generation and carries a structured reason such as unsupported target or temporary admission refusal. It is authoritative only when it is a well-formed response from the authenticated worker endpoint. The server may then apply the reason’s retry or obsolescence policy without treating the callback outcome as ambiguous. A generic HTTP status is not this protocol response.

RFD 5 owns the local developer lifecycle and configuration, and RFD 6 owns the transport lifecycle. This section fixes the language-neutral semantics every SDK implements for one invocation, so Rust and Elixir can present different syntax over the same protocol outcomes.

A registry declares logical functions by exact string identifier, the exact-name event triggers that select them, and an optional whole-run timeout per function. Identifiers are unique and non-empty, and a trigger names a declared function and a non-empty event name. The registry is complete desired state, not a subscription: the manifest the SDK publishes is exactly the current registry.

step.run is the base API and compensation is optional. An SDK exposes at minimum:

Step identifiers are explicit strings in the initial SDKs; a deterministic source-derived default identifier is deferred. Callbacks are synchronous in the Rust SDK for this milestone. An SDK may schedule callbacks on its own executor as long as one invocation runs at most one callback at a time.

One SDK invocation is exactly one discovery pass or one targeted forward or compensation execution. In discovery, the first unresolved step.run suspends the invocation, declares that step, and no later callback runs. Accepted replay entries return their recorded values without executing callbacks, and a replayed terminal step failure is reproduced at the same call through the SDK’s error channel. A step identifier encountered twice in one pass returns the structured definition blocker before the second callback executes: the blocker is the decision machine’s response even when the first encounter already suspended the pass, so a host that keeps declaring on the machine reaches the conflict. A pass that has stopped is different at the SDK layer: it answers no later declaration at all — the declaration never reaches the decision machine — so a workflow that catches the suspension and declares the same identifier again cannot replace the boundary the machine already recorded. In a targeted execution only the named step’s current callback runs; a target the current code no longer emits returns target_not_found instead of failing the run, and a compensator is invoked only when the replay history carries its step’s accepted forward result. The suspension used to stop a pass is an SDK-internal control signal: it must not escape step.run as an application error, and a callback failure must remain distinguishable from it.

Operation identity and attempt metadata are server-supplied and immutable. The operation ID identifies the logical forward or compensation operation across physical attempts; the attempt ordinal identifies one physical attempt. An SDK passes both to callbacks unchanged and never derives identity from wall-clock time, process state, or source location.

Application errors use one structured record: a stable category, a message, a retryability flag, and optional string-keyed details. Each SDK maps that record onto its idiomatic error channel and maps a caught callback failure back into the same record. Rust returns Result values from synchronous callbacks in this milestone; Elixir returns {:ok, value} and {:error, reason} tagged tuples. The protocol outcome is identical and only the language surface differs. A callback failure is never reported as an uncaught workflow failure, and only a failure escaping the function entry adapter is.

Callbacks are never serialized, and an SDK must not make any state durable outside the protocol: anything retained across invocations is a cache of durable history, never an authority.

Protocol values use a JSON-compatible tagged value model. The initial encoding must distinguish:

Maps with non-string keys and arbitrary language objects are not portable and are rejected unless an SDK converts them explicitly. Values have canonical size limits and hashing rules so retries across languages identify the same payload.

An SDK converts between idiomatic language values and the tagged model in both directions and must preserve the distinctions above: null versus an absent member, signed and unsigned integers outside JavaScript’s safe range, finite floats only, UTF-8 strings versus byte strings, arrays versus string-keyed objects, instants and durations, and blob references. A value the tagged model cannot carry — a map with non-string keys, an arbitrary language object, a closure, or a non-finite float — is refused with a conversion error unless the caller converts it explicitly; an SDK must not coerce it into a portable value. A byte string carries the exact bytes and is never transcoded.

An SDK may add an explicit JSON convenience for callers that work in JSON. It converts a JSON document to or from the byte-string variant only and never rewrites bytes the caller supplied. It cannot express instants, durations, blob references, or integer widths the JSON document does not preserve, and it must refuse a non-finite number literal rather than coerce it. Decoding bytes that are not a valid JSON document is an error, never a silent string. The convenience is opt-in: it is never applied to a tagged value the caller passed or to an accepted step result.

Errors are structured records containing a stable category, retryability, message, optional language-specific details, and an optional serialized cause. Stack traces are diagnostic and never control retry behavior.

An SDK must distinguish a callback-attempt failure from an uncaught workflow failure in its response. After retry exhaustion, the server supplies the recorded callback failure to a later discovery invocation. Only a failure that then escapes the workflow entry adapter is reported as an uncaught workflow failure and may transition the run into compensation.

Every request and response names a protocol version. Registration negotiates optional capabilities, while the initial required message set evolves only additively within a major version. Unknown fields are ignored; unknown required enum variants or capabilities fail explicitly.

Protocol conformance is tested by a language-neutral fake peer that drives registration, discovery, memoized replay, targeted forward execution, compensation, duplicate requests, missing targets, malformed values, and protocol negotiation. Discovery fixtures allow ambient branches to change step declarations, require duplicate IDs in one pass to return the same definition blocker in every language, and prove accepted callbacks remain memoized after a fix. Transport lifecycle conformance belongs to RFD 6.

The fake peer and its fixtures are language-neutral and are a prerequisite for SDK acceptance: every SDK passes the shared fixture matrix before its real-server acceptance workflow in RFD 5 runs. Fixtures change with the protocol, not per SDK.

The canonical documents live in crates/protocol/fixtures and are indexed by the shared conformance harness in crates/conformance, which also fixes the registry contract every SDK’s conformance suite registers; the harness README is the authoritative interface for a test runner. Machine-readable JSON Schema publication remains open: no schema generator is a project dependency in this milestone, so the fixtures and the harness’s machine-readable matrix are the published schema surface until that changes.

This RFD does not standardize SDK syntax, application deployment, framework adapters, worker connection lifecycle, CEL syntax, or storage schemas. It does not require deterministic control flow or rediscovery to use a specific coroutine or exception mechanism.

The first implementation settles the tagged representation as the versioned type/value encoding in the protocol crate and its fixtures, and the payload limits as the execution-request, replay, and nesting bounds recorded in RFD 6. Step identifiers are explicit strings in the initial SDKs; a deterministic source-derived default remains open for later SDKs.

Wire schemas, fixtures, and compatibility tests are tracked in RFD 2 implementation checklist.