Workflow execution and compensation semantics

Define the user-visible contract for durable workflows before choosing protocol messages or storage layouts. Ryusen should preserve the simple step.run mental model while automatically undoing completed work when an optional compensator exists.

step.run is the only initial execution primitive. A step may optionally add a compensation callback; a separate saga API is not required.

The server durably schedules both forward and compensation attempts. Callback code remains in the user’s deployed function and is selected by the SDK during replay.

Ryusen may adopt mechanisms proven by other workflow systems only when they strengthen the step.run contract without making users manage the mechanism by default. Correctness is automatic, operational detail is observable, and configuration appears only when a developer needs control.

Internal distinctions such as discovery versus effect tasks, delivery queues, leases, history epochs, visibility projections, worker affinity, and transport sessions do not become required workflow APIs. The server chooses safe defaults and the development UI explains their behavior. SDKs expose idiomatic language interfaces rather than reproducing server terminology.

Advanced retry, timeout, heartbeat, and checkpoint controls may be added through progressive disclosure on step.run. A basic step does not require task-queue selection, deployment versions, patch markers, explicit history rollover, or manual heartbeat code.

An architectural pattern is rejected when its guarantee requires routine code version pinning, deterministic migration annotations, restoring old workers, or additional infrastructure for local development. These remain possible operational techniques outside the base programming model, not prerequisites for using Ryusen correctly.

A step has a stable identifier within a function. Once the server has accepted a successful result for that identifier, later replays return the recorded result and do not rerun its forward callback.

Physical invocation is at least once. A worker may perform an external effect and disappear before the server accepts its result. Ryusen must not describe this as exactly once. An attempt whose external outcome cannot be established is represented as Unknown; it is not silently treated as either success or failure.

Workflow discovery is repeatable orchestration, not Temporal-style deterministic replay. Code outside step.run callbacks may inspect clocks, randomness, environment, mutable state, or external reads, and later discovery may therefore emit different step IDs. Ryusen treats that like optimistic code evolution: previously accepted results remain durable and newly encountered IDs may execute. If a user needs one durable choice, that choice belongs in run input or a recorded step.run result.

External effects outside step.run callbacks remain outside Ryusen’s retry, ambiguity, and compensation guarantees. Forward and compensation callbacks are the tracked effect boundaries. The SDK makes this rule implicit in the ordinary step.run structure rather than requiring a separate saga API, but cannot enforce it against arbitrary user code.

Transient failures consume the step’s forward retry policy. A terminal uncaught failure evaluates recovery after authorized work has settled. Run cancellation and timeout first stop new forward scheduling and settle work that was already authorized. An empty unwind terminalizes without entering Compensating; a nonempty unwind begins compensation. Errors handled by the function do not trigger compensation. Successful runs do not execute compensators.

A newly created run is immediately Running with its first discovery work. Cancellation or timeout before that discovery receives start authorization fences it without invoking user code and terminalizes the empty unwind as NotRequired. Queue delay or unavailable workers are operational conditions, not a separate run phase.

Runs have no deadline by default. A developer may declare a run timeout, and a deployment may provide a visible inherited default. Hosted policy may impose a configurable maximum, but Ryusen rejects a declaration above that maximum rather than silently clamping it; the local server has no policy ceiling by default. The effective absolute deadline and its source are observable. Run timeout is separate from callback-attempt timeout, lease, and heartbeat policy, and it stops new forward execution without cutting short compensation recovery.

A callback failure is not exposed to workflow control flow while the server can retry it. When the retry policy is exhausted, the server durably records one terminal failure for the logical step and schedules discovery instead of immediately failing the run. On replay, the matching step.run produces that recorded failure through the language’s idiomatic error channel:

The wire protocol represents success and failure explicitly and does not require every SDK to emulate exceptions. A failure is handled when user code consumes or transforms it and continues or completes. It is uncaught only when it reaches the workflow entry adapter as that language’s function-level failure. That adapter response, rather than exhaustion by itself, atomically marks the run failed and begins compensation. A handled failure remains visible in step and attempt history even when the run later completes successfully.

A step ID names one logical operation for the life of a run. During each discovery pass, the SDK tracks every encountered ID. Encountering the same ID a second time returns a structured duplicate_step_id definition conflict before the second callback can execute. Ryusen blocks new forward effects instead of silently returning the first result at a second call or immediately failing and compensating the run.

The blocker identifies the step ID and, where the SDK can provide them, the first and duplicate source locations and display names. The first occurrence may already have a durable result; blocking never invalidates it. Mutually exclusive branches that do not encounter the ID twice in one pass are not rejected, but the shared ID still means they address the same logical operation and reuse its accepted result.

Rediscovery after a fix uses current code and normal memoization:

A definition blocker is recoverable and does not start compensation. A newly preferred connected release wakes blocked runs for its functions. Because HTTP and unversioned workers cannot reliably announce new code, a capped low-frequency probe and an explicit operator retry also rediscover current code. Repeatedly observing the same conflict updates its diagnostics without executing callbacks or creating repeated terminal failures.

Cancellation and run timeout record a durable stop intent; acknowledgement does not claim that an already-authorized effectful callback has stopped. Forward work that has not received durable start authorization is fenced without execution. Authorized repeatable discovery may be discarded. An authorized effectful attempt may already have changed an external system and must settle before reverse recovery begins.

The engine accepts a valid late success or structured failure from an effectful attempt that was authorized before the stop intent. It does not rediscover new forward work afterward. A success becomes a completed step and participates in reverse unwind. A reported application failure follows normal failed-attempt semantics; as with any callback failure, Ryusen cannot prove that application code produced no partial external effect.

Loss of contact with authorized effectful work becomes Unknown and blocks compensation. A valid late result may resolve it until an operator resolution commits. An operator may supply a success result or explicitly declare failure; declaring failure accepts the risk of proceeding without treating that step as completed. Compensation starts only after every effectful attempt authorized before cancellation or timeout has a durable resolved outcome.

Operator resolution always names the exact attempt and generation and is idempotent. Supplying success accepts a portable result as authoritative; declaring failure applies normal phase-specific failure semantics. Explicit retry is available only while forward execution or compensation remains active. It accepts duplicate-effect risk, fences the ambiguous physical attempt, and starts one new attempt with the same stable operation ID. Cancellation never permits a new forward retry.

Explicit retry settles generation G as RetryAuthorized; that settlement is not a successful or failed outcome for the logical target. The same transaction fences G and creates generation G+1. G+1 may then settle normally and commit the target outcome. Any result from G arriving after G+1 exists is retained as a stale diagnostic submission and cannot change the step, unwind, or run. Each generation settles at most once, while successive explicitly authorized generations preserve one operation ID so external idempotency can span them.

Every completed step enters the unwind plan. When its target executes, the SDK asks current code for that step’s compensator. If one exists, Ryusen executes it; if none exists, target_not_found records a durable skip. Compensation remains optional and the missing callback is a logical no-op rather than a workflow failure.

The initial engine visits completed steps sequentially in reverse completion order. A compensator receives the recorded forward result and its own stable operation identifier. It has an independent retry policy, timeout, attempt history, and result. It must tolerate duplicate physical invocation for the same reason as a forward callback.

Recovery outcomes follow this table:

NotRequired for an empty unwind means only that no successful completed step was available to reverse. It does not prove that failed application code created no partial external effect; the attempt history retains that uncertainty.

Loss of an authorized compensation result becomes Unknown; the remaining unwind stops because Ryusen cannot know whether the reverse effect happened or safely invoke it again. A valid late result or explicit operator resolution settles the attempt. A confirmed compensation failure follows its retry policy, while a confirmed success advances the unwind.

The original execution outcome remains separate from this recovery outcome. A run can therefore report "cancelled, fully compensated" or "timed out, partially compensated" without losing why compensation began.

Ryusen never pins a run to a deployment. Every discovery, retry, resume, and compensation selects a currently eligible implementation of the logical function. Connected execution prefers the newest compatible live release and converges as older workers drain; HTTP revision selection belongs to the configured endpoint. Release and build identifiers remain attempt metadata and never become run affinity.

The current compatible fleet is authoritative and execution proceeds optimistically:

The compensation-presence bit recorded during forward declaration is historical diagnostic data, not affinity to that code. Once a compensation success, failure, or skip commits, later deployments do not reopen that durable recovery transition.

Generation fencing prevents obsolete attempts from committing new workflow state. It cannot stop an already-running external effect, so live deployment changes can still produce an ambiguous outcome. During connected rolling overlap, two releases that advertise the same execution contract may both run work until the older release drains; Ryusen cannot infer arbitrary semantic incompatibility from user code.

This RFD does not promise distributed ACID rollback, isolation from intermediate effects, or exactly-once execution against arbitrary systems. It does not define parallel compensation, nested compensation scopes, custom unwind policies, managed integrations, or historical artifact retention.

Most users only learn step.run; adding compensate improves failure handling without changing the base abstraction. Optimistic code evolution lets a new deployment repair old runs, but stable step identifiers become an important application-level compatibility convention rather than an enforced pin.

The executable semantics and failure cases are tracked in RFD 1 implementation checklist.