Durable Coroutines for TypeScript: Fault-tolerant Agents without the Overhead

We added durable coroutines to Restate's TypeScript SDK: persistent, concurrent, multi-step routines that run inside a long-lived durable function. They provide the same persistence and recovery semantics as workflows, but with much lower overhead and a durable process-like abstraction. The implementation makes heavy use of TypeScript's generator functions to implement a deterministic scheduler controlling the execution.

Durable Coroutines grew out of our users' requirements when building durable agents. Modern agent harnesses persist and track their progress in fine-grained steps: chunks of LLM results, multiple steps per tool call (intent, guardrail, result chunks). That granularity gives agents a stronger model when recovering from failures. But this is hard to achieve with a durable execution model that schedules and dispatches activities to workers for each persistent step. We built durable coroutines to fill that gap.

Background: Workflows vs. Durable Functions

Durable execution originally came in the shape of workflows and activities: if you want a persistent step, you need to create an activity, which gets scheduled on some worker. In systems like Temporal, that means the workflow task cedes execution (returns to the workflow orchestrator), the activity is sent to a worker to execute and return a result, and the workflow task gets executed again¹. Parallel work means setting up multiple activities, and when the workflow task cedes execution, all activities start running. This approach fits a task-queue & worker architecture.

1export async function agent(prompt: string) {2  const context: Message[] = [userMessage(prompt)];3 4  while (true) {5    const next = await callModel(context);6    if (next.done) return next.answer;7 8    const calls = next.tools.map((t) =>9      callTool(t));10    const results = await Promise.all(calls);11    context.push(...results);12  }13}

1export function* agent(prompt: string) {2  const msgs: Message[] = [userMessage(prompt)];3 4  while (true) {5    const next = yield* run(() => callModel(msgs));6    if (next.done) return next.answer;7 8    const calls = next.tools.map((t) =>9      spawn(callTool(t)));10    const results = yield* all(calls);11    msgs.push(...results);12  }13}

An alternative approach that Restate and other systems implemented is often described as Durable Functions. The functions can have durable steps (ctx.run(...), plus durable signals, sleeps, promises, ...) and compose via RPC (calling other durable functions). Durable steps are faster than activities (or calling other durable functions) because they need to do less work: Restate maintains a streaming connection to the executing functions, sending steps and acks between the application and Restate Server. Restate's entire architecture is designed to make step commits fast.

Durable functions give developers the choice between two models: putting a unit of work (that should be persisted) into separate functions or activities is useful for heavier tasks, or when the work needs to be queued and dispatched to different workers/deployments and run with specific rate or concurrency limits.

In contrast, programs written with durable steps & coroutines feel more like writing durable processes. Except in actual failure/recovery cases, the whole execution is one continuous run in the same process, letting the code reuse local resources like cached files/results or authenticated connections. Restate has always supported durable steps, but the addition of Durable Coroutines fixes the limited support for concurrency in that API.

Enter Durable Coroutines

Restate's Durable Coroutines library takes the idea of coroutines from programming languages like Go, Rust, or Kotlin and implements a deterministic scheduler to make them compatible with journal-based durable execution.

There are only a few important concepts:

• Everything is a generator function because that allows Restate to tightly control the execution (see How it works). If you've never worked with generator functions, you can squint your eyes and pretend that yield* fulfills a similar purpose here as await does for Promises.
• run(...) is a single durable step, like in Restate's standard API.
• spawn(...) creates a new concurrently executed coroutine (Task), which can have its own sequence of durable actions and can spawn further Tasks.
• select(), all(), and other combinators combine steps and tasks in the same way as promise/future combinators do.

Let's dive into an example: an agent loop that spawns tools as concurrent sub-workflows and listens for steering signals.

function* agent(userMessage: string): restate.Operation<string> {
  const sandbox = yield* connect(); // re-usable connection across tool calls
  const messages: Message[] = [{role: "user", content: userMessage}];

  const tasks = new Map<string, restate.Task<ToolResult>>(); // tasks in-flight
  let steer = restate.signal<string>("steer"); // to wait for a note from the user

  // Durable agent loop
  while (true) {
    // 1. Durable LLM call
    const action = yield* restate.run(async () => llm(messages));
    if (action.type === "final") {
      return action.message;
    }
    messages.push({role: "assistant", calls: action.calls});

    // 2. Spawn parallel tool calls (each of them possibly a multi-step sequence)
    for (const call of action.calls) {
      tasks.set(call.id, restate.spawn(performCall(call, sandbox)));
    }
    messages.push({
      role: "tool",
      results: action.calls.map(({id}) => ({id, result: "task created"})),
    });

    // 3. Race tool calls with new steering input
    const next = yield* restate.select({...Object.fromEntries(tasks), steer});
    if (next.tag === "steer") {
      // process steering note
      messages.push({role: "user", content: yield* steer});
      steer = restate.signal<string>("steer"); // renew the signal
    } else {
      // process completed tool call
      const result = yield* tasks.get(next.tag)!;
      tasks.delete(next.tag);
      messages.push({role: "tool", results: [result]});
    }
  }
}

To see how far you can go with Durable Coroutines, have a look at Restate's agent reference architecture, which uses them to compose steering, interruption, sync/async tools, programmatic tool calling, and more.

How it works

The library uses TypeScript generators instead of Promises (async/await) to give the SDK control over coroutine execution. With async/await, the event loop decides how the execution of different concurrent code paths interleaves. Generators instead pause at each yield until the caller explicitly resumes them. This lets the SDK decide which coroutine continues next and what result it receives. By recording these decisions in the journal, the SDK can replay execution deterministically.

Take a tool with two durable steps: a guardrail and the actual tool execution. For each step, yield* pauses the coroutine until the result is there:

function* runTool(tool, params) {
  const allowed = yield* run(() => guardrail(tool, params)); // true/false
  if (!allowed) {
      return "Tool call rejected by guardrail";
  }
  return yield* run(() => callTool(tool, params));
}

// SDK drives execution via:
const task = runTool(tool, params); // runs nothing yet
task.next(); // starts guardrail step and pauses
task.next(true); // guardrail passed: starts tool call step and pauses
task.next(toolResult); // returns

run() doesn't return its result directly. Like all actions (a call to another service, a timer, or a signal), it returns a Future. yield* hands that Future to the SDK (the scheduler) and pauses the coroutine. The scheduler waits for the Future to complete, then calls next() with the result, which becomes the value of the yield* expression.

Concurrency

The code may start additional concurrent coroutines (Tasks) via spawn(...). The scheduler advances all coroutines (repeatedly calls next(...) across yield points) until each coroutine has yielded a future to a Restate action (e.g., a run(...) step, a signal, RPC, ...), meaning everyone is currently waiting.

This is the critical point where we need to ensure that we record the order in which these actions complete, so we can deterministically replay them on recovery. The scheduler does that by running RestatePromise.race(pendingActions), which automatically records the winning completion in the durable execution journal. It then resumes the waiting coroutine and starts advancing coroutines again. Exactly one coroutine executes at a time, but the I/O in their durable steps overlaps.

To illustrate this, here is the execution of an agent running two concurrent tools, each consisting of a guardrail step and a tool call step. Note how the SDK runs race(...) each time all coroutines are waiting on a durable step (run(...)).

function* agent(prompt) { const plan = yield* run(() => llm(prompt)); const webSearch = spawn(searchTool(plan.query)); const weather = spawn(weatherTool(plan.city)); const [a, b] = yield* all([webSearch, weather]); return summarize(a, b);}

Inside the scheduler

The scheduler wraps every coroutine in a fiber: the generator plus a state that is ready, waiting, or done. Ready fibers sit in a FIFO queue. The scheduler advanced a fiber by calling next() on its generator, which runs the coroutine's code synchronously up to its next yield*. Along the way, the code can start new work without waiting for it:

• run() creates the durable step and returns its Future right away without awaiting it
• spawn() creates a fiber and appends it to the ready queue. The new coroutine starts only once the current one stops running.

Let's zoom in on how the scheduler moves each coroutine forward until it waits on a Restate action. Here is its core, simplified from scheduler.ts and fiber.ts:

const fibers = new Set(); // all live coroutines
const ready = []; // fibers that can run

async function execute(handler) {
  const main = spawn(handler);
  drain();
  while (!main.isDone()) {
    // the only place the durable execution journal is used:
    // the first pending action that completes gets added
    const { fiber, result } = await RestatePromise.race(pendingActions());
    wake(fiber, result);
    drain();
  }
  return main.result;
}

// The Restate actions that fibers wait on
function pendingActions() {
  return [...fibers]
    .filter((fiber) => fiber.waitingOn?.isRestateAction)
    .map((fiber) =>
      fiber.waitingOn.promise.map((result) => ({ fiber, result })),
    );
}

// Advance fibers one at a time until all of them are waiting
function drain() {
  while (ready.length > 0) advance(ready.shift());
}

// Run one coroutine until it waits on something unfinished
function advance(fiber) {
  let input = fiber.input; // result of what it last waited on
  while (true) {
    const step = fiber.generator.next(input); // runs user code
    if (step.done) {
      return finish(fiber, step.value);
    }
    const future = step.value; // the Future passed to yield*
    if (future.isRestateAction) { // durable action: set up for next RestatePromise.race()
      fiber.waitingOn = future;
      return;
    }
    if (future.isDone()) {
      input = future.result; // in-memory and completed: fall through loop
      continue;
    }
    fiber.waitingOn = future; // in-memory and pending, put aside
    future.onDone((result) => wake(fiber, result));
    return;
  }
}

function wake(fiber, result) {
  fiber.waitingOn = null;
  fiber.input = result;
  ready.push(fiber);
}

function finish(fiber, result) {
  fibers.delete(fiber);
  fiber.task.complete(result); // calls the onDone callbacks
}

function spawn(coroutine) {
  const fiber = new Fiber(coroutine);
  fibers.add(fiber);
  ready.push(fiber); // starts once the current coroutine stops
  return fiber.task;
}

function run(action) {
  // Creates the durable step, but journals only once the scheduler awaits
  // the race. Directly supplies result if action is already complete (replay).
  return restateActionFuture(ctx.run(action));
}

At a yield*, the yielded value tells the scheduler which Future the coroutine is waiting for. Because yield* delegates through nested generator calls, a run() three generator functions deep still surfaces at the fiber's next(). Every Future is backed either by a Restate action or by an in-memory result: a Task, or a channel that passes a value from one coroutine to another. That backing is the one thing the scheduler checks, which leaves four cases:

• A Restate action (a step, timer, call, or signal): the fiber records it as the action it waits on, and the scheduler moves on to the next ready fiber. Only these actions go into the race.
• An in-memory Future that is already done (a finished Task, or a channel that holds its value): the scheduler calls next() again right away with the result. The coroutine keeps running without touching the journal.
• An in-memory Future that is still pending: the fiber registers to be woken when it completes, then waits. It adds nothing to the race.
• The end of the function: the fiber is done, and completing its Task moves every fiber waiting on it back into the ready queue.

Cancellation

The use of generators also gives the SDK a natural way to cancel coroutines. When a cancellation signal arrives, the SDK calls throw(TerminalError) on each coroutine instead of next(). The TerminalError is thrown at the yield* in the coroutine, and gives it a chance to run any cleanup logic in the catch or finally blocks. In-flight run closures get an aborted AbortSignal.

To cancel a coroutine spawned by you, you can call task.interrupt(error) on the Task returned from spawn. This throws a terminal error at its next yield* that cascades to the coroutines it spawned.

Start Building

Durable Coroutines are already running in production in various companies. For example, Replit Agent is built on durable coroutines. If you like this, give it a spin and let us know what you think.

Restate is open, free, and available on GitHub. To learn more about Durable Coroutines, install the dependency: npm install @restatedev/restate-sdk @restatedev/restate-sdk-gen or get started with the template npx @restatedev/restate example typescript-hello-world-gen, and check the README for a quick tour.

If you like what we are doing, give us a star on GitHub, and join the community on Discord or Slack.