Replit Agent switched to Restate for faster durable execution, cost efficiency, and BYOC

Replit is a leading AI coding platform with more than 50 million users. Replit Agent lets them build entire applications from a plain-English prompt. Behind that simple interface is a long-running agent that can work for hours, span many turns, and perform thousands of steps. Users can give feedback, approve actions, and steer or stop work as it happens. Supporting that experience at Replit’s scale requires fast, efficient orchestration that recovers from failures without losing progress.

Replit initially used Temporal for this, but switched to Restate for three main reasons:

• Lower latency and cost efficiency: Replit Agent executes thousands of fast, inline durable steps per turn, enabling a simpler, more resilient architecture.
• Unique building blocks for stateful, concurrent agents: Virtual Objects model stateful agents and sessions, while Durable Coroutines run concurrent tasks efficiently within one process, simplifying coordination and making the architecture easier to extend.
• Restate BYOC: Fully managed deployments in Replit's own cloud account keep sensitive data private and support both its multi-tenant cloud stack and dedicated enterprise offerings.

The results:

• Agent executions with thousands of inline durable steps, each adding only a few milliseconds of latency overhead.
• Tens of Restate cells across regions and availability zones in Replit's Kubernetes environment.
• Weeks after migration, a major promotional traffic spike reached close to 25,000 durable actions per second per cell.

Why Replit moved to Restate

1. Fast, efficient durable execution inside the agent loop

Restate's low overhead per durable step makes it practical for Replit to use durable execution within the agent loop, rather than around it. Replit Agent can run executions with thousands of durable steps, recording progress throughout model calls, guardrails, tool operations, and state changes. Fine-grained durable steps mean less work needs to be repeated after failures. Restate makes this practical by executing durable steps inline, avoiding the overhead of scheduling and dispatching each step as a separate activity.

Keeping execution within the same process also simplifies resource sharing. Sandbox connections and channels for user updates can stay open throughout a conversation turn, reducing repeated connection setup and coordination between separately scheduled activities.

2. Unique building blocks for stateful, concurrent agents

Replit Agent retains conversations, runs tasks concurrently, and responds to steering, approvals, and cancellations while working. In a workflow-centric model, this pushes the architecture toward an indefinitely running workflow, with each new turn or steering instruction delivered as a signal. Subagents are subworkflows, and cancellation and steering must propagate across them, further increasing complexity.

Restate offered building blocks that fit these needs more directly. Virtual Objects make agents and sessions stateful, long-lived entities. Durable Coroutines let concurrent tasks run efficiently within one process, while durable communication primitives support invocation, steering, cancellation, and approvals across failures and restarts. These building blocks simplify coordination and the overall architecture, making it easier for Replit to extend the platform in the future.

3. BYOC for enterprise deployments and data privacy

Coding agents handle source code and other sensitive customer data. Replit wanted that data to remain inside its infrastructure without taking on the operational burden of self-managing a durable execution system.

With Restate bring-your-own-cloud (BYOC), Replit gets managed Restate in its own cloud account. Replit now has tens of Restate cells across regions and availability zones, and supporting both its multi-tenant cloud stack and dedicated enterprise offerings

How Replit builds on Restate

Replit models agents and sessions as Restate Virtual Objects: durable, addressable entities with state that can be invoked repeatedly and automatically manage concurrency. The UI invokes a chat Virtual Object to start, steer or stop agent execution via Restate's durable communication primitives. Restate records communication alongside execution, so interactions survive failures and process restarts.

For example, when a user cancels a task, Restate's durable cancellation signal propagates automatically through all nested subagents, stopping their work while allowing cleanup tasks such as returning resources.

A Replit-specific agent harness

The platform team built a Replit-specific abstraction over Restate so agent engineers don't each have to manage determinism, retry choices, and low-level cancellation behavior. This harness coordinates model inference, conventional and programmatic tool execution, sandbox commands, conversation state and compaction, parallel subagents, steering, cancellation, guardrails, and cleanup.

Restate supplies the durable state, communication, execution, and recovery primitive. Replit's platform team defines how its agent behaves, while its AI engineers work through an interface designed specifically for Replit Agent.

Durable coroutines and inline execution

Replit and Restate worked together on a TypeScript SDK extension called Durable Coroutines. It lets concurrent branches run sequences of durable steps within the same process. This lets Replit express the agent as a stateful, concurrent program that starts operations in parallel and responds as they complete, rather than expressing all concurrency as a fan-out of independently scheduled activities.

Each durable step sends an event over a bidirectional HTTP2 stream to the Restate runtime, which appends it to its distributed log, replicates it to a minimum number of followers, and acknowledges it. This takes a few milliseconds, avoiding the latency of scheduling each action as remote work and waiting for a worker to pick it up.

When the agent needs to wait for human approval, execution can suspend, and it resumes when approval arrives (potentially months later).

Versioned deployments

Replit uses tunnel deployments managed through the Restate Kubernetes Operator. A tunnel is an outbound connection from the Kubernetes environment to Restate Cloud, allowing Cloud to invoke services without exposing them to the public internet. The operator tracks service revisions, drains previous revisions, and integrates with Kubernetes autoscaling, supporting the rollout of new agent versions alongside ongoing work.

Serving millions of users in production

Replit developed and tested the new Restate-based agent over approximately six weeks before launching it internally. It then moved a small percentage of users to the Restate-based agent and increased traffic gradually over roughly two to three weeks.