Building stateful serverless applications with Knative and Restate

Knative revolutionized developing and operating serverless applications on Kubernetes, but it is still quite challenging to build stateful applications on top of it.

Let’s say you want to build an application that needs to persist some state. In order to do so, you might need to connect your service to a database, and when doing so, you’ll need to deal with retries, duplicate events, double writes, and all sorts of other distributed systems issues.

As another example, assume you want to build a service orchestrator that needs to invoke a set of services and if one of them fails, it needs to compensate the previous operations. Ideally, you just want to write some sequential code that executes one operation after another and performs a rollback if one of them fails. In practice, this won’t be so easy though because you need to take care of retries when invoking downstream services, possible failures of the orchestrator service itself, and arbitrarily long waiting times when invoking downstream services.

What if you could embed the app state and execute complex service coordination all within your Knative services, without having to deal with any of those issues?

Enter Restate #

Restate is an open source Durable Execution Engine to build stateful serverless applications. You write code that looks like usual RPC services and is executed durably. The engine stores the execution progress. After a crash, the engine restores the application to the previous state and resumes the execution from the point where it left off.

Another aspect of recording the execution progress is that in case of a long waiting time, e.g. due to a service being slow to respond, the engine automatically suspends the execution, to avoid wasting compute resources. In practice this means that during “waiting time”, the application can be scaled down to zero!

By using Restate and Knative together you can develop stateful entities, orchestrate microservices, implement saga patterns, and deduplicate events, while being able to scale-to-zero when no work is required. Restate will take care of the hard distributed systems problems such as state consistency, cross-service communication, failure recovery, and so on.

With Restate, you build applications using one of the available Restate SDKs, and then deploy your services as a serverless/stateless HTTP servers, for example as Knative services. Right now Restate supports Golang, Java, Kotlin, Typescript, Rust and Python. To invoke your services, you send requests to Restate rather than to your service directly, such that Restate acts like a “proxy” between your clients and your services.

Restate is a single binary and can be deployed as a stateful deployment on your Kubernetes cluster, similarly to how you would deploy a database, or you can use Restate Cloud if you want a managed service. For more info, check the deployment documentation.

Signup flow example #

To give you an idea of how it works, let’s have a look at a signup flow using Knative and Restate together. The example application is composed as follows:

• A user service, where we store the user information.
• A signup service, which implements the signup of a new user, sends a confirmation email, and then activates the user.

If you want to follow along locally, install the Restate CLI, and then clone the example locally:

restate example go-knative-go && cd go-knative-go

User service #

Let’s start with the user service.

To build it, we’ll create a Restate Virtual Object, that is an abstraction to encapsulate a set of RPC handlers with a K/V store associated with it. Virtual Objects are addressable by a key, which you provide when invoking one of their handlers. Moreover, Virtual Objects have an intrinsic lock per key, meaning Restate will make sure at most one request can run at the same time for a given key, and any additional request will be enqueued in a per-key queue.

Let’s start by defining the user Virtual Object:

// Struct to encapsulate the user virtual object logic
type userObject struct{}

This struct will be used as receiver for grouping the handlers together.

Let’s define our first handler to Initialize the user:

// User struct definition, (de)serializeable with json
type User struct {
    Name     string `json:"name"`
    Surname  string `json:"surname"`
    Password string `json:"password"`
}

// Initialize will initialize the user object
func (t *userObject) Initialize(ctx restate.ObjectContext, user User) error {
	// Check if the user doesn't exist first
	usr, err := restate.Get[*User](ctx, "user")
	if err != nil {
		return err
	}
	if usr != nil {
		return restate.TerminalError(fmt.Errorf("the user was already initialized"))
	}

	// Store the user
	restate.Set(ctx, "user", user)

	// Store the unactivated status
	restate.Set(ctx, "activated", false)
	
	return nil
}

Each Restate handler is called with a Context, an interface encapsulating the various features Restate exposes to developers. This context is different depending on the type of handler. In case of virtual object handlers we use restate.ObjectContext.

In this handler we use restate.Get and restate.Set to read and write from Restate’s Virtual Object K/V store. We first check if a user was already initialized, and then we write the new user and activation boolean to Restate’s K/V store.

Then, we implement the handler to Activate a user after it has been initialized:

// Activate will signal the user is activated
func (t *userObject) Activate(ctx restate.ObjectContext) error {
	// Check if the user exists first
	usr, err := restate.Get[*User](ctx, "user")
	if err != nil {
		return err
	}
	if usr == nil {
		return restate.TerminalError(fmt.Errorf("the user doesn't exist"))
	}

	// Store the activated status
	restate.Set(ctx, "activated", true)

	return nil
}

Lastly, we define the handler to get the user data:

func (t *userObject) Get(ctx restate.ObjectSharedContext) (User, error) {
	return restate.Get[User](ctx, "user")
}

This handler uses restate.ObjectSharedContext to read the K/V state without locking the virtual object instance.

We’re now ready to implement the signup service.

Signup service #

The signup service has a single handler that orchestrates the signup:

func (t *signupService) Signup(ctx restate.Context, newUser NewUser) (string, error) {
	// Initialize the newUser first
	user := User{
		Name:     newUser.Name,
		Surname:  newUser.Surname,
		Password: newUser.Password,
	}
	_, err := restate.Object[restate.Void](ctx, "User", newUser.Username, "Initialize").Request(user)
	if err != nil {
		return "", err
	}

	// Prepare an awakeable to await the email activation
	awakeable := restate.Awakeable[restate.Void](ctx)

	// Send the activation email
	_, err = restate.Run[restate.Void](ctx, func(ctx restate.RunContext) (restate.Void, error) {
		return restate.Void{}, sendEmail(newUser.Username, awakeable.Id())
	})
	if err != nil {
		return "", err
	}

	// Await the activation
	_, err = awakeable.Result()
	if err != nil {
		return "", err
	}

	// Activate the user
	_, err = restate.Object[restate.Void](ctx, "User", newUser.Username, "Activate").Request(user)
	if err != nil {
		return "", err
	}

	return fmt.Sprintf("The new user %s is signed up and activated", newUser.Username), nil
}

Using restate.Call, we can invoke other Restate services. These requests are guaranteed to be executed exactly once.

With restate.Awakeable we can wait for an event to happen. You can complete requests simply sending HTTP requests to Restate providing the Awakeable id. In our example, the email will embed a link containing the Awakeable id, which will be completed once the user clicks on the verification button.

With restate.Run we can execute any arbitrary piece of code and memoize the result, such that in case of a crash, Restate won’t re-execute that chunk of code, but will load the stored result and use it for the subsequent operations.

Start the HTTP service and deploy it with Knative #

Expose the services using HTTP:

func main() {
	// Read PORT env injected by Knative Serving
	port := os.Getenv("PORT")
	if port == "" {
		port = "9080"
	}
	bindAddress := fmt.Sprintf(":%s", port)

	// Bind services to the Restate HTTP/2 server
	srv := server.NewRestate().
		Bind(restate.Reflect(&userObject{})).
		Bind(restate.Reflect(&signupService{}))

	// Start HTTP/2 server
	if err := srv.Start(context.Background(), bindAddress); err != nil {
		slog.Error("application exited unexpectedly", "err", err.Error())
		os.Exit(1)
	}
}

Follow the steps in the example readme to run the example locally with kind.

You can build the container image using your tools, e.g. with ko:

ko build main.go -B

And deploy it with kn:

kn service create signup \
  --image $MY_IMAGE_REGISTRY/main.go \
  --port h2c:8080

Before sending requests, you need to tell Restate about your new service deployment:

restate deployments register http://signup.default.svc

And this is it! You’re now ready to send requests:

curl http://localhost:8080/Signup/Signup --json '{"username": "slinkydeveloper", "name": "Francesco", "surname": "Guardiani", "password": "Pizza-without-pineapple"}'

Please note: some parts of the code example are omitted for brevity, check the full example for more details and how to run this locally with kind.

We got your back #

Let’s assume for a second that the sendEmail function in the Signup flow fails the first time we try the signup, what would it happen?

Without Restate, you would need to retry executing sendEmail a couple of times in a loop. But what if, while retrying to execute sendEmail, the signup service crashes or goes away? In that case, you’ll lose track of the signup progress, and next time the user presses F5, you’ll need some logic to reconstruct the state of the previous signup and/or discard it.

With Restate, if sendEmail fails, it will be automatically retried, and all the operations that have been executed previously, in this case the call to the User/Initialize handler, won’t be executed again, but their result values will simply be restored.

This is possible thanks to Restate’s Durable Execution Engine, that records the progress of your application, and in case of a crash restarts it from the point where it was interrupted.

On top of that, Restate is able to suspend the execution when no progress can be made, e.g. in case of a long sleep, or when waiting for a response from another service, all of that without splitting your business logic in a sequence of different handlers. While waiting your Knative service can scale down to zero!

What’s next #

In this post we’ve looked at how to build a stateful entity and a simple orchestration flow using Restate and deploy it on Knative.

By combining Restate and Knative together you get the best of both worlds, as you can build serverless application with the ease of developing stateful applications.

With Restate and Knative together you can build much more: workflows, sagas, stateful event processing (combining Knative Eventing too!) just to name few ideas. Check out the Restate examples to get a grasp of what’s possible to build with it.