# Step-by-Step Guide: Setting Up WASI Node Pools for AKS and Running WASM Spin Applications all with Pulumi

## TL;DR: Code

%[https://github.com/dirien/pulumi-aks-wasm-spin] 

## Introduction

In this blog post, I am going to show you how to run a [Fermyon](https://www.fermyon.com/) [Spin](https://www.fermyon.com/spin) WebAssembly (WASM) application on a WebAssembly System Interface (WASI) node pool.

**Everything will be powered by using** [**Pulumi**](https://www.pulumi.com/)**.**

Microsoft has revealed their decision to transition from krustlet and adopt [containerd shims](https://github.com/deislabs/containerd-wasm-shims) for operating WASM workloads in their WASI node pools within AKS.

The objective of [containerd shims](https://github.com/deislabs/containerd-wasm-shims) is to offer implementations capable of executing WASM/WASI workloads utilizing the [runwasi](https://github.com/deislabs/runwasi) library. By installing these shims on Kubernetes nodes, a runtime class can be added to Kubernetes, enabling the scheduling of Wasm workloads on the respective nodes. This allows your Wasm pods and deployments to function similarly to container workloads.

[Runwasi](https://github.com/deislabs/runwasi) is a project focused on operating wasm workloads on Wasmtime, a rapid and secure WebAssembly runtime managed by containerd.

We will focus this article on the Spin shim which is powered by the [Fermyon Spin](https://github.com/fermyon/spin) engine.

## What is WASM?

WebAssembly (WASM) is a binary format that is designed for maximum execution speed and portability using a WASM runtime. The WASM runtime is designed to run on a target architecture and execute WebAssemblies in a sandboxed environment to ensure security at near-native speed.

The WebAssembly System Interface (WASI) establishes a standardized connection between the WASM runtime and the host system, granting access to system resources like the file system or network.

## What is Fermyon Spin?

Fermyon [Spin](https://www.fermyon.com/spin) is a framework for building cloud-native applications with WebAssembly components. It is created by Fermyon and is fully open source. You can find the source code on [GitHub](https://github.com/fermyon/spin).

## Prerequisites

* [Pulumi](https://www.pulumi.com/docs/get-started/install/)
    
* [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)
    
* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
    
* [Rust installed](https://www.rust-lang.org/tools/install)
    
* [Go installed](https://golang.org/doc/install)
    
* IDE of your choice (VS Code, IntelliJ, etc.)
    

## Activate Preview Features for AKS

To install the `aks-preview` extension for Azure CLI, run the following command:

```bash
az extension add --name aks-preview
```

or update your existing `aks-preview` extension to the latest version:

```bash
az extension update --name aks-preview
```

## Register the `WasmNodePoolPreview` Feature

You may need to register the `WasmNodePoolPreview` feature for your subscription by simply running the following command:

```bash
az feature register --namespace "Microsoft.ContainerService" --name "WasmNodePoolPreview"
```

This will take a few minutes to complete. You can check the status of the feature registration by running the following command:

```bash
az feature show --namespace "Microsoft.ContainerService" --name "WasmNodePoolPreview"
```

Once the `state` property of the feature is `Registered`, you can create a WASI node pool for your AKS cluster.

## Set Up Your AKS Cluster

Let's start a new Pulumi project with the following command, using the `pulumi-azure-native` provider and `Go` as the language of choice:

```bash
mkdir pulumi-aks-wasm-spin && cd pulumi-aks-wasm-spin
pulumi new azure-go --force
```

You will need to provide some details about your project. You can use the default values for all questions for this demo.

We can now generate our AKS cluster using the subsequent code:

```go
package main

import (
	"encoding/base64"

	containerservice "github.com/pulumi/pulumi-azure-native-sdk/containerservice/v20230101"
	resources "github.com/pulumi/pulumi-azure-native-sdk/resources/v20220901"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		// Create an Azure Resource Group
		resourceGroup, err := resources.NewResourceGroup(ctx, "wasm-aks-rg", &resources.ResourceGroupArgs{
			ResourceGroupName: pulumi.String("wasm-aks-rg"),
		})
		if err != nil {
			return err
		}

		wasmCluster, err := containerservice.NewManagedCluster(ctx, "wasm-aks-cluster", &containerservice.ManagedClusterArgs{
			ResourceGroupName: resourceGroup.Name,
			KubernetesVersion: pulumi.String("1.25.5"),
			ResourceName:      pulumi.String("wasm-aks-cluster"),
			Identity: &containerservice.ManagedClusterIdentityArgs{
				Type: containerservice.ResourceIdentityTypeSystemAssigned,
			},
			DnsPrefix: pulumi.String("wasm-aks-cluster"),
			AgentPoolProfiles: containerservice.ManagedClusterAgentPoolProfileArray{
				&containerservice.ManagedClusterAgentPoolProfileArgs{
					Name:         pulumi.String("agentpool"),
					Mode:         pulumi.String("System"),
					OsDiskSizeGB: pulumi.Int(30),
					OsType:       pulumi.String("Linux"),
					Count:        pulumi.Int(1),
					VmSize:       pulumi.String("Standard_B4ms"),
				},
			},
		})
		if err != nil {
			return err
		}

		wasmPool, err := containerservice.NewAgentPool(ctx, "wasm-aks-agentpool", &containerservice.AgentPoolArgs{
			AgentPoolName:     pulumi.String("wasmpool"),
			ResourceGroupName: resourceGroup.Name,
			ResourceName:      wasmCluster.Name,
			WorkloadRuntime:   pulumi.String("WasmWasi"),
			Count:             pulumi.Int(1),
			VmSize:            pulumi.String("Standard_B4ms"),
			OsType:            pulumi.String("Linux"),
		})
		if err != nil {
			return err
		}

		kubeconfig := pulumi.All(wasmCluster.Name, resourceGroup.Name).ApplyT(func(args []interface{}) (*string, error) {
			clusterName := args[0].(string)
			resourceGroupName := args[1].(string)
			creds, err := containerservice.ListManagedClusterUserCredentials(ctx, &containerservice.ListManagedClusterUserCredentialsArgs{
				ResourceGroupName: resourceGroupName,
				ResourceName:      clusterName,
			})
			if err != nil {
				return nil, err
			}
			decoded, err := base64.StdEncoding.DecodeString(creds.Kubeconfigs[0].Value)
			if err != nil {
				return nil, err
			}
			return pulumi.StringRef(string(decoded)), nil
		}).(pulumi.StringPtrOutput)

		ctx.Export("resourceGroupName", resourceGroup.Name)
		ctx.Export("wasmClusterName", wasmCluster.Name)
		ctx.Export("wasmAgentPoolName", wasmPool.Name)
		ctx.Export("kubeconfig", pulumi.ToSecret(kubeconfig))

		return nil
	})
}
```

The code above will create a basic AKS cluster with a single node pool.

Then we will add a WASI node pool using the `containerservice.NewAgentPool` function. It is very important that the `WorkloadRuntime` property is set to `WasmWasi` to enable the `containerd-wasm-shims` on the node pool. Also, the `OsType` property must be set to `Linux` and the `WASM/WASI` node pool can't be used as a system node pool.

The last part of the code above is exporting the `kubeconfig` of the cluster. This is needed to connect to the cluster if you want to use the `kubectl` CLI or `k9s` to inspect the cluster.

The `ctx.Export` statements are exporting for example the `kubeconfig` as a secret and some other information about the infrastructure. You can run the following Pulumi command to see all the exported values.

```bash
pulumi stack output
```

Before I continue with the next steps regarding the infrastructure, I think it is a good time to switch to `Spin` to create the workload for our WASI node pool.

## Developing a WASM Application Using Fermyon Spin

### Installation of the Spin CLI

Before we can start building our application, we need to install the Spin CLI. To do so, we can run the following command:

```bash
curl -fsSL https://developer.fermyon.com/downloads/install.sh | bash
sudo mv spin /usr/local/bin
```

There are more installation options available on the [install Spin](https://developer.fermyon.com/spin/install) documentation page depending on your operating system. I am using macOS, so I will use the above command.

### Start a New Spin Project

Now with the Spin CLI installed, we can create our project. Spin supports multiple languages, but not all features are available in every language. Please check the [language Support](https://developer.fermyon.com/spin/language-support-overview) page for more detailed information.

In this blog post, I will be using the Rust language. I start by installing the Spin templates.

> Note: You do not need any templates to create a Spin application, but they will help you to get started quicker as everything is already configured.

```bash
spin templates install --git https://github.com/fermyon/spin --update
Copying remote template source
Installing template redis-rust...
Installing template static-fileserver...
Installing template http-grain...
Installing template http-swift...
Installing template http-php...
Installing template http-c...
Installing template redirect...
Installing template http-rust...
Installing template http-go...
Installing template http-zig...
Installing template http-empty...
Installing template redis-go...
Installed 12 template(s)

+------------------------------------------------------------------------+
| Name                Description                                        |
+========================================================================+
| http-c              HTTP request handler using C and the Zig toolchain |
| http-empty          HTTP application with no components                |
| http-go             HTTP request handler using (Tiny)Go                |
| http-grain          HTTP request handler using Grain                   |
| http-php            HTTP request handler using PHP                     |
| http-rust           HTTP request handler using Rust                    |
| http-swift          HTTP request handler using SwiftWasm               |
| http-zig            HTTP request handler using Zig                     |
| redirect            Redirects a HTTP route                             |
| redis-go            Redis message handler using (Tiny)Go               |
| redis-rust          Redis message handler using Rust                   |
| static-fileserver   Serves static files from an asset directory        |
+------------------------------------------------------------------------+
```

To build in Rust Spin components, we need to install the `wasm32-wasi` target for Rust. To install the target, run following command:

```bash
rustup target add wasm32-wasi
```

Now we can call the `spin new` command to create a new Spin application:

```bash
spin new http-rust 
Enter a name for your new application: aks-spin-demo
Description: Demo Spin application for AKS WASI node pool
HTTP base: /
HTTP path: /api/figlet
```

This should generate all the files we need and a directory called `aks-spin-demo`. The `spin.toml` file contains the configuration for the application.

Let's take a look at the `spin.toml` file:

```toml
spin_manifest_version = "1"
authors = ["Engin Diri"]
description = "Demo Spin application for AKS WASI node pool"
name = "aks-spin-demo"
trigger = { type = "http", base = "/" }
version = "0.1.0"

[[component]]
id = "aks-spin-demo"
source = "target/wasm32-wasi/release/aks_spin_demo.wasm"
allowed_http_hosts = []
[component.trigger]
route = "/api/figlet"
[component.build]
command = "cargo build --target wasm32-wasi --release"
```

As we want to build a nice Figlet application, we need to add the following dependencies to the `Cargo.toml` file:

```toml
figlet-rs = "0.1.5"
```

We then need to change the code in the `src/lib.rs` file to the following:

```rust
use anyhow::Result;
use spin_sdk::{
    http::{Request, Response},
    http_component,
};
use figlet_rs::FIGfont;

/// A simple Spin HTTP component.
#[http_component]
fn handle_aks_spin_demo(_: Request) -> Result<Response> {
    let standard_font = FIGfont::standard().unwrap();
    let figure = standard_font.convert("Hello, Fermyon on Azure AKS!");
    Ok(http::Response::builder()
        .status(200).body(Some(figure.unwrap().to_string().into()))?)
}
```

### Build and Run the Application Locally

You can try out the application locally by running the following command:

```bash
spin build -u -f spin-local.toml
```

This will build the application and start a local web server. You can run a curl command to test the application:

```bash
curl http://127.0.0.1:3000/api/figlet
```

You should see the following output:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1679863779234/c940dc85-df3a-40ea-8d41-9841dd760ab0.png align="center")

### Publish the Application to Azure Container Registry (ACR)

Before we can publish the application to the ACR, we need to create the ACR first. We will do this by extending the existing Pulumi program.

Add the `containerregistry` package to our `go.mod` file, by running the following command:

```bash
go get -u github.com/pulumi/pulumi-azure-native-sdk/containerregistry
go get -u github.com/pulumi/pulumi-azure-native-sdk/authorization
go get -u github.com/pulumi/pulumi-docker/sdk/v4
```

Now we can add the following code to the `main.go` file:

```go
package main

// ... Omited code

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		// ... Omited code

		registry, err := v20230101preview.NewRegistry(ctx, "wasm-aks-registry", &v20230101preview.RegistryArgs{
			ResourceGroupName: resourceGroup.Name,
			Location:          resourceGroup.Location,
			RegistryName:      pulumi.String("wasmaksregistry"),
			AdminUserEnabled:  pulumi.Bool(true),
			Sku: &v20230101preview.SkuArgs{
				Name: pulumi.String("Standard"),
			},
		})
		if err != nil {
			return err
		}

		credentials := pulumi.All(resourceGroup.Name, registry.Name).ApplyT(func(args []interface{}) (*v20230101preview.ListRegistryCredentialsResult, error) {
			return v20230101preview.ListRegistryCredentials(ctx, &v20230101preview.ListRegistryCredentialsArgs{
				ResourceGroupName: args[0].(string),
				RegistryName:      args[1].(string),
			})
		})
		if err != nil {
			return err
		}
		adminUsername := credentials.ApplyT(func(result interface{}) (string, error) {
			credentials := result.(*v20230101preview.ListRegistryCredentialsResult)
			return *credentials.Username, nil
		}).(pulumi.StringOutput)
		adminPassword := credentials.ApplyT(func(result interface{}) (string, error) {
			credentials := result.(*v20230101preview.ListRegistryCredentialsResult)
			return *credentials.Passwords[0].Value, nil
		}).(pulumi.StringOutput)

		definition, err := v20220401.LookupRoleDefinition(ctx, &v20220401.LookupRoleDefinitionArgs{
			RoleDefinitionId: "7f951dda-4ed3-4680-a7ca-43fe172d538d",
		})
		if err != nil {
			return err
		}

		_, err = v20220401.NewRoleAssignment(ctx, "wasm-aks-role-assignment", &v20220401.RoleAssignmentArgs{
			PrincipalId:      wasmCluster.IdentityProfile.MapIndex(pulumi.String("kubeletidentity")).ObjectId().Elem(),
			PrincipalType:    pulumi.String(v20220401.PrincipalTypeServicePrincipal),
			RoleDefinitionId: pulumi.String(definition.Id),
			Scope:            registry.ID(),
		}, pulumi.DependsOn([]pulumi.Resource{registry}))
		if err != nil {
			return err
		}

		image, err := docker.NewImage(ctx, "wasm-spin-demo-image", &docker.ImageArgs{
			ImageName: pulumi.Sprintf("%s.azurecr.io/aks-wasm-spin-demo:latest", registry.Name),
			Build: &docker.DockerBuildArgs{
				Dockerfile:     pulumi.String("aks-spin-demo/Dockerfile"),
				Context:        pulumi.String("aks-spin-demo"),
				BuilderVersion: docker.BuilderVersionBuilderBuildKit,
				Platform:       pulumi.String("linux/amd64"),
			},
			Registry: &docker.RegistryArgs{
				Server:   pulumi.Sprintf("%s.azurecr.io", registry.Name),
				Username: adminUsername,
				Password: adminPassword,
			},
		}, pulumi.DependsOn([]pulumi.Resource{wasmCluster, wasmPool, registry}))

		return nil
	})
}
```

This code will create the ACR resource with admin user enabled. We will use the admin user to push our Spin image to the ACR. We also create a role assignment to allow the AKS cluster to pull images from the newly created ACR. With this role assignment in place, we don't need to create a pull secret for the AKS cluster or a specific service account.

The last part of the code will build the image using the `pulumi-docker` provider. I have created a Dockerfile in the `aks-spin-demo` folder which is a multi-stage build. The first stage will build the Rust application and the second stage will create a minimal image with the compiled binary and the `spin.toml` file.

> **Attention**: The tag `spin_manifest_version` has to be renamed to `spin_version`, otherwise the shim will not work!

The minimal image is created by using the Chainguard `cgr.dev/chainguard/static` image. The `cgr.dev/chainguard/static` image is a base image with just enough files to run static binaries!

```bash
FROM --platform=${BUILDPLATFORM} rust:1.68.1 AS build
WORKDIR /opt/build
COPY . .
RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release

FROM cgr.dev/chainguard/static:latest
COPY --from=build /opt/build/target/wasm32-wasi/release/aks_spin_demo.wasm .
COPY --from=build /opt/build/spin.toml .
```

### Deploy the Application to the AKS Cluster

Now we can head over to the deployment of the Spin application on our AKS cluster. For this step, we will use the `pulumi-kubernetes` provider. With this provider, we can use `go` to create the Kubernetes resources.

The resources we will create are:

* A `namespace` for the application, we name it `wasm-demo`
    
* A `deployment` for the application, important is here to set the `command` to `/`
    
* A `service` for the application of type `LoadBalancer`
    

Add the `pulumi-kubernetes` provider to your `go.mod` file:

```bash
go get -u github.com/pulumi/pulumi-kubernetes/sdk/v3
```

```go
package main

// ... Omited code

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		// ... Omited code
		k8s, err := kubernetes.NewProvider(ctx, "wasm-aks-provider", &kubernetes.ProviderArgs{
			Kubeconfig:            kubeconfig,
			EnableServerSideApply: pulumi.Bool(true),
		}, pulumi.DependsOn([]pulumi.Resource{wasmCluster, wasmPool, registry}))
		if err != nil {
			return err
		}

		_, err = core.NewNamespace(ctx, "wasm-aks-namespace", &core.NamespaceArgs{
			Metadata: &meta.ObjectMetaArgs{
				Name: pulumi.String("wasm-demo"),
			},
		}, pulumi.Provider(k8s))

		deployment, err := apps.NewDeployment(ctx, "wasm-aks-deployment", &apps.DeploymentArgs{
			Metadata: &meta.ObjectMetaArgs{
				Name:      pulumi.String("wasm-demo"),
				Namespace: pulumi.String("wasm-demo"),
				Annotations: pulumi.StringMap{
					"pulumi.com/skipAwait": pulumi.String("true"),
				},
			},
			Spec: &apps.DeploymentSpecArgs{
				Selector: &meta.LabelSelectorArgs{
					MatchLabels: pulumi.StringMap{
						"app": pulumi.String("wasm-demo"),
					},
				},
				Replicas: pulumi.Int(1),
				Template: &core.PodTemplateSpecArgs{
					Metadata: &meta.ObjectMetaArgs{
						Labels: pulumi.StringMap{
							"app": pulumi.String("wasm-demo"),
						},
					},
					Spec: &core.PodSpecArgs{
						RuntimeClassName: pulumi.String("wasmtime-spin-v1"),
						Containers: core.ContainerArray{
							&core.ContainerArgs{
								Name:  pulumi.String("wasm-demo"),
								Image: image.ImageName,
								Command: pulumi.StringArray{
									pulumi.String("/"),
								},
								Resources: &core.ResourceRequirementsArgs{
									Requests: pulumi.StringMap{
										"cpu":    pulumi.String("10m"),
										"memory": pulumi.String("10Mi"),
									},
									Limits: pulumi.StringMap{
										"cpu":    pulumi.String("500m"),
										"memory": pulumi.String("64Mi"),
									},
								},
							},
						},
					},
				},
			},
		}, pulumi.Provider(k8s), pulumi.DependsOn([]pulumi.Resource{wasmCluster, wasmPool, registry, image}))

		_, err = core.NewService(ctx, "wasm-aks-service", &core.ServiceArgs{
			Metadata: &meta.ObjectMetaArgs{
				Name:      pulumi.String("wasm-demo"),
				Namespace: pulumi.String("wasm-demo"),
				Annotations: pulumi.StringMap{
					"pulumi.com/skipAwait": pulumi.String("true"),
				},
			},
			Spec: &core.ServiceSpecArgs{
				Type: core.ServiceSpecTypeLoadBalancer,
				Ports: core.ServicePortArray{
					&core.ServicePortArgs{
						Name:       pulumi.String("http"),
						Protocol:   pulumi.String("TCP"),
						Port:       pulumi.Int(8080),
						TargetPort: pulumi.Int(80),
					},
				},
				Selector: pulumi.StringMap{
					"app": deployment.Spec.Selector().MatchLabels().MapIndex(pulumi.String("app")),
				},
			},
		}, pulumi.Provider(k8s))
		if err != nil {
			return err
		}
		return nil
	})
}
```

### Test the Application

Now we can run the `pulumi up` command to deploy the application to the AKS cluster.

```bash
pulumi up
```

After the deployment is finished, we can get the public IP of the service with the following command:

```bash
kubectl get svc -n wasm-demo wasm-demo -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
20.101.12.51
```

With this IP address, we can now use `curl` to test the application:

```bash
curl http://20.101.12.51:8080/api/figlet
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1679863898454/2cfb57ba-2134-44a0-8568-01da5d09294e.png align="center")

And it works!

## Housekeeping

To clean up the resources, we can run the `pulumi destroy` command. This will delete all the resources that we just created.

## Conclusion

AKS support for WASM remains in its preview stage, yet deploying WASM applications to AKS is feasible by activating the `WasmNodePoolPreview` feature flag.

I think that WASM is a very interesting technology and seeing that major cloud providers like Azure are already starting to support it is very exciting and a step in the right direction.

I eagerly anticipate the advancements in store for WASM and am confident that we will observe an increasing number of WASM integrations from all the major cloud providers.

## Resources

* https://www.pulumi.com/docs/get-started/install/
    
* https://developer.fermyon.com/spin/index
    
* https://www.pulumi.com/registry/packages/azure-native/
    
* https://www.pulumi.com/registry/packages/kubernetes/
    
* https://www.pulumi.com/registry/packages/docker/