exec container process `/wasi_example_main.wasm`: Exec format error

[Tej-Singh-Rana]

Hi Team,

I was playing with the kwasm operator. I followed the steps in the official docs but encountered an error. Can you please assist me with this?
A few days ago, it was working fine. Today, I checked again and encountered an error.

Environment Details:

Kubernetes Version: v1.28
Environment Type: Sandbox
CRI:  containerd 1.6.6 

Working:

Not Working:

Error Output:

controlplane ~ ➜  kubectl logs pod/wasm-test-gmzpv
{"msg":"exec container process `/wasi_example_main.wasm`: Exec format error","level":"error","time":"2023-09-25T17:42:14.765200Z"}

Earlier, crun is managed by the kwasm operator; now, I have to install it manually. Did we update anything recently? Can you share any URL where I can read about this?

Thanks & Regards,

[0xE282B0]

Hi @Tej-Singh-Rana,
I'm sorry for the inconvenience. The installer has been updated to 0.3.0 To streamline the installation process, the crun has been removed in favor of the containerd-shim-wasmedge. Also, all other runtimes can now work with sidecars. All examples that used crun before should still work if you change the RuntimeClass to wasmedge.

The website has not been updated yet. I just fixed the website a minute ago to make the examples work again.

Thanks for your detailed bug reports! If you find anything that does not work with the wasmedge RuntimeClass, or if you explicitly need crun as a runtime (the shims use libcontainer from the youki project), please let me know!

[Tej-Singh-Rana]

Hi @0xE282B0 ,
Thanks for your reply.

I tried the above steps, but still failing for me.

updated the installer,

Here is the config.toml content.

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.lunatic]
    runtime_type = "/opt/kwasm/bin/containerd-shim-lunatic-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.slight]
    runtime_type = "/opt/kwasm/bin/containerd-shim-slight-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.spin]
    runtime_type = "/opt/kwasm/bin/containerd-shim-spin-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wws]
    runtime_type = "/opt/kwasm/bin/containerd-shim-wws-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmedge]
    runtime_type = "/opt/kwasm/bin/containerd-shim-wasmedge-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmer]
    runtime_type = "/opt/kwasm/bin/containerd-shim-wasmer-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmtime]
    runtime_type = "/opt/kwasm/bin/containerd-shim-wasmtime-v1"

Binaries:

Thanks & Regards,

[0xE282B0]

Interesting, everything looks as it should:

• Binaries at /opt/kwasm/bin
• Containerd conf.toml entries
• RuntimeClass wasmedge
• according to the error meddage the pod is configured with runtimeClassName: wasmedge

Which kubernetes distribution do you use? - I tested some here: KWasm/kwasm-node-installer#43
Did containerd restart corectly after config change?

[Tej-Singh-Rana]

I deployed K8s cluster using the kubeadm tool. Yes, I did restart containerd service.

[0xE282B0]

Do you have a default conf.toml or is the plugin configuration the only content in the conf file?

[Tej-Singh-Rana]

is the plugin configuration the only content in the conf file?

Only this one.

[0xE282B0]

Could you generate a default config and run the installer again?
containerd config default > /etc/containerd/config.toml

[Tej-Singh-Rana]

Sure, I will try this and let you know.

Regards,

[Tej-Singh-Rana]

1. Ran the following command:

containerd config default > /etc/containerd/config.toml

2. Restarted the containerd service.

Content of the config.toml file:

disabled_plugins = []
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/containerd"
state = "/run/containerd"
temp = ""
version = 2

[cgroup]
  path = ""

[debug]
  address = ""
  format = ""
  gid = 0
  level = ""
  uid = 0

[grpc]
  address = "/run/containerd/containerd.sock"
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216
  tcp_address = ""
  tcp_tls_ca = ""
  tcp_tls_cert = ""
  tcp_tls_key = ""
  uid = 0

[metrics]
  address = ""
  grpc_histogram = false

[plugins]

  [plugins."io.containerd.gc.v1.scheduler"]
    deletion_threshold = 0
    mutation_threshold = 100
    pause_threshold = 0.02
    schedule_delay = "0s"
    startup_delay = "100ms"

  [plugins."io.containerd.grpc.v1.cri"]
    device_ownership_from_security_context = false
    disable_apparmor = false
    disable_cgroup = false
    disable_hugetlb_controller = true
    disable_proc_mount = false
    disable_tcp_service = true
    enable_selinux = false
    enable_tls_streaming = false
    enable_unprivileged_icmp = false
    enable_unprivileged_ports = false
    ignore_image_defined_volumes = false
    max_concurrent_downloads = 3
    max_container_log_line_size = 16384
    netns_mounts_under_state_dir = false
    restrict_oom_score_adj = false
    sandbox_image = "k8s.gcr.io/pause:3.6"
    selinux_category_range = 1024
    stats_collect_period = 10
    stream_idle_timeout = "4h0m0s"
    stream_server_address = "127.0.0.1"
    stream_server_port = "0"
    systemd_cgroup = false
    tolerate_missing_hugetlb_controller = true
    unset_seccomp_profile = ""

    [plugins."io.containerd.grpc.v1.cri".cni]
      bin_dir = "/opt/cni/bin"
      conf_dir = "/etc/cni/net.d"
      conf_template = ""
      ip_pref = ""
      max_conf_num = 1

    [plugins."io.containerd.grpc.v1.cri".containerd]
      default_runtime_name = "runc"
      disable_snapshot_annotations = true
      discard_unpacked_layers = false
      ignore_rdt_not_enabled_errors = false
      no_pivot = false
      snapshotter = "overlayfs"

      [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
        base_runtime_spec = ""
        cni_conf_dir = ""
        cni_max_conf_num = 0
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_path = ""
        runtime_root = ""
        runtime_type = ""

        [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options]

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]

        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
          base_runtime_spec = ""
          cni_conf_dir = ""
          cni_max_conf_num = 0
          container_annotations = []
          pod_annotations = []
          privileged_without_host_devices = false
          runtime_engine = ""
          runtime_path = ""
          runtime_root = ""
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            BinaryName = ""
            CriuImagePath = ""
            CriuPath = ""
            CriuWorkPath = ""
            IoGid = 0
            IoUid = 0
            NoNewKeyring = false
            NoPivotRoot = false
            Root = ""
            ShimCgroup = ""
            SystemdCgroup = false

      [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
        base_runtime_spec = ""
        cni_conf_dir = ""
        cni_max_conf_num = 0
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_path = ""
        runtime_root = ""
        runtime_type = ""

        [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options]

    [plugins."io.containerd.grpc.v1.cri".image_decryption]
      key_model = "node"

    [plugins."io.containerd.grpc.v1.cri".registry]
      config_path = ""

      [plugins."io.containerd.grpc.v1.cri".registry.auths]

      [plugins."io.containerd.grpc.v1.cri".registry.configs]

      [plugins."io.containerd.grpc.v1.cri".registry.headers]

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]

    [plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
      tls_cert_file = ""
      tls_key_file = ""

  [plugins."io.containerd.internal.v1.opt"]
    path = "/opt/containerd"

  [plugins."io.containerd.internal.v1.restart"]
    interval = "10s"

  [plugins."io.containerd.internal.v1.tracing"]
    sampling_ratio = 1.0
    service_name = "containerd"

  [plugins."io.containerd.metadata.v1.bolt"]
    content_sharing_policy = "shared"

  [plugins."io.containerd.monitor.v1.cgroups"]
    no_prometheus = false

  [plugins."io.containerd.runtime.v1.linux"]
    no_shim = false
    runtime = "runc"
    runtime_root = ""
    shim = "containerd-shim"
    shim_debug = false

  [plugins."io.containerd.runtime.v2.task"]
    platforms = ["linux/amd64"]
    sched_core = false

  [plugins."io.containerd.service.v1.diff-service"]
    default = ["walking"]

  [plugins."io.containerd.service.v1.tasks-service"]
    rdt_config_file = ""

  [plugins."io.containerd.snapshotter.v1.aufs"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.btrfs"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.devmapper"]
    async_remove = false
    base_image_size = ""
    discard_blocks = false
    fs_options = ""
    fs_type = ""
    pool_name = ""
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.native"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.overlayfs"]
    root_path = ""
    upperdir_label = false

  [plugins."io.containerd.snapshotter.v1.zfs"]
    root_path = ""

  [plugins."io.containerd.tracing.processor.v1.otlp"]
    endpoint = ""
    insecure = false
    protocol = ""

[proxy_plugins]

[stream_processors]

  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
    path = "ctd-decoder"
    returns = "application/vnd.oci.image.layer.v1.tar"

  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
    path = "ctd-decoder"
    returns = "application/vnd.oci.image.layer.v1.tar+gzip"

[timeouts]
  "io.containerd.timeout.bolt.open" = "0s"
  "io.containerd.timeout.shim.cleanup" = "5s"
  "io.containerd.timeout.shim.load" = "5s"
  "io.containerd.timeout.shim.shutdown" = "3s"
  "io.containerd.timeout.task.state" = "2s"

[ttrpc]
  address = ""
  gid = 0
  uid = 0

3. Installed the operator from the below link:

https://kwasm.sh/quickstart/#installation

4. Updated the installer with tag v0.3.0

https://github.com/KWasm/kwasm-node-installer/releases/tag/v0.3.0

5. Restarted the containerd service.

Content of the config.toml file:

disabled_plugins = []
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/containerd"
state = "/run/containerd"
temp = ""
version = 2

[cgroup]
  path = ""

[debug]
  address = ""
  format = ""
  gid = 0
  level = ""
  uid = 0

[grpc]
  address = "/run/containerd/containerd.sock"
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216
  tcp_address = ""
  tcp_tls_ca = ""
  tcp_tls_cert = ""
  tcp_tls_key = ""
  uid = 0

[metrics]
  address = ""
  grpc_histogram = false

[plugins]

  [plugins."io.containerd.gc.v1.scheduler"]
    deletion_threshold = 0
    mutation_threshold = 100
    pause_threshold = 0.02
    schedule_delay = "0s"
    startup_delay = "100ms"

  [plugins."io.containerd.grpc.v1.cri"]
    device_ownership_from_security_context = false
    disable_apparmor = false
    disable_cgroup = false
    disable_hugetlb_controller = true
    disable_proc_mount = false
    disable_tcp_service = true
    enable_selinux = false
    enable_tls_streaming = false
    enable_unprivileged_icmp = false
    enable_unprivileged_ports = false
    ignore_image_defined_volumes = false
    max_concurrent_downloads = 3
    max_container_log_line_size = 16384
    netns_mounts_under_state_dir = false
    restrict_oom_score_adj = false
    sandbox_image = "k8s.gcr.io/pause:3.6"
    selinux_category_range = 1024
    stats_collect_period = 10
    stream_idle_timeout = "4h0m0s"
    stream_server_address = "127.0.0.1"
    stream_server_port = "0"
    systemd_cgroup = false
    tolerate_missing_hugetlb_controller = true
    unset_seccomp_profile = ""

    [plugins."io.containerd.grpc.v1.cri".cni]
      bin_dir = "/opt/cni/bin"
      conf_dir = "/etc/cni/net.d"
      conf_template = ""
      ip_pref = ""
      max_conf_num = 1

    [plugins."io.containerd.grpc.v1.cri".containerd]
      default_runtime_name = "runc"
      disable_snapshot_annotations = true
      discard_unpacked_layers = false
      ignore_rdt_not_enabled_errors = false
      no_pivot = false
      snapshotter = "overlayfs"

      [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
        base_runtime_spec = ""
        cni_conf_dir = ""
        cni_max_conf_num = 0
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_path = ""
        runtime_root = ""
        runtime_type = ""

        [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options]

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]

        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
          base_runtime_spec = ""
          cni_conf_dir = ""
          cni_max_conf_num = 0
          container_annotations = []
          pod_annotations = []
          privileged_without_host_devices = false
          runtime_engine = ""
          runtime_path = ""
          runtime_root = ""
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            BinaryName = ""
            CriuImagePath = ""
            CriuPath = ""
            CriuWorkPath = ""
            IoGid = 0
            IoUid = 0
            NoNewKeyring = false
            NoPivotRoot = false
            Root = ""
            ShimCgroup = ""
            SystemdCgroup = false

      [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
        base_runtime_spec = ""
        cni_conf_dir = ""
        cni_max_conf_num = 0
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_path = ""
        runtime_root = ""
        runtime_type = ""

        [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options]

    [plugins."io.containerd.grpc.v1.cri".image_decryption]
      key_model = "node"

    [plugins."io.containerd.grpc.v1.cri".registry]
      config_path = ""

      [plugins."io.containerd.grpc.v1.cri".registry.auths]

      [plugins."io.containerd.grpc.v1.cri".registry.configs]

      [plugins."io.containerd.grpc.v1.cri".registry.headers]

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]

    [plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
      tls_cert_file = ""
      tls_key_file = ""

  [plugins."io.containerd.internal.v1.opt"]
    path = "/opt/containerd"

  [plugins."io.containerd.internal.v1.restart"]
    interval = "10s"

  [plugins."io.containerd.internal.v1.tracing"]
    sampling_ratio = 1.0
    service_name = "containerd"

  [plugins."io.containerd.metadata.v1.bolt"]
    content_sharing_policy = "shared"

  [plugins."io.containerd.monitor.v1.cgroups"]
    no_prometheus = false

  [plugins."io.containerd.runtime.v1.linux"]
    no_shim = false
    runtime = "runc"
    runtime_root = ""
    shim = "containerd-shim"
    shim_debug = false

  [plugins."io.containerd.runtime.v2.task"]
    platforms = ["linux/amd64"]
    sched_core = false

  [plugins."io.containerd.service.v1.diff-service"]
    default = ["walking"]

  [plugins."io.containerd.service.v1.tasks-service"]
    rdt_config_file = ""

  [plugins."io.containerd.snapshotter.v1.aufs"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.btrfs"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.devmapper"]
    async_remove = false
    base_image_size = ""
    discard_blocks = false
    fs_options = ""
    fs_type = ""
    pool_name = ""
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.native"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.overlayfs"]
    root_path = ""
    upperdir_label = false

  [plugins."io.containerd.snapshotter.v1.zfs"]
    root_path = ""

  [plugins."io.containerd.tracing.processor.v1.otlp"]
    endpoint = ""
    insecure = false
    protocol = ""

[proxy_plugins]

[stream_processors]

  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
    path = "ctd-decoder"
    returns = "application/vnd.oci.image.layer.v1.tar"

  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
    path = "ctd-decoder"
    returns = "application/vnd.oci.image.layer.v1.tar+gzip"

[timeouts]
  "io.containerd.timeout.bolt.open" = "0s"
  "io.containerd.timeout.shim.cleanup" = "5s"
  "io.containerd.timeout.shim.load" = "5s"
  "io.containerd.timeout.shim.shutdown" = "3s"
  "io.containerd.timeout.task.state" = "2s"

[ttrpc]
  address = ""
  gid = 0
  uid = 0

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.lunatic]
    runtime_type = "/opt/kwasm/bin/containerd-shim-lunatic-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.slight]
    runtime_type = "/opt/kwasm/bin/containerd-shim-slight-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.spin]
    runtime_type = "/opt/kwasm/bin/containerd-shim-spin-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wws]
    runtime_type = "/opt/kwasm/bin/containerd-shim-wws-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmedge]
    runtime_type = "/opt/kwasm/bin/containerd-shim-wasmedge-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmer]
    runtime_type = "/opt/kwasm/bin/containerd-shim-wasmer-v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmtime]
    runtime_type = "/opt/kwasm/bin/containerd-shim-wasmtime-v1"

6. Ran the sample tests:

https://kwasm.sh/quickstart/#wasm-runtime-configuration

7. This time, I got a different error.

[0xE282B0]

Thats odd!
The good thing is, that error message comes from the shim. But it's very unspecific error message.
Is the Spin container working or one of the others?

You can test that with these manifests:

kubectl apply -f https://github.com/KWasm/kwasm-node-installer/releases/download/v0.3.0/deployment.yaml
kubectl apply -f https://github.com/KWasm/kwasm-node-installer/releases/download/v0.3.0/runtimeclass.yaml

[Tej-Singh-Rana]

I will check and let you know.

[Tej-Singh-Rana]

I tried it again and deployed the above manifest files.

Runtimeclasses:

controlplane ~ ➜  kubectl get runtimeclasses
NAME       HANDLER    AGE
lunatic    lunatic    4m20s
slight     slight     4m20s
spin       spin       4m20s
wasmedge   wasmedge   4m20s
wasmer     wasmer     4m20s
wasmtime   wasmtime   4m20s
wws        wws        4m20s

Deployments:

controlplane ~ ➜  kubectl get po -A
NAMESPACE      NAME                                   READY   STATUS              RESTARTS      AGE
default        lunatic-demo-6475554875-7vxml          0/2     RunContainerError   6 (9s ago)    79s
default        wasm-slight-6467bcc5bc-w5h5d           0/2     CrashLoopBackOff    6 (13s ago)   78s
default        wasm-spin-74c4cf5c77-lf25l             0/2     CrashLoopBackOff    6 (14s ago)   80s
default        wasm-wws-888f6bc4b-22kwz               0/2     RunContainerError   6 (14s ago)   79s
default        wasmedge-demo-5ff758d79-gb2xx          0/2     CrashLoopBackOff    6 (18s ago)   80s
default        wasmer-demo-857f947cb7-tm9m9           0/2     CrashLoopBackOff    6 (16s ago)   80s
default        wasmtime-demo-56c78ddd95-grm2v         0/2     CrashLoopBackOff    6 (15s ago)   79s

Inspected the pods:

Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  5m11s                  default-scheduler  Successfully assigned default/wasm-wws-888f6bc4b-22kwz to controlplane
  Warning  Failed     5m5s                   kubelet            Error: failed to start containerd task "wws-hello": Others("failed to create container: failed to receive. \"waiting for intermediate process\". BrokenChannel"): unknown
  Normal   Pulled     5m5s                   kubelet            Successfully pulled image "ghcr.io/deislabs/containerd-wasm-shims/examples/wws-js-hello:latest" in 902ms (4.077s including waiting)
  Normal   Pulled     4m53s                  kubelet            Successfully pulled image "redis" in 314ms (11.079s including waiting)
  Normal   Pulled     4m52s                  kubelet            Successfully pulled image "ghcr.io/deislabs/containerd-wasm-shims/examples/wws-js-hello:latest" in 226ms (1.445s including waiting)
  Normal   Created    4m49s (x2 over 4m53s)  kubelet            Created container redis
  Warning  Failed     4m49s (x2 over 4m53s)  kubelet            Error: failed to create containerd task: failed to create shim task: Other: mount process exit unexpectedly, exit code: ENOENT: No such file or directory: unknown
  Normal   Pulled     4m49s                  kubelet            Successfully pulled image "redis" in 331ms (2.28s including waiting)
  Warning  BackOff    4m48s (x2 over 4m49s)  kubelet            Back-off restarting failed container redis in pod wasm-wws-888f6bc4b-22kwz_default(fa4e4906-cd03-4a61-aa4e-6f701d212849)
  Normal   Pulling    4m36s (x3 over 5m5s)   kubelet            Pulling image "redis"
  Normal   Created    4m36s (x3 over 5m5s)   kubelet            Created container wws-hello
  Normal   Pulling    4m36s (x3 over 5m9s)   kubelet            Pulling image "ghcr.io/deislabs/containerd-wasm-shims/examples/wws-js-hello:latest"
  Warning  Failed     4m36s (x2 over 4m52s)  kubelet            Error: failed to create containerd task: failed to create shim task: Other: mount process exit unexpectedly, exit code: ENOENT: No such file or directory: unknown
  Normal   Pulled     4m36s                  kubelet            Successfully pulled image "ghcr.io/deislabs/containerd-wasm-shims/examples/wws-js-hello:latest" in 256ms (256ms including waiting)
  Warning  BackOff    3s (x25 over 4m49s)    kubelet            Back-off restarting failed container wws-hello in pod wasm-wws-888f6bc4b-22kwz_default(fa4e4906-cd03-4a61-aa4e-6f701d212849)

[Tej-Singh-Rana]

Is the Spin container working or one of the others?

Sorry, I didn't get your point. Which containers?