nerdctl compose fails, profile 'nerdctl-default' does not exist

[nicolasespiau]

Description

I'm developing a nodejs app suite with some microservices and an API.

In order to perform integration testing before any deployment, I have a step in my Gitlab CI that uses a docker-compose.yaml to spawn all these MS and API and launch some tests against the API.

All of this happens in a Kube cluster based on containerd, so I'm trying to use nerdctl compose.

When my CI steps run, it fails with some time="2022-07-15T16:17:33Z" level=debug msg="failed to run [aa-exec -p nerdctl-default -- true]: \"[90] aa-exec: ERROR: profile 'nerdctl-default' does not exist\\n\"" error="exit status 1" errors.

Steps to reproduce the issue

Describe the results you received and expected

Here is my docker-compose.yaml:

version: "3.9"
networks:
  testinteg:
    name: testinteg
services:
  mongosrv:
    container_name: "mongosrv"
    image: registry.gitlab.com/casetonpote1/ctp-utils/mongosrv:testinteg
    command: [ "--auth", "--dbpath", "/mongo/data/db" ]
    pull_policy: always
    networks:
      - testinteg
  users:
    container_name: "users"
    image: registry.gitlab.com/casetonpote1/backend-users:dev
    environment:
      NODE_ENV: "dev"
      PORT: 80
      LOG_LEVEL: "debug"
      LOG_FORMAT: "splat,simple"
      PASSWORD_JWT_SECRET: "anothersecurestring"
      PASSWORD_JWT_TTL: "30s"
      SSL_ENABLED: "false"
      MOCK_DB: "false"
      MONGO_DB: "casetespotes"
      MONGO_HOST: "mongosrv"
      MONGO_USER: "usersRW"
      MONGO_PASSWD: "password"
    depends_on:
      - mongosrv
    networks:
      - testinteg
  stories:
    container_name: "stories"
    image: registry.gitlab.com/casetonpote1/backend-stories:dev
    environment:
      NODE_ENV: "dev"
      PORT: 80
      LOG_LEVEL: "debug"
      LOG_FORMAT: "splat,simple"
      SSL_ENABLED: "false"
      MOCK_DB: "false"
      MONGO_DB: "casetespotes"
      MONGO_HOST: "mongosrv"
      MONGO_USER: "storiesRW"
      MONGO_PASSWD: "password"
    depends_on:
      - mongosrv
    networks:
      - testinteg
  profiles:
    container_name: "profiles"
    image: registry.gitlab.com/casetonpote1/backend-profiles:dev
    environment:
      NODE_ENV: "dev"
      PORT: 80
      LOG_LEVEL: "debug"
      LOG_FORMAT: "splat,simple"
      SSL_ENABLED: "false"
      MOCK_DB: "false"
      MONGO_DB: "casetespotes"
      MONGO_HOST: "mongosrv"
      MONGO_USER: "profilesRW"
      MONGO_PASSWD: "password"
      MOCK_GEO: "true"
    depends_on:
      - mongosrv
    networks:
      - testinteg
  sessions:
    container_name: "sessions"
    image: registry.gitlab.com/casetonpote1/backend-sessions:dev
    environment:
      NODE_ENV: "dev"
      PORT: 80
      LOG_LEVEL: "debug"
      LOG_FORMAT: "splat,simple"
      APP_JWT_SECRET: "anothersecurestring"
      APP_JWT_TTL: "30s"
      REFRESH_ALLOWED: "7d"
      SSL_ENABLED: "false"
      MOCK_DB: "false"
      MONGO_DB: "casetespotes"
      MONGO_HOST: "mongosrv"
      MONGO_USER: "authenticater"
      MONGO_PASSWD: "password"
      MS_USERS_ENDPOINT: "http://users"
    depends_on:
      - mongosrv
    networks:
      - testinteg
  linkings:
    container_name: "linkings"
    image: registry.gitlab.com/casetonpote1/backend-linkings:dev
    environment:
      NODE_ENV: "dev"
      PORT: 80
      LOG_LEVEL: "debug"
      LOG_FORMAT: "splat,simple"
      SSL_ENABLED: "false"
      MOCK_DB: "false"
      MONGO_DB: "casetespotes"
      MONGO_HOST: "mongosrv"
      MONGO_USER: "linkingsRW"
      MONGO_PASSWD: "password"
    depends_on:
      - mongosrv
    networks:
      - testinteg
  api-gateway:
    container_name: "api-gateway"
    image: registry.gitlab.com/casetonpote1/api-gateway-public:test
    environment:
      NODE_ENV: "dev"
      PORT: 80
      LOG_LEVEL: "debug"
      LOG_FORMAT: "splat,simple"
      SSL_ENABLED: "false"
      AUTH_SOURCE: "api-public-dev"
      MS_STORIES_ENDPOINT: "http://stories"
      MS_USERS_ENDPOINT: "http://users"
      MS_PROFILES_ENDPOINT: "http://profiles"
      MS_SESSIONS_ENDPOINT: "http://sessions"
      MS_LINKINGS_ENDPOINT: "http://linkings"
    depends_on:
      - users
      - stories
      - profiles
      - sessions
      - linkings
    networks:
      - testinteg

And here is my CI step:

integrationtesting:
  tags:
    - kubernetes
  image: ghcr.io/containerd/nerdctl:v0.21.0
  stage: tests
  before_script:
    - nerdctl compose -f test_integ/dependencies/docker-compose.yaml up -d --debug-full
  script:
    - nerdctl compose -f test_integ/dependencies/tester-compose.yaml up --debug-full
  after_script:
    - nerdctl compose -f test_integ/dependencies/docker-compose.yaml -f test_integ/dependencies/tester-compose.yaml down
  dependencies:
    - buildapitest
    - buildtestertest

Where dependencies are steps where required images are created. These steps are successful.

All the images pulling seems to be ok, but when creating the containers I get this error messages in the CI logs:

time="2022-07-15T16:17:33Z" level=debug msg="failed to run [aa-exec -p nerdctl-default -- true]: \"[80] aa-exec: ERROR: profile 'nerdctl-default' does not exist\\n\"" error="exit status 1"
108time="2022-07-15T16:17:33Z" level=debug msg="verification process skipped"
109time="2022-07-15T16:17:33Z" level=debug msg="failed to run [aa-exec -p nerdctl-default -- true]: \"[86] aa-exec: ERROR: profile 'nerdctl-default' does not exist\\n\"" error="exit status 1"
110time="2022-07-15T16:17:33Z" level=debug msg="verification process skipped"
111time="2022-07-15T16:17:33Z" level=debug msg="failed to run [aa-exec -p nerdctl-default -- true]: \"[82] aa-exec: ERROR: profile 'nerdctl-default' does not exist\\n\"" error="exit status 1"
112time="2022-07-15T16:17:33Z" level=debug msg="verification process skipped"
113time="2022-07-15T16:17:33Z" level=fatal msg="no such file or directory"
114time="2022-07-15T16:17:33Z" level=debug msg="failed to run [aa-exec -p nerdctl-default -- true]: \"[89] aa-exec: ERROR: profile 'nerdctl-default' does not exist\\n\"" error="exit status 1"
115time="2022-07-15T16:17:33Z" level=debug msg="verification process skipped"
116time="2022-07-15T16:17:33Z" level=fatal msg="no such file or directory"
117time="2022-07-15T16:17:33Z" level=debug msg="failed to run [aa-exec -p nerdctl-default -- true]: \"[90] aa-exec: ERROR: profile 'nerdctl-default' does not exist\\n\"" error="exit status 1"
118time="2022-07-15T16:17:33Z" level=debug msg="verification process skipped"
119time="2022-07-15T16:17:33Z" level=fatal msg="no such file or directory"
120time="2022-07-15T16:17:33Z" level=fatal msg="no such file or directory"
121time="2022-07-15T16:17:33Z" level=debug msg="failed to run [aa-exec -p nerdctl-default -- true]: \"[91] aa-exec: ERROR: profile 'nerdctl-default' does not exist\\n\"" error="exit status 1"
122time="2022-07-15T16:17:33Z" level=debug msg="verification process skipped"
123time="2022-07-15T16:17:34Z" level=fatal msg="no such file or directory"
124time="2022-07-15T16:17:34Z" level=debug msg="failed to run [aa-exec -p nerdctl-default -- true]: \"[95] aa-exec: ERROR: profile 'nerdctl-default' does not exist\\n\"" error="exit status 1"
125time="2022-07-15T16:17:34Z" level=debug msg="verification process skipped"
126time="2022-07-15T16:17:34Z" level=fatal msg="no such file or directory"
127time="2022-07-15T16:17:34Z" level=fatal msg="no such file or directory"
128time="2022-07-15T16:17:34Z" level=fatal msg="error while creating container users: exit status 1"

What version of nerdctl are you using?

ghcr.io/containerd/nerdctl:v0.21.0

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

No response

Host information

$ nerdctl info
Client:
Namespace: default
Debug Mode: false
Server:
Server Version: v1.6.6
Storage Driver: overlayfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Log: fluentd journald json-file
Storage: aufs native overlayfs
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.13.0-1017-aws
Operating System: Ubuntu 22.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.687GiB
Name: runner-dqoqvrt-project-25793245-concurrent-0ht8p8
ID: d4341431-1b20-4fc6-b55e-89dd1d3edd57
WARNING: IPv4 forwarding is disabled

[Zheaoli]

Seems like you need to follow the following guideline to add AppArmor Profile for nerdctl

Applying an (existing) AppArmor profile to rootless containers: nerdctl run --security-opt apparmor=<PROFILE>. Use `sudo nerdctl apparmor load` to load the nerdctl-default profile.

https://github.com/containerd/nerdctl#features-present-in-nerdctl-but-not-present-in-docker