Fedora Silverblue support?

[garrett]

Silverblue doesn't have a "normal" package manager. All installations are the same. One can "overlay" RPMs (and their dependencies), but most of the time, all the work happens in Flatpaks and podman containers (especially toolbox containers, which are a special type of long-term podman contaner that simulates a standard Fedora installation, with dnf/yum and so on).

What's probably needed is to have podman support (#15) and ensure that nothing else is needed from the host system.

[inercia]

If you cannot run a docker client in Fedora Silverblue then I think this will be blocked by #15...

[garrett]

I currently have Docker as an overlay on Silverblue, and this is enough to get the (now deprecated) cockpit-docker add-on up and running, but it might not be working well enough for k3d. (I guess it isn't.)

There's the whole cgroups v1 vs. v2 issue, which may be problematic with Docker (Moby) still.

[inercia]

@garrett Could you try to install k3d and see if it works from command line? If that works then there must be a problem with k3x...

[garrett]

$ k3d cluster create mycluster
ERRO[0000] Failed to list containers                    
ERRO[0000] Failed to get nodes for cluster 'mycluster'  
ERRO[0000] Failed to list docker networks               
ERRO[0000] Failed to create cluster network             
ERRO[0000] Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/networks?filters=%7B%22name%22%3A%7B%22k3d-mycluster%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied 
ERRO[0000] Failed to create cluster >>> Rolling Back    
INFO[0000] Deleting cluster 'mycluster'                 
ERRO[0000] Failed to delete container ''                
WARN[0000] Failed to delete node '': Try to delete it manually 
INFO[0000] Deleting cluster network 'k3d-mycluster'     
WARN[0000] Failed to delete cluster network 'k3d-mycluster': 'Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Delete "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/networks/k3d-mycluster": dial unix /var/run/docker.sock: connect: permission denied' 
ERRO[0000] Failed to delete 1 nodes: Try to delete them manually 
FATA[0000] Cluster creation FAILED, also FAILED to rollback changes! 

With sudo:

$ sudo k3d cluster create mycluster
INFO[0000] Created network 'k3d-mycluster'              
INFO[0000] Created volume 'k3d-mycluster-images'        
INFO[0001] Creating node 'k3d-mycluster-server-0'       
INFO[0002] Pulling image 'docker.io/rancher/k3s:v1.18.6-k3s1' 
ERRO[0014] Failed to start container                    
ERRO[0014] Failed to create node 'k3d-mycluster-server-0' 
ERRO[0014] Failed to create node                        
ERRO[0014] Error response from daemon: OCI runtime create failed: this version of runc doesn't work on cgroups v2: unknown 
ERRO[0014] Failed to create cluster >>> Rolling Back    
INFO[0014] Deleting cluster 'mycluster'                 
INFO[0014] Deleted k3d-mycluster-server-0               
INFO[0014] Deleting cluster network '59275b244928c7f7aea159077836aeb0a607f666567dff29800eafc6b8f6de5e' 
FATA[0014] Cluster creation FAILED, all changes have been rolled back! 

And here's a basic docker command:

sudo docker run -it fedora
Unable to find image 'fedora:latest' locally
latest: Pulling from library/fedora
c7def56d621e: Pull complete 
Digest: sha256:d6a6d60fda1b22b6d5fe3c3b2abe2554b60432b7b215adc11a2b5fae16f50188
Status: Downloaded newer image for fedora:latest
docker: Error response from daemon: OCI runtime create failed: this version of runc doesn't work on cgroups v2: unknown.
ERRO[0019] error waiting for container: context canceled 

Docker does look like it's having issues on cgroups v2 and that is what's fouling up k3d. And k3d needs root access, which k3x in Flatpak doesn't have, right?

Anyway, I guess we just need to wait for k3d to get podman support and then hopefully everything will "magically" work, even on cgroups v2, possibly even rootless. 😁

[inercia]

Not sure how things work in Silverblue, but you would need to be part of the right group for accessing /var/run/docker.sock. Once you fix that, you should be able to run k3d without sudo.

But besides that, this OCI runtime create failed: this version of runc doesn't work on cgroups v2: unknown message is really concerning. But it is strange because there must be some way to run Docker in Silverblue...

[garrett]

But it is strange because there must be some way to run Docker in Silverblue...

Running Docker in Silverblue is basically called podman, as it's docker-compatible clone of Docker, but with additional features (such as being able to run it as a user without root privs). Docker (Moby) hasn't been ported to cgroups v2 yet, so it only works for basic commands and not running containers (which is the important part of course 😉).

[inercia]

Running Docker in Silverblue is basically called podman, as it's docker-compatible clone of Docker, but with additional features (such as being able to run it as a user without root privs). Docker (Moby) hasn't been ported to cgroups v2 yet, so it only works for basic commands and not running containers (which is the important part of course wink).

Yeah, but the important thing here would be to have a Docker-like API that the go docker client library could use. AFAIK, podman does not provide this, so it think it will not be possible to add support for podman in the near future. The solution I was proposing was to run the real-thing in Silverblue, but I don't know if that is possible or not...

[garrett]

Podman does actually provide a Docker-like API in Podman 2.0. https://podman.io/blogs/2020/07/01/rest-versioning.html

I think the API docs have the docker-compatible API under "compat" @ https://docs.podman.io/en/latest/_static/api.html (podman also has its own API to do additional things like handle pods)

I saw in a comment elswhere on GitHub that getting a podman service up an running is as running:

podman system service --time=0 &
export DOCKER_HOST=unix:/$XDG_RUNTIME_DIR/podman/podman.sock

That's for running podman without requiring root (in a user session), as it references $XDG_RUNTIME_DIR.

For system containers, it's:

sudo podman system service --time=0 &
export DOCKER_HOST=unix:/run/podman/podman.sock

To start up the service and specify a special URI, such as the Docker URI:

sudo podman system service --time=0 unix:/var/run/docker.sock

I found out some of this in the docs for podman system service. It's the same as running man podman-system-serice (with podman installed). There's help at the command line too: podman system service --help

Additionally, I saw someone say that installing the cockpit-podman compatibility package should set up a link too. (But the service would still have to be run, as docker, by default, isn't running any service in the background, as it doesn't need it for standard operations. So the compat package doesn't really apply here, FWIW.)

[inercia]

Thanks for the clarification @garrett. Maybe it would be worth adding all this information in the k3d issue, as this is something that does not really need to be supported by k3x but by k3d...

[garrett]

@inercia: Thanks! Right, it should be talked about there. I copy/pasted/edited the comment to move it there.

Additionally, while podman support might just get Silverblue support working, there might be some additional issues due to the immutable filesystem and perhaps not having system packages. In other words, solving podman support will probably solve Silverblue support (but not ncessarily). That's why I opened up multiple issues. We could close this one and just assume podman support will make it on Silverblue or keep both open until we see what happens with podman.

Thanks again for your effort for on k3x and these issues! (I'm looking forward to being able to try it out someday.)

[tidux]

I got this same traceback on Fedora Kinoite (Silverblue KDE) 35 running under moby-engine. It is most certainly not due to podman or a cgroups problem as shown below:

$ flatpak run com.github.inercia.k3x
GI search path:
[PRE-MAIN]  [search-path] - /usr/lib/x86_64-linux-gnu/girepository-1.0
[PRE-MAIN]  [search-path] - /app/lib/girepository-1.0
[PRE-MAIN]  [search-path] - /app/lib/girepository-1.0
[PRE-MAIN] Current LD_LIBRARY_PATH: /app/lib:/app/lib/x86_64-linux-gnu:/lib64:/lib:/usr/lib64:/usr/lib:/usr/local/lib
2022-04-13 22:00:09,131 [MainThread  ] [DEBUG]  [MAIN] Creating settings manager...
2022-04-13 22:00:09,139 [MainThread  ] [DEBUG]  Creating settings in delayed mode...
2022-04-13 22:00:09,140 [MainThread  ] [INFO ]  Changing loglevel to 20
2022-04-13 22:00:09,140 [MainThread  ] [INFO ]  [MAIN] Using icon /home/tidux/.var/app/com.github.inercia.k3x/cache/icons/app.svg
2022-04-13 22:00:09,140 [MainThread  ] [INFO ]  [MAIN] Starting appindicator.Indicator
2022-04-13 22:00:09,158 [MainThread  ] [INFO ]  Creating/recreating docker client with DOCKER_HOST=unix:///var/run/docker.sock

(k3x:2): Gtk-CRITICAL **: 22:00:09.259: gtk_entry_set_text: assertion 'text != NULL' failed
Traceback (most recent call last):
  File "/app/bin/k3x", line 62, in <module>
    sys.exit(main.main(VERSION))
  File "/app/share/k3x/k3x/main.py", line 139, in main
    _indicator = Indicator(version=version)  # NOTE: assign for keeping the object alive
  File "/app/share/k3x/k3x/main.py", line 113, in __init__
    self._keybinder = Keybindings(self._settings, self._shortcuts)
  File "/app/share/k3x/k3x/keybindings.py", line 85, in __init__
    self._keybinder = SystemHotkey(check_queue_interval=0.01)
  File "/app/lib/python3.7/site-packages/system_hotkey/system_hotkey.py", line 614, in __init__
    self.conn = xcffib.connect()
  File "/app/lib/python3.7/site-packages/xcffib/__init__.py", line 527, in __init__
    self.invalid()
  File "/app/lib/python3.7/site-packages/xcffib/__init__.py", line 559, in invalid
    raise ConnectionException(err)
xcffib.ConnectionException: Connection closed, error during parsing display string.
$ ls -l /var/run/docker.sock
srw-rw----. 1 root docker 0 Apr 13 21:55 /var/run/docker.sock
$ groups
tidux wheel docker
$ docker run -it alpine cat /etc/os-release 
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
df9b9388f04a: Pull complete 
Digest: sha256:4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454
Status: Downloaded newer image for alpine:latest
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.15.4
PRETTY_NAME="Alpine Linux v3.15"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
$