Skip to content

Commit 1131f32

Browse files
CypressXtCypressXt
authored
Ansible runner deployment (#8)
* Initial ansible deployment playbook for runners (deb|rpm based OS) * Added ansible playbook documentation * Fixed 'when' typo, added gitignore for ansible * Reduce amount of podman pull * Updated ansible documentation * touch empty env file only if non-existing
1 parent c1e74c1 commit 1131f32

14 files changed

Lines changed: 295 additions & 0 deletions

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
ansible/host.ini
2+
ansible/vars/runner.yml

‎README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,8 @@
11
This repo contains the Dockerfiles for the Meshtastic self-hosted runners.
2+
3+
4+
5+
6+
## Ansible deployment
7+
8+
Runners can be deployed using ansible, check the [documentation here](./ansible/README.md).

‎ansible/README.md‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# Github runner ansible deployment
2+
3+
This ansible deployment allows you to deploy self-hosted github runner used by the [meshtastic firmware project](https://github.com/meshtastic/firmware).
4+
Those runners are building firmware for meshtastic devices.
5+
6+
## What do I need to use it ?
7+
8+
### On your local machine
9+
10+
To be able to use this playbook you need have [Ansible installed](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) on your local machine.
11+
A github token provided by the meshtastic crew is also required.
12+
13+
### On your runner's remote machine
14+
15+
Currently only a debian or fedora based system is supported.
16+
You need to have a SSH remote access user able to escalate using sudo.
17+
This is required as this playbook will install [required tools](./roles/basic-os/tasks/basic-os.yml#L1-L22) and configure your system to be able to run podman containers (the actual github runner containers).
18+
19+
## What does it do ?
20+
21+
* create an unprivileged `github` user: [here](./roles/basic-os/tasks/basic-os.yml#L24-L29)
22+
* configure podman: [here](./roles/basic-os/tasks/basic-os.yml#L31-L47)
23+
* create a podman container for each runner: [here](./roles/runner/tasks/github-runner.yml#L20-34)
24+
* create a systemd service for each runner container: [here](./roles/runner/tasks/github-runner.yml#L36-55)
25+
* create a script for creating runner secrets on each run [here](./roles/runner/tasks/github-runner.yml#L1-9)
26+
* create a script for checking every 24hrs if an update is available for the podman image `ghcr.io/meshtastic/gh-runners:latest`: [here](./roles/runner/tasks/github-runner.yml#L11-18)
27+
* create a secret file for each runner container: [here](./roles/runner/tasks/github-runner.yml#L1-18)
28+
29+
In your newly created `github` user's home, you will find 2 files per runners and one file common for all of them:
30+
```
31+
github@runner-meshtastic:~$ ls -la
32+
.check_podman_image.sh # podman image update check script (common for all runners)
33+
.<your-runners-name>-meshtastic.env # current running job secrets
34+
.<your-runners-name>-meshtastic.make_env.sh # script generating the "curent running job secrets" file
35+
```
36+
37+
Secrets are generated for each runner container by the `.<your-runners-name>-meshtastic.make_env.sh` script triggered by the [systemd service](./roles/runner/files/github-runner-meshtastic.service.j2#L10) and are written in `.<your-runners-name>-meshtastic.env`.
38+
39+
##### Typical operation
40+
1. Before a container is started, the file `.<your-runners-name>-meshtastic.make_env.sh` is executed and write token values in `.<your-runners-name>-meshtastic.env`
41+
2. Before a container is started, the file `.check_podman_image.sh` is executed and check every 24hrs if the podman image `ghcr.io/meshtastic/gh-runners:latest` has an updated available
42+
3. The container starts, the file `.<your-runners-name>-meshtastic.env` is picked up inside the container [as a volume](./roles/runner/tasks/github-runner.yml#L30)
43+
4. The container authenticate on github using the secrets stored in `.<your-runners-name>-meshtastic.env`
44+
5. The meshtastic firmware job is executed
45+
6. The container terminates
46+
7. If the container [terminated smoothly](./roles/runner/files/github-runner-meshtastic.service.j2#L16), back to step `1.`
47+
48+
## What do I need to adjust before deployment ?
49+
50+
* In the [host.ini](./hosts.ini) file add the hostname FQDN on which you wish to install runners.
51+
* Adjust the [the runner.yml vars](./vars/runner.yml) accordingly:
52+
```
53+
github_token: meshtastic-github-token # Provided by the meshtastic crew
54+
user_nickname: github-username # Your github nickname
55+
runner_servers:
56+
your-first-system-fqdn.tld: # Your host as written in the host.ini file
57+
runners:
58+
- name: runner-1 # The name of your runner
59+
runner_location: datacenter-alpha # The location of your runner
60+
- name: runner-2
61+
runner_location: datacenter-bravo
62+
your-second-system-fqdn.tld:
63+
runners:
64+
- name: runner-3
65+
runner_location: datacenter-charlie
66+
- name: runner-4
67+
runner_location: datacenter-delta
68+
```
69+
70+
## How to trigger the deployment ?
71+
72+
You're now ready to deploy your runners !
73+
74+
You can trigger a "dry-run" and visualize the changes without applying them like this:
75+
```
76+
ansible-playbook -i hosts.ini runner-meshtastic.yml -DKC
77+
```
78+
79+
If you're happy with the result, trigger and apply changes as follow:
80+
```
81+
ansible-playbook -i hosts.ini runner-meshtastic.yml -DK
82+
```
83+
84+
## And then ?
85+
86+
Do a little review of the system once again to ensure that files and services are populated properly.
87+
Once confident, you can start your runners for the first time like this:
88+
89+
```
90+
sudo systemctl start github-<your-runner-1-name>-meshtastic.service
91+
sudo systemctl start github-<your-runner-2-name>-meshtastic.service
92+
...
93+
```
94+
95+
If you need to debug something you can check your runner logs like that:
96+
```
97+
sudo -u github podman logs --tail=40 -f <your-runner-name>-meshtastic
98+
```

‎ansible/ansible.cfg‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[defaults]
2+
allow_world_readable_tmpfiles = true # necessary to allow unprivileged user with ansible

‎ansible/hosts.ini‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[all]
2+
your-system-fqdn.tld

‎ansible/roles/basic-os/tasks/basic-os.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
- name: "[Basic-OS] Install default tools for common OS"
2+
ansible.builtin.package:
3+
name:
4+
- podman
5+
- jq
6+
- curl
7+
- xxd
8+
- acl
9+
state: latest
10+
11+
- name: "[Basic-OS] Install specific tools for deb-based OS"
12+
ansible.builtin.package:
13+
name:
14+
- bsdmainutils
15+
state: latest
16+
when: ansible_distribution == 'Debian' or ansible_distribution == 'Ubuntu'
17+
18+
- name: "[Basic-OS] Install specific tools for rpm-based OS"
19+
ansible.builtin.package:
20+
name:
21+
- slirp4netns
22+
state: latest
23+
when: ansible_distribution == 'CentOS' or ansible_distribution == 'Fedora'
24+
25+
- name: "[Basic-OS] Create github user"
26+
ansible.builtin.user:
27+
name: github
28+
shell: /bin/bash
29+
password_lock: yes
30+
create_home: yes
31+
32+
- name: "[Basic-OS] Check if github user is lingering"
33+
ansible.builtin.stat:
34+
path: "/var/lib/systemd/linger/github"
35+
register: user_lingering
36+
37+
- name: "[Basic-OS] Enable lingering for github"
38+
ansible.builtin.command: "loginctl enable-linger github"
39+
when:
40+
- not user_lingering.stat.exists
41+
42+
- name: "[Basic-OS] Set user.max_user_namespaces"
43+
ansible.posix.sysctl:
44+
name: user.max_user_namespaces
45+
value: '28633'
46+
state: present
47+
reload: true
48+
sysctl_file: /etc/sysctl.d/userns.conf

‎ansible/roles/basic-os/tasks/main.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
- name: basic-os
2+
include_tasks:
3+
file: basic-os.yml
4+
apply:
5+
become: yes
6+
tags:
7+
- basic-os

‎ansible/roles/runner/files/check_podman_image.sh.j2‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/bin/bash
2+
3+
SECS=`date +%s`
4+
IMAGE_EXPIRATION=86400
5+
IMAGE_UPDATED_FILE_PATH=/home/github/.image-updated
6+
PODMAN_IMAGE=ghcr.io/meshtastic/gh-runners:latest
7+
8+
if [ ! -f $IMAGE_UPDATED_FILE_PATH ]; then
9+
echo $SECS > $IMAGE_UPDATED_FILE_PATH
10+
fi
11+
OLDSECS=`cat $IMAGE_UPDATED_FILE_PATH`
12+
if [ $SECS -gt `expr $OLDSECS + $IMAGE_EXPIRATION` ] ; then
13+
podman pull $PODMAN_IMAGE
14+
echo $SECS > $IMAGE_UPDATED_FILE_PATH
15+
fi

‎ansible/roles/runner/files/github-runner-meshtastic.service.j2‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
[Unit]
2+
Description=Meshtastic Github {{item.name}}
3+
Documentation=man:podman-generate-systemd(1)
4+
Wants=network-online.target
5+
After=network-online.target
6+
RequiresMountsFor=/var/run/container/storage
7+
8+
[Service]
9+
ExecStartPre=/home/github/.check_podman_image.sh
10+
ExecStartPre=/home/github/.{{item.name}}-meshtastic.make_env.sh
11+
ExecStart=/usr/bin/podman start -a {{item.name}}-meshtastic
12+
ExecStop=/usr/bin/podman stop -t 10 {{item.name}}-meshtastic
13+
User=github
14+
Group=github
15+
Type=simple
16+
Restart=on-success
17+
18+
[Install]
19+
WantedBy=default.target

‎ansible/roles/runner/files/make_env.sh.j2‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/bin/bash
2+
3+
env_file_path=/home/github/.{{item.name}}-meshtastic.env
4+
bearer={{ github_token }}
5+
6+
TOKEN=$(curl -L -sS -X POST -H "Accept: application/vnd.github+json" -H "Authorization: Bearer $bearer" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/orgs/meshtastic/actions/runners/registration-token | jq '.token')
7+
printf "export UID_RANDOM=`xxd -p -l2 /dev/urandom`\nexport TOKEN=$TOKEN\n" | tr -d '"' > $env_file_path

0 commit comments

Comments
 (0)