Keep in sync the alpine image with the renovate repology datasource

[orlandoalexandrescu]

What would you like Renovate to be able to do?

Some of our docker images are based on an Alpine Linux version, like here:

FROM docker-upstream.artifactory.ourhostnetwork.net/postgres:11.17-alpine3.16 as upstream

In order to update alpine packages with renovate, we did this:

# renovate: datasource=repology depName=alpine_3_16/python3-dev versioning=loose
ARG PYTHON3_DEV_VERSION="3.10.5-r0"
# renovate: datasource=repology depName=alpine_3_16/py3-setuptools versioning=loose
ARG PY3_SETUPTOOLS_VERSION="59.4.0-r0"

We would like to keep the "3.16" bit from the image tag ("postgres:11.17-alpine3.16") in sync with the depname "alpine_3_16" bit ("depName=alpine_3_16/python3-dev").

Also, this would be useful as well in our "--repository" flag, passed to the apk command:

RUN apk add --update --no-cache \
                python3-dev=${PYTHON3_DEV_VERSION} \
                py3-setuptools=${PY3_SETUPTOOLS_VERSION} \
                --repository=https://artifactory.ourhostnetwork.net:443/artifactory/alpine-upstream/v3.16/main 

If you have any ideas on how this should be implemented, please tell us here.

A possible idea is to have an output variable of some kind, that can be configured in the renovate.json.

E.g. COMPATIBILITY_OS_VERSION, which for postgres:11.17-alpine3.16 will return 3.16.

The depName in the renovate comments would be automatically updated to "alpine_3_16".

Is this a feature you are interested in implementing yourself?

No

[github-actions[bot]]

Hi there,

Get your issue fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction.

We may close the issue if you, or someone else, haven't created a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[reitzig]

I'd propose resolving ${ARG_VAR} in regexManagers match strings -- but here that's not sufficient, as we see 3.16 vs 3_16. 🤔

[taki-sahli]

I was searching for the same thing today, the workaround that I'm using is like this,

I add a new customManagers to update alpine and repology repository:

  "customManagers": [
    {
      "description": "Update `alpine` repology",
      "customType": "regex",
      "fileMatch": "^build.env$",
      "matchStrings": [
        " renovate: dependency=alpine_(?<currentValue>(?<major>\\d+)_(?<minor>\\d+))/"
      ],
      "depNameTemplate": "docker.io/alpine",
      "datasourceTemplate": "docker",
      "versioningTemplate": "docker",
      "autoReplaceStringTemplate": " renovate: dependency=alpine_{{{replace '\\.' '_' newValue}}}/",
      "currentValueTemplate": "{{{major}}}.{{{minor}}}"
    }
  ]

And the file that contains the versions is like this:

# renovate: dependency=docker.io/alpine datasource=docker versioning=docker
ALPINE_VERSION="3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099"

# renovate: dependency=krallin/tini
TINI_VERSION="v0.19.0"

# renovate: dependency=alpine_3_21/gcc datasource=repology versioning=loose
GCC_VERSION="14.2.0-r4"

# renovate: dependency=alpine_3_21/icu datasource=repology versioning=loose
ICU_VERSION="74.2-r0"

# renovate: dependency=alpine_3_21/sqlite datasource=repology versioning=loose
SQLITE_VERSION="3.47.1-r0"

When there's an update, I got diff file like this:

# renovate: dependency=docker.io/alpine datasource=docker versioning=docker
ALPINE_VERSION="3.22.1@sha256:4bcff63911fcb4448bd4fdacec207030997caf25e9bea4045fa6c8c44de311d1"

# renovate: dependency=krallin/tini
TINI_VERSION="v0.19.0"

# renovate: dependency=alpine_3_22/gcc datasource=repology versioning=loose
GCC_VERSION="14.2.0-r4"

# renovate: dependency=alpine_3_22/icu datasource=repology versioning=loose
ICU_VERSION="74.2-r0"

# renovate: dependency=alpine_3_22/sqlite datasource=repology versioning=loose
SQLITE_VERSION="3.47.1-r0"

Then I manually fix the alpine packages version by checking in https://pkgs.alpinelinux.org/packages?name=&branch=v3.22&repo=&arch=x86_64&origin=&flagged=&maintainer=

In case there's future updates of alpine packages, I group them together:

  "packageRules": [
    {
      "groupName": "`alpine packages`",
      "matchDatasources": [
        "repology"
      ],
      "matchPackageNames": [
        "alpine_*/*"
      ]
    }
  ]

If anyone has a better way, please let me know.
Thank you.

[gizero]

Hi @taki-sahli! Thanks for sharing your config. I'm facing the same problem and have been thinking to a similar solution. I see how your approach can help, but I'm not sure how you are going to orchestrate renovate's PRs when a new alpine version comes out.

Given renovate PRs go through a quite standard CICD pipeline that tries to build the image to be successful, how would you order merging the relevant PRs? I can imagine something like this:

1. As soon as a new alpine release comes out renovate will open two PRs: one for updating alpine base image version to the new distro version (pipeline fails) and the Update alpine repology one from you custom manager (no code changes, pipeline succeeds)
2. The Update alpine repology first can be merged
3. Wait for the next renovate run to suggest updating alpine_*/* packages to versions from the new alpine distro version (pipeline will fail because of the base image version still being the old one)
4. Retarget PR from 4 against the one that updates alpine base image version to merge their changes (pipeline will succeed)

Is this the process you are using to complete the distro upgrade within your dockerfiles?

[taki-sahli]

Hello @gizero,

Actually no, because my process requires manually fixing the pr,

in 1 the custom manager will update the repology against old version of alpine.
so 2 if it was merged or not it doesn't matter, because the packages will still be outdated since renovate will the use old base version.
In 3 indeed, the merge will fail since the packages doesn't match the new version, so I checkout in git to the renovate branch, and manually I update the versions, then I force push.
This will make the pull request pipeline succeeds.
There's no step 4, because the pr for repology doesn't match the pr for upgrading alpine version.

My process only makes sure that alpine_*/* matches alpine's docker version, because I already have a repo with unmatched versions.

But after this, the alpine packages will be auto updated in renovate since they target the new base version.
If a new version of alpine is out, I need to intervene to fix the pr.

1. The PR (the alpine docker version and alpine_*/* are grouped together):

2. Pipeline fails:

3. I manually fix the versions and force push:

4. Pipeline fixed and the PR can be merged:

[reitzig]

Hm, if there was a way to re-run (some) managers on that file (until no further changes manifest) ... 🤔 I guess postUpgradeTasks is there, and maybe we could shell out to renovate itself with a tailored config (one manager, one file)? 🤔