Error when mirroring from Harbor registry

[pat-s]

Trying to set up a mirror from a Harbor registry results in the following error:

skopeo-list tags failed: [...] invalid reference format

The Harbor registry is controlled by myself and on the latest version (2.13). I can pull image from the repository just fine.
I am not in control of the Quay registry and cannot tell which version it is on.

I can't really find anything helpful on the web related to this error. Any help would be appreciated!

[mtrmac]

Thanks for reaching out.

What version of Skopeo is this? I’d expect a more specific error (registry returned invalid tag …).

[pat-s]

I don't know unfortunately, as I am just an unprivileged user in the environment where Quay is running. I could reach out to the admins though.

Also tried to infer it through the API but didn't have a token for it. Any way I can "easily" find out?

[mtrmac]

I mean, if you have access neither to the Skopeo version shipped, nor to the registry (to possibly run your own Skopeo), I don’t know what diagnostic steps you could possibly take.

It’s likely that the cause is that the repository contains a tag that is not valid for images. But without a more specific error, or debug-level logging output, there’s no way to tell for sure.

[pat-s]

I have tried the following from a personal VM of mine

skopeo --version
skopeo version 1.18.1

skopeo --debug list-tags \
  --authfile ~/.docker/config.json \
  docker://<repo>

    "Tags": [
        "2025-08-01-rhel9-2025.07.0-preprod",
        "2025-08-01-rhel9-2025.07.0-prod",
        "cache",
        "cache-preprod"
    ]

Given that this works, does it makes sense to bisect older skopeo versions until I find a potential match for the error thrown by quay?

[mtrmac]

If the current version of Skopeo works, we are not very likely to fix an older version of Skopeo without coordination with Quay — and it wouldn’t help you if you are not an admin of the Quay instance anyway. The way to fix that would probably be to either update Quay directly to some version that contains an updated Skopeo, or perhaps to coordinate a backport with the Quay product (I have no idea what the versions involved are and whether that’s an option).

Sure, it would still be relevant to figure out out which Skopeo version is required.

There haven’t been that many changes to the tag listing code over time; there is only one cause where invalid tags no longer cause a failure ( #2409 ), and that situation should be reported in the skopeo --debug output. (The output above quotes none of the debug logs.)

[pat-s]

Yeah I understand.

Interestingly, I couldn't reproduce the error with the docker images down to v1.1.0.

This is strange, I don't expect the quay registry in question to run an even older version of skopeo than this version.
I am aware that RH products are rather conservative with their updates and the same often goes for internally hosted registries, but...

In the quay reg in question, I have to use a proxy for public requests but that shouldn't play a role in all of this.

[mtrmac]

It’s possible that the root cause is something else. Maybe the host name of the sync source / destination has a typo in it, or is somehow formatted incorrectly by Quay? invalid reference format without quoting the invalid string is less than helpful in that respect.

[pat-s]

I double-checked everything multiple times already, will check again. But yeah, unfortunately I don't see anything else in the one-line stderr output from quay than this. At least, without access to the admin logs.

I suspect that the admins are not happy when I ask them to debug this for Harbor and will just say: "Use DockerHub". Anyhow, thanks again for your help, I think there's no way around asking the admins or doing more hello-world debug examples.

[mtrmac]

If you track the cause down, I’d be very interested in hearing about it, to make sure the error is enhanced with more context.

[pat-s]

I meanwhile found out that Quay 3.8.9 is in use which seems to be running skopeo 1.10 (from 2022).

[github-actions]

A friendly reminder that this issue had no activity for 30 days.