Skip to content

fix(replication): Normalise CORE_URL comparison in isLocalHarbor - #873

Merged
bupd merged 2 commits into
mainfrom
fix/islocalharbor-core-url-normalize
Sep 14, 2026
Merged

bupd merged 2 commits into
mainfrom
fix/islocalharbor-core-url-normalize

Conversation

@bupd

@bupd bupd commented Sep 11, 2026

Copy link
Copy Markdown
Member

Stacked PR — merge bottom-up. All four come from the 2.15.3 → 2.15.8 integration test.

  1. fix(replication): Normalise CORE_URL comparison in isLocalHarbor #873 — isLocalHarbor CORE_URL normalisation (base main) → this PR
  2. fix(api): Reject query operands the column cannot take #874 — query operand validation (base fix/islocalharbor-core-url-normalize)
  3. fix(registry): Answer 400 when the endpoint health check cannot reach the URL #875 — registry health check 400 (base fix/query-filter-value-validation)
  4. fix(db): Widen execution.revision to bigint #876 — execution.revision bigint (base fix/registry-health-check-bad-request)

The four changes touch disjoint files and do not depend on each other; the stack only fixes a review and merge order.
Merge bottom-up and let GitHub walk each base down. Do not delete a base branch before the PR above it has been retargeted — on this repo that closes the dependent PR permanently.

What

isLocalHarbor compared the replication registry URL to CORE_URL as a raw string:

func isLocalHarbor(url string) bool { return url == os.Getenv("CORE_URL") }

Core stamps the local replication source URL from its own CORE_URL; jobservice, which runs the replication job, compares that value against its own. When the two differ only by an explicit default port the compare fails, the secret authorizer is skipped, and the adapter falls back to basic auth with an empty username.

The effect is asymmetric and easy to misread: public projects keep replicating because anonymous pull is allowed, while every private project fails with 401 unauthorized to access repository: <repo>, action: pull.

How it was found

A 2.15.3 → 2.15.8 tenant upgrade on a real deployment. The chart renders http://harbor-core for both components, but the SBOM workaround for images below v2.15.5 sets core.extraEnv CORE_URL=http://harbor-core:80 — so only core's value carried the port. web-proxy (public) replicated Succeed 4/4 while base-os (private) Failed 1/1, both to a remote registry and to itself. Setting jobservice's CORE_URL to the identical string made the private policy Succeed 1/1 immediately.

Reproduced on 2.15.8

Confirmed independently on a Harbor Next 2.15.8 stood up on k3d from this chart, replicating to a plain registry:2 destination — the failure is on the replication source, so no second Harbor is needed. All cases run back-to-back on one cluster.

CORE_URL: core / jobservice private project public project
default chart (both portless) Succeed 1/1 Succeed 1/1
:80 / default Failed 0/1, 401 Succeed 1/1
:80 / :80 Succeed 1/1 Succeed 1/1

The asymmetry is the tell: only the private project fails, because the fallback to anonymous basic auth still satisfies a public project's anonymous pull.

jobservice, in the failing case:

[INFO]  transfer.go:140: client for source registry [type: harbor, URL: http://harbor-core:80, insecure: true] created
[ERROR] transfer.go:468: failed to pull the manifest of artifact privproj/busybox: http status code: 401,
        body: {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized to access repository: privproj/busybox, action: pull"}]}

The adapter.go:39 warning is a clean oracle for which branch ran:

  • default config — Detected LOCAL Harbor instance (URL=http://harbor-core matches CORE_URL) present, both projects succeed
  • one-sided :80 — warning absent, private project 401s
  • both :80 — warning returns as URL=http://harbor-core:80 matches CORE_URL, zero errors

Chain confirmed live: core stamps the local registry URL from its own CORE_URL (getLocalRegistry() in src/pkg/reg/manager.go:171config.InternalCoreURL()); jobservice compares that against its own os.Getenv("CORE_URL"). On mismatch adapter.New skips the secret authorizer and falls back to basic.NewAuthorizer(AccessKey, ...), where AccessKey is empty for CredentialTypeSecret — anonymous. Public projects still replicate, private ones 401.

Fix

Parse both URLs and compare scheme, host and port, with :80 made explicit for http and :443 for https. Trailing slashes are ignored. If either side fails to parse, or has no scheme or host, the exact string compare still decides — a bare harbor-core parses into Path with empty scheme and host, which would otherwise compare equal to an unset CORE_URL.

This is the same equivalence the v2auth middleware already applies when matching a request host against the configured URL (match() in src/server/middleware/v2auth/auth.go:141), reached from the other direction: that one strips a default port, this one adds it. isLocalHarbor was the last raw == comparison of this kind left in the tree. Whether the two should share one helper is worth doing, but not inside a bugfix that would then touch the auth middleware — filed as #880 rather than bundled here.

Test

TestIsLocalHarbor, 18 cases. The two that encode the reproduced failure:

// registry.URL "http://harbor-core:80" (stamped by core) vs CORE_URL "http://harbor-core" (jobservice) -> want true
{"core pins the port through extraEnv, jobservice does not", "http://harbor-core", "http://harbor-core:80", true},
{"jobservice pins the port through extraEnv, core does not", "http://harbor-core:80", "http://harbor-core", true},

plus both https/:443 directions, trailing slash, uppercase host, non-default ports, different host, different scheme, different path, unparseable and schemeless input, and an empty CORE_URL.

$ go test ./pkg/reg/adapter/harbor/...
ok  	github.com/goharbor/harbor/src/pkg/reg/adapter/harbor/base	0.008s

Not changed

The chart already renders one consistent CORE_URL for both components:

$ helm template t deploy/chart --set externalURL=... | grep CORE_URL | sort | uniq -c
      2 CORE_URL: "http://t-harbor-core"

The divergence only appears when an operator overrides core's value through core.extraEnv. That workaround is obsolete at HEADaccessoryRef in src/pkg/scan/util.go now handles a portless single-label host, which was the reason for pinning :80 in the first place. Confirmed on a 2.15.8 cluster: with the default portless config and no override anywhere, SBOM generation on a private project succeeds and pushes its sbom.harbor accessory (28447 B).

Remediation for a tenant already carrying the override — order matters

The override is still load-bearing below v2.15.5. Do not delete it as a standalone values cleanup.

It works around SBOM push sending robot credentials to Docker Hub when CORE_URL is portless, fixed by #465 ("Keep SBOM accessory push on the local registry when CORE_URL has no port") in v2.15.5. Every tenant known to carry the override is unpinned and therefore at the global harbor_version: v2.15.3 — below that fix. The 2.15.8 evidence above says the workaround is dead at 2.15.8; it says nothing about where those tenants are today. Delete it from a v2.15.3 tenant and SBOM push regresses: one silent breakage traded for another.

On a build that predates this fix, removal and the move to ≥ v2.15.5 must be one atomic change — delete early and SBOM push regresses on v2.15.3; delete late and private replication stays broken in between.

The two halves of that trap rest on different kinds of evidence, which is worth keeping straight:

  • Delete latereproduced live. The :80 / default row in the matrix above was run on 2.15.8, i.e. already past fix(scan): Keep SBOM accessory push on the local registry when CORE_URL has no port #465 and without this fix, and private replication failed 401 while public succeeded.
  • Delete earlyestablished from the code at the tags, not measured. Nobody ran SBOM on a v2.15.3 deployment with a portless CORE_URL. What is verifiable is the mechanism: at v2.15.3, GenAccessoryArt built the reference as name.ParseReference(fmt.Sprintf("%s/%s@%s", sq.Registry.URL, ...)). ParseReference only treats the first path segment as a registry if it contains a . or a :, so a portless single-label host like harbor-core parses as a Docker Hub namespace and the push leaves the cluster. v2.15.5 replaced that with name.NewRegistry(registryURL) + reg.Repo(repository) and added the portless single-label host stays the registry test case — absent at v2.15.3 and v2.15.4, present at v2.15.5 and v2.15.8.

A build carrying this fix removes the need to coordinate at all. The one-sided mismatch becomes inert, so the upgrade can be taken on its own with the override left in place, and the values cleanup scheduled separately as ordinary hygiene.

That regime does not yet exist for any affected tenant, and merging this PR alone will not create it. This PR targets main. Every tenant carrying the override is on the 2.15 line and their realistic target is v2.15.8 — also a 2.15 build. So the relaxation above is conditional on a release-2.15 backport, not on this merging.

The precedent is #465 itself, the fix this whole argument leans on: it needed both a main PR (#465, merged 2026-07-21) and a release-2.15 backport (#581, merged 2026-08-07) to reach v2.15.8. This change currently has only the main half, and no backport PR for it exists.

So: a release-2.15 backport is required for this fix to reach any currently-affected tenant. Until one lands and those tenants are moved onto a build carrying it, the pre-fix atomic constraint governs their v2.15.8 upgrade unchanged. Not opened here because the repo's pattern is to backport after the main PR merges — see #867, #868, #870, all titled (backport #N).

End state is still deleted from both components, not mirrored onto jobservice. Mirroring works — the :80 / :80 row above succeeds — but it pins an obsolete workaround into the tenant files and re-breaks the moment someone edits one side.

Tenant list and the pre-fix constraint: herdr-cc/tasks/featgate-status.md §5.

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 39 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: c94fd4cd-d2ef-49e7-af8a-a25e209ac4cd

📥 Commits

Reviewing files that changed from the base of the PR and between 6965746 and 76997b4.

📒 Files selected for processing (2)
  • src/pkg/reg/adapter/harbor/base/adapter.go
  • src/pkg/reg/adapter/harbor/base/adapter_test.go

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the tests label Sep 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This change may need patch-release backports. Comment with one of these commands to open a cherry-pick PR:

/backport v2.15

@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@bupd
bupd force-pushed the fix/islocalharbor-core-url-normalize branch from 7d36fd9 to 57ad94b Compare September 11, 2026 07:22

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/pkg/reg/adapter/harbor/base/adapter.go
@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Warning

Outdated: this PR is not the top of stack #907, so the stack's preview image is published on #876. What follows is from an earlier revision of this PR.

Preview images for this PR are available in 8gears.container-registry.com/8gcr-pr with tag pr-873, built from 3453274:

  • 8gears.container-registry.com/8gcr-pr/harbor-core:pr-873
  • 8gears.container-registry.com/8gcr-pr/harbor-jobservice:pr-873
  • 8gears.container-registry.com/8gcr-pr/harbor-registryctl:pr-873
  • 8gears.container-registry.com/8gcr-pr/harbor-exporter:pr-873
  • 8gears.container-registry.com/8gcr-pr/harbor-portal:pr-873
  • 8gears.container-registry.com/8gcr-pr/harbor-registry:pr-873
  • 8gears.container-registry.com/8gcr-pr/trivy-adapter:pr-873

Verify a preview image:

cosign verify \
  --certificate-identity-regexp="https://github.com/container-registry/harbor-next/.github/workflows/pr-ci.yml@.*" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  8gears.container-registry.com/8gcr-pr/harbor-core:pr-873

Verify the SBOM attestation:

cosign verify-attestation \
  --certificate-identity-regexp="https://github.com/container-registry/harbor-next/.github/workflows/pr-ci.yml@.*" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  --type spdxjson \
  8gears.container-registry.com/8gcr-pr/harbor-core:pr-873

@bupd
bupd force-pushed the fix/islocalharbor-core-url-normalize branch from 57ad94b to 3453274 Compare September 11, 2026 08:46
@bupd bupd added prio/P1 Wanted this cycle; schedule it into a merge wave state/needs-review Green and rebased; waiting on a human labels Sep 14, 2026
@bupd
bupd added this pull request to stack #907 September 14, 2026 07:42
Core stamps the local replication source URL from its own CORE_URL and
jobservice compares the result against its own. A deployment that sets an
explicit default port on only one of the two — http://harbor-core versus
http://harbor-core:80, which the documented SBOM workaround did — made the
raw string compare fail. The secret authorizer was then skipped and the
adapter fell back to basic auth with an empty username, so every private
project failed replication with 401 while public ones still worked through
anonymous pull.

Parse both URLs and compare scheme, host and port with :80 and :443 made
explicit for http and https. If either side fails to parse, or carries no
scheme or host, the exact string compare still decides.

Signed-off-by: Prasanth Baskar <prasanth@8gears.com>
@bupd
bupd force-pushed the fix/islocalharbor-core-url-normalize branch from 3453274 to d57e422 Compare September 14, 2026 07:42
@bupd bupd removed the state/needs-review Green and rebased; waiting on a human label Sep 14, 2026
Userinfo, query, or fragment now only match through the exact string
compare, so a registry URL carrying embedded credentials can no longer
be classified local by the normalized path.

Signed-off-by: Prasanth Baskar <prasanth@8gears.com>
@bupd

bupd commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

Addressed the review and tightened the code: normalization now applies only to bare scheme://host[:port][/path] URLs (new bareURL guard) — userinfo, query, or fragment must match through the exact string compare, so a URL with embedded credentials can no longer be classified local via the normalized path. Comments trimmed to the essentials, error/shape guards folded together, three test cases added for the userinfo/query paths.

@bupd
bupd merged commit 51dda34 into main Sep 14, 2026
24 checks passed
@bupd
bupd deleted the fix/islocalharbor-core-url-normalize branch September 14, 2026 07:55
@bupd

bupd commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

/backport v2.15

@github-actions

Copy link
Copy Markdown
Contributor

Opened backport PR for release-2.15: #928

bupd added a commit that referenced this pull request Sep 16, 2026
Signed-off-by: Prasanth Baskar <prasanth@8gears.com>
(cherry picked from commit 51dda34)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@github-actions github-actions Bot mentioned this pull request Sep 14, 2026
bupd added a commit that referenced this pull request Sep 16, 2026
## Release Notes

## Commercial Features

_No changes this release._

## What's Changed

### Fixes

- **api:** Reject query operands the column cannot take (backport
[#925](#925))
([30d06ea](30d06ea))
- **compose:** Preserve the request port in the nginx Host header in
[#926](#926)
([310d6e0](310d6e0))
- **db:** Widen execution.revision to bigint (backport
[#924](#924))
([1d5719e](1d5719e))
- **deps:** Bump go dependencies minor versions (backport
[#895](#895)) by
@bupd in
[#899](#899)
([a4b7e1c](a4b7e1c))
- **deps:** Bump vendored builds: trivy 0.74.0, adapter v0.41.0, pin
cleanup (backport
[#893](#893)) by
@bupd in
[#898](#898)
([5bcfece](5bcfece))
- **deps:** Upgrade Go to 1.26.8 (backport
[#892](#892)) by
@bupd in
[#897](#897)
([ff01ea1](ff01ea1))
- **quota:** Enable retry backoff on quota usage updates (backport
[#716](#716))
([75fab53](75fab53))
- **registry:** Answer 400 when the endpoint URL cannot be reached in
[#923](#923)
([07d4436](07d4436))
- **replication:** Normalise CORE_URL comparison in isLocalHarbor in
[#873](#873)
([49cd3cb](49cd3cb))
- **scan:** Keep the last report when the raw report is empty (backport
[#863](#863))
([1433e30](1433e30))

### Upstream

- add size limit for audit log payload
(goharbor/harbor[#23755](https://github.com/container-registry/harbor-next/issues/23755))
by @stonezdj in
[goharbor/harbor#23755](goharbor/harbor#23755)
([45e97fd](45e97fd))
- fix(i18n): improve Korean translation of FULL_NAME
(goharbor/harbor[#22701](https://github.com/container-registry/harbor-next/issues/22701))
by @InGyu-Moon in
[goharbor/harbor#22701](goharbor/harbor#22701)
([9ea0d71](9ea0d71))
- Prevent 500 if no project members have been provided
(goharbor/harbor[#23642](https://github.com/container-registry/harbor-next/issues/23642))
by @Yanni8 in
[goharbor/harbor#23642](goharbor/harbor#23642)
([c518ae7](c518ae7))

---

## Container Images

Multi-arch images (`linux/amd64`, `linux/arm64`) signed with
[cosign](https://github.com/sigstore/cosign).

| Image | Reference |
|-------|-----------|
| `harbor-core` |
`8gears.container-registry.com/8gcr/harbor-core:v2.15.9` |
| `harbor-jobservice` |
`8gears.container-registry.com/8gcr/harbor-jobservice:v2.15.9` |
| `harbor-registryctl` |
`8gears.container-registry.com/8gcr/harbor-registryctl:v2.15.9` |
| `harbor-exporter` |
`8gears.container-registry.com/8gcr/harbor-exporter:v2.15.9` |
| `harbor-portal` |
`8gears.container-registry.com/8gcr/harbor-portal:v2.15.9` |
| `harbor-registry` |
`8gears.container-registry.com/8gcr/harbor-registry:v2.15.9` |
| `trivy-adapter` |
`8gears.container-registry.com/8gcr/trivy-adapter:v2.15.9` |

**Verify an image signature:**
```sh
cosign verify \
  --certificate-identity "https://github.com/container-registry/harbor-next/.github/workflows/release-please.yml@refs/heads/release-2.15" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  8gears.container-registry.com/8gcr/harbor-core:v2.15.9
```

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Vadim Bauer <vadim@8gears.com>
Co-authored-by: Prasanth Baskar <prasanth@8gears.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working prio/P1 Wanted this cycle; schedule it into a merge wave tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant