feat: optimize ls-remote (#28650)

[kirillbilchenko]

Large repositories may advertise hundreds of thousands of refs even though most Applications only use branches and tags. This adds an opt-in repo-server path that uses native Git protocol v2 to query configured ref namespaces, avoiding unrelated refs such as pull-request refs while preserving the current behavior by default.

Fixes #28650.

Implementation of the proposal

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
• The title of the PR states what changed and the related issues number (used for the release note).
• The title of the PR conforms to the Title of the PR
• I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
• I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
• Does this PR require documentation updates?
• I've updated documentation as required by this PR.
• I have signed off all my commits as required by DCO
• I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
• My build is green (troubleshooting builds).
• My new feature complies with the feature status guidelines.
• I have added a brief description of why this PR is necessary and/or what this PR solves.
• Optional. My organization is added to USERS.md.
• Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

[bunnyshell]

❗ Preview Environment stop on Bunnyshell failed

See: Environment Details | Pipeline Logs

Available commands (reply to this comment):

• 🔴 /bns:stop to stop again the environment
• 🔵 /bns:start to start the environment
• 🚀 /bns:deploy to redeploy the environment
• ❌ /bns:delete to remove the environment

[codecov]

Bundle Report

Changes will increase total bundle size by 155 bytes (0.0%) ⬆️. This is within the configured threshold ✅

Detailed changes

Bundle name	Size	Change
argo-cd-ui-array-push	11.11MB	155 bytes (0.0%) ⬆️

Affected Assets, Files, and Routes:

view changes for bundle: argo-cd-ui-array-push

Assets Changed:

Asset Name	Size Change	Total Size	Change (%)
main.*.js	155 bytes	3.58MB	0.0%

[codecov]

Codecov Report

❌ Patch coverage is 79.56522% with 47 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (master@bf8bdaa). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
util/git/client.go	78.24%	32 Missing and 15 partials ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             master   #28667   +/-   ##
=========================================
  Coverage          ?   65.30%           
=========================================
  Files             ?      426           
  Lines             ?    60142           
  Branches          ?        0           
=========================================
  Hits              ?    39274           
  Misses            ?    17260           
  Partials          ?     3608           

Flag	Coverage Δ
e2e	26.82% <25.65%> (?)
unit-tests	60.76% <77.82%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[nandotorres]

Nice work! I'd like to propose one refinement that makes the design cover three goals at once without them being in tension:

1. Fast/efficient ls-remote via v2 (heads + tags narrowing).
2. Fall back to v0 (full advertisement) and pay the price if the server doesn't support v2. Fine also to don't implement a fallback, as the ArgoCD admin should know if v2 is supported on the server.
3. Still let users resolve a filtered/"ghost" ref (e.g. refs/pull/x/head) when they pass a fully-qualified ref even while v2 is enabled and we normally only fetch heads + tags.

The key insight is that these are not actually in conflict:

A fully-qualified ref is self-narrowing under v2.

git -c protocol.version=2 ls-remote <url> refs/pull/999/head sends refs/pull/999/head as a server-side ref-prefix and returns exactly that one ref (plus its ^{} peel). So a ghost ref requested explicitly is just as cheap as heads/tags and never has to go through the v0 full advertisement. The allow-list only needs to govern the bulk enumeration entry (used for names we cannot narrow in advance); it should not gate an explicit fully-qualified request.

---

Proposed routing in lsRemote(revision)

  1. full SHA?                    -> return as-is                     (no network)

  if optimized enabled:
  2. fully-qualified (refs/.../)? -> v2 TARGETED query for THAT ref     [goal #3]
        git -c protocol.version=2 ls-remote --symref <url> <revision>
        works for refs/pull/*, refs/merge-requests/*, anything;
        no allow-list membership required, still fast
        -> if the ref genuinely does not exist: return a real "not found" error
           (do NOT fall back to v0 — it cannot find it either)

  3. short name / semver constraint -> resolve from the shared            [goal #1]
        `--heads --tags` bulk entry (allow-listed namespaces, cached once per repo)
        -> miss (branch outside heads/tags, truncated SHA) -> fall to #4

  else / on v2 transport failure:
  4. v0 full advertisement (go-git) -> pay the 800k-ref price             [goal #2]

[kirillbilchenko]

Nice work! I'd like to propose one refinement that makes the design cover three goals at once without them being in tension:

1. Fast/efficient ls-remote via v2 (heads + tags narrowing).
2. Fall back to v0 (full advertisement) and pay the price if the server doesn't support v2. Fine also to don't implement a fallback, as the ArgoCD admin should know if v2 is supported on the server.
3. Still let users resolve a filtered/"ghost" ref (e.g. refs/pull/x/head) when they pass a fully-qualified ref even while v2 is enabled and we normally only fetch heads + tags.

The key insight is that these are not actually in conflict:

A fully-qualified ref is self-narrowing under v2.

git -c protocol.version=2 ls-remote <url> refs/pull/999/head sends refs/pull/999/head as a server-side ref-prefix and returns exactly that one ref (plus its ^{} peel). So a ghost ref requested explicitly is just as cheap as heads/tags and never has to go through the v0 full advertisement. The allow-list only needs to govern the bulk enumeration entry (used for names we cannot narrow in advance); it should not gate an explicit fully-qualified request.

Proposed routing in lsRemote(revision)

  1. full SHA?                    -> return as-is                     (no network)

  if optimized enabled:
  2. fully-qualified (refs/.../)? -> v2 TARGETED query for THAT ref     [goal #3]
        git -c protocol.version=2 ls-remote --symref <url> <revision>
        works for refs/pull/*, refs/merge-requests/*, anything;
        no allow-list membership required, still fast
        -> if the ref genuinely does not exist: return a real "not found" error
           (do NOT fall back to v0 — it cannot find it either)

  3. short name / semver constraint -> resolve from the shared            [goal #1]
        `--heads --tags` bulk entry (allow-listed namespaces, cached once per repo)
        -> miss (branch outside heads/tags, truncated SHA) -> fall to #4

  else / on v2 transport failure:
  4. v0 full advertisement (go-git) -> pay the 800k-ref price             [goal #2]

Thanks for the suggestion. I tested this with Git packet tracing, and unfortunately git ls-remote refs/pull/... does not send the ref as a v2 ref-prefix; the server still advertises all refs and Git filters
locally. Only --heads and --tags provide real server-side narrowing.

For this PR, I’d keep the optimized heads/tags path and the existing full fallback for custom refs. True targeted custom-ref support would require lower-level ls-refs handling and is better suited to a follow-up.

[kirillbilchenko]

Applied the fix on staging env, and has visible difference now, this is much much smaller scale but still playing a role

[kirillbilchenko]

@agaudreault sorry for tagging you, I just saw you did a review for the pr that I referenced about cache, so maybe this is something you can also check if this one makes sense?