Make the NATS Broker control plane scalable

[kahirokunn]

Why

The shared Broker ingress handles traffic for every NATS JetStream Broker in the cluster, but its Deployment was fixed at one replica. The Broker controller already supports leader election, but the manifest also fixed its replica count and did not express a scheduling preference for additional replicas.

The manifests should keep the one-replica Kubernetes default while allowing the ingress and controller to scale independently.

Proposed Changes

• Add CPU-based autoscaling for the shared Broker ingress.
• Allow the Broker controller to scale independently, with replicas preferably scheduled across nodes.

Release Note

The NATS Broker shared ingress now autoscales based on CPU utilization, with support for operator-managed scaling and replica spreading across nodes.

[knative-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kahirokunn
Once this PR has been reviewed and has the lgtm label, please assign matzew for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.50%. Comparing base (2c86680) to head (898382c).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #798   +/-   ##
=======================================
  Coverage   43.50%   43.50%           
=======================================
  Files          68       68           
  Lines        4259     4259           
=======================================
  Hits         1853     1853           
  Misses       2282     2282           
  Partials      124      124           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kahirokunn]

/ok-to-test

[kahirokunn]

@astelmashenko Hi ✋
If “replicas” is omitted, the default value is 1.
Since omitting it allows integration with HPA and KEDA, what do you think?
Thank you 🙏

[astelmashenko]

@kahirokunn , thank you for the PR. I'm sorry, but I'm busy on this week, will try to find time on the next week to review all the PRs.

[knative-prow-robot]

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.