feat(delivery): support maximum backoff duration

[kahirokunn]

Related upstream: knative/eventing#9278, implemented by knative/eventing#9279.

Proposed Changes

This PR adds native NATS support for Eventing's experimental DeliverySpec.backoffMax. The Eventing revision selected by go.mod is the merge commit of knative/eventing#9279, which defines backoffMax and retryAfterMax and their feature-gated validation.

• Apply backoffMax to Broker and Trigger delivery, and expose it through NatsJetStreamChannel defaults and subscriber delivery.
• Run the complete upstream DeliverySpec validation with Eventing feature flags available to the channel webhook.
• Map JetStream's one-based delivery count to Eventing's zero-based backoff attempt, and pass subscriber response status and headers into Eventing's backoff calculation.

Retry behavior

Field	Controls
backoffMax	Delays calculated from backoffDelay and backoffPolicy
retryAfterMax	Delays requested by Retry-After on HTTP 429 and 503 responses

The limits are independent. Eventing caps each delay separately and uses the larger result. Operators enable delivery-backoff-max, and additionally enable delivery-retryafter when using retryAfterMax.

Tests

Unit tests cover retry numbering, delivery propagation, feature-gated validation, and response-aware delay calculation. Experimental E2E tests cover backoffMax and retryAfterMax through both the NATS Broker and NatsJetStreamChannel, following Eventing's upstream backoffMax scenario and Retry-After assertions.

The KinD workflow follows Eventing's Kubernetes and suite matrix, Kind setup, and experimental-suite execution.

Eventing installation	Suites	Kubernetes
knative-v1.23.0 release	Regular E2E	v1.34.x, v1.35.x
Revision selected by go.mod	Regular and experimental E2E	v1.34.x, v1.35.x

Release Note

🎁 Add experimental DeliverySpec.backoffMax support to native NATS Broker and NatsJetStreamChannel delivery, and honor retryAfterMax when scheduling JetStream redeliveries.

Docs

Provider documentation is included for Broker delivery and NatsJetStreamChannel delivery. Upstream user documentation is tracked in knative/docs#6686.

[knative-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kahirokunn
Once this PR has been reviewed and has the lgtm label, please assign aliok for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

❌ Patch coverage is 66.34615% with 70 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.83%. Comparing base (0e4ba1f) to head (f85abce).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
test/experimental/features/delivery/feature.go	44.76%	53 Missing and 5 partials ⚠️
pkg/broker/filter/handler.go	0.00%	4 Missing ⚠️
test/experimental/features/delivery/matchers.go	88.88%	2 Missing and 2 partials ⚠️
pkg/webhook/controller.go	0.00%	3 Missing ⚠️
...channel/jetstream/dispatcher/message_dispatcher.go	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #802      +/-   ##
==========================================
+ Coverage   43.50%   44.83%   +1.33%     
==========================================
  Files          68       72       +4     
  Lines        4259     4447     +188     
==========================================
+ Hits         1853     1994     +141     
- Misses       2282     2321      +39     
- Partials      124      132       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[astelmashenko]

@kahirokunn , AFAIK knative-eventing does not support DeliverySpec.backoffMax please check https://knative.dev/docs/eventing/reference/eventing-api/#duck.knative.dev/v1.DeliverySpec

[kahirokunn]

@astelmashenko Thank you for your review 🙏
backoffMax is supported on Eventing main since knative/eventing#9279, and eventing-natss already depends on that merge commit via #804. It is an Alpha feature guarded by delivery-backoff-max, so it is not present in the currently published v1.23 API reference yet.
I would appreciate it if you could leave another review.

[kahirokunn]

@Cali0707 Since you seem to have looked at my issue, I'll ping you! 🙏