Report terminating instances as gone in AWS HasInstance

[ribhavjain]

What type of PR is this?

/kind bug

What this PR does / why we need it:

Implements the AWS provider part of the HasInstance() quick-fix instructions from #9877.

Today the AWS HasInstance() returns true for any instance present in the ASG cache, regardless of its state. After NodeGroup.DeleteNodes():

• the cached maps aren't updated, so HasInstance() keeps returning true until the next cache regeneration, and
• the ASG API keeps listing the instance in the Terminating lifecycle state until its termination completes, so every cache regeneration re-adds it and HasInstance() keeps returning true that whole time.

Per #9877, this makes ClusterStateRegistry keep categorizing scaled-down Nodes as Ready/Unready after they are already excluded from the target size, which undercounts upcoming nodes and triggers the same unnecessary extra scale-ups as #9813.

This change:

• DeleteInstances() now marks the instance as Terminating in the lifecycle cache immediately after the successful TerminateInstanceInAutoScalingGroup call, so HasInstance() starts responding false right after DeleteNodes() returns, before any cache regeneration.
• HasInstance() now returns false for instances whose cached lifecycle state is Terminating, Terminating:Wait, Terminating:Proceed or Terminated (the same set DeleteInstances() already treats as "already terminating"), so the false response survives cache regenerations for as long as the ASG API still lists the instance.

Behavior for non-autoscaled nodes (not present in the cache) and for autoscaled nodes still included in the target size is unchanged.

Which issue(s) this PR fixes:

Part of #9877 (AWS provider quick-fix; not closing the umbrella issue)

Special notes for your reviewer:

Per the contributor guide's AI guidance: AI tooling was used in preparing this change. I have reviewed it and can explain every line.

Does this PR introduce a user-facing change?

AWS: `HasInstance()` now reports instances as gone as soon as they start terminating (e.g. immediately after a scale-down), preventing unnecessary extra scale-ups caused by counting terminating instances as ready.

Summary by CodeRabbit

• Bug Fixes

  • Improved detection of instances being terminated.
  • Deleted instances are now reported as unavailable immediately, even when AWS returns stale status information.
  • Prevented terminating instances from incorrectly appearing active during cache refreshes.

• Tests

  • Added coverage for instance availability before, during, and after deletion.
  • Verified unaffected instances continue to be reported as available.

[kubernetes-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ribhavjain
Once this PR has been reviewed and has the lgtm label, please assign gjtempleton for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• cluster-autoscaler/cloudprovider/aws/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kubernetes-prow]

This issue is currently awaiting triage.

If SIG Autoscaling contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[kubernetes-prow]

Hi @ribhavjain. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5c4884ab-77f4-4e6e-99da-38fde8409a96

📥 Commits

Reviewing files that changed from the base of the PR and between 5e07dd5 and a4e5a39.

📒 Files selected for processing (3)

• cluster-autoscaler/cloudprovider/aws/auto_scaling_groups.go
• cluster-autoscaler/cloudprovider/aws/aws_cloud_provider.go
• cluster-autoscaler/cloudprovider/aws/aws_cloud_provider_test.go

---

📝 Walkthrough

Walkthrough

The AWS provider now tracks terminating instances in the ASG cache. HasInstance immediately reports them as absent. Cache regeneration preserves terminating state when AWS returns stale active data. Tests cover deletion and refresh scenarios.

Changes

AWS terminating instance handling

Layer / File(s)	Summary
Cache lifecycle tracking and deletion updates cluster-autoscaler/cloudprovider/aws/auto_scaling_groups.go	The ASG cache centralizes terminating-state detection, records successfully terminated instances, and preserves terminating state during cache regeneration.
Instance presence validation and regression coverage cluster-autoscaler/cloudprovider/aws/aws_cloud_provider.go, cluster-autoscaler/cloudprovider/aws/aws_cloud_provider_test.go	HasInstance reports terminating instances as absent before status lookup. Tests cover cached termination, deletion, stale InService data, and terminating-state refreshes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to a4e5a

The AWS provider now treats terminating instances as gone sooner, preventing unnecessary scale-ups; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: choraden

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change to report terminating AWS instances as absent in HasInstance.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ribhavjain]

/assign @gjtempleton

[ribhavjain]

@gjtempleton gentle ping — anything you need from me to unlock CI here? This implements the AWS quick-fix from #9877 (same for the small kubemark build fix in #10148).