added a warning when n_trials budget is depleted but ask still return trials

[sagar1sharma]

…s trials

Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: https://github.com/automl/SMAC3/blob/main/CONTRIBUTING.md

Reference Issues/PRs

Fixes: #1296

Description, Motivation and Context

This PR fixes ask/tell behavior after budget exhaustion and makes the post-budget behavior explicit and configurable via warn modes.
Previously, ask() could continue returning trials after budget exhaustion without a strict guard in ask/tell usage, with stopping primarily enforced in the optimize() loop.
This PR adds explicit post-budget handling in ask() (warning/exception modes), updates docs, and adds tests for the same.

Changes

Added post-budget handling in SMBO.ask() for modes: warn_once, warn_never, warn_always, and exception.
Added dedicated exception class: AskAndTellBudgetExhaustedError.
Added/updated warn mode plumbing through facade to optimizer.
Updated ask-and-tell docs to describe post-budget behavior and warn modes.
Added/updated tests in both:
tests/test_ask_and_tell/test_ask_and_tell_intensifier.py
tests/test_ask_and_tell/test_ask_and_tell_successive_halving.py

Type Of Changes

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] My change is based on the latest stage of the develop branch.
[x] My change required a change of the documentation, which has been done.
[ ] I checked that the documentation can be build, visualizes everything as expected, and does not contain any warnings.
[x] I have added/adapted tests to cover my changes.
[x] The tests can be executed successfully.
[ ] I have added a description of the changes to CHANGELOG.rst.

[Copilot]

Pull request overview

This pull request adds a warning mechanism to alert users when ask() is called after the optimization budget has been exhausted. The change maintains backward compatibility by continuing to return trials while warning users about this behavior, and updates documentation to explain the ask-and-tell interface behavior.

Changes:

• Added a warning flag and logic to warn users once when calling ask() after budget exhaustion
• Updated documentation to clarify ask-and-tell behavior after budget depletion
• Properly reset the warning flag in the reset() method

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
smac/main/smbo.py	Added _warned_on_ask_after_budget_exhausted flag, warning logic in ask() method, and reset logic
docs/advanced_usage/5_ask_and_tell.md	Added warning section explaining ask-and-tell behavior after budget exhaustion

Comments suppressed due to low confidence (4)

smac/main/smbo.py:164

• The warning message doesn't specify which type of budget was exhausted. Since budget_exhausted can be True due to walltime, cputime, or trial count limits, the message should clarify which specific budget(s) caused the exhaustion. Consider adding information about remaining_walltime and remaining_cputime in addition to remaining_trials, similar to the logging in the optimize() method at lines 350-354.

            logger.warning(
                "ask() was called after the scenario budget was exhausted."
                f"remaining trials: {self.remaining_trials}). "
                "SMAC will continue returning trials for backward compatibility."
            )

smac/main/smbo.py:165

• The new warning functionality when ask() is called after budget exhaustion lacks test coverage. While test_ask_and_tell_after_optimization in tests/test_ask_and_tell/test_ask_and_tell_intensifier.py exercises this code path, it doesn't verify that the warning is emitted or that the warning flag is set correctly. Consider adding a test that captures and verifies the logger.warning call.

        if self.budget_exhausted and not self._warned_on_ask_after_budget_exhausted:
            logger.warning(
                "ask() was called after the scenario budget was exhausted."
                f"remaining trials: {self.remaining_trials}). "
                "SMAC will continue returning trials for backward compatibility."
            )
            self._warned_on_ask_after_budget_exhausted = True

docs/advanced_usage/5_ask_and_tell.md:21

• The documentation states "SMAC does not hard-stop ask() when n_trials is depleted" but this is misleading. The budget_exhausted check includes walltime, cputime, and n_trials budgets, not just n_trials. The documentation should clarify that ask() continues to return trials after any budget type is exhausted, not just when n_trials is depleted.

    In pure ask-and-tell usage, SMAC does not hard-stop `ask()` when `n_trials` is depleted.
    This means `ask()` can still return additional trials after budget exhaustion.

smac/main/smbo.py:82

• Trailing whitespace detected at the end of line 82. This should be removed to maintain code cleanliness and avoid potential issues with version control and linting tools.

        self._warned_on_ask_after_budget_exhausted = False  

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.