Skip to content

[BUG] analyze_market_regimes fabricates uniform probabilities and misreports method after silent fit fallback #247

Description

@A1-NWS-Dev1

🐛 Bug Description

backtesting_analyze_market_regimes can silently run entirely on the rule-based threshold classifier while still reporting method: "hmm"/"kmeans" and returning a fabricated uniform [0.333, 0.333, 0.333] probability distribution for every day in the history, regardless of the regime actually assigned.

Root cause: MarketRegimeDetector.fit_regimes has several "not enough data" early-return branches (e.g. fewer than max(50, n_regimes*20) valid feature windows -- which the tool's own default 365-day lookback usually does not clear for n_regimes=3). Each of these sets self.is_fitted = True without ever fitting self.scaler/self.model. Downstream, get_regime_probabilities then hits self.scaler.transform(...), raises NotFittedError on the never-fitted scaler, and its broad except fabricates np.ones(n) / n. Independently, detect_current_regime falls back to the rule-based threshold classifier through its own, separately-triggered exception handling -- so the regime label varies plausibly day-to-day while the probability stays fake-uniform, and self.method is never updated, so the tool keeps reporting the originally-requested method.

💰 Financial Disclaimer Acknowledgment

  • I understand this is educational software and not financial advice
  • I am not expecting investment recommendations or guaranteed returns
  • This bug report is about technical functionality, not financial performance

📋 Reproduction Steps

  1. Call backtesting_analyze_market_regimes with symbol="AAPL", method="hmm", n_regimes=3, using the tool's default lookback (no start_date/end_date).
  2. Inspect the returned method field and every recent_regime_history[*].probabilities entry.

🎯 Expected Behavior

method should report what was actually used ("threshold" when the fit fell back), and probabilities should either be the fitted model's real posterior or an honest one-hot vector at the assigned regime.

Actual behavior: method still reports "hmm", and every probabilities entry is exactly [0.3333333333333333, 0.3333333333333333, 0.3333333333333333] regardless of which regime was assigned that day.

Separately: method="hmm" is also a misnomer -- it fits sklearn.mixture.GaussianMixture (static clustering, no transition-probability/temporal modeling), not a genuine Hidden Markov Model. This should be documented even where a real fit succeeds.

💻 Environment Information

  • OS: Windows 11
  • Python Version: 3.12.10
  • MaverickMCP Version: main branch, HEAD 95c919f8ed22aa4dd0b58525900769c30a32ad3d at time of report
  • Installation Method: uv sync --extra backtesting

📊 Error Messages/Logs

No exception is raised. The tool returns a successful response with a misleading method field and fabricated confidence values.

🔧 Additional Context

Confirmed by direct source inspection of maverick/backtesting/strategies/ml/regime_detector.py::MarketRegimeDetector and by empirical reproduction (instrumented calls showing NotFittedError being swallowed). A fix, accurate documentation, and regression tests are proposed in PR (linked from this issue).

🏷️ Bug Classification

Severity:

  • Medium (feature partially working) -- silently fabricates analytical confidence

Component:

  • MCP server/tools (backtesting domain)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions