Skip to content

Add LightGBM as third model candidate in regression pipeline#43

Open
agentksimha wants to merge 3 commits intohumanai-foundation:mainfrom
agentksimha:refactor/regression_model
Open

Add LightGBM as third model candidate in regression pipeline#43
agentksimha wants to merge 3 commits intohumanai-foundation:mainfrom
agentksimha:refactor/regression_model

Conversation

@agentksimha
Copy link
Copy Markdown

This PR adds LightGBMRegressor as a third competing model alongside RandomForest and XGBoost in the regression pipeline.

What changed:

  • Added LGBMRegressor import
  • Added param_grid_lgbm with learning_rate, n_estimators, max_depth, num_leaves
  • Added LightGBM as a third entry in model_list so it participates in the same GridSearchCV and MSE-based selection logic

Justification:
LightGBM uses leaf-wise tree growth instead of depth-wise (XGBoost/RF), which is better suited for the feature-engineered dataset containing ratio and difference features with skewed distributions. It is faster to train and more memory-efficient on tabular data, making it a justified addition without disrupting the existing pipeline structure.

Results:
R² improved across all four non-special targets (CR21, CR54, CR13, CR23). Threshold accuracy showed mixed results — improved on CR21 and CR54, slightly lower on CR13 and CR23 — reflecting LightGBM's tradeoff of minimising squared error over threshold-based accuracy on those columns.

No changes to preprocessing, feature engineering, special targets, or output format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants