Supervised learning, unsupervised learning, model evaluation and selection, data preprocessing, and pipelines and composition. Per-topic detail is in the other reference files in this directory.
Comprehensive algorithms for classification and regression tasks.
Key algorithms:
- Linear models: Logistic Regression, Linear Regression, Ridge, Lasso, ElasticNet
- Tree-based: Decision Trees, Random Forest, Gradient Boosting
- Support Vector Machines: SVC, SVR with various kernels
- Ensemble methods: AdaBoost, Voting, Stacking
- Neural Networks: MLPClassifier, MLPRegressor
- Others: Naive Bayes, K-Nearest Neighbors
When to use:
- Classification: Predicting discrete categories (spam detection, image classification, fraud detection)
- Regression: Predicting continuous values (price prediction, demand forecasting)
See: references/supervised_learning.md for detailed algorithm documentation, parameters, and usage examples.
Discover patterns in unlabeled data through clustering and dimensionality reduction.
Clustering algorithms:
- Partition-based: K-Means, MiniBatchKMeans
- Density-based: DBSCAN, HDBSCAN, OPTICS
- Hierarchical: AgglomerativeClustering
- Probabilistic: Gaussian Mixture Models
- Others: MeanShift, SpectralClustering, BIRCH
Dimensionality reduction:
- Linear: PCA, TruncatedSVD, NMF
- Manifold learning: t-SNE, Isomap, LLE, MDS, ClassicalMDS (1.8+)
- External (install separately): UMAP (
umap-learn) - Feature extraction: FastICA, LatentDirichletAllocation
When to use:
- Customer segmentation, anomaly detection, data visualization
- Reducing feature dimensions, exploratory data analysis
- Topic modeling, image compression
See: references/unsupervised_learning.md for detailed documentation.
Tools for robust model evaluation, cross-validation, and hyperparameter tuning.
Cross-validation strategies:
- KFold, StratifiedKFold (classification)
- TimeSeriesSplit (temporal data)
- GroupKFold (grouped samples)
Hyperparameter tuning:
- GridSearchCV (exhaustive search)
- RandomizedSearchCV (random sampling)
- HalvingGridSearchCV (successive halving)
Metrics:
- Classification: accuracy, precision, recall, F1-score, ROC AUC, confusion matrix
- Regression: MSE, RMSE, MAE, R², MAPE
- Clustering: silhouette score, Calinski-Harabasz, Davies-Bouldin
When to use:
- Comparing model performance objectively
- Finding optimal hyperparameters
- Preventing overfitting through cross-validation
- Understanding model behavior with learning curves
See: references/model_evaluation.md for comprehensive metrics and tuning strategies.
Transform raw data into formats suitable for machine learning.
Scaling and normalization:
- StandardScaler (zero mean, unit variance)
- MinMaxScaler (bounded range)
- RobustScaler (robust to outliers)
- Normalizer (sample-wise normalization)
Encoding categorical variables:
- OneHotEncoder (nominal categories)
- OrdinalEncoder (ordered categories)
- LabelEncoder (target encoding)
Handling missing values:
- SimpleImputer (mean, median, most frequent)
- KNNImputer (k-nearest neighbors)
- IterativeImputer (multivariate imputation)
Feature engineering:
- PolynomialFeatures (interaction terms)
- KBinsDiscretizer (binning)
- Feature selection (RFE, SelectKBest, SelectFromModel)
When to use:
- Before training any algorithm that requires scaled features (SVM, KNN, Neural Networks)
- Converting categorical variables to numeric format
- Handling missing data systematically
- Creating non-linear features for linear models
See: references/preprocessing.md for detailed preprocessing techniques.
Build reproducible, production-ready ML workflows.
Key components:
- Pipeline: Chain transformers and estimators sequentially
- ColumnTransformer: Apply different preprocessing to different columns
- FeatureUnion: Combine multiple transformers in parallel
- TransformedTargetRegressor: Transform target variable
Benefits:
- Prevents data leakage in cross-validation
- Simplifies code and improves maintainability
- Enables joint hyperparameter tuning
- Ensures consistency between training and prediction
When to use:
- Always use Pipelines for production workflows
- When mixing numerical and categorical features (use ColumnTransformer)
- When performing cross-validation with preprocessing steps
- When hyperparameter tuning includes preprocessing parameters
See: references/pipelines_and_composition.md for comprehensive pipeline patterns.