Diabetes Risk Prediction Platform

This project started as a college assignment and I further developed it into a small, end-to-end ML project: it trains three classifiers on the Pima Indians Diabetes dataset, serves the best one behind a Flask API, and exposes a lightweight browser UI. I built the serving and containerization layers to learn what happens to a model after the notebook.

Problem Statement

Given age, glucose, insulin, and BMI, predict whether a patient tested positive for diabetes — a binary classification task. The dataset cannot distinguish type 1 from type 2, so the model does not attempt to.

Beyond accuracy, the project has two engineering goals: training should be reproducible from a config file, and the preprocessing applied at inference must be provably identical to the one fitted at training time.

Dataset

• Source: Pima Indians Diabetes dataset (Kaggle/UCI) at data/diabetes.csv. If absent, a synthetic dataset is generated so the pipeline stays runnable — but real results require the real CSV.
• Size: 768 rows; 500 negative / 268 positive (34.9% positive).
• Features: Age (years), Glucose (mg/dL), Insulin (µU/mL), BMI (kg/m²); target Outcome → type ∈ {0: negative, 1: positive}.

Missing data (the main data-quality issue)

The dataset encodes missing measurements as 0, which is physiologically impossible for these fields:

Column	Zeros	% of rows
Insulin	374	48.7%
BMI	11	1.4%
Glucose	5	0.7%

Nearly half the Insulin column is missing. Treating those zeros as real readings pulls the feature distribution toward zero and biases every model trained on it. The pipeline converts them to NaN and median-imputes them inside the model Pipeline, so the fill values are learned from the training split only — imputing before the split would leak test-set statistics into training.

Models

Model	Rationale
Gaussian Naive Bayes	Fast baseline, probabilistic outputs, good for imbalanced small datasets.
MLPClassifier (sklearn)	Modern non-linear baseline with automatic differentiation and regularization.
Custom two-layer MLP	Educational implementation that exposes weight serialization and manual training loops.

Training & Evaluation

1. training/train.py is the CLI entry point that orchestrates data loading, preprocessing (standardization), model training, hold-out validation, and artifact persistence (models/*.pkl).
2. training/evaluate_and_report.py now loads trained artifacts, runs a hold-out evaluation, saves confusion matrices + ROC curves under reports/, and prints summary metrics. (Legacy REPORT.md generation can be re-enabled if needed.)
3. Metrics tracked: accuracy, weighted precision/recall/F1, plus recall and precision on the positive class and ROC-AUC. On a 65/35 split the weighted averages are dominated by the negative class, so they are reported but not relied on.
4. Every CLI training run logs hyperparameters, metrics, and models to MLflow (experiment diabetes-risk-local, SQLite backend at mlflow.db). Inspect it with mlflow ui --backend-store-uri sqlite:///mlflow.db.

Experiment tracking and the model registry

Models are logged with mlflow.sklearn.log_model rather than as loose pickle files, which means each run records the model's input signature and an input example — MLflow can then reject malformed payloads at load time instead of failing deep inside a NumPy call.

The winning pipeline is registered under the name diabetes-risk-model, giving it a versioned identity independent of any filename:

import mlflow
mlflow.set_tracking_uri("sqlite:///mlflow.db")
model = mlflow.sklearn.load_model("models:/diabetes-risk-model/1")
model.predict([[50, 150, 0, 33]])   # raw features; preprocessing is inside

The tracking backend is SQLite rather than the bare mlruns/ file store for a concrete reason: the file store cannot serve the Model Registry (no versioned names, no stage transitions), and MLflow has deprecated it.

By default the API loads the pickles in models/, so the container runs without a tracking server. Set MODEL_REGISTRY_URI to serve the registered model instead:

MODEL_REGISTRY_URI=models:/diabetes-risk-model/2 \
MLFLOW_TRACKING_URI=sqlite:///mlflow.db \
python -m backend.app

Why dependencies are pinned

requirements.txt pins exact versions, which matters because models/*.pkl are committed. Pickled sklearn estimators are version-sensitive: loading a model built under sklearn 1.8.0 in a 1.9.0 environment emits InconsistentVersionWarning — might lead to breaking code or invalid results. Unpinned, a fresh pip install today resolves sklearn 1.9.0 / numpy 2.5.1, so the Docker image would deserialize artifacts built by a different library than it runs. Verified: a clean venv from the unpinned file produced 6 such warnings; with pins it produces 0. Retrain after bumping any pin.

Current Results (hold-out, 20% stratified, seed 42)

Model	Accuracy	F1 (wtd)	Recall (pos)	Precision (pos)	ROC-AUC
GaussianNB	0.7078	0.7008	0.5000	0.6000	0.7537
MLP (sklearn)	0.7338	0.7306	0.5741	0.6327	0.8154
MLP (custom)	0.7143	0.7130	0.5741	0.5962	0.7861

Reproduce with python -m training.train --config configs/base.yaml.

The accuracy number is misleading, and that is the most interesting result here. At the default 0.5 threshold the best model reaches 73% accuracy but only 0.57 recall on the positive class — it misses about 43% of patients who actually have diabetes. For a screening tool that is the expensive kind of error, and accuracy hides it entirely.

Fixing missing values helped, but modestly:

Metric (MLP)	Zeros as real values	Zeros imputed
Accuracy	0.7208	0.7338
Recall (pos)	0.5370	0.5741
ROC-AUC	0.8080	0.8154

The larger lever is the decision threshold, not the model. Sweeping it on the imputed MLP:

Threshold	Accuracy	Recall (pos)	Precision (pos)
0.50	0.7338	0.5741	0.6327
0.40	0.7208	0.6852	0.5873
0.30	0.6948	0.7407	0.5479
0.25	0.7273	0.8333	0.5769

Lowering the threshold to 0.25 raises recall from 0.57 to 0.83 — catching 83% of diabetics instead of 57% — while accuracy stays essentially flat. The cost is precision (more false positives, i.e. more unnecessary follow-up tests), which for a screening test is usually the right trade.

The API exposes this as a request parameter, so the operating point is a deployment decision rather than a hardcoded constant:

# Same patient, different decision rule
curl -X POST localhost:5000/predict -H 'Content-Type: application/json' \
  -d '{"model_type":"mlp_sklearn","age":45,"glucose":135,"insulin":90,"bmi":30}'
# {"diabetes_type":0,"probability":0.437,"confidence":0.563,"threshold":0.5}

curl -X POST localhost:5000/predict -H 'Content-Type: application/json' \
  -d '{"model_type":"mlp_sklearn","age":45,"glucose":135,"insulin":90,"bmi":30,"threshold":0.3}'
# {"diabetes_type":1,"probability":0.437,"confidence":0.437,"threshold":0.3}

The response always includes the raw probability, so a caller can apply its own cutoff without re-querying. The browser UI has a threshold slider that demonstrates the same trade interactively.

Each run also drops confusion matrices and ROC curves to reports/<model>_{confusion_matrix,roc_curve}.png, so reviewers can inspect the visuals without re-training (regenerate anytime with python -m training.evaluate_and_report).

System Architecture

1. Training pipeline: Config-driven (configs/base.yaml); writes each model plus the fitted preprocessor into models/.
2. Model loading: Flask loads the serialized estimators and preprocessor once at startup.
3. REST API: /predict validates input with Pydantic, applies the preprocessor, and returns the prediction with a confidence score. /health and /models support monitoring.
4. Frontend UI: Static HTML/JS client (frontend/), served by the same Flask process so its /predict calls are same-origin.

data → training scripts → model artifacts → Flask API → browser UI

Preprocessing travels with the model. Each artifact in models/ is a single sklearn Pipeline:

0 → NaN  →  SimpleImputer(median)  →  StandardScaler  →  estimator

fitted end-to-end on the training split. There is no separate scaler.pkl to keep in sync, and the API passes raw patient values straight to .predict() — the model applies exactly the transform it was trained with. Adding a preprocessing step in training propagates to inference with no change to backend/.

Making this work meant giving the hand-written SimpleMLP a proper sklearn interface (BaseEstimator/ClassifierMixin, weights built in fit, classes_ exposed), so the custom model composes into a Pipeline exactly like the library ones instead of needing its own loading path.

I learned why this matters by breaking it. Initially the 0 → NaN conversion happened at CSV-load time, outside the Pipeline. Training therefore replaced sentinel zeros with the median, but the API — which never touches the loader — scaled a user-supplied insulin=0 as a literal zero. Same patient, two different feature vectors:

POST /predict  {"age":50,"glucose":150,"insulin":0,"bmi":33}   → 0 (negative)
POST /predict  {"age":50,"glucose":150,"insulin":80,"bmi":33}  → 1 (positive)

A field that was missing for 48.7% of the training data flipped the prediction at inference. Moving the conversion inside the Pipeline fixed it with metrics unchanged, because the transform was relocated rather than altered. The lesson: any preprocessing that is not inside the persisted artifact is a training/serving skew waiting to happen.

All commands must be run from the repository root; backend/ and training/ are Python packages.

Quickstart

pip install -r requirements.txt

# Train and persist models (also logs to mlruns/)
python -m training.train --config configs/base.yaml

# Generate evaluation plots (writes PNGs to reports/)
python -m training.evaluate_and_report

# Optional: inspect MLflow dashboard locally
mlflow ui --backend-store-uri sqlite:///mlflow.db

# Run the Flask API (serves frontend as static files)
python -m backend.app

# Visit the UI
open frontend/index.html  # or navigate to http://127.0.0.1:5000

The frontend JavaScript calls the API via same-origin relative paths (/predict), so it works unchanged whether served locally, inside Docker, or behind a reverse proxy.

Containerized Run

The curated .dockerignore keeps bytecode, MLflow runs, pickled artifacts, and virtualenvs out of the build context, so Docker layers stay lean.

# Build the production image locally
docker build -t diabetes-risk-prod .

# Run it with Gunicorn listening on $PORT (defaults to 5000)
docker run --rm -p 5000:5000 --env PORT=5000 diabetes-risk-prod

# Or rely on the provided compose file for repeatable dev/prod parity
docker compose up --build

The compose stack exposes the API on http://127.0.0.1:5000 and serves the static frontend via the same container. Override PORT or FLASK_DEBUG in docker-compose.yml or with --env flags if needed.

Testing

Pytest covers the input validation helpers (tests/test_input_validation.py) and the Flask /predict endpoint (tests/test_api.py).

python -m pytest

Developer Tooling

• Install formatter/linter/test extras with pip install -r requirements-dev.txt (includes pytest, black, isort, flake8, and pre-commit).
• Enable Git hooks by running pre-commit install once; enforce them manually anytime with pre-commit run --all-files.
• Black + isort keep the code style consistent, while flake8 prevents lint regressions before CI even runs.

Limitations

Stated plainly, because these are the things I would ask about:

• Recall is the real weakness. At the default 0.5 threshold the model misses ~43% of positive cases. The API exposes threshold so a caller can trade precision for recall, but the default is still 0.5 — choosing a clinically appropriate default would need a real cost model for false negatives vs false positives.
• Model selection uses accuracy (selection_metric: accuracy in the config), which — per the analysis above — is the wrong objective for this problem. ROC-AUC or recall-at-fixed-precision would be more defensible. The config makes this a one-line change; the reported numbers are honest about the current setting.
• Single train/test split, no cross-validation. With 768 rows, the reported metrics have meaningful variance; a seed change moves them by a few points. Repeated stratified k-fold would give a more trustworthy estimate.
• No hyperparameter search. The MLP's architecture (one hidden layer of 50) was chosen by hand, not tuned.
• 48.7% of Insulin is imputed, so that feature carries far less information than its presence suggests. Dropping it, or adding a missingness indicator, is worth testing.
• Not clinically validated. This is a learning project on a small public dataset and must not be used for medical decisions.
• Experiment tracking is a local MLflow file store only; there is no model registry, monitoring, or drift detection.