Risk-stratification model for heart-attack prediction on 356K patient records (RandomForest, ROC AUC 0.80).

By — Chetan Sarda, Zixuan Zhu, Yinong Yao, Randeep Singh

Final ROC curve on held-out test set — Random Forest, ROC AUC 0.801

Results at a glance

• Test ROC AUC: 0.801 on held-out data (primary metric, chosen over accuracy due to ~90/10 class imbalance)
• Best CV ROC AUC (grid search): 0.883 — Random Forest tied with Logistic Regression and SVM at the top
• Recall on heart-attack class: 0.81 — the model misses few true positives (intentional cost of lower precision in a screening context)
• Fairness audit (Fairlearn): Female ROC AUC 0.844 vs Male ROC AUC 0.833 — no material disparate performance detected
• Risk tiering: predicted probability mapped to Low (<0.5), Medium (0.5–0.7), High (>0.7) for triage

Final confusion matrix

Fairness metrics by sex

Background — why I picked this one

This project came out of my MBAN capstone. We were handed a fictional client — Star City General Hospital — and told to build something that would actually help a hospital operate better. Most of my team gravitated toward churn or marketing-style problems; I pushed for heart disease because the public-health numbers were striking and the imbalanced-classification problem looked like a meaningful technical challenge.

The CDC figures are stark. Heart disease is the leading cause of death in the US — one person every 33 seconds. Roughly 47% of Americans have at least one of the three primary risk factors (high blood pressure, high cholesterol, smoking). Then you add diabetes, obesity (high BMI), low physical activity, excessive alcohol — and the list of people who should be flagged in a preventative-care program is enormous. The NHLBI's Framingham work has been pointing at obesity, sedentary lifestyle, and smoking as the dominant levers for decades. None of this is new. What's new-ish is that a hospital now has the data infrastructure to actually act on it at the individual-patient level instead of relying on population-level guidance.

Personally, I wanted to use this project to learn a few specific things:

1. How to actually handle severe class imbalance end-to-end, not just in a toy notebook. The "HadHeartAttack" target is roughly 10% positive, which is enough to tank any model that optimises for accuracy.
2. How to build a real sklearn Pipeline + ColumnTransformer setup that doesn't leak between train/test folds. I'd been burned by fit-on-everything mistakes before.
3. How to talk about fairness in a model output before a stakeholder asks. I'd read about Fairlearn and wanted an excuse to actually use it.
4. How to move from "here's a model" to "here's a triage workflow the hospital can run" — the gap between a Jupyter notebook and something a clinician could plug into.