Back to the work
Biostatistics · Researcher

Metabolic Syndrome Risk Validation.

An external, longitudinal validation of two metabolic-syndrome risk scores on the US NHANES Linked Mortality File, benchmarked against established clinical equations (Framingham, ACC/AHA PCE, FINDRISC) with survey-weighted competing-risks survival analysis.

Role
Researcher
When
2026
Stack
R, Python, survival, XGBoost
Scale
17,031 adults NHANES cohort
View live demo ↗GitHub ↗
Metabolic Syndrome Risk Validation previewNHANES · Fine-Gray · external validation
17,031 adultsNHANES cohort
5 risk scoreshead-to-head
Fine-Graycompeting risks
Pre-registeredand reproducible

The problem

Two recently published metabolic-syndrome risk scores were developed and tuned on Korean cohorts. The open question was whether they hold up out of sample: do they predict long-term mortality in a large, survey-sampled US population, and do they add anything over the clinical risk equations doctors already use? Answering that honestly means a pre-registered external validation, not a re-fit on convenient data.

What it does

  1. Implements five risk scores from scratch in R, the metabolic-syndrome-derived RMRS (a triangular-areal-similarity method) and a B9 decision tree, plus the ACC/AHA Pooled Cohort Equations, Framingham 2008, and FINDRISC as comparators, each with unit tests.
  2. Runs them on a NHANES 1999 to 2018 cohort (17,031 adults) linked to the 2019 Mortality File, for all-cause, cardiovascular, and diabetes-related mortality.
  3. Uses survey-weighted Fine-Gray competing-risks and Cox models, IPCW time-dependent AUC, competing-risks decision-curve analysis, and a 500-replicate PSU-cluster bootstrap for confidence intervals, the inference a clinical journal would expect.
  4. Diagnoses why a score underperforms: an XGBoost ceiling on the same five inputs and a CART refit on a held-out NHANES split separate 'the signal is weak' from 'the imported splits do not transport'.

Impact

  • An honest, mixed result rather than a flattering one: for cardiovascular and all-cause mortality the established clinical equations dominate (Framingham 2008 reaches AUC 0.858 at 9.5 years for cardiovascular mortality versus 0.660 for RMRS), while for diabetes-related mortality RMRS competes with FINDRISC (AUC 0.752 vs 0.770) and adds modest incremental value (NRI and IDI confidence intervals exclude zero).
  • Localizes the B9 decision tree's weakness to transportability: a CART refit on held-out NHANES data recovers RMRS-level discrimination, so the Korean-calibrated splits, not the tree method, are the problem.
  • Pre-registered and fully reproducible: an OSF analysis-plan draft, a 15-step R/Python pipeline, renv-pinned dependencies, and a Makefile that regenerates the cohort and every result table from raw NHANES.