Student Dropout AnalysisPoisson & Negative Binomial Modeling

The problem

Which factors are most associated with student dropout across New York State? I analyzed 39,674 NYSED school district records across 140 variables, then validated findings with cross-validated statistical models suited to count outcomes.

Data preparation

Raw percentage fields arrived as strings with "%" signs and dashes for privacy-suppressed values. After cleaning:

• 39,674 valid records retained for modeling
• ~45% missingness in some fields driven by privacy suppression, not data absence — suppressed values reconstructed where appropriate
• Multicollinearity reduced before modeling
• Count variables extremely right-skewed; log(enroll_cnt) used as an exposure offset so models learn rates, not raw size artifacts

What the data showed

• Migrant students and English language learners face over 3× higher dropout rates
• NYC districts average 13.2% dropout versus 7.5% elsewhere
• Enrollment size is the strongest driver of dropout volume

Model bake-off

Poisson, Negative Binomial, and OLS families were compared with consistent metrics. Important trap: OLS on a log-rate can post a tiny RMSE on a transformed scale while failing the actual task of predicting raw dropout_cnt.

Poisson P2 — categorical context + numeric covariates + county fixed effects + enrollment offset — is the preferred count model (RMSE ≈ 14.97, MAE ≈ 5.52). Enrollment size, GED attainment, still-enrolled counts, and subgroup effects drive dropout volume.

Why this matters

The method matters because the finding only becomes usable when the model matches the data type. Count outcomes need count models — and stakeholder recommendations need the subgroup and geography story, not just a leaderboard metric.

Interested in similar work? Get in touch.