Propensity score model overfitting led to inflated variance of estimated odds ratios

Propensity score model overfitting led to inflated variance of estimated odds ratios

Overview

What is the issue?
  • Simulation studies suggest that the ratio of the number of events to the number of estimated parameters in a logistic regression model should be not less than 10 or 20 to 1 to achieve reliable effect estimates.
  • Applications of propensity score approaches for confounding control in practice, however, do often not consider these recommendations.
How was the study conducted?
  • We conducted extensive Monte Carlo and plasmode simulation studies to investigate the impact of propensity score model overfitting on the performance in estimating conditional and marginal odds ratios using different established propensity score inference approaches.
  • We assessed estimate accuracy and precision as well as associated type I error and type II error rates in testing the null hypothesis of no exposure effect.
What did the study find?
  • For all inference approaches considered, our simulation study revealed considerably inflated standard errors of effect estimates when using overfitted propensity score models.
  • Overfitting did not considerably affect type I error rates for most inference approaches. However, because of residual confounding, estimation performance and type I error probabilities were unsatisfactory when using propensity score quintile adjustment.
Implications
  • Overfitting of propensity score models should be avoided to obtain reliable estimates of treatment or exposure effects in individual studies.

Manuscripts

Schuster T, Lowe W, Platt RW. Propensity score model over-fitting leads to inflated variance of estimated odds ratios. Journal of Clinical Epidemiology. Journal of Clinical Epidemiology. J Clin Epidemiol. 2016 Dec;80:97-106.

Presentations

Project Team

Project Lead
Tibor Schuster PhD
CPRD
Collaborator
Wilfrid Kouokam Lowé MSc
CPRD
Collaborator
Robert W. Platt PhD
CPRD