Comparison of methods for analysing cluster randomized trials: An example involving a factorial design

Background. Studies involving clustering effects are common, but there is little consistency in their analysis. Various analytical methods were compared for a factorial cluster randomized trial (CRT) of two primary care-based interventions designed to increase breast screening attendance. Methods. Three cluster-level and five individual-level options were compared in respect of log odds ratios of attendance and their standard errors (SE), for the two intervention effects and their interaction. Cluster-level analyses comprised: (C1) unweighted regression of practice log odds; (C2) regression of log odds weighted by their inverse variance; (C3) random-effects meta-regression of log odds with practice as a random effect. Individual-level analyses comprised: (I1) standard logistic regression ignoring clustering; (I2) robust SE; (I3) generalized estimating equations; (I4) random-effects logistic regression; (I5) Bayesian random-effects logistic regression. Adjustments for stratification and baseline variables were investigated. Results. As expected, method I1 was highly anti-conservative. The other, valid, methods exhibited considerable differences in parameter estimates and standard errors, even between the various random-effects methods based on the same statistical model. Method I4 was particularly sensitive to between-cluster variation and was computationally stable only after controlling for baseline uptake. Conclusions. Commonly used methods for the analysis of CRT can give divergent results. Simulation studies are needed to compare results from different methods in situations typical of cluster trials but when the true model parameters are known.