Stepped wedge designs have gained recognition as a method for rigorously assessing implementation of evidence-based quality improvement interventions (QIIs) across multiple healthcare sites. In theory, this design uses random assignment of sites to successive QII implementation start dates based on a timeline determined by evaluators. However, in practice, QII timing is often controlled more by site readiness. We propose an alternate version of the stepped wedge design that does not assume the randomized timing of implementation while retaining the method's analytic advantages and applying to a broader set of evaluations. To test the feasibility of a nonrandomized stepped wedge design, we developed simulated data on patient care experiences and on QII implementation that had the structures and features of the expected data from a planned QII. We then applied the design in anticipation of performing an actual QII evaluation.,We used simulated data on 108,000 patients to model nonrandomized stepped wedge results from QII implementation across nine primary care sites over 12 quarters. The outcome we simulated was change in a single self-administered question on access to care used by Veterans Health Administration (VA), based in the United States, as part of its quarterly patient ratings of quality of care. Our main predictors were QII exposure and time. Based on study hypotheses, we assigned values of 4 to 11% for improvement in access when sites were first exposed to implementation and 1 to 3 % improvement in each ensuing time period thereafter when sites continued with implementation. We included site-level (practice size) and respondent-level (gender, race/ethnicity) characteristics that might account for nonrandomized timing in site implementation of the QII. We analyzed the resulting data as a repeated cross-sectional model using HLM 7 with a three-level hierarchical data structure and an ordinal outcome. Levels in the data structure included patient ratings, timing of adoption of the QII, and primary care site.,We were able to demonstrate a statistically significant improvement in adoption of the QII, as postulated in our simulation. The linear time trend while sites were in the control state was not significant, also as expected in the real life scenario of the example QII.,We concluded that the nonrandomized stepped wedge design was feasible within the parameters of our planned QII with its data structure and content. Our statistical approach may be applicable to similar evaluations.

CONTEXT: Previous studies testing continuous quality improvement (CGI) for depression showed no effects. Methods for practices to self-improve depression care performance are needed. The authors assessed the impacts of evidence-based quality improvement (EBGI), a modification of CGI, as carried out by 2 different health care systems, and collected qualitative data on the design and implementation process. OBJECTIVE: Evaluate impacts Of EBGI on practice-wide depression care and outcomes. DESIGN: Practice-level randomized experiment comparing EBGI With usual Care. SETTING: Six Kaiser Permanente of Northern California and 3 Veterans Administration primary care practices randomly assigned to EBGI teams (6 practices) or usual care (3 practices). Practices included 245 primary care clinicians and 250,000 patients. INTERVENTION: Researchers assisted system senior leaders to identify priorities for EBGI teams; initiated the manual-based EBGI process; and provided references and tools. EVALUATION PARTICIPANTS: Five hundred and sixty-seven representative patients with major depression. MAIN OUTCOME MEASURES: Appropriate treatment, depression, functional status, and satisfaction. RESULTS: Depressed patients In EBGI practices showed a trend toward more appropriate treatment compared with those in usual care (46.0% vs 39.9% at 6 months, P=.07), but no significant improvement in 12-month depression symptom outcomes (27.0% vs 36.1% poor depression outcome, P=.18). Social functioning improved significantly (mean score 65.0 vs 56.8 at 12 months, P=.02); physical functioning did not. CONCLUSION: Evidence-based quality improvement had perceptible, but modest, effects on practice performance for patients with depression. The modest improvements, along with qualitative data, identify potential future directions for improving CGI research and practice.