An Investigation of Integrating Program- and System-level Datasets for Examining Relationships Among Variables of a Complex Health System

Abstract

The potential for integrated program- and system-level datasets for generating previously inaccessible evaluative insights remains untapped. This study, situated within a yearlong evaluation of a lifestyle intervention program for patients with metabolic syndrome (MetS), provides an illustrative example of using an integrated dataset to identify relationships among the different levels of healthcare and examining the statistical findings with key informant interviews to extended interpretations beyond statistical significance towards clinical significance. The analysis of the program-level dataset revealed that the lifestyle intervention program was an effective treatment for MetS, while the additional analysis of the system-level dataset found an increase in patients’ ED utilization post-enrolment. Further, the new evaluative insight from the integrated dataset of no significant relationship among the clinical indicators and ED utilization begins to situate the program within the larger system context, suggesting some relationship between lifestyle intervention programs and ED utilization unrelated to clinical indicators. In so doing, this study advances a design for evaluating health programs with implications for enhancing program-level evaluations within complex health systems through integration of system-level datasets, and for increasing clinical validity of evaluation findings through inclusion of clinical experts as members of the evaluation team.