Browse the corpus

Non-pharmaceutical interventions: evaluating challenges and priorities for future health shocks. Non-pharmaceutical interventions implemented during health shocks such as the covid-19 pandemic require rapid, robust, and rigorous evaluation that can generate timely evidence to guide government policy and maintain public confidence, say Azeem Majeed and colleagues

Evaluating NPIs during a fast moving crisis presents formidable challenges for policy makers, researchers, clinicians, and public health specialists. The need to act urgently may outpace the generation of robust evidence, as happened in the covid-19 pandemic, requiring decisions to be made in the face of uncertainty and with incomplete data, leading to tension between the need for timely action and the aim of evidence based policy making. In the covid-19 pandemic, simultaneous implementation of multiple NPIs across different populations created a complex set of interactions that obscured the impact of individual measures.10 This complexity was compounded by variability in implementation. For example, differences in how interventions are applied, such as the strictness of lockdowns or the enforcement of mask mandates, can affect outcomes and make evaluation challenging. Furthermore, public adherence to NPIs is influenced by cultural, social, and psychological factors, including trust in the government and in science, which can vary widely between societies and change over time. The impact of NPIs was also influenced by the stage of the pandemic at which they were implemented. For example, interventions earlier in the pandemic might have had different effects from those introduced later when population immunity through infection and vaccination was greater.14

en societies and change over time. The impact of NPIs was also influenced by the stage of the pandemic at which they were implemented. For example, interventions earlier in the pandemic might have had different effects from those introduced later when population immunity through infection and vaccination was greater.14 Without evidence from randomised controlled trials for many interventions,15 16 policy makers and researchers have to rely on observational studies, which are vulnerable to bias and confounding. For instance, during covid-19, the impact of school closures on community infection transmission rates was difficult to isolate when concurrent measures, such as workplace closures and travel restrictions, were in place. Modelling studies may also be important in predicting outcomes and guiding policy, as during covid-19, but they depend on the quality of the data and the assumptions built into the models, which can lead to large uncertainties in their predictions. Finally, in-depth qualitative interviews and lived experiences engaging patients and the public with particular emphasis on vulnerable and marginalised population can be useful in framing research questions for subsequent analysis and in dissemination and implementation of the findings.

The UK’s rapid action in generating and disseminating research findings during the pandemic shows the value of having a robust public health infrastructure that can generate data for planning and evaluation quickly and inform public health policy.17 18 Data from NHS electronic health records played a key role in the UK’s response and enhanced the quality of observational studies by providing timely, high quality data across the entire population. These records allow for more nuanced analysis of the impact of NPIs, with adjustment for individual characteristics of patients and healthcare interactions, which can help to mitigate some of the biases and confounding present in observational data or in ecological studies. Furthermore, data were released at varying degrees of aggregation, allowing anonymised data to be accessed more freely than individual level data. The dismantling of these systems since the pandemic is concerning, however.19 A national pandemic preparedness body should be established to map potential future threats, identify continuously updated pools of theme specific multidisciplinary experts, explore relevant data sources including any barriers to linkage with other data (for example, NHS electronic health records), design a safe research environment for efficient data sharing agreements that maintain patients’ confidentiality while facilitating essential research, establish a comprehensive framework for unified rapid ethical approval across the data owners/custodians, and secure a commitment for rapid funding.

Routine NHS data, although not designed for research, offer an essential platform to understand service provision and the effects of public health interventions in health system shocks. Such data were widely used during the covid-19 pandemic in addition to informing clinical practice to track health system performance, monitor disease transmission and testing, and determine who was at risk of severe outcomes, who should shield, and who should be vaccinated and when.20 The UK was in a strong position to use healthcare data at the start of the pandemic given the availability of linked data and governance processes for secure data access. However, architectural and infrastructure changes still had to be made to exponentiate use of the data. Rapid creation of a national English data resource including more than 96% of the English population for covid-19 and cardiovascular research,21 in addition to models such as the OpenSAFELY platform,22 a secure and transparent open source mechanism for analysis of routine healthcare data, paved the way for rapid analytics to be undertaken to inform policy and decision making. Many international collaborative efforts were also made, such as the International Consortium for Clinical Characterisation of covid-19 by electronic health records and the International Covid-19 Data Alliance, which led to trustworthy international research partnerships.

ndertaken to inform policy and decision making. Many international collaborative efforts were also made, such as the International Consortium for Clinical Characterisation of covid-19 by electronic health records and the International Covid-19 Data Alliance, which led to trustworthy international research partnerships. Global data access was not always straightforward, however, and the quality of data was not equitable during the pandemic, with much more limited data from low income countries, leading to discrepancies in mapping disease trajectories and deaths internationally.23 24 Additionally, data that would have been helpful for pandemic preparedness (such as occupation) were not always well recorded. Although equity in access to healthcare services and health outcomes may well be lacking during a health system shock such as a pandemic, routine data can help to determine where mitigation measures need to be implemented, allowing regional and demographic differences to be explored. Details around ethnicity, deprivation, or occupation have not always been captured; and missing data, where not considered, or unmeasured confounding in some early studies led to biased estimates of effect when not interpreted in the context needed. Without knowing the subtleties of who used healthcare, who was tested, and changes in the availability of testing over the course of the pandemic, interpretation of who was at greatest risk of poor outcomes could be erroneous.

e early studies led to biased estimates of effect when not interpreted in the context needed. Without knowing the subtleties of who used healthcare, who was tested, and changes in the availability of testing over the course of the pandemic, interpretation of who was at greatest risk of poor outcomes could be erroneous. Ultimately, data contributed to evidence synthesis and guideline development, including data on psychological and socioeconomic impacts and unintended consequences, balancing competing pressures and informing public communication. Concerns about privacy sometimes raised by the public were less of a problem than before the pandemic, partially owing to better messaging and communication than previously and a greater understanding among the public of the importance of data for rapid decision making at the time. Health Data Research UK (HDR UK) played a pivotal role in healthcare data infrastructure in the pandemic, bringing the research community together and highlighting the importance of collaborating at scale.25 For example, through HDR UK initiatives, people were rapidly identified to take part in clinical trials (for example, RECOVERY, which started in the UK as the Randomised Evaluation of COVID-19 Therapy as a clinical trial testing treatments for patients admitted to hospital with covid-19 pneumonia26).

rating at scale.25 For example, through HDR UK initiatives, people were rapidly identified to take part in clinical trials (for example, RECOVERY, which started in the UK as the Randomised Evaluation of COVID-19 Therapy as a clinical trial testing treatments for patients admitted to hospital with covid-19 pneumonia26). Responses to future health system shocks will benefit from the trusted research environments that were created quickly during the covid-19 pandemic to allow federated analytics. Collaboration across the research data community facilitated the prioritisation of research questions to be answered by the HDR UK covid-19 response team. Although data were centralised, at least in the UK, standardised coding and algorithms did not always exist, leading to different results and difficulty in tracking across time in a rapidly moving clinical landscape. This was particularly challenging at the start of the pandemic. As time went on, common data models became more readily available with more data standardised (for example, using the Observational Medical Outcomes Partnership Common Data Model; https://www.ohdsi.org/data-standardization/), making cross country comparisons easier, without losing the granularity of the data that is essential in decision making.

a models became more readily available with more data standardised (for example, using the Observational Medical Outcomes Partnership Common Data Model; https://www.ohdsi.org/data-standardization/), making cross country comparisons easier, without losing the granularity of the data that is essential in decision making. As part of preparedness for future health shocks, we need more robust and standardised data collection. Paper based medical data collection and multiple electronic health record systems in England make federated analytics challenging. Standardisation of coding for recording of diseases would allow comparisons to be made more easily and the effects of implementations assessed more clearly. A lack of availability of and access to social care data also remains. Further data linkage including social care data as well as laboratory data with patient record data could help future responses. Proper and regular communication with patients and the public to acknowledge and tackle their concerns will uphold and enhance public trust and help to minimise the effect of misinformation and disinformation campaigns.27

The socioeconomic and psychological effects of NPIs during the covid-19 pandemic were substantial and multifaceted, affecting many aspects of life globally. These included the large reduction in economic activity seen in the UK and many other countries early in the pandemic. Industries such as travel, hospitality, and retail faced severe losses, and some businesses were forced to shut down permanently, leading to a rise in unemployment. The pandemic also exacerbated existing income inequalities. People in poorer paid jobs generally faced greater financial instability and job losses, whereas many higher income workers could work remotely.28 School closures affected children’s learning worldwide, with potential long term implications. The shift to online learning highlighted the digital divide, as children without access to adequate technology or the internet at home faced significant disadvantages. Access to healthcare, particularly elective care, was also affected.29 30 Psychological effects caused by isolation, fear of infection, economic stress, and uncertainty led to a rise in mental health problems, including anxiety, depression, and stress related disorders.13 Lockdowns and quarantine measures led to an increased risk of domestic violence as victims found themselves trapped with abusers with less access to support services. The long term socioeconomic and psychological effects of NPIs will continue to be studied for many years to come, as societies grapple with and adapt to the changes brought about during the pandemic.

creased risk of domestic violence as victims found themselves trapped with abusers with less access to support services. The long term socioeconomic and psychological effects of NPIs will continue to be studied for many years to come, as societies grapple with and adapt to the changes brought about during the pandemic. In light of lessons learnt about the populations disproportionately affected by the pandemic, future pandemic preparedness should focus on the collection and availability of up-to-date data on occupation, employment, income, social care needs, and mental health (including psychotherapy and counselling) and on strengthening healthcare to provide services remotely when needed. Effective communication about individual inconvenience and collective altruism and resilience is needed, along with a pragmatic plan to offer proportionate financial and other social support to vulnerable groups.31

Although NPIs can be effective during health shocks, they come with substantial economic costs owing to reduced economic activity in addition to the direct costs of their implementation. Many studies on the cost effectiveness of NPIs are based on models that make assumptions about infection rates, the effectiveness of interventions, compliance levels, and the economic value of health outcomes. Real world data can differ from these models, which means that the cost effectiveness of NPIs can vary substantially from estimates.32 Some studies have shown NPIs to be cost effective in many scenarios, especially when considering the value of lives saved and healthcare costs averted, but the overall picture is complex. The effectiveness and economic impact of these interventions depend on the context in which they are applied, how they are implemented, and the behaviour of the population. Policy decisions in a future pandemic should consider cost effectiveness early to allow policy makers to implement NPIs that are most cost effective and minimise the negative outcomes from NPIs.

The covid-19 pandemic has underscored the interconnectedness of global health and the need for international cooperation in managing public health crises. It has also stressed the importance of robust healthcare systems and the requirement for ongoing investment in public health infrastructure and preparedness for future pandemics, including the capacity for rapid research on and evaluation of NPIs. This includes the timely and transparent sharing of information on the nature of the disease and its transmission and global collaboration on the evaluation of NPIs. This underpins the importance of international bodies such as the World Health Organization and the need for strong collaboration between countries such as adhering to the proposed WHO pandemic agreement.33

NPIs are critical components of the public health arsenal in managing infectious disease outbreaks and health shocks with other causes, particularly in the early stages when fewer effective options for prevention and treatment are available, as the covid-19 pandemic highlighted. As the pandemic evolved and more tools became available to manage covid-19, including vaccination and antiviral drugs, along with greater immunity from infection, reliance on NPIs decreased. NPIs such as face masks and school closures remain controversial. The endpoints for evaluating NPIs are typically defined in terms of the effects on transmission, morbidity, and mortality. However, we must also consider the potential for unintended consequences, such as the impact of school closures on children’s mental health, risk of obesity, educational attainment, and future economic prospects.2 3 These unintended consequences can be difficult to measure, but they are an important part of the overall impact of NPIs.

also consider the potential for unintended consequences, such as the impact of school closures on children’s mental health, risk of obesity, educational attainment, and future economic prospects.2 3 These unintended consequences can be difficult to measure, but they are an important part of the overall impact of NPIs. Future health shocks may well be more general than acute infection emergencies and have natural or geopolitical causes. The experience during the covid-19 pandemic provides many lessons for preparing. Firstly, having a robust system for evaluating NPIs that can rapidly assess the effectiveness of new interventions and identify unintended consequences is important. Secondly, a clear understanding of the potential benefits and harms of different NPIs is needed. This will help governments to make informed decisions about which interventions to implement. Thirdly, international collaboration in the evaluation of NPIs is important. Such collaborations will help to ensure that the best available evidence is used to inform policy decisions. Engaging key stakeholders including patients and the public, especially those at high risk of medical complications and death, in the decision making process is critical. As shown by the covid-19 experience, investing in systems that allow for the agile, accurate, and ethical evaluation of interventions to inform policy decisions and protect public health is essential.

e public, especially those at high risk of medical complications and death, in the decision making process is critical. As shown by the covid-19 experience, investing in systems that allow for the agile, accurate, and ethical evaluation of interventions to inform policy decisions and protect public health is essential. Many non-pharmaceutical interventions (NPIs) were effective in reducing the transmission of SARS-CoV-2, helping to “flatten the curve” of covid-19 with subsequent reductions in hospital admissions and deaths Policy makers should consider the unintended consequences of NPIs, such as the effects on the economy, educational attainments of children, and mental health and wellbeing Systems for evaluating NPIs should be flexible enough to provide essential data to guide policy for future health shocks with different causes, including non-infectious as well as infectious shocks (for example, environmental shocks such as extremes of temperature and floods and human-made disasters such as wars) A national pandemic preparedness body could map potential future threats, convene multidisciplinary expert pools, explore data sources, design a safe research environment for efficient data sharing agreements, establish a comprehensive framework for unified rapid ethical approval across the data owners/custodians, and secure research funding Evaluation of NPIs demands more focus on collection and availability of up-to-date data on occupation, employment, income, social care needs, and mental health (including psychotherapy and counselling)

A national pandemic preparedness body could map potential future threats, convene multidisciplinary expert pools, explore data sources, design a safe research environment for efficient data sharing agreements, establish a comprehensive framework for unified rapid ethical approval across the data owners/custodians, and secure research funding Evaluation of NPIs demands more focus on collection and availability of up-to-date data on occupation, employment, income, social care needs, and mental health (including psychotherapy and counselling) Stakeholder engagement, including with the most vulnerable groups in society, is essential in ensuring that public confidence is maintained and any proposed interventions are publicly acceptable