Disruptions in clinical trials may be due to external events, such as pandemics, warfare, and natural disasters. Resulting complications might include site closures, supply chain interruptions, and travel restrictions, and lead to unforeseen intercurrent events (events that occur after treatment initiation and affect the interpretation of the clinical question of interest and/or the existence of the measurements associated with it). In Uschner et al. (2023), the randomization test is presented as a strategy to address clinical trial disruptions. Inspired by Van Lancker et al. (2023), both Uschner et al. (2023) and Van Lancker et al. (2023) consider the guiding principles set forth in The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) E9(R1) and discuss how trial disruptions can be incorporated as intercurrent events in the statistical analysis plan (SAP) (ICH E9(R1), 2021). Uschner et al. (2023) draws on the hypothetical estimand strategies introduced in Van Lancker et al. (2023) for the causal intention-to-treat (ITT) estimand. Causal estimands are defined in terms of counterfactual outcomes, and to express them as a statistical estimand that can be estimated from data, some understanding of causal inference frameworks and identification results is necessary (Hernán and Robins, 2020; Pearl, 2009). Van Lancker et al. (2023) provides an overview of causal inference and missing data methodologies; therefore, we recommended reviewing this in advance of Uschner et al. (2023).
In Uschner et al. (2023), several example clinical trial disruptions are described, including treatment effect drift, population shift, change of care, change of data collection, and change of availability of study medication. A complex setting is presented — a randomized controlled trial (RCT) with (i) planned intercurrent events and (ii) unplanned intercurrent events and other complications brought on by external disturbances — and the clinical question of interest corresponds to the causal ITT Comment on “Randomization Tests to Address Disruptions in Clinical Trials... Preprint 1 estimand. Randomization tests are then presented as a means for non-parametric inference that is robust to violations of assumptions typically made in clinical trials. These assumptions are not explicitly mentioned, however. More generally, we do not see where the authors make the case that one should be going for a randomization test in a disrupted RCT with planned and unplanned intercurrent events. Even in the case where an external disruption does not occur, it is not clear how the randomization test is useful in an RCT with planned intercurrent events, while Targeted Learning estimation methods are valid in such settings (Gruber et al., 2022a,b; Gruber et al., 2023). The randomization test is limited in its applicability and “poor experimental design may render it not useful” (Rosenberger, 2019, p. 28; Louis, 2019). We therefore request in the authors’ rejoinder a clear theoretical demonstration in specific examples in this setting (RCT with planned and unplanned intercurrent events) or the simpler setting (RCT with planned intercurrent events) that a randomization test is the only valid inferential method relative to an estimation method following the Targeted Learning Roadmap.
In this invited discussion, we comment on the appropriateness of Targeted Learning (TL) and the accompanying TL Roadmap in the context of disrupted clinical trials. We also highlight a few key articles related to the broad applicability of TL for RCTs and real-world data (RWD) analyses with planned and/or unplanned intercurrent events. We begin by introducing TL and motivating its utility in Section 2, and in Section 3, we provide a brief overview of the TL Roadmap for causal inference. In Section 4, we recite the example clinical trial disruptions presented in the Introduction of Uschner et al. (2023), and then discuss considerations and solutions based on the principles of TL. Concluding remarks are then given in Section 5.