Introduction
What happens when an emerging programme of medical research overlaps with a surging social movement? Often dated from a seminal 2006 publication from the now Center for Psychedelic & Consciousness Research at Johns Hopkins University (Griffiths, Richards, McCann, & Jesse, Reference Griffiths, Richards, McCann and Jesse2006), the revival of interest in psychedelic research has led to over 20 published clinical trials targeting a host of indications including depression (Goodwin et al., Reference Goodwin, Aaronson, Alvarez, Arden, Baker, Bennett and Malievskaia2022), post-traumatic stress disorder (Mitchell et al., Reference Mitchell, Bogenschutz, Lilienstein, Harrison, Kleiman, Parker-Guilbert and Doblin2021), anxiety in terminal cancer (Griffiths et al., Reference Griffiths, Johnson, Carducci, Umbricht, Richards, Richards and Klinedinst2016), and substance use disorder (Bogenschutz et al., Reference Bogenschutz, Ross, Bhatt, Baron, Forcehimes, Laska and Worth2022). During the same decade and a half, the hype surrounding psychedelic therapeutics has grown rapidly, becoming in itself a much-discussed topic amongst the scientific research community (e.g. Hall & Humphreys, Reference Hall and Humphreys2022), journalists (e.g. Love, Reference Love2022) and psychedelic researchers themselves (Butler, Jelen, & Rucker, Reference Butler, Jelen and Rucker2022; Yaden, Potash, & Griffiths, Reference Yaden, Potash and Griffiths2022).
One perspective on this increasing hype is that it obscures a clear understanding of how and to what extent psychedelic-assisted therapy (PAT) works. Recently, Yaden et al. (Reference Yaden, Potash and Griffiths2022) suggested that psychedelic therapies should be viewed neither ‘super-enthusiastically’ nor ‘super-skeptically’. The authors explain that for psychedelics, this means viewing them neither as wonder drugs nor as producing psychotic-like states of delirium. They call on their fellow psychedelic researchers to help ‘burst’ the bubble through better science communication practices. Using terminology from the Gartner Consultancy's Hype Cycle, they suggest that PAT is moving from a peak of inflated expectations to a trough of disillusionment, and only after this could a ‘well-calibrated’ assessment of the evidence regarding psychedelics occur, enabling the field to settle in a plateau of productivity (ibid.: E1).
The use of the Gartner Hype Cycle does not consider factors that may be specific to the particular case of psychedelics (e.g. see Borup, Brown, Konrad, & van Lente, Reference Borup, Brown, Konrad and van Lente2006, pp. 291–292). As such, these analyses are limited to how hype shapes wider expectations of PAT efficacy, without sufficient appreciation of how these effects in turn impact upon the findings of psychedelic therapy trials. Our aim in this article is to take this next step, effectively formalising the phenomenon by which the rapid mass-popularisation of psychedelic therapeutics has the potential to reshape the nature of the interventions themselves. This phenomenon was described as the Pollan Effect (Noorani, Reference Noorani2020), in reference to the popular impact of Michael Pollan's bestseller book on PAT, ‘How to Change Your Mind’ (Pollan, Reference Pollan2018).
This article focuses on PAT and consequently includes data and illustrative examples from trials involving 3,4-methylenedioxymethamphetamine (MDMA) and classical hallucinogens which combine psychedelic administration with psychotherapy. We omit ketamine trial data because most of the ketamine trials to date do not explicitly entail a therapy component and thus their protocols tend to diverge in key respects from PAT trials. First, we describe the two components of the Standard Unit Treatment Variable Assumption (SUTVA), a central plank in causal inference based on randomised controlled trial (RCT) data. We argue that violations of SUTVA – amplified by unblinding – make interpretation of treatment effects in psychedelic RCTs to date problematic. Next, we consider a distinctive – if not unique – feature of psychedelic experiences, borrowing the term ‘chemosociality’ from the anthropological literature to describe forms of sociality born of shared chemical exposure. We propose the concept of dark loops to describe forms of sociality variously emerging from and impacting participant experiences in RCTs. These dark loops are not recorded, let alone incorporated into the causal pathways in the interpretation of psychedelic trial data to date. As such, while agreeing with recent calls to neither overhype nor disregard the findings of existing trials (Butler et al., Reference Butler, Jelen and Rucker2022; Yaden et al., Reference Yaden, Potash and Griffiths2022), our goal is to fold in an appreciation of how the hype continues to shape the phenomena under study in complex ways, even as trials become increasingly large and rigorous in their design. We end with three positions researchers might take in response to the preceding arguments: chemosocial minimisation, where the research is designed to attenuate or eliminate the effects of dark loops in RCTs; chemosocial description, where the dark loops (and their impacts) are openly and candidly documented and chemosocial valorisation, where dark loops are hypothesised to contribute to trial outcomes and actively drawn upon for positive effect.
To illuminate the influence of dark loops between and within individual trials, key factors to note are both the expansion in the number and size of PAT clinical trials being conducted in recent years, and the length of time for which large clinical trials typically run. In terms of already published results, before 2015, five RCTs comprising a total of 62 patients were published (Bouso, Doblin, Farré, Alcázar, & Gómez-Jarabo, Reference Bouso, Doblin, Farré, Alcázar and Gómez-Jarabo2008; Gasser et al., Reference Gasser, Holstein, Michel, Doblin, Yazar-Klosinski, Passie and Brenneisen2014; Grob et al., Reference Grob, Danforth, Chopra, Hagerty, McKay, Halberstadt and Greer2011; Mithoefer, Wagner, Mithoefer, Jerome, & Doblin, Reference Mithoefer, Wagner, Mithoefer, Jerome and Doblin2011; Oehen, Traber, Widmer, & Schnyder, Reference Oehen, Traber, Widmer and Schnyder2013). Between 2016 and 2020, seven RCTs were published, including 193 participants (Danforth et al., Reference Danforth, Grob, Struble, Feduccia, Walker, Jerome and Emerson2018; Griffiths et al., Reference Griffiths, Johnson, Carducci, Umbricht, Richards, Richards and Klinedinst2016; Mithoefer et al., Reference Mithoefer, Mithoefer, Feduccia, Jerome, Wagner, Wymer and Doblin2018; Ot'alora et al., Reference Ot'alora, Grigsby, Poulter, Van Derveer, Giron, Jerome and Doblin2018; Palhano-Fontes et al., Reference Palhano-Fontes, Barreto, Onias, Andrade, Novaes, Pessoa and Araújo2019; Ross et al., Reference Ross, Bossis, Guss, Agin-Liebes, Malone, Cohen and Schmidt2016; Wolfson et al., Reference Wolfson, Andries, Feduccia, Jerome, Wang, Williams and Doblin2020). In the past two years alone, six RCTs have been published, between them including 514 participants (Bogenschutz et al., Reference Bogenschutz, Ross, Bhatt, Baron, Forcehimes, Laska and Worth2022; Carhart-Harris et al., Reference Carhart-Harris, Giribaldi, Watts, Baker-Jones, Murphy-Beiner, Murphy and Nutt2021; Davis et al., Reference Davis, Barrett, May, Cosimano, Sepeda, Johnson and Griffiths2021; Goodwin et al., Reference Goodwin, Aaronson, Alvarez, Arden, Baker, Bennett and Malievskaia2022; Mitchell et al., Reference Mitchell, Bogenschutz, Lilienstein, Harrison, Kleiman, Parker-Guilbert and Doblin2021; Schindler et al., Reference Schindler, Sewell, Gottschalk, Luddy, Flynn, Lindsey and D'Souza2021). While a full accounting of unpublished and ongoing trials is beyond the current scope, we are aware of at least three large RCTs (i.e. 100–200 participants) of MDMA-assisted psychotherapy either ongoing or in initiation (AUTHOR Bedi is leading one of these), and several RCTs in progress testing lysergic acid diethylamide (LSD), psilocybin, and N,N-dimethyltryptamine (DMT) for various indications (see https://psychedelicalpha.com/data/psychedelic-drug-development-tracker).Footnote 1 Larger RCTs of PAT, which are becoming increasingly common as the field develops, typically take years from the first to the last participant visit. For instance, the recent Phase 3 study of MDMA-assisted therapy for post-traumatic stress disorder (PTSD) took 1 year and 9 months from start of recruitment until the last visit (Mitchell et al., Reference Mitchell, Bogenschutz, Lilienstein, Harrison, Kleiman, Parker-Guilbert and Doblin2021), and a Phase 2b trial of psilocybin-assisted therapy for treatment-resistant depression took 2 ½ years from initiation to completion of data collection (Goodwin et al., Reference Goodwin, Aaronson, Alvarez, Arden, Baker, Bennett and Malievskaia2022). As further outlined below, both the increasing number of trials being conducted and the size and length of trials ensures that there are ample opportunities for dark loops to form between different trials and within individual studies.
Causal inference and the Standard Unit Treatment Value Assumption
Establishment of efficacy (and safety) in clinical trials relies on demonstrating a causal link between treatment and outcomes. The modern science of causal inference, based on a counterfactual framework, provides a theoretical foundation for identification of causal effects and makes explicit the assumptions on which this identification is based. Consider a dichotomous treatment A which can take values a = [0,1] and an outcome measure Y. For any individual i, there are two potential outcomes of treatment prior to application of the treatment: Yi(a = 0) and Yi(a = 1). However, in the real world, for any individual, only one of these outcomes can be observed and the other outcome is counterfactual. Hence, identification of individual treatment effects (ITEs) is not possible from observed data. This is termed the fundamental problem of causal inference (Holland, Reference Holland1986; Rubin, Reference Rubin1974). As the ITE cannot be identified, the aim of clinical trials is to identify the average treatment across a set of individuals with defined characteristics. The average treatment effect (ATE) can be defined as:
This identification of average causal effects relies on four key assumptions (Hernan & Robins, Reference Hernan and Robins2020). The first of these is termed positivity and is the assumption that all treatment states are possible. It can be written as:
The next assumption is that of exchangeability or ‘no unmeasured confounders’. Given any set of confounders ©, it can be written as:
where ⫫ indicates independence. In randomized controlled trials, the exchangeability and positivity assumptions are satisfied by the random allocation of participants to treatments. The next two assumptions form the Standard Unit Treatment Value Assumption (SUTVA) (Rubin, Reference Rubin1980). The first of these, consistency, is also termed ‘no hidden variation of treatments’ in which treatments should be well defined in order to have well-defined counterfactuals. That is, they should meet the criterion:
Observational study designs in the investigation of psychedelics – including ‘citizen science’ approaches – suffer from violations of the consistency assumption, making causal inference less certain. However, the need for consistency in treatments is also problematic for psychedelic RCTs, which share the challenges of psychotherapy trials where the treatment is forced into a straitjacket requiring manualised consistency but in clinical practice benefits from more responsiveness in the moment to the needs of the individual patient being treated.
The second part of SUTVA is the no-interference assumption. Cox (Reference Cox1958) described this as:
‘the requirement that the observation on one unit should be unaffected by the particular assignment of treatments to the other units, i.e. that there is no “interference” between different units’ (ibid.: 19).
From the perspective of causal inference, violation of non-interference is problematic because there can be many potential outcomes for two treatments depending on the network of interference effects (Rosenbaum, Reference Rosenbaum2007). Further, many of the statistical tests used for inference also rely on independence of data points, violations of which potentially lead to erroneous statistical interpretations in either direction (ibid.). Interference of experimental effects between units (usually individual participants) has been considered previously from a counterfactual perspective. For example, consider a household pair (i and j) who both could potentially receive a randomly allocated treatment (e.g. a vaccine). The counterfactuals for Y in this case of limited interference can be expanded to [Yi(Ai = 0, Aj = 0), Yi(Ai = 1, Aj = 0), Yi(Ai = 0, Aj = 1), Yi(Ai = 1, Aj = 1)] and can be mathematically tractable. However, as interference becomes more widespread such approaches quickly become insoluble – particularly when the network of interactions is not well described. When interference can occur between any units in the study sample then there is said to be full interference (Ogburn & VanderWeele, Reference Ogburn and VanderWeele2014).
Also termed spillover, Ogburn and VanderWeele (Reference Ogburn and VanderWeele2014) define three types of interference effects, two of which are relevant for our consideration of the current wave of PAT RCTs. Firstly, direct interference, where there is an effect of one unit's treatment on another unit's outcome that is independent of the outcome of the first unit. For example, if one member of a household pair receives information about a dietary intervention, that information might spill over to the other pair member regardless of the effect of the intervention on the first unit. Secondly, contagion where one individual's treatment outcome can affect the outcome of another unit. As the name intuitively suggests this type of interference is particularly common in the study of infectious diseases. For example, one individual in a house who is vaccinated decreases the probability that they will contract the disease and pass it onto the other member of the household.
With regard to non-interference, it is useful to consider the boundaries of interference. These are the units that are used to estimate the treatment effects – i.e., the sample of participants in a trial. But – as noted above – clinical trials can take years to complete from first patient intervention to last outcome measurement. Interference can potentially occur over this entire time scale – for example, the first person in a trial talking to the last person in a trial. Furthermore, given that group-level estimates and even individual participant data are now often combined across trials in meta- and mega-analyses, the non-interference assumption can be violated over even greater temporal durations.
Unblinding and expectancy in amplifying SUTVA breach concerns
When considering the effects of violations of SUTVA in PAT RCTs, it is important to consider the critical role that unblinding (or unmasking) plays in amplifying interference effects. In clinical trials, the role of randomisation is to break the backdoor path between any known and unknown confounders and treatment allocation, validating the exchangeability assumption (Hernan & Robins, Reference Hernan and Robins2020) (Fig. 1a). In the case of clinical trials, this means that potential confounders such as expectancy become covariates and can be adjusted for (if required in statistical analysis) to estimate treatment effects (Fig. 1b). However, the widespread breaking of double-blinding in psychedelic trials means that the expectancy needs to be more accurately treated as a mediator. This expectancy mediator provides a pathway for interference effects to be magnified. As in mediation analysis, in Fig. 1c, the total treatment effect is now c = c′ + ab. Indeed, it remains unclear in psychedelic RCTs whether any treatment is mediated by the direct pathway independently of expectancy [see Muthukumaraswamy, Forsyth, & Lumley (Reference Muthukumaraswamy, Forsyth and Lumley2021) for details].
Figure 1. (a) The role of randomisation in clinical trials is to break backdoor pathways and distribute known and unknown confounders across trial arms so that they can now be considered as covariates (b). Unblinding effects open an indirect causal pathway between treatment and outcome (c).
A fuller account of the failures in blinding in psychedelic trials to date must be connected with consideration of interference and contagion effects. We can reasonably expect unblinding in trials to produce greater interference and contagion effects as participants confident in their treatment allocation seek to connect and share experiences, thereby reshaping their and others' experiences, while also scripting those of prospective participants. Moreover, any increased familiarity with psychedelic experiences thus gained may further increase unblinding rates. In the following section, we provide a foundation for, and unpack, some of these processes.
Chemosocialities
When researched through clinical trial methodologies, certain substances are more prone to breaching SUTVA than others. We suggest that the current revival of clinical research into PAT may offer a useful upper limit case of such breaches. This claim hinges on appreciating how the resurgence of PAT research has been central to a wider set of intersecting social movements that have formed around psychedelic substances. These movements have gathered together interest and attention, individuals and communities, knowledges, resources, and political stakes. They encompass numerous, oft-conflicting perspectives, including those related to indigenous exploitation, countercultural association, traditional and underground use, psychedelic orientalism in relation to the exoticized ‘other’, psychedelic exceptionalism vis-à-vis widespread drug prohibition and the war on drugs and so forth (cf. Dumit & Sanabria, Reference Dumit, Sanabria, Bruun, Wahlberg, Douglas-Jones, Hasse, Hoeyer, Kristensen and Winthereik2022, p. 301).
These processes can be described in terms of psychedelics' chemosocial properties. Anthropologists Nicholas Shapiro and Eben Kirksey (Reference Shapiro and Kirksey2017) coined the term chemosociality to describe
‘the longstanding relationships and emergent social forms that arise from chemical exposures and dependencies’.
They continue
‘If biosociality involves social relationships that emerge from biological conditions and the science and technology through which they are known...then chemosociality involves novel, altered, attenuated, or augmented relationships that emerge from shared and shifting chemical ecologies.’
We note that Eben and Kirksey and other scholars (e.g. Fortun, Reference Fortun2012; Murphy, Reference Murphy2006) use ideas such as chemosociality to refer to the present as a living history of dynamic relations. Part of the demands of tracing chemosocialities, then, is past-oriented. In relation to psychedelic science, we might ask what or who ‘haunts’ the social spaces in and through which there is currently a surge of research interest in psychedelics. Sociologist Danielle Giffort (Reference Giffort2020) recently offered an elaborated study of the extent and ways in which Harvard psychedelic researcher-turned-countercultual guru Timothy Leary continues to haunt today's psychedelic research. Further historical scholarship might attend to a wide array of hauntings: of the continued exploitation and expropriation of indigenous peoples and knowledges, of the war on drugs, of how psychedelic history or ‘lore’ has itself been purposefully constructed, of the unethical actions of Big Pharma,Footnote 2 of a locale where a new RCT is being set up, of memories of what has happened to that locale's most marginalized and so forth.
The concept of chemosocialities also points forward, to the unfolding of new socialities produced by psychedelics. As anthropologist Jason Pine (Reference Pine2016) has described in relation to ‘the ecstatic abandon of methamphetamines and other recreational drugs’,
‘…solipsistic spaces often open shared bubbles of reverberating affect’ (ibid.: 310).
The affective components of psychedelic experiences – of awe, love, fear, mystery, deeply felt connection – impel reaching out, connecting with others, sharing stories, making sense in new ways, and further exploration in collective contexts. All of this renders psychedelics exemplary in their capacity to drive the creation of chemosocialities.
The fact that psychedelics appear to be such chemosocial agentsFootnote 3 in and of itself suggests that they will cultivate hype and hope beyond the confines of clinical trials. This is particularly so if they are framed as new and paradigm-shifting medical treatments for prospective patient–consumers, who may feel invited to become part of a ‘movement’. Recently psychedelic researchers have shared concerns over the sheer number of emails from desperate people reacting to the ‘psychedelic panacea’ media coverage and seeking relief through PAT for their suffering (Kent, Reference Kent2022). Acknowledging this, we propose that we should be expecting numerous breaches in SUTVA, which constitute the dark loops of trial circuitry. With the term ‘dark loops’ we describe collective feedback structures emerging from and returning into trials, which are hidden from view while potentially reshaping trial outcomes in systematic ways. In what follows, we describe some of the SUTVA breaches most relevant to psychedelic research trials (see Fig. 2). In later sections, we will offer three approaches for how researchers might respond to breaches in SUTVA.
Figure 2. Illustration of unmeasured direct interference and contagion effects in trials of psychedelic-assisted psychotherapy since circa 2006, potentially confounding individual trial findings and meta-analyses to date.
Community recruitment
When participants are recruited from the same communities, they are already connected in ways that enable them to share, discuss and make sense of their psychedelic experiences in real time in parallel to their progression through the trial. These communities may be geographically local, family- and friend-based and/or online and primarily driven through social media networks. These forms of contagion may shape participant expectations, as well as how they interpret and integrate their experiences in the post-acute phase – before primary and secondary endpoints. We can expect this to be exacerbated both in early phase trials, where snowball sampling is common and the many implications of self-selection are most pronounced, and in relation to the longer term endpoints (for instance, one-year follow-ups) that may come to inform comparative cost-effectiveness analyses.
New participant-community groups
The deep connections forged between participants and their study team (particularly, their therapists or guides) can lead to new friendships and enlarged community-based networks forming around the trials (see Noorani, Reference Noorani2021, pp. 209–211). In their qualitative study, Watts, Day, Krzanowski, Nutt, and Carhart-Harris (Reference Watts, Day, Krzanowski, Nutt and Carhart-Harris2017) document the significance of these new connections, wherein participants described the bonds with their guides resulting from having ‘been through something substantial together’ (ibid.: 550). Particularly in light of the low acceptance rates into psychedelic clinical trials, we can expect some of these strong connections to form well before the sharing of the participants' first psychedelic experience. This is bolstered by the positive attitudes towards the trial team and the broader psychedelic research agenda evinced by those participants most likely to survive selection processes where exclusion criteria include, for instance, those ‘judged to be incompatible with establishment of rapport’ with the study team (Protocol for Carhart-Harris et al., Reference Carhart-Harris, Giribaldi, Watts, Baker-Jones, Murphy-Beiner, Murphy and Nutt2021, p. 8).
If participants are involved in these community networks and groups while participating in trials (particularly until the study primary endpoints, but also through long-term follow-up), we should expect unquantified spillover effects to influence the outcomes of the trials. As part of this, those who through their participation in trials (come to) want the research programmes to continue and the substances to be legalized or medicalized can be expected to convey this messaging more widely, including to prospective participants in shared community groups and networks. Some trial participants have gone on to form community-based groups, such as psychedelic societies, integration groups, patient advocacy groups like PsyPAN in the UK, or the ‘MAPS Participants’ Twitter collective formed in part in response to concerns over psychedelic medicalisation. For these former participants, part of the value of the trial intervention might be traced precisely through the sense of purpose and new self-identities fashioned through this onward work towards building, improving or challenging problematic aspects of the wider psychedelic ecosystem.Footnote 4 We may differentiate between (a) those who socialize and collectivize outside of the strict parameters of the trials, (b) those who are want but are unable to do so (perhaps because of unwillingness to disclose the problems that the trials are set up to treat, or the continued stigma for some over psychedelic use), and (c) those who do not want to speak about their experiences outside of the trial settings (perhaps due to a desire for privacy concerning their psychedelic experiences). It remains an open question whether these variables are themselves mediators of symptom reduction and well-being outcomes, as well as the ‘fruits’ of onward integration processes.
Centre effects
Variability across a treatment protocol's multiple research sites breach SUTVA's consistency assumption and as such are important to document. Yet while researchers' own therapy allegiances have been shown to account for a large part of outcome variance in comparative psychotherapy studies (Luborsky et al., Reference Luborsky, Diguer, Seligman, Rosenthal, Krause, Johnson and Schweizer1999), the wider ‘culture’ of each site tends not to be described, whether in pre-registration, ethical review applications, or trial write-ups – either quantitatively or qualitatively. A site's culture might include the larger-than-life personae of key researchers, self-styled images from each site of what it means to do ‘cutting-edge’ science, and any mechanistic or interpretive tropes favoured by the site – such as the healing power of the mystical-type experience or the quietening of the ego. Centre effects have been cited as a concern from a regulatory perspective (for instance, European Medicines Agency, 2013, p. 11) but are rarely investigated as a source of variation in their own right. In the case of PAT, influential psychedelic researcher Matthew Johnson at Johns Hopkins University issued a ‘warning signal’ in 2020 for investigators and clinicians in psychedelic research to avoid introducing their own religious/spiritual beliefs or iconography into their research participants' treatment experiences (Johnson, Reference Johnson2021, pp. 579–580), which suggests that this is happening and likely contributing to centre effects. Any uneven influences across research sites may be experienced in heightened form due to the increased suggestibility commonly accorded to psychedelic states of consciousness (e.g. see Carhart-Harris et al., Reference Carhart-Harris, Kaelen, Whalley, Bolstridge, Feilding and Nutt2015).
Patient testimonials
The affectively powerful testimonials of former participants have since 2006 been largely very positive and proselytising, while a few provide cautionary tales based on participants' negative experiences. These testimonials can affect future participants directly if, for instance, they are shared online, or at local psychedelic society or integration group events. They can also be reported through influential journalistic accounts. That these testimonials can then be consumed by current and future participants in the lead up to, or immediate integration period after, their PAT sessions, is a clear example of contagion. This might occur as a result of participants seeking out such stories and accounts, or being brought into contact with them through media algorithms. Given the richness of psychedelic experiences reported in qualitative research with trial participants, others' testimonials can be powerful aides in dialogical modes of sense-making. The degree of exposure of current participants to former participants' testimonies and accounts is likely to be shaped by the level of hype, the maturity of the research field, awareness of active RCTs, and key media events, such as the launch of the Psymposia and New York Times Magazine production Cover Story in the autumn of 2021 or the Netflix adaptation of How to Change your Mind in the summer of 2022.Footnote 5
Hype and volatility in a maturing research field
It has been argued that there is no pragmatic or epistemic need to separate expectancy effects from true treatment effects in psychedelic medicine (e.g. Schenberg, Reference Schenberg2021). However, such an approach creates the unusual situation where the ‘efficacy’ of a medical intervention is unstable over time and potentially at the whim of social zeitgeist. Instability of efficacy estimates has been noted before and techniques (e.g. recursive cumulative meta-analyses) exist to track them over time (Ioannidis & Lau, Reference Ioannidis and Lau2001). However, in the cases considered by these authors, effect sizes converge on a ‘true’ effect size over time as data accumulate – whereas in the case of psychedelic expectancy confounds there will be inherent instability of effect sizes. Such instability should be problematic from a conceptual level for regulatory agencies such as the United States Food & Drug Administration and the European Medicines Agency because an intervention approved in 2024 may not have any efficacy in 2034 depending on external factors, if no causal pathway independent of ab (Fig. 1c) and of sufficient strength has been established. Such instability of effect sizes is also problematic for health economists and those stakeholders – such as insurance companies and health services – that pay for healthcare.
The above considerations help us analyse how the outcomes of psychedelic experiences produced through RCTs may change over time. With a large and growing media infrastructure for the spread of expectations, any one story may cascade across the whole network, potentially swinging from positive to negative coverage – even leading once again to regulatory clampdown and another period of overground research dormancy. Conversely, if a certain saturation point is reached where psychedelic experiences are normalised through a sufficiently elaborated media infrastructure, this may stabilise overall expectations.
Several explanations are commonly proffered for an anticipated reduction in efficacy of novel treatments. One is an increase in sample sizes by which usually one might expect treatment effect sizes to converge on a ‘true’ effect size, that is, reflective of the heterogeneity of the population it is intended to treat (Pereira, Horwitz, & Ioannidis, Reference Pereira, Horwitz and Ioannidis2012). A second is that the hype, which contributed to expectation effects enhancing outcomes in earlier trials, will reduce as the treatments become less novel. According to this explanation, the initial surge in optimism surrounding the healing modality – and possible subsequent deflation in expectations, as is suggested by the Gartner Hype Cycle – is followed by it becoming normalized, common or even mundane.
To these we propose an additional hypothesis, that as the psychedelic clinical trial field matures, breaches in SUTVA will decrease and become less direct: the rate of formation of participant-community groups will fall, participants will be drawn from more heterogenous backgrounds and locations, there will be more standardisation in research procedures, and less of a sense of contributing to a shared movement around the medicalisation of psychedelics. While direct breaches in SUTVA are problematic for individual trials, the longer looping effects constitutive of indirect breaches – such as when testimonies from one trial are published after trial participation has ended – will continue to confound meta- and mega-analyses across successive trials.
Implications and future directions
We consider the above concerns with SUTVA breaches as applicable across RCTs for pharmacological and psychotherapeutic interventions. However, given psychedelics' particularly chemosocial nature and apparent sensitivity to expectations, alongside the high rates of unblinding in clinical trials, psychedelic trials are especially well-placed to analyse RCTs in their social and cultural context. In this final section, we offer three distinct but not necessarily mutually exclusive approaches that trial researchers might consider in response to the above analyses of the hidden chemosocial effects of PAT, illustrating each with examples.
Chemosocial minimisation
The methodologically and epistemically conservative response is to endeavour to attenuate or eliminate the inconsistencies and dark loops that breach SUTVA. This approach re-emphasises the need for control in the design and conduct of RCTs, and guards against the increased costs of powering trials to compensate for additional covariates. We call this chemosocial minimisation. In line with this approach, current trial participants might be requested to not attend community-based integration groups, psychedelic societies, or emerging patient advocacy groups, until their final trial endpoints.Footnote 6 They could also be asked to refrain from consuming media depicting or explaining psychedelic experiences once their trials are underway. In all these cases enforcement will be an ongoing challenge, and ethically fraught given the potential but underappreciated value of such dark loops in providing informal modes of support and harm reduction.
A second strategy for chemosocial minimisation is to dampen down expectancy effects during trial recruitment. Recruitment is a key time during which expectancies can be both communicated and selected for. To date most PAT trials have recruited directly from the community, increasing the likelihood of recruiting participants who specifically want psychedelic treatment – presumably because of their positive expectations around this modality. Recruitment material identifying the treatment being offered and media pieces designed to boost recruitment also likely drive selective recruitment of people with positive expectancies. Conversely, recruitment from existing clinical services reduces this likelihood, increasing the chances of recruiting participants who are more interested in obtaining effective treatment than a particular type of care. This is consistent with approaches taken in other fields of medicine, such as surgery, where experimental treatment is usually offered as an extension of existing care (Butler et al., Reference Butler, Jelen and Rucker2022). In addition to selection effects, recruitment materials themselves can be powerful in shaping expectancies of prospective participants in unacknowledged ways. Recruitment materials that do not specify the treatment type – although potentially less attractive to participants – are unlikely to drive inflated expectations of treatment.
Other approaches to minimising the effects of SUTVA violations including improving blinding procedures. While it remains to be seen whether effective blinding can be achieved for psychedelic substances, several approaches – including using active control conditions with acute effects that overlap with the psychedelic being used – have yet to be fully investigated. Effective blinding would allow the cultivation of a positive set towards the therapeutic approach – itself a form of positive expectancy – without sacrificing the benefits of the randomised design. In terms of centre effects, these can be reduced by manualising and explicitly constraining treatment protocols used across sites and ensuring shared trainings to harmonise the approaches of site researchers and staff.
Chemosocial description
A second response to breaches of the SUTVA assumptions is to document the inconsistencies and dark loops in trial circuitry. We call this approach chemosocial description. In PAT RCTs, this involves openly and candidly depicting any chemosocial effects, both within and beyond the confines of the trials. In turn, this requires ringfencing funding when conducting trials, both for the methodologically individualist research program of understanding individual participants' experiences and behaviours beyond the proverbial ‘door’ to the trial sites, as well as methodologically holist research (including through ethnography and case series) into the moods, atmospherics, imagery, and so forth unique to the culture of each research site (cf. Ballou et al., Reference Ballou, Kaptchuk, Hirsch, Nee, Iturrino, Hall and Davis2017; Muthukumaraswamy et al., Reference Muthukumaraswamy, Forsyth and Lumley2021). These are reasons to involve former participants in data interpretation, as it is they who have traced the different components of their healing journeys in and beyond the trial sites themselves, and therefore can help to document the dark loops emerging from trial circuitry that were unanticipated at the trial design stage.
Such a proposal for supplementing methodologies for the study of clinical trials has precedent in earlier psychedelic research. Reflecting back on her influential career in the so-called ‘first wave’ of psychedelic research, Betty Eisner coined the term matrix as a complement to [mind]set and setting, to describe
‘…the environment from which the subject comes: the environment surrounding the subject before and after the session, and the larger environment to which the subject returns’ (Reference Eisner1997, p. 214).
Psychedelic chemosocialities, then, can be understood as driving transformations in the matrix of trial participants, which in turn further reveal psychedelics' chemosocial qualities.
One way to illuminate dark loops and inconsistencies across trial sites would be to thoroughly measure expectations. This will enable the correlation of expectations against outcomes, measure changes in expectations across participant journeys, investigate the relative impact of the expectations of different relevant actors (e.g. participants, therapists, researchers) in participant outcomes and – across meta-analyses and systematic reviews – consider how large-scale cultural expectations are associated with different outcomes over time. Focused attention is currently being given to expectancy phenomena, and numerical scales are being debated (for recent extended discussions, see Aday et al., Reference Aday, Heifets, Pratscher, Bradley, Rosen and Woolley2022 and Muthukumaraswamy et al., Reference Muthukumaraswamy, Forsyth and Lumley2021). Yet, expectancy remains a broad construct and we suggest that more focused qualitative study into expectancy will be required. We propose three avenues for further exploration: firstly, differentiating expectations of acute subjective effect from expectations of treatment outcome may reveal complex relationships between these phenomena. For instance, expectations about acute effects may contribute to unblinding, driving placebo or nocebo responding but only in the presence of positive expectancies about treatment outcomes. Secondly, in the case of PAT, expectations of different mechanisms of change may help to parse different meaning responses (Hartogsohn, Reference Hartogsohn2018) to the intervention. Indeed, Gukasyan and Nayak (Reference Gukasyan and Nayak2021) have recommended the development of a tailored scale that
‘could be helpful especially in assessing personal knowledge and connotations of psychedelics (60's counterculture, indigenous mysticism, or cutting-edge science).’
Thirdly and particularly for chronic conditions such as complex PTSD or treatment-resistant depression, it may be valuable to measure hope as distinct from positive expectations of outcome. Another understudied construct, hope has been articulated through qualitative research into participants' experiences of RCTs as
‘…a kind of passive volitional intention combined with imagination, will and acceptance. Hope was not so much a prediction as an existential stance: a lifejacket against despair’ (Kaptchuk et al., Reference Kaptchuk, Shaw, Kerr, Conboy, Kelley, Csordas and Jacobson2009)
In terms of considerations beyond the above challenges in understanding expectation, chemosocial description suggests publications report the time periods during which data was generated to enable a secondary consideration of contextual factors. Lastly, given the interpersonal dynamics of the selection process for what have become highly-coveted places on a PAT RCT, chemosocial description suggests the need for a more textured description of recruitment and selection processes. In addition to reporting acceptance rates, trial reports might include recruitment flyers and the study team's phone scripts. Trials could also convene post-study focus groups with staff involved in the selection process, to identify and reflect upon any informal decision-making procedures that emerged to supplement the formal, pre-established inclusion and exclusion criteria.
Chemosocial valorisation
The third response we offer to breaches in SUTVA is to embrace the chemosocial properties of PAT, and to attempt to optimise the dark loops of trials as they come to light. Distinct from, but building upon, the call to document these effects through chemosocial description, we call this chemosocial valorisation. It hypothesises that the impressive results in participants' symptom reduction and overall healing reported from the early-stage trials might be partially explained by breaches in SUTVA. Attempts to optimise these healing processes would then lean into them, rather than preventing them through restrictions in trial design, ignoring them through strategic choice of what to measure, or denying them outright. Indeed, Eisner explained
‘When a matrix is working properly, it also becomes a process: the setting becomes one in which individuals can change and mature, and as they grow, the matrix expands to contain the additional growth. There is also continual reinforcement towards change’ (ibid.: 215).
Her own thinking about the matrix arose in response to the therapeutic community that formed out of her clinical research, as patients coming from out of town stayed in one of four ‘communes’ formed around the trial sites, which Eisner described as ‘dedicated to change, and thus, creative changes in an individual undergoing drug-potentiated therapy were greeted enthusiastically’ (ibid.). These phenomena comport with claims that psychedelic research can produce psychedelic communitas, a construct that appears predictive of enduring changes in psychological well-being and social connectedness (Kettner et al., Reference Kettner, Rosas, Timmermann, Kärtner, Carhart-Harris and Roseman2021).
Chemosocial valorisation celebrates the breaches of SUTVA as so many components in a collective healing venture, constituted of all involved in the RCTs and their wider networks, and the pursuit of chemosocial minimisation as work that ironically whittles away at these healing effects in the name of an overly-narrow conception of methodological rigour. Functionally-speaking, the early formation of community groups and the rise of the hope-filled psychedelic medicalization movement may then be viewed as doing what is often sought of the post-approval clinician's craft, in for instance being responsive to trial participants, personalizing healing, and offering continued care.
While retaining key characteristics of RCT design, such as shared initial conditions across trial arms, systematic comparisons and the use of statistical analysis, chemosocial valorisation invites an epistemic break in our understanding of the value of clinical trials (cf. Burke & Blumberger, Reference Burke and Blumberger2021, p. 1688). Rather than determining scaleable interventions undergirded by generalisable knowledge, these trials would be locally-bound and prefigurative experiments in what is possible, strictly-speaking only supporting ‘“it-works-somewhere” claims’ (Cartwright, Reference Cartwright2011, p. 1401). Participatory action research (PAR) principles could come into their own, with participants and researchers adapting ongoing protocols in an iterated manner dependent upon chemosocial formations emerging in real time. Through a lens of chemosocial valorisation, the size and components of dark loops offer one key way to learn about collective processes of healing and harm reduction – as well as hidden sources of harm – set in motion by the ongoing RCTs.
Additional guardrails for this kind of research would be important, and could be sourced from the expertise of PAR researchers, together with former participants. Possible guardrails include ensuring continual engagement with ethical review boards and broader clinical research governance structures as the research evolves, establishing a charter that lists the agreed-upon goals of the research as well as the responsibilities and commitments of researchers, and setting up an expert advisory board constituted of both researchers and local communities (see Fogg et al., Reference Fogg, Lanning, Shoebridge, Longstaff, De Vos, Dawson-Taylor and Chauhan2022; Khanlou & Peter, Reference Khanlou and Peter2005). Public Patient Involvement panels could be retooled for and/or integrated into these processes. It could be argued as an advantage of chemosocial valorisation that if researchers identify emergent feedback loops with potential ethical and/or safety implications while a trial is underway, they can implement adaptations in the support they offer participants without undue pressure over compromising the integrity of the study design.
We can read an increasing number of trialists as turning towards chemosocial effects in some form, potentially in the language of ‘group cohesion effects’ (Burlingame, McClendon, & Yang, Reference Burlingame, McClendon, Yang, Norcross and Lambert2019; Gukasyan & Nayak, Reference Gukasyan and Nayak2021). For instance, in a psilocybin-assisted therapy trial for demoralized long-term AIDS survivors (Anderson et al., Reference Anderson, Danforth, Daroff, Stauffer, Ekman, Agin-Liebes and Woolley2020), mutual support among participants was encouraged, including inviting participants to exchange phone numbers and meet outside of the trial context and having participants meet in pairs both before and after having their (separate) psychedelic sessions. Or in a prospective study of bipolar 2 disorder patients underway, researchers mindful of working with a high-risk population state that they ‘will require participants to have both community and psychological support in place to manage any lingering effects of the intervention after the study is completed’ (Gard et al., Reference Gard, Pleet, Bradley, Penn, Gallenstein, Riley and Woolley2021, p. 8).The distinction between such ‘investigator-initiated’ trials and drug development registration trials may explain the extra methodological latitude afforded the former, with SUTVA breaches built into their designs. The impact of this distinction – and the dual-track regulatory system it holds in place – upon the question of which evidence counts in drug approval processes is an important area for future research.
Conclusion
Amidst much discussion of the relationship between RCTs and real-world evidence, this article begins with the recognition that RCTs themselves sit in the real world. Dismissing RCTs in favour of real-world evidence is premature – and in line with arguments that they can work in dynamic interplay (e.g. Rudrapatna & Butte, Reference Rudrapatna and Butte2020; Schlag et al., Reference Schlag, Zafar, Lynskey, Athanasiou-Fragkouli, Phillips and Nutt2022), we suggest closer attention to what is happening in RCTs will reveal potential inconsistencies and dark loops that can be further studied through real-world data. We have limited our analyses here to PAT RCTs, but the same concerns with the assumptions of causal inference apply more broadly. This includes pragmatic clinical trials, where the treatment arms that are compared with one another could each be assessed for their own chemosocial affordances.
Overall, our argument can be understood as a formalisation of the Pollan Effect, timely in the wake of the 2022 launch of the Netflix series adaptation of Pollan (Reference Pollan2018). Our concern with the proliferation of psychedelic RCTs that are currently underway is that they will achieve little for understanding the complex chemosocial properties of psychedelic interventions. With breaches of SUTVA and rampant unblinding, the effect size estimates from these trials will lack validity in terms of traditional assessment of treatment-efficacy. At the same time, if these trials proceed without clear acknowledgement and examination of the dark loops out of and back into psychedelic RCT designs, the opportunity for a more fulsome study of psychedelics as chemosocial phenomena is being missed.
Acknowledgements
We thank Alexandre Guerin and Rachel Liebert for providing adaptations to Fig. 1.
Financial support
Noorani has been supported in the writing of this article by a stipend from Tactogen and prior employment as Senior Lecturer in Psychology at the University of East London. Bedi has received funding from the Medical Research Future Fund (MRFF), US National Institute of Drug Abuse, the Victorian Department of Health and Human Services Alcohol and Drug Research Innovation Agenda, Gandel Philanthropy, the Marian and EH Flack Trust, the Youth Support and Advocacy Service, and the National Centre for Clinical Research on Emerging Drugs (NCCRED). Muthukumaraswamy has received research grants from MindBio Therapeutics Ltd, atai Life Sciences and the Health Research Council of New Zealand.
Conflict of interest
Noorani is a part-time Scholar-in-Residence at Tactogen Public Benefit Corporation.
Open access
