How Psychiatrists Responded to the Launch of Our New ECT Survey

The Joys of Announcing a Survey Launch on Social Media

Online surveys are increasingly being used as a legitimate research tool. This article discusses their strengths and weaknesses. It was prompted, in part, by events on social media last month.

On January 17^th three ECT recipients and three psychologists, including myself, launched the first-ever international online survey for ECT recipients and their families,¹ which will be open for at least three more months.

Within minutes of announcing the launch on X/Twitter, several British psychiatrists and their supporters began to vilify the survey, my University, its Ethics Committee, and myself. One senior pro-ECT psychiatrist tweeted ‘When we read the paper, which will be announced with the usual immodest fanfare, it will be important to remember just how weak and subject to bias the design can be seen to be’. He went on to thank someone with a positive experience of ECT for completing the survey, adding ‘You will have given them more honest answers than many’. He went on to express his doubts about ‘whether it will shed any useful light on the true experiences of people who have received ECT’. The next day he added ‘I hope nobody who is the slightest bit suggestible or with an axe to grind about Psychiatry or ECT feels tempted to participate.’

The same psychiatrist posted a snapshot of our question about the gender of the prescriber, sarcastically adding ‘Smash the Patriarchy.’ Another antagonist alleged there were ‘incredibly poor ethical & procedural standards in use’. Our asking whether people were ‘forced’ or ‘pressured’ into having ECT was criticised for somehow ‘leading the witness’. Furthermore, I was apparently going to ‘profit off’ the survey because it is part of my paid work for my University.

The fact that we are inviting family and friends to participate was described as ‘Dubious ethically re: reporting on behalf of others.’ Several proudly admitted filling out the survey themselves. One even boasted about doing so twice, once ‘on behalf of my Asian son from Germany who is the same age as me.’

Another pro-ECT psychiatrist had a more productive response to the allegation that our survey was bound to elicit a biased sample: ‘I guess the correct way is to perform a representative survey of ECT patients. We will look into this.’ Good.

Amidst these rather unpleasant, unprofessional jibes, there were some legitimate points. Perhaps the most important was the issue of online surveys not being representative of the larger population in question and, therefore, potentially biased. Before discussing that, two other potential weaknesses of this research method, and some of its strengths, I will summarise three previous online surveys with which I have been involved.

Three Examples of Online Surveys

A survey of antidepressant users

Towards the end of my 20 years at the University of Auckland, two colleagues and I, alarmed at the ever-increasing prescription rates of antidepressants, in New Zealand and around the world, were trying to figure out how to research the issues while struggling to find the time to meet our basic academic duties. The resulting online questionnaire had 47 questions, covering demographics; the prescribing process; information about antidepressant usage and perceptions of their effectiveness; side-effects; benefits; experiences of alternative treatment options; and beliefs about the causes of depression. The questionnaire consisted of multiple-choice questions, rating scales and a few open-ended questions.²

More than 1800 antidepressant users in New Zealand responded. The first paper focussed on the adverse effects. Eight of the 20 adverse effects studied were reported by over half the respondents, most frequently ‘Sexual Difficulties’ (62%) and ‘Feeling Emotionally Numb’ (60%). Other findings included ‘Withdrawal Effects’ (55%); ‘Feeling Not Like Myself’ (52%), ‘Reduction in Positive Feelings’ (42%), ‘Caring Less About Others’ (39%), and, surprisingly for a drug that was supposed to reduce depression, ‘Suicidality’ (39%). With restrained understatement, my Kiwi colleagues and I concluded ‘The adverse effects of antidepressants may be more frequent than previously reported, and include emotional and interpersonal effects’.

This survey produced such rich and broad data that it eventually spawned 10 other papers in peer-reviewed scientific journals, on topics from causal beliefs³ and gender issues^4,5 to the finding that a positive relationship with the prescriber predicted a positive outcome.⁶ But perhaps the most impactful follow-up paper was the one taking a closer look at withdrawal.⁷ Besides the 55% reporting withdrawal effects, 27% said they felt ‘addicted’. The longer people had been on the drugs the more likely they were to report addiction and withdrawal when they tried to stop. Paroxetine was found to be particularly likely to cause withdrawal effects. Most alarming, perhaps, was the finding that only 1% had been told anything about withdrawal or the need to come off slowly.

Including antipsychotics

That first New Zealand survey was so productive I decided to replicate it internationally, when I moved, in 2015, to Swinburne University in Melbourne. Only this time I included antipsychotics, with which I was far more familiar than antidepressants.  More than 1500 people from 38 countries responded.

The antidepressant half of the survey confirmed that the alarming findings of the first survey were not specific to New Zealand.⁸ This time our conclusions were less guarded:

‘Asking people directly reveals far higher rates of adverse responses to antidepressants than previously understood, especially in the under researched emotional, psychological and interpersonal domains. Given recent findings that antidepressants are, at best, only marginally more effective than placebo, these findings, especially in relation to suicidality and addiction, produce a cost-benefit analysis that cannot justify the continued high prescription rates for these drugs.’

The antipsychotic section, responded to by over 800 people, from 30 countries, led to five papers, reporting some surprising and invaluable findings. For example, 14 adverse effects were reported by more than half of respondents, most commonly: ‘Drowsiness, feeling tired, sedation’ (92%), ‘Loss of motivation’ (86%), ‘Slowed thoughts’ (86%), and ‘Emotional numbing’ (85%).⁹ Fifty-eight percent reported that the drugs made them feel suicidal. Most (70%) had not been told anything about any of these, or any other, adverse effects. While 35% reported that their ‘quality of life’ was ‘improved’, 54% reported that it was made ‘worse’. Importantly, duration of treatment was unrelated to positive outcomes but significantly related to negative outcomes. Most respondents (70%) had tried to stop taking the drugs. The most common reason they had wanted to stop was the debilitating adverse effects (64%) of these powerful tranquillising drugs.

The value of including open-ended questions in online surveys was demonstrated by responses to the questions “Overall in my life antipsychotic medications have been _____?” and “Is there anything else you would like to say, or emphasise, about your experiences with antipsychotic drugs?”¹⁰ Thematic analysis identified 749 negative, 180 positive, and 53 mixed statements. The two positive themes were ‘symptom reduction’ and ‘sleep’. The four negative themes were: ‘adverse effects’, ‘interactions with prescriber’, ‘withdrawal/difficult to get off them’, and ‘ineffective’. The one mixed theme was ‘short-term good, long-term bad’. We concluded:

‘Open questions can add to findings from methodologies focused on symptom reduction. Clinicians should pay more attention to the need for respectful and collaborative patient-prescriber relationships. At the point of prescription, this must include providing the full range of information about antipsychotics, including potential benefits and harms, difficulties withdrawing, and information on alternatives treatments such as psychological therapies.’

The paper included more than 100 quotes from the respondents, which brought to life the people behind the statistics. For example:

• ‘Having the medication makes it possible to thrive instead of just surviving.
• ‘Maybe helpful for a short time, but likely not worth it in the end they clouded my perceptions and sense of self very badly’
• ‘They are not “antipsychotic” they just helped me sleep.’
• ‘I put on 27 kg and developed diabetes.’
• ‘Severe sedation made it impossible to continue.’
• ‘They dumb me down & numb me up, I have no happiness or joy.’
• ‘They made me feel less than human, dead inside.’

Another paper confirmed that, like the public but unlike most psychiatrists, the majority of people who are prescribed antipsychotics have psycho-social rather than bio-genetic causal explanations for their difficulties¹¹. For example:

• ‘I believe being raised in a house with a dad with anger issues led me to live in constant fear as a child. But I didn’t understand that fear so instead it changed to paranoid thinking like monsters or demons were coming to kill me. Then I developed protective delusions (such as me being a werewolf or God).’
• ‘I suffered abuse as a child which I think caused abnormalities in my beliefs, self image, perception of the world, and my emotional responses.’
• ‘Past trauma and environment, isolation, father’s suicide, holocaust 1st generation German Jew, seeing my mom pass away from a heart attack.’

The two other papers addressed ‘How important are informed consent, informed choice, and patient-doctor relationships, when prescribing antipsychotic medication?’¹² and the virtually taboo topic of withdrawal from antipsychotics.¹³ Most people (72%) reported withdrawal effects when they tried to come off their antipsychotics, but none had been told anything about withdrawal effects or the need to reduce gradually. On a more positive note, 26% reported positive effects of withdrawing, such as feeling more alive.

These first two surveys highlighted the high incidence of withdrawal effects. When included in a broader review of the relevant research,¹⁴ they informed a seminal Public Health England review of prescribed drug dependence¹⁵ and radical improvements in the previously misleading guidance on antidepressant withdrawal from the UK’s Royal College of Psychiatrists and National Institute for Health and Care Excellence (NICE).¹⁶

What do people trying to withdraw from psychiatric drugs need?

The third survey, designed and conducted with two British psychiatrists, led to the 2023 paper ‘Designing Withdrawal Support Services for Antidepressant Users: Patients’ Views on Existing Services and What They Really Need’¹⁷. Whereas the two previous surveys recruited openly via social media and ‘snowballing’, this one invited only the members of 15 online support groups for people stopping antidepressants. From 49 countries, 1276 people responded.

Most (71%) found their doctors’ advice regarding stopping an antidepressant unhelpful; the main reasons being ‘Recommended a reduction rate that was too quick for me’, ‘Not familiar enough with withdrawal symptoms to advise me’ and ‘Suggested stopping antidepressants would not cause withdrawal symptoms’. A range of potential professional services were rated ‘very useful’, most frequently: ‘Access to smaller doses (e.g. tapering strips, liquid, smaller dose tablets) to ensure gradual reduction’ (88%) and ‘A health professional providing a personalised, flexible reduction plan’ (79%). We concluded:

‘Most participants reported their prescribers were unable to help them safely stop antidepressants, compelling them to turn to online peer-support groups instead. Our findings indicate, in keeping with previous studies, that clinicians require upskilling in safe tapering of antidepressants, and that patients need specialised services to help them stop safely.’

The findings of all three surveys are now integral to our ongoing efforts to compel the UK National Health Service to provide support for the, literally, millions of people trying to get off psychiatric drugs with no, or ill-informed, support from their prescribers.

The Limitations of Online Surveys

Amidst the rather venomous attacks on these surveys, and the current ECT survey, from those who like to dismiss their findings and our work as ‘anti-psychiatry’, ‘pill-shaming’ etc. there are two very reasonable concerns. Because online surveys are typically not completed by random samples of the population in question (i.e. all antidepressant users, all ECT recipients), but by ‘convenience samples’, they are wide open to generating biased, unrepresentative samples. Secondly, anybody can fill out the survey, perhaps multiple times, which casts serious doubt over the integrity of the data.

Unrepresentative, biased samples

One inevitable bias is that not everyone has access to the internet. And can a survey really be called ‘international’ if we haven’t provided multiple translations? Although the last two surveys summarised above elicited responses from many countries, most came from the UK or the USA and were predominantly from white people.

The major concern raised by critics, however, is that the surveys may be disproportionately responded to by people with bad experiences of the treatment in question, or with ‘an axe to grind’. This is a particularly legitimate worry where recruitment occurs partly via social media. It should be noted, however, that the bias could be in either direction. For example, in the ‘Limitations’ section of the first paper from the New Zealand survey,² we wrote:

‘The very high frequencies of adverse effects raise the issue of whether recipients who were dissatisfied with their medication were more likely to participate. This seems very unlikely, however, given that the majority (82.8%) reported that they believed the drugs had reduced their depression, a rate far higher than most conventional efficacy studies of ADs. If the survey had attracted a disproportionate number of unsatisfied people one might have expected a much higher rate reporting a reduced quality of life while on ADs (8.9%). Thus the sample appears to be biased towards people who had a positive response to ADs.’

In the international survey^8,9 over half (56%) thought the antipsychotic drugs reduced the problems they were prescribed for, and two-thirds (65%) reported that the antidepressants had reduced their depression. Again, both rates of positive outcomes are higher than found in traditional drug trials.

The current ECT survey is clearly biased towards those who had a positive response to, and/or were least harmed by, ECT. Those still severely depressed or suicidal, and those with severe cognitive damage, will be less likely or able to take part in the study. And, of course, those for whom the ECT did not prevent them from killing themselves will not be represented (except via their relatives’ reports). Nor will the smaller number who have died as a result of the ECT itself, usually via cardiovascular failure.¹⁸

Anyone can complete it, repeatedly

It is true that people who have not received the treatment in question can fill out an online survey. What might motivate someone to do so? Perhaps they had a relative who had a bad experience, and they want that to be represented? Perhaps they have personally had a bad experience with mental health services but not with this particular treatment? Perhaps they are somehow ideologically opposed to treating human distress with chemicals and electricity? Perhaps they are a psychiatrist, or a drug company executive or employee, with a vested financial or professional interest?

The survey platform, Qualtrics, does flag up respondents that engage in what they call ‘straight lining’, meaning responses with little variation. For example, we could exclude people who respond to all 25 possible adverse effects of ECT with the same choice from ‘not at all’ ‘mild’ ‘moderate’ or ‘severe’.  That decision, however, is also open to the effects of bias, on the researchers’ part. It would rule out people who genuinely had none of the adverse effects. If such decisions are to be made, they should be made before seeing the responses. It must be acknowledged that there is no surefire way to prevent people wo do not meet the inclusion criteria for a survey taking part, or to eliminate their responses once submitted.

Another concern expressed by critics is that people with a negative view, or ‘an axe to grind’ may be motivated to take the survey multiple times. This can and does happen, but all multiple responses from the same Internet Protocol (IP) address, unique to every device connected to computer network, are deleted (unless they have markedly differing responses indicating the use of the same computer by more than one person). This was the case for two out of 1,831 respondents in the New Zealand study. Checking IPs also reduces any contamination from people who have not received the treatment in question since they cannot submit multiple times. (In the case of the critic who boasted on X/Twitter that he had filled out the survey twice, we will probably delete both).

Self-report

Online surveys report the experience, opinion or feelings of the person completing the survey. I am always puzzled by the notion that we should not rely on self-report because it is subjective. Most research in mental health relies on either the self-report of a patient answering questions (sometimes from a structured test) from a psychiatrist or mental health professional, or on the self-report of a researcher or mental health professional expressing their opinion about a patient’s recovery etc. (again, sometimes from a structured list of questions). Who knows better how they are feeling, or whether their memory had been damaged, than the person themselves? Not that their perceptions are indisputably accurate, of course. But who is likely to be more accurate?

A more legitimate concern is that of memory, especially, ironically, when the treatment in question, like ECT, causes so much memory damage. One option is to gather data from a second person, as we are doing by including relatives and friends in our ECT survey. But their memory is fallible too.

Strengths of Online Surveys

Weighed against these important limitations are several strengths.

• Online surveys can generate numbers far larger than traditional studies such as randomised controlled trials (RCTs).
• They can gather data on people who have been on the drug in question for a range of periods, up to several years, as compared to the 8 or 12 weeks typically used in RCTs
• They are not subject to the bias that has been demonstrated in drug trials run by drug companies.
• By the use of open questions, they can generate rich, nuanced, quantitative data of the kind generated by small studies using interviews, but with hundreds of people rather than eight or so.
• Completely open questions, like ‘In my life antidepressants/ECT has been ………………….’ can genuinely give people a voice to say what they want to say. rather than provide data that answers a researcher’s preconceived hypothesis.
• They are easier, and take less time, to conduct than some other types of studies.
• They are cheap, and therefore do not usually require research grant applications.

Conclusion

I am not suggesting that online surveys are somehow inherently better than other methodologies. I believe in a ‘horses for courses’ approach whereby whichever methodology best (and most easily) answers the research questions should be adopted. I am suggesting, however, that online surveys have, if they pay attention to their limitations, earned the right to be on the menu.

Editor’s Note: The online survey for ECT recipients and their families can be found here: https://uelpsych.eu.qualtrics.com/jfe/form/SV_57KmQWiynXIhMNw

References

1. READ, J. (2024). International survey of electroconvulsive therapy. Psychology Today, Jan. 29. https://www.psychologytoday.com/gb/blog/psychiatry-through-the-looking-glass/202401/international-survey-of-electroconvulsive-therapy
2. READ, J., et al. (2014). Adverse emotional and interpersonal effects reported by 1,829 New Zealanders while taking antidepressants. Psychiatry Research 216, 67-73.
3. READ, J., et al. (2014). Beliefs of people taking antidepressants about causes of depression and reasons for increased prescribing rates. Journal of Affective Disorders, 168, 236-242.
4. CARTWRIGHT, C., et al. (2018). Personal agency in women’s recovery from depression: The impact of antidepressants and women’s personal efforts. Clinical Psychologist, 22, 72-82.
5. GIBSON, K., et al. (2018). Conflict in men’s experiences with antidepressants. American Journal of Men’s Health, 12, 104-116
6. READ, J., et al. (2015). Understanding the non-pharmacological correlates of self-reported efficacy of antidepressants. Acta Psychiatrica Scandinavica 131, 434-445.
7. READ, J., et al. (2018). How many of 1,829 antidepressant users report withdrawal symptoms or addiction? International Journal of Mental Health Nursing, 27, 1805-1815.
8. READ, J., WILLIAMS, J. (2018). Adverse effects of antidepressants reported by a large international cohort: Emotional blunting, suicidality, and withdrawal effects. Current Drug Safety, 13, 176-186
9. READ, J., WILLIAMS, J. (2019). Positive and negative effects of antipsychotic medication: an international online survey of 832 recipients. Current Drug Safety, 14, 173-181.
10. READ, J., SACIA, A. (2020). Using open questions to understand 650 people’s experiences with antipsychotic drugs. Schizophrenia Bulletin, 46, 896-604.
11. READ, J. (2020). Bad things happen and can drive you crazy: The causal beliefs of 701 people taking antipsychotics. Psychiatry Research, 285, 112754.
12. READ, J. (2022). How important are informed consent, informed choice, and patient-doctor relationships, when prescribing antipsychotic medication? Journal of Mental Health, doi:1080/09638237.2022.2069708
13. READ, J. (2022). The experiences of 585 people when they tried to come off antipsychotic drugs. Addictive Behaviors Reports, 15,
14. DAVIES, J., READ, J. (2019). A systematic review into the incidence, severity and duration of antidepressant withdrawal effects: Are guidelines evidence-based? Addictive Behaviours, 97, 111-121.
15. TAYLOR, S., et al. (2019). Dependence and withdrawal associated with some prescribed medicines: An evidence review. London: Public Health England, 2019.
16. HENGARTNER, M., et al. (2020). Antidepressant withdrawal – the tide is finally turning. Epidemiology and Psychiatric Sciences, 29: e52, 1-3.
17. READ, J., et al. (2023). Designing withdrawal support services for antidepressant users: Patients’ views on existing services and what they really need. Journal of Psychiatric Research, 161, 298-306.
18. READ, J., et al. (2019). Electroconvulsive Therapy for depression: A Review of the quality of ECT vs sham ECT trials and meta-analyses. Ethical Human Psychiatry and Psychology, 21, 64-103.