The Joys of Announcing a Survey Launch on Social Media
Online surveys are increasingly being used as a legitimate research tool. This article discusses their strengths and weaknesses. It was prompted, in part, by events on social media last month.
On January 17th three ECT recipients and three psychologists, including myself, launched the first-ever international online survey for ECT recipients and their families,1 which will be open for at least three more months.
Within minutes of announcing the launch on X/Twitter, several British psychiatrists and their supporters began to vilify the survey, my University, its Ethics Committee, and myself. One senior pro-ECT psychiatrist tweeted âWhen we read the paper, which will be announced with the usual immodest fanfare, it will be important to remember just how weak and subject to bias the design can be seen to beâ. He went on to thank someone with a positive experience of ECT for completing the survey, adding âYou will have given them more honest answers than manyâ. He went on to express his doubts about âwhether it will shed any useful light on the true experiences of people who have received ECTâ. The next day he added âI hope nobody who is the slightest bit suggestible or with an axe to grind about Psychiatry or ECT feels tempted to participate.â
The same psychiatrist posted a snapshot of our question about the gender of the prescriber, sarcastically adding âSmash the Patriarchy.â Another antagonist alleged there were âincredibly poor ethical & procedural standards in useâ. Our asking whether people were âforcedâ or âpressuredâ into having ECT was criticised for somehow âleading the witnessâ. Furthermore, I was apparently going to âprofit offâ the survey because it is part of my paid work for my University.
The fact that we are inviting family and friends to participate was described as âDubious ethically re: reporting on behalf of others.â Several proudly admitted filling out the survey themselves. One even boasted about doing so twice, once âon behalf of my Asian son from Germany who is the same age as me.â
Another pro-ECT psychiatrist had a more productive response to the allegation that our survey was bound to elicit a biased sample: âI guess the correct way is to perform a representative survey of ECT patients. We will look into this.â Good.
Amidst these rather unpleasant, unprofessional jibes, there were some legitimate points. Perhaps the most important was the issue of online surveys not being representative of the larger population in question and, therefore, potentially biased. Before discussing that, two other potential weaknesses of this research method, and some of its strengths, I will summarise three previous online surveys with which I have been involved.
Three Examples of Online Surveys
A survey of antidepressant users
Towards the end of my 20 years at the University of Auckland, two colleagues and I, alarmed at the ever-increasing prescription rates of antidepressants, in New Zealand and around the world, were trying to figure out how to research the issues while struggling to find the time to meet our basic academic duties. The resulting online questionnaire had 47 questions, covering demographics; the prescribing process; information about antidepressant usage and perceptions of their effectiveness; side-effects; benefits; experiences of alternative treatment options; and beliefs about the causes of depression. The questionnaire consisted of multiple-choice questions, rating scales and a few open-ended questions.2
More than 1800 antidepressant users in New Zealand responded. The first paper focussed on the adverse effects. Eight of the 20 adverse effects studied were reported by over half the respondents, most frequently âSexual Difficultiesâ (62%) and âFeeling Emotionally Numbâ (60%). Other findings included âWithdrawal Effectsâ (55%); âFeeling Not Like Myselfâ (52%), âReduction in Positive Feelingsâ (42%), âCaring Less About Othersâ (39%), and, surprisingly for a drug that was supposed to reduce depression, âSuicidalityâ (39%). With restrained understatement, my Kiwi colleagues and I concluded âThe adverse effects of antidepressants may be more frequent than previously reported, and include emotional and interpersonal effectsâ.
This survey produced such rich and broad data that it eventually spawned 10 other papers in peer-reviewed scientific journals, on topics from causal beliefs3 and gender issues4,5 to the finding that a positive relationship with the prescriber predicted a positive outcome.6 But perhaps the most impactful follow-up paper was the one taking a closer look at withdrawal.7 Besides the 55% reporting withdrawal effects, 27% said they felt âaddictedâ. The longer people had been on the drugs the more likely they were to report addiction and withdrawal when they tried to stop. Paroxetine was found to be particularly likely to cause withdrawal effects. Most alarming, perhaps, was the finding that only 1% had been told anything about withdrawal or the need to come off slowly.
Including antipsychotics
That first New Zealand survey was so productive I decided to replicate it internationally, when I moved, in 2015, to Swinburne University in Melbourne. Only this time I included antipsychotics, with which I was far more familiar than antidepressants. Â More than 1500 people from 38 countries responded.
The antidepressant half of the survey confirmed that the alarming findings of the first survey were not specific to New Zealand.8 This time our conclusions were less guarded:
âAsking people directly reveals far higher rates of adverse responses to antidepressants than previously understood, especially in the under researched emotional, psychological and interpersonal domains. Given recent findings that antidepressants are, at best, only marginally more effective than placebo, these findings, especially in relation to suicidality and addiction, produce a cost-benefit analysis that cannot justify the continued high prescription rates for these drugs.â
The antipsychotic section, responded to by over 800 people, from 30 countries, led to five papers, reporting some surprising and invaluable findings. For example, 14 adverse effects were reported by more than half of respondents, most commonly: âDrowsiness, feeling tired, sedationâ (92%), âLoss of motivationâ (86%), âSlowed thoughtsâ (86%), and âEmotional numbingâ (85%).9 Fifty-eight percent reported that the drugs made them feel suicidal. Most (70%) had not been told anything about any of these, or any other, adverse effects. While 35% reported that their âquality of lifeâ was âimprovedâ, 54% reported that it was made âworseâ. Importantly, duration of treatment was unrelated to positive outcomes but significantly related to negative outcomes. Most respondents (70%) had tried to stop taking the drugs. The most common reason they had wanted to stop was the debilitating adverse effects (64%) of these powerful tranquillising drugs.
The value of including open-ended questions in online surveys was demonstrated by responses to the questions âOverall in my life antipsychotic medications have been _____?â and âIs there anything else you would like to say, or emphasise, about your experiences with antipsychotic drugs?â10 Thematic analysis identified 749 negative, 180 positive, and 53 mixed statements. The two positive themes were âsymptom reductionâ and âsleepâ. The four negative themes were: âadverse effectsâ, âinteractions with prescriberâ, âwithdrawal/difficult to get off themâ, and âineffectiveâ. The one mixed theme was âshort-term good, long-term badâ. We concluded:
âOpen questions can add to findings from methodologies focused on symptom reduction. Clinicians should pay more attention to the need for respectful and collaborative patient-prescriber relationships. At the point of prescription, this must include providing the full range of information about antipsychotics, including potential benefits and harms, difficulties withdrawing, and information on alternatives treatments such as psychological therapies.â
The paper included more than 100 quotes from the respondents, which brought to life the people behind the statistics. For example:
- âHaving the medication makes it possible to thrive instead of just surviving.
- âMaybe helpful for a short time, but likely not worth it in the end they clouded my perceptions and sense of self very badlyâ
- âThey are not âantipsychoticâ they just helped me sleep.â
- âI put on 27 kg and developed diabetes.â
- âSevere sedation made it impossible to continue.â
- âThey dumb me down & numb me up, I have no happiness or joy.â
- âThey made me feel less than human, dead inside.â
Another paper confirmed that, like the public but unlike most psychiatrists, the majority of people who are prescribed antipsychotics have psycho-social rather than bio-genetic causal explanations for their difficulties11. For example:
- âI believe being raised in a house with a dad with anger issues led me to live in constant fear as a child. But I didn’t understand that fear so instead it changed to paranoid thinking like monsters or demons were coming to kill me. Then I developed protective delusions (such as me being a werewolf or God).â
- âI suffered abuse as a child which I think caused abnormalities in my beliefs, self image, perception of the world, and my emotional responses.â
- âPast trauma and environment, isolation, father’s suicide, holocaust 1st generation German Jew, seeing my mom pass away from a heart attack.â
The two other papers addressed âHow important are informed consent, informed choice, and patient-doctor relationships, when prescribing antipsychotic medication?â12 and the virtually taboo topic of withdrawal from antipsychotics.13 Most people (72%) reported withdrawal effects when they tried to come off their antipsychotics, but none had been told anything about withdrawal effects or the need to reduce gradually. On a more positive note, 26% reported positive effects of withdrawing, such as feeling more alive.
These first two surveys highlighted the high incidence of withdrawal effects. When included in a broader review of the relevant research,14 they informed a seminal Public Health England review of prescribed drug dependence15 and radical improvements in the previously misleading guidance on antidepressant withdrawal from the UKâs Royal College of Psychiatrists and National Institute for Health and Care Excellence (NICE).16
What do people trying to withdraw from psychiatric drugs need?
The third survey, designed and conducted with two British psychiatrists, led to the 2023 paper âDesigning Withdrawal Support Services for Antidepressant Users: Patientsâ Views on Existing Services and What They Really Needâ17. Whereas the two previous surveys recruited openly via social media and âsnowballingâ, this one invited only the members of 15 online support groups for people stopping antidepressants. From 49 countries, 1276 people responded.
Most (71%) found their doctors’ advice regarding stopping an antidepressant unhelpful; the main reasons being âRecommended a reduction rate that was too quick for meâ, âNot familiar enough with withdrawal symptoms to advise meâ and âSuggested stopping antidepressants would not cause withdrawal symptomsâ. A range of potential professional services were rated âvery usefulâ, most frequently: âAccess to smaller doses (e.g. tapering strips, liquid, smaller dose tablets) to ensure gradual reductionâ (88%) and âA health professional providing a personalised, flexible reduction planâ (79%). We concluded:
âMost participants reported their prescribers were unable to help them safely stop antidepressants, compelling them to turn to online peer-support groups instead. Our findings indicate, in keeping with previous studies, that clinicians require upskilling in safe tapering of antidepressants, and that patients need specialised services to help them stop safely.â
The findings of all three surveys are now integral to our ongoing efforts to compel the UK National Health Service to provide support for the, literally, millions of people trying to get off psychiatric drugs with no, or ill-informed, support from their prescribers.
The Limitations of Online Surveys
Amidst the rather venomous attacks on these surveys, and the current ECT survey, from those who like to dismiss their findings and our work as âanti-psychiatryâ, âpill-shamingâ etc. there are two very reasonable concerns. Because online surveys are typically not completed by random samples of the population in question (i.e. all antidepressant users, all ECT recipients), but by âconvenience samplesâ, they are wide open to generating biased, unrepresentative samples. Secondly, anybody can fill out the survey, perhaps multiple times, which casts serious doubt over the integrity of the data.
Unrepresentative, biased samples
One inevitable bias is that not everyone has access to the internet. And can a survey really be called âinternationalâ if we havenât provided multiple translations? Although the last two surveys summarised above elicited responses from many countries, most came from the UK or the USA and were predominantly from white people.
The major concern raised by critics, however, is that the surveys may be disproportionately responded to by people with bad experiences of the treatment in question, or with âan axe to grindâ. This is a particularly legitimate worry where recruitment occurs partly via social media. It should be noted, however, that the bias could be in either direction. For example, in the âLimitationsâ section of the first paper from the New Zealand survey,2 we wrote:
âThe very high frequencies of adverse effects raise the issue of whether recipients who were dissatisfied with their medication were more likely to participate. This seems very unlikely, however, given that the majority (82.8%) reported that they believed the drugs had reduced their depression, a rate far higher than most conventional efficacy studies of ADs. If the survey had attracted a disproportionate number of unsatisfied people one might have expected a much higher rate reporting a reduced quality of life while on ADs (8.9%). Thus the sample appears to be biased towards people who had a positive response to ADs.â
In the international survey8,9 over half (56%) thought the antipsychotic drugs reduced the problems they were prescribed for, and two-thirds (65%) reported that the antidepressants had reduced their depression. Again, both rates of positive outcomes are higher than found in traditional drug trials.
The current ECT survey is clearly biased towards those who had a positive response to, and/or were least harmed by, ECT. Those still severely depressed or suicidal, and those with severe cognitive damage, will be less likely or able to take part in the study. And, of course, those for whom the ECT did not prevent them from killing themselves will not be represented (except via their relativesâ reports). Nor will the smaller number who have died as a result of the ECT itself, usually via cardiovascular failure.18
Anyone can complete it, repeatedly
It is true that people who have not received the treatment in question can fill out an online survey. What might motivate someone to do so? Perhaps they had a relative who had a bad experience, and they want that to be represented? Perhaps they have personally had a bad experience with mental health services but not with this particular treatment? Perhaps they are somehow ideologically opposed to treating human distress with chemicals and electricity? Perhaps they are a psychiatrist, or a drug company executive or employee, with a vested financial or professional interest?
The survey platform, Qualtrics, does flag up respondents that engage in what they call âstraight liningâ, meaning responses with little variation. For example, we could exclude people who respond to all 25 possible adverse effects of ECT with the same choice from ânot at allâ âmildâ âmoderateâ or âsevereâ. Â That decision, however, is also open to the effects of bias, on the researchersâ part. It would rule out people who genuinely had none of the adverse effects. If such decisions are to be made, they should be made before seeing the responses. It must be acknowledged that there is no surefire way to prevent people wo do not meet the inclusion criteria for a survey taking part, or to eliminate their responses once submitted.
Another concern expressed by critics is that people with a negative view, or âan axe to grindâ may be motivated to take the survey multiple times. This can and does happen, but all multiple responses from the same Internet Protocol (IP) address, unique to every device connected to computer network, are deleted (unless they have markedly differing responses indicating the use of the same computer by more than one person). This was the case for two out of 1,831 respondents in the New Zealand study. Checking IPs also reduces any contamination from people who have not received the treatment in question since they cannot submit multiple times. (In the case of the critic who boasted on X/Twitter that he had filled out the survey twice, we will probably delete both).
Self-report
Online surveys report the experience, opinion or feelings of the person completing the survey. I am always puzzled by the notion that we should not rely on self-report because it is subjective. Most research in mental health relies on either the self-report of a patient answering questions (sometimes from a structured test) from a psychiatrist or mental health professional, or on the self-report of a researcher or mental health professional expressing their opinion about a patientâs recovery etc. (again, sometimes from a structured list of questions). Who knows better how they are feeling, or whether their memory had been damaged, than the person themselves? Not that their perceptions are indisputably accurate, of course. But who is likely to be more accurate?
A more legitimate concern is that of memory, especially, ironically, when the treatment in question, like ECT, causes so much memory damage. One option is to gather data from a second person, as we are doing by including relatives and friends in our ECT survey. But their memory is fallible too.
Strengths of Online Surveys
Weighed against these important limitations are several strengths.
- Online surveys can generate numbers far larger than traditional studies such as randomised controlled trials (RCTs).
- They can gather data on people who have been on the drug in question for a range of periods, up to several years, as compared to the 8 or 12 weeks typically used in RCTs
- They are not subject to the bias that has been demonstrated in drug trials run by drug companies.
- By the use of open questions, they can generate rich, nuanced, quantitative data of the kind generated by small studies using interviews, but with hundreds of people rather than eight or so.
- Completely open questions, like âIn my life antidepressants/ECT has been âŚâŚâŚâŚâŚâŚâŚ.â can genuinely give people a voice to say what they want to say. rather than provide data that answers a researcherâs preconceived hypothesis.
- They are easier, and take less time, to conduct than some other types of studies.
- They are cheap, and therefore do not usually require research grant applications.
Conclusion
I am not suggesting that online surveys are somehow inherently better than other methodologies. I believe in a âhorses for coursesâ approach whereby whichever methodology best (and most easily) answers the research questions should be adopted. I am suggesting, however, that online surveys have, if they pay attention to their limitations, earned the right to be on the menu.
Editor’s Note: The online survey for ECT recipients and their families can be found here: https://uelpsych.eu.qualtrics.com/jfe/form/SV_57KmQWiynXIhMNw
References
- READ, J. (2024). International survey of electroconvulsive therapy. Psychology Today, Jan. 29. https://www.psychologytoday.com/gb/blog/psychiatry-through-the-looking-glass/202401/international-survey-of-electroconvulsive-therapy
- READ, J., et al. (2014). Adverse emotional and interpersonal effects reported by 1,829 New Zealanders while taking antidepressants. Psychiatry Research 216, 67-73.
- READ, J., et al. (2014). Beliefs of people taking antidepressants about causes of depression and reasons for increased prescribing rates. Journal of Affective Disorders, 168, 236-242.
- CARTWRIGHT, C., et al. (2018). Personal agency in womenâs recovery from depression: The impact of antidepressants and womenâs personal efforts. Clinical Psychologist, 22, 72-82.
- GIBSON, K., et al. (2018). Conflict in menâs experiences with antidepressants. American Journal of Menâs Health, 12, 104-116
- READ, J., et al. (2015). Understanding the non-pharmacological correlates of self-reported efficacy of antidepressants. Acta Psychiatrica Scandinavica 131, 434-445.
- READ, J., et al. (2018). How many of 1,829 antidepressant users report withdrawal symptoms or addiction? International Journal of Mental Health Nursing, 27, 1805-1815.
- READ, J., WILLIAMS, J. (2018). Adverse effects of antidepressants reported by a large international cohort: Emotional blunting, suicidality, and withdrawal effects. Current Drug Safety, 13, 176-186
- READ, J., WILLIAMS, J. (2019). Positive and negative effects of antipsychotic medication: an international online survey of 832 recipients. Current Drug Safety, 14, 173-181.
- READ, J., SACIA, A. (2020). Using open questions to understand 650 peopleâs experiences with antipsychotic drugs. Schizophrenia Bulletin, 46, 896-604.
- READ, J. (2020). Bad things happen and can drive you crazy: The causal beliefs of 701 people taking antipsychotics. Psychiatry Research, 285, 112754.
- READ, J. (2022). How important are informed consent, informed choice, and patient-doctor relationships, when prescribing antipsychotic medication? Journal of Mental Health, doi:1080/09638237.2022.2069708
- READ, J. (2022). The experiences of 585 people when they tried to come off antipsychotic drugs. Addictive Behaviors Reports, 15,
- DAVIES, J., READ, J. (2019). A systematic review into the incidence, severity and duration of antidepressant withdrawal effects: Are guidelines evidence-based? Addictive Behaviours, 97, 111-121.
- TAYLOR, S., et al. (2019). Dependence and withdrawal associated with some prescribed medicines: An evidence review. London: Public Health England, 2019.
- HENGARTNER, M., et al. (2020). Antidepressant withdrawal â the tide is finally turning. Epidemiology and Psychiatric Sciences, 29: e52, 1-3.
- READ, J., et al. (2023). Designing withdrawal support services for antidepressant users: Patientsâ views on existing services and what they really need. Journal of Psychiatric Research, 161, 298-306.
- READ, J., et al. (2019). Electroconvulsive Therapy for depression: A Review of the quality of ECT vs sham ECT trials and meta-analyses. Ethical Human Psychiatry and Psychology, 21, 64-103.
I’m sorry, but in a system that labels a person non complaint when they don’t take said treatment, isn’t this a bit like going to a country where it’s against the law to speak against something and expecting the responses to be what they really think when asked about what they’re not allowed to speak against. Let alone, whether they are allowed to think, and were they how they would respond.
I can’t believe the trite responses shared in the first paragraph, but then I’m not surprised either.
Report comment
Yes, and within the paternalistic systems of America and Western civilization, legitimately concerned mothers are NOT allowed to speak about concerns of child abuse.
While the primary actual societal function of both the psychological and psychiatric industries of America and Western civilization is, in fact, systemically covering up child abuse and rape … in my case for my ex-religion … which is a very “hard pill to swallow,” but sadly true.
https://www.indybay.org/newsitems/2019/01/23/18820633.php?fbclid=IwAR2-cgZPcEvbz7yFqMuUwneIuaqGleGiOzackY4N2sPeVXolwmEga5iKxdo
https://www.madinamerica.com/2016/04/heal-for-life/
https://books.google.com/books?id=xI01AlxH1uAC&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q&f=false
Many thanks to you, John Read, for all the truth seeking and speaking you do. Your research and truth telling is invaluable!
Report comment
I think the psychiatrists who behaved like spoiled children in response to your survey only respect research paid for by pharma and the makers of ECT machines. That research just has a certain rigor that’s absent from all “critical psychiatry” research. And if they don’t like the results of the pharma funded research, they can just throw out the study, and all of the accompanying data (and shred it twice), and start fresh.
Report comment
I look forward to reading the results of this study.
Report comment
Permanently disabled, catastrophically harmed = having an axe to grind, according to the entrenched dominant narrative of the psycho pharmaceutical complex.
They might as well say, “if you were harmed by our drugs or our shock treatments or by anything else psychiatry subjected you to, keep quiet as you don’t matter in the least. No filling out of surveys for you.”
Report comment
â¨ď¸
Report comment
It’s very telling to me that they try to write off patient narratives in advance of publication. If the treatment were so great, why wouldn’t they be proudly promoting the survey and more exploration? Surely all the patients would talk about how great it is too.
Report comment
I’m about to get kicked off of disability caused by ECT and psych polypharmacy and will be without any income and alone at 57. There was *never* a path to return to the work force.
How many are there like me? Alone and broken by psychiatry, all the while psychiatrists make jokes like adolescents about filling out the survey twice. Disgusting. I hope a special place in hell awaits them.
Report comment
Sadly, tons – I’m an historian and the memories I’ve lost are devestating.
Report comment
The fact that the US federal government gave Yale a million dollar grant to study ECT on dementia patients when there is no evidence showing that ECT is safe or effective for any person shows that something extremely rotten is going on.
Report comment
I had never heard of this outrageous grant before. ECT on dementia patients? And we criticize Mengele and other Nazi doctors for experimenting on helpless inmates? Can you provide a link on this subject?
Report comment
Yes, here it is. I think MIA published something about it also.
https://medicine.yale.edu/news-article/wilkinson-rhee-awarded-nih-research-project-grants/
Report comment
That’s right. There is no evidence that ECT is safe and effective. The FDA said so itself. “…the long-term safety and effectiveness of ECT treatment has not been demonstrated.” https://www.ecfr.gov/current/title-21/chapter-I/subchapter-H/part-882/subpart-F/section-882.5940
Report comment
I had between 20-30 bilateral ECTs, the most damaging kind, when I was 16 at a state hospital built in the 19th century which didnât even have heat in the bedrooms in winter. They were administered in the most cruel and sadistic way with an injection of muscle relaxant without sedation, resulting in excruciating pain and fears of dying. I have written before about my experiences with ECT and the state hospital for Mad in America (see Michael Sturman, Jan.23, 2020, and June 1, 2023), and about the failure of the shock device manufacturers and the FDA to provide acceptable scientific evidence of their safety and effectiveness (May 24, 2022). I am not surprised that Mr. Readâs on-line study of shock recipients was met by some in the industry with derision. This has been their attitude all along. In the meanwhile, countless patients undergoing ECT are risking brain damage which has been in some cases so severe that they can no longer resume their educational or career pursuits. In my own case, I experienced difficulties in finding words in simple daily conversations and in writing, and an overall loss of what I had learned before in school. It has taken me years to recover. In the meanwhile, I suffered embarrassment because of my speech and difficulties in my academic pursuits. In addition, I suffered from symptoms of PTSD as a result of my experiences for many years, including painful daily recollections, nightmares, flashbacks, and fears of further state hospitalizations and shock treatments. As matters now stand, I donât believe shock treatment has a legitimate place in this era of scientific medicine. I took Dr. Readâs on-line survey and encourage others to do the same. I believe it will show a large number of shock recipients have also experienced cognitive difficulties following shock– far more than the shock industry is willing to admit in their greedy pursuit of patient dollars and their disregard of what comes after ECT.
Report comment
The PTSD is so real – and now the general mistrust of ‘medical’ treatments – but of course, doctors want to say I’m paranoid, but if I only hold these fears about those who have actually hurt me and caused life-altering changes, that does not meet the criteria for paranoia in a clinical sense – all psychology/psychiatry is experimental and as a field it should be labeled as such – your doctor should be Dr. Sally Smith, Experimental Psychiatrist.
Report comment
My favorite is on your charts if you say you don’t smoke they mark: denies
There is a presumption you’re telling the truth about symptoms and lying about everything else – well, that’s actually crazy!
Report comment
I’m about to get kicked off disability because I won’t subject myself to a medical appointment where a stranger doctor looks me over and then writes down what he thinks. “Is the patient lying, exaggerating, malingering? Does the patient show signs of mental disorder? Is the patient on drugs? How much can she lift? How Long can she stand? How far can she crawl?
What are her diagnoses?”
I guess Yale doesn’t remember diagnosing me with major depressive disorder and then (after ECT) borderline personality disorder and fibromyalgia and arthritis and poly-substance abuse.
Report comment