Thank you for your feedback regarding my article. I want to clarify that I am presenting the data from the study as I read it; MIA is not suggesting anything, nor am I. I am presenting the facts: the study found that psychiatrists delivered the worst quality healthcare of any medical specialty in all domains, and the authors suggest that the solution would be ceasing to measure the quality of psychiatric care.
You are correct that the authors argue that these measures are not relevant to psychiatry, and in fact that there is no way to measure whether psychiatrists are delivering quality care at all (which seems problematic to me personally, but I reported it as is).
I notice that one of the measures psychiatry failed at is documenting patients’ medication, on which psychiatrists scored almost 10 points lower than other specialties. Surely you don’t mean to suggest that documenting patients’ medication is irrelevant to psychiatry as currently practiced?
Best,
Peter
Cabrogal, I actually have one more comment. I re-read the study yet again since I was thinking about this today, and I discovered that your point about the low dose of ayahuasca is not actually really a limitation.
Yes, the researchers in this paper CALLED it a limitation. However, while the researchers called this a lower dose, there’s really nothing to compare it to. It’s not as if there is an established dose of ayahuasca for clinical work. So, the researchers note that their dose was just somewhat lower than in two other studies.
However, the researchers also say that the amount they used was actually the *usual dose* of ayahuasca and they were not responsible for preparing it, the people in charge of the ritual were. So, really, this study used the standard dose of the drugâand those two other studies gave *excessive* amounts of the drug, which is actually a way of biasing a study. So, no, this is not a low/placebo dose, it is a standard dose, and it actually makes the study stronger.
From the study: “Study participants received 7 capsules with the option of taking 3 additional ones as a booster, after about 2 h of the first dose. A dose of 7 capsules was portrayed by the host organization as similar to as [sic] regular volume of ayahuasca brew.”
Thanks again for your questions. It’s an interesting topic.
I agree entirely, Steve. Great point.
Dear Cabrogal,
Thank you for your critique. I would like to clarify some of the issues you brought up.
1. I wrote âA study on the mental health effects of the psychedelic drug ayahuasca found that the drug was no better than a placeboâ
You wrote, “Err, no it didnât.”
Yes, it did. The drug was no better than placebo on the outcomes of anxiety, depression, or stress. Yes, the researchers tested all sorts of outcomes (for instance, measures of âmindfulnessâ; padding their study so that somethingâanything at allâis likely to be positive, just by chance) and there was one outcome that showed a difference. That, as you correctly noticed, was “emotional empathy,” which I considered to be irrelevant, poorly operationalized, and likely a false positive anyway, so I did not think it even worth mentioning.
To be clear: They found one irrelevant outcome to be slightly better for ayahuasca in a study with many outcomes, including highly relevant ones, that were no different between placebo and drug.
On the RELEVANT outcomes, the researchers wrote: “Compared to baseline, symptoms reduced in both groups after the ceremony, INDEPENDENT OF TREATMENT” (emphasis mine).
2. I wrote: âIn fact, both groups experienced about the same level of psychedelic effects, too. The researchers write that âparticipants in both groups experienced altered states of consciousness during the ceremony.â
You wrote: “Thatâs not at all what it says either. âContact highsâ are a thing, so youâd expect both groups to experience altered states. But unless the dose is quite small they would definitely not experience the same level of psychedelic effects. And sure enough, according to Fig 2 the ayahuasca group experienced significantly greater psychedelic effects than the placebo group in all categories except âEgo Dissolution Inventoryâ and âreduction of vigilanceâ.”
Nope. There were two measures of the psychedelic experience, the EDI (ego dissolution inventory) and the 5-Dimensional Altered States of Consciousness Rating Scale (5D ASC). You correctly identified that there was no difference between ayahuasca and placebo on the EDI. However, you wrote that there were significantly greater psychedelic effects in subscales of the 5D ASC. Iâm afraid thatâs just not true. There are 16 different subscales of the 5D ASC, all reported on in the supplemental materials, and only one subscale reached p<0.05 (the most liberal definition of statistical significance)âand that was âaudio visual synesthesia.â The other 15 subscales did NOT demonstrate a statistically significant difference between ayahuasca and placebo.
The researchers themselves admit this: âMean ratings of EDI and total 5D-ACS (dimensions and subscales) did not significantly differ between conditions and did not significantly interact with ayahuasca use experience of the study participants.â
So, yes, it is accurate for me to write that the two groups experienced the same level of psychedelic effect.
3. You wrote: âThat said, itâs important to remember that itâs not the drug that effects healing in psychedelic therapy. The drug (in sufficiently high doses) merely temporarily knocks down the ego so the sufferer can gain insights into her condition that were obscured by her own self-image and notions of how she relates to her suffering and the aspects of her self/experience/environment that give rise to it. Itâs up to the sufferer herself to decide what to do with those insights.â
Except that this study explicitly showed that the drug DID NOT AFFECT EGO DISSOLUTION any more than the placebo did (see my response #2, above).
4. You wrote: âItâs also misleading to imply âritual effectâ=âplacebo effectâ. In fact the ritual has important socio-spiritual components that serve to reintegrate the sufferer with his community and environment, thereby addressing aspects of disorders typically neglected by Western medicine.â
This is an interesting comment. In this case, I used the term placebo effect/response for two reasons: one, that is how the researchers used it in this paper; and two, because the study was about comparing ayahuasca versus placebo drug specifically.
The term placebo effect is generally used to include a variety of things, including expectation effects, and usually it also helps control for things like regression to the mean which, obviously, is not an âeffectâ of the placebo. In its strictest sense there is no such thing as a placebo âeffectâ because by definition, placebos are substances without an effect. But that usage strikes me as pedantic. The term placebo effect, when used to encompass all of the aspects of the difference between a drug group and a control group, is helpful and, I think, operationalized well enough to be clear.
In a comparison like this, the response of the group taking the placebo includes the expectation effect, and one could argue that for most Americans, the medical field carries more expectation of benefit than a mystic ritual would. So Iâd suggest that all medical placebo effects are due at least in part, and Iâd say in my opinion greatly, to faith in medical science.
Thus, the argument that the ritual shouldnât be called a âplacebo effectâ is a semiotics question, but not a helpful distinction when the point of the study is comparing the drug + ritual versus the ritual alone.
4. You wrote: âIn other words they were using sub-therapeutic doses, so it would have been quite surprising to see a strong drug-mediated response, especially as the subjects werenât even suffering from the disorders used as response measures.â
This is true, and I think it is a legitimate limitation of the study. Good job noticing that one.
I just wanted to clarify that the requirement for 2 statistically significant results is only for the *regular* approval process. As the article states, the accelerated approval pathway (81% of current drug applications) does *not* have this requirement.
Hi Sandy,
Thank you for your feedback regarding my article. I want to clarify that I am presenting the data from the study as I read it; MIA is not suggesting anything, nor am I. I am presenting the facts: the study found that psychiatrists delivered the worst quality healthcare of any medical specialty in all domains, and the authors suggest that the solution would be ceasing to measure the quality of psychiatric care.
You are correct that the authors argue that these measures are not relevant to psychiatry, and in fact that there is no way to measure whether psychiatrists are delivering quality care at all (which seems problematic to me personally, but I reported it as is).
I notice that one of the measures psychiatry failed at is documenting patients’ medication, on which psychiatrists scored almost 10 points lower than other specialties. Surely you don’t mean to suggest that documenting patients’ medication is irrelevant to psychiatry as currently practiced?
Best,
Peter
Cabrogal, I actually have one more comment. I re-read the study yet again since I was thinking about this today, and I discovered that your point about the low dose of ayahuasca is not actually really a limitation.
Yes, the researchers in this paper CALLED it a limitation. However, while the researchers called this a lower dose, there’s really nothing to compare it to. It’s not as if there is an established dose of ayahuasca for clinical work. So, the researchers note that their dose was just somewhat lower than in two other studies.
However, the researchers also say that the amount they used was actually the *usual dose* of ayahuasca and they were not responsible for preparing it, the people in charge of the ritual were. So, really, this study used the standard dose of the drugâand those two other studies gave *excessive* amounts of the drug, which is actually a way of biasing a study. So, no, this is not a low/placebo dose, it is a standard dose, and it actually makes the study stronger.
From the study: “Study participants received 7 capsules with the option of taking 3 additional ones as a booster, after about 2 h of the first dose. A dose of 7 capsules was portrayed by the host organization as similar to as [sic] regular volume of ayahuasca brew.”
Thanks again for your questions. It’s an interesting topic.
I agree entirely, Steve. Great point.
Dear Cabrogal,
Thank you for your critique. I would like to clarify some of the issues you brought up.
1. I wrote âA study on the mental health effects of the psychedelic drug ayahuasca found that the drug was no better than a placeboâ
You wrote, “Err, no it didnât.”
Yes, it did. The drug was no better than placebo on the outcomes of anxiety, depression, or stress. Yes, the researchers tested all sorts of outcomes (for instance, measures of âmindfulnessâ; padding their study so that somethingâanything at allâis likely to be positive, just by chance) and there was one outcome that showed a difference. That, as you correctly noticed, was “emotional empathy,” which I considered to be irrelevant, poorly operationalized, and likely a false positive anyway, so I did not think it even worth mentioning.
To be clear: They found one irrelevant outcome to be slightly better for ayahuasca in a study with many outcomes, including highly relevant ones, that were no different between placebo and drug.
On the RELEVANT outcomes, the researchers wrote: “Compared to baseline, symptoms reduced in both groups after the ceremony, INDEPENDENT OF TREATMENT” (emphasis mine).
2. I wrote: âIn fact, both groups experienced about the same level of psychedelic effects, too. The researchers write that âparticipants in both groups experienced altered states of consciousness during the ceremony.â
You wrote: “Thatâs not at all what it says either. âContact highsâ are a thing, so youâd expect both groups to experience altered states. But unless the dose is quite small they would definitely not experience the same level of psychedelic effects. And sure enough, according to Fig 2 the ayahuasca group experienced significantly greater psychedelic effects than the placebo group in all categories except âEgo Dissolution Inventoryâ and âreduction of vigilanceâ.”
Nope. There were two measures of the psychedelic experience, the EDI (ego dissolution inventory) and the 5-Dimensional Altered States of Consciousness Rating Scale (5D ASC). You correctly identified that there was no difference between ayahuasca and placebo on the EDI. However, you wrote that there were significantly greater psychedelic effects in subscales of the 5D ASC. Iâm afraid thatâs just not true. There are 16 different subscales of the 5D ASC, all reported on in the supplemental materials, and only one subscale reached p<0.05 (the most liberal definition of statistical significance)âand that was âaudio visual synesthesia.â The other 15 subscales did NOT demonstrate a statistically significant difference between ayahuasca and placebo.
The researchers themselves admit this: âMean ratings of EDI and total 5D-ACS (dimensions and subscales) did not significantly differ between conditions and did not significantly interact with ayahuasca use experience of the study participants.â
So, yes, it is accurate for me to write that the two groups experienced the same level of psychedelic effect.
3. You wrote: âThat said, itâs important to remember that itâs not the drug that effects healing in psychedelic therapy. The drug (in sufficiently high doses) merely temporarily knocks down the ego so the sufferer can gain insights into her condition that were obscured by her own self-image and notions of how she relates to her suffering and the aspects of her self/experience/environment that give rise to it. Itâs up to the sufferer herself to decide what to do with those insights.â
Except that this study explicitly showed that the drug DID NOT AFFECT EGO DISSOLUTION any more than the placebo did (see my response #2, above).
4. You wrote: âItâs also misleading to imply âritual effectâ=âplacebo effectâ. In fact the ritual has important socio-spiritual components that serve to reintegrate the sufferer with his community and environment, thereby addressing aspects of disorders typically neglected by Western medicine.â
This is an interesting comment. In this case, I used the term placebo effect/response for two reasons: one, that is how the researchers used it in this paper; and two, because the study was about comparing ayahuasca versus placebo drug specifically.
The term placebo effect is generally used to include a variety of things, including expectation effects, and usually it also helps control for things like regression to the mean which, obviously, is not an âeffectâ of the placebo. In its strictest sense there is no such thing as a placebo âeffectâ because by definition, placebos are substances without an effect. But that usage strikes me as pedantic. The term placebo effect, when used to encompass all of the aspects of the difference between a drug group and a control group, is helpful and, I think, operationalized well enough to be clear.
In a comparison like this, the response of the group taking the placebo includes the expectation effect, and one could argue that for most Americans, the medical field carries more expectation of benefit than a mystic ritual would. So Iâd suggest that all medical placebo effects are due at least in part, and Iâd say in my opinion greatly, to faith in medical science.
Thus, the argument that the ritual shouldnât be called a âplacebo effectâ is a semiotics question, but not a helpful distinction when the point of the study is comparing the drug + ritual versus the ritual alone.
4. You wrote: âIn other words they were using sub-therapeutic doses, so it would have been quite surprising to see a strong drug-mediated response, especially as the subjects werenât even suffering from the disorders used as response measures.â
This is true, and I think it is a legitimate limitation of the study. Good job noticing that one.
I just wanted to clarify that the requirement for 2 statistically significant results is only for the *regular* approval process. As the article states, the accelerated approval pathway (81% of current drug applications) does *not* have this requirement.