As a professor and psychologist with a specialization in evaluation, research methods and statistical analysis, as the actor Liam Neeson has shared in his portrayal of Bryan Mills in the collection of Taken movies, I too have a very particular set of skills. And yes, I have acquired these skills over what at times feels like a very long career. Due to the new movement across so many sectors to become evidence-based and hopefully evidence-driven, I am often contacted by various organizations seeking help with documenting how their efforts make a statistically significant difference when it comes to their clients’ success. After receiving a call from the Alterative to Meds Center (ATMC), a small residential treatment center in Sedona, Arizona, and learning about their approach to holistic medication tapering and addiction treatment efforts, I was intrigued.
Most of my research typically focuses on child development, at-risk youth, and organizations just trying to help kids experience a productive and enjoyable childhood. I have consulted for more than 150 school systems and organizations, and managed the evaluation efforts for more than $19.5 million in research funded by the National Science Foundation (NSF), U.S. Department of Education and the Department of Justice. But this request from ATMC was a little different. This request was coming from a treatment center focused on helping adults with their pharmaceutical drug dependency issues. And with my interest in helping kids not be drugged for acting like kids, and recently losing one of my best friends to a long battle with insomnia and endless medications which I believe led to his suicide, I felt compelled enough to say yes.
With so many other promising holistic avenues to drug dependency treatment available across the USA, and yet still in need of documenting how their treatment approaches are successful, I thought that sharing the methodological and statistical approaches I took, and lessons learned along the way, could be of benefit to treatment centers seeking to find such statistical confirmation. But I also hope this summary can help individuals and their families get a better understanding of what an evidence-based practice should offer as assurance to their clients. So let’s take a look at some of the essential aspects that must be considered for those seeking documentation of evidence-based treatment.
Please note, the Alternative to Meds Center has been in business since 2006, and has treated more than 1,600 clients since opening its doors. Assuming that not all of their clients experiencing serious psychological and dependency challenges had success in ending such reliance on the pharmaceutical industry, I felt compelled to advise them upfront (as I do with all clients) that there was a possibility the results might not statistically show they were making a significant positive difference. I also explained that it would require a good amount of work on their part to help coordinate and implement sound data collection efforts. And I warned that, as with every study, there will always be barriers faced and change needed due to unforeseen obstacles. We did indeed encounter numerous challenges along the way. But as I will share shortly, for the Alternative to Meds Center and the vast majority of their clients, this story has a happy ending.
First Things First… Measuring the Outcomes
In the world of academia, we often give our graduate students the following advice as they begin their thesis or dissertation efforts: “Start with the end in mind.” In other words, start by visualizing what facts or outcomes you want to document when your study is complete. I quite often give this same advice to my clients. In the case of the Alternative to Meds Center, I asked them, what outcomes do you want to find that would document that your efforts are successful? The variables they wanted to explore for such evidence of their success were: 1) to what extent their clients were able to stop the use of pharmaceutical drugs; 2) how the symptoms clients were having trouble with were reduced; 3) the improvement in their clients’ sleep quality; and 4) their clients’ overall improvement in wellbeing.
After deciding upon the outcome variables, the next step was to determine reliable (consistent) and valid (accurate) ways to go about documenting such outcomes utilizing a sound methodological statistical approach. I know this last sentence might sound a little ivory tower to some, but it is paramount to understand first and foremost. Why? Because in order to perform any type of statistical analysis seeking to document statistical significance — aka showing with numbers that there is high probability that the results did not happen by chance — ethical analysts are required to use measurements (i.e. statistics, numbers) and assessment tools or scales which have established reliability and validity. In the world of psychology, these are often referred to as psychometrics.
If you cannot be sure that what you are using to measure the outcomes you wish to document are actually consistently measuring the same thing over and over again (reliability), or more importantly measuring what you think you are measuring (validity), you might as well flip a coin, and save time and money. Because without highly reliable measurements being used which truly measure what we claim to measure, the 50/50 chance of flipping a coin and calling heads or tails is possibly just as accurate as a flawed approach to statistical analysis. Sadly, in a world where not enough psychology and medical journals require the documentation of the assessment tools’ reliability and validity, there is far too much misinformation and disinformation being shared with the public.
Testing 1, 2, & 3
Additionally, in order to perform an analysis that shows a client has experienced successful outcomes, at the bare minimum we must collect a baseline assessment (pretest), and perform a posttest assessment. These tests administered at pretest and posttest must also be identical tests. You must use the same statistics, scales, test, or survey if you will, throughout the effort. If you want to show change from intake to discharge, you have to consistently measure how the client changed during that period of time.
And if we are going to pursue what some consider a golden standard in psychological or behavioral research, we not only want to document that clients made progress from intake to discharge, but also document that the improvements continued or at least were sustained for months if not years after the treatment. So the first thing organizations seeking to establish evidence-based practices must do is ensure they are consistently using the same quality psychometric assessments, at intake, discharge and follow-up periods, to document their clients’ initial progress, as well as on-going wellbeing and sustained improvement.
Gathering Enough Data… the Right Way
To begin this initial study of ATMC’s efforts, there was ample data already collected to assess the effectiveness of the program from intake to discharge on most of the outcomes where they hoped to find evidence supportive of their treatment efforts. But what they were missing were follow-up assessments on how well their clients were doing months after care.
Thus, the next major task at hand for ATMC was to do follow-up assessments on their large sample of clients who had graduated from the program. After helping them build an online survey encompassing the existing and additional assessments needed to measure their outcomes (via a website from which I could collect the responses), the staff sent this survey link to clients that had been away from the ATMC residential treatment center for more than six months but less than four years. Another important caveat at this point, however, is to make sure you have your clients use the patient code or number connected to their treatment records, so you can connect the follow-up survey efforts to each individual client. It is also important to make sure you set up the survey link using this code number to provide access to the online survey, and as a quality control procedure to make sure they only complete the follow-up survey one time.
Due to the challenge of having up-to-date addresses and contact information for this sample of clients, this equated to about 300 clients they sent either an email or a letter to asking for them to help ATMC determine its strengths, challenges, and ways it can improve. And unfortunately, due to ATMC experimenting with differing approaches to what tools and assessments they used over the past few years, there were a number of data inconsistencies that caused us to discard a portion of the responses.
Please note, this follow-up assessment process did take several months at two different times to achieve. Part of this was due to their already busy staff not being readily available to perform the time-intensive task of reaching out and requesting the clients’ participation. The other reason for adding on a second phase was due to the need for more participants to respond. In other words, if at first you don’t succeed (at getting a large enough sample to analyze), try try again!
Why? Not to get all professorial on you again, but this is a very important part of a study’s worth and success. Because in order to perform statistical analysis, an analyst will need approximately 20 to 40 respondents for each outcome variable you are assessing with the survey. In the statistical world, this is called cell size. So for a survey that has assessments looking at analyzing three variables, you would need 60 (3 x 20) to 120 (3 x 40) respondents. Such numbers are needed to ensure that you have the power to perform such analysis. And if you don’t have a large enough sample, you could end up running into what is called Type I or Type II error. This respectively refers to assuming you found significant findings when you really didn’t, or not finding significant findings when you really did. In other words, publishing a study on the results of 20 or so clients is not enough to hang your hat on, or have clients feel confident about the treatment success.
The Long and Short of It
I could play the role of Nyquil and sedate you with an endless array of additional statistical specifics essential to truly documenting a case for evidence-based treatment practices. But basically, the long and the short of it is that in order to get started and take the best path forward to collecting data worthy and strong enough to power a sound analysis, or assessing if the research is worthy of believing, a study will need to focus on the following:
- Determine what the outcome variables are that you wish to document evidence of successful treatment with.
- Only collect data using assessments with established statistical evidence of reliability and validity; and avoid use of tools developed in-house when possible.
- Make sure to (at a minimum) pretest before treatment begins, after treatment ends and do a follow-up at least six months after the discharge from treatment.
- Seek to collect data from as many clients as you can to ensure adequate power for the analysis.
- And when possible, collect the data in a convenient format to reduce the effort needed by your clients and staff to collect the feedback (e.g., online survey portal).
- Find an ethical analyst to provide the outside evaluation and analysis of your efforts; aka avoid bias by not collecting, analyzing and producing the results in-house.
As a side note, such an approach, even if it doesn’t produce positive outcomes, can be quite beneficial to organizations seeking to determine what they are doing well and what needs to be revisited or improved. Such outcomes, good or not so good, can be used as operational analytics that on a regular basis can provide you with the information you need to pursue continuous improvement efforts.
Signs of Success for ATMC
So how did the ATMC study turn out? Well, the results suggest that ATMC has an excellent success rate. Upwards of 87.5% of their clients sharing feedback had discontinued the use of pharmaceuticals and remained med free for extended periods of time — ranging from six to forty-seven months. The analysis documents that ATMC’s holistic treatment process can significantly reduce a client’s dependency on pharmaceuticals, if not eliminate use entirely.
Additionally, for the large percentage of the clients who were able to dramatically reduce or end their pharmaceutical dependency during the ATMC treatment process, the study documents that the results are long lasting. Evidence also suggests that for a majority of the clients, the more time away equates to continued reduction in pharmaceutical medication use.
This study also identified a statistically significant decrease in symptom severity scores from intake to discharge from the clinic. Furthermore, a total of 83.5% of the responding clients shared that the improved symptom levels experienced before graduating from ATMC had continued for six months to four years. Meanwhile, 71.9% of ATMC clients report experiencing more positive social relationships and interactions in life after graduating. And more than 75% of ATMC graduates report experiencing quality sleep patterns continuing six months to four years after their residential care in the center.
Now of course, with any center seeking to help clients who have wrestled with extreme emotional, traumatic or psychological challenges for various amounts of extended periods of time, thus leading to a dependency on pharmaceutical drugs so strong that they needed treatment, one cannot assume everyone who leaves the doors of a clinic is going to experience positive outcomes. From the responses and data I reviewed, I must share that this was the case for a portion of ATMC clients who responded to the study and completed the follow-up surveys. And in the follow-up survey some of these clients expressed their unhappiness with the efforts provided by ATMC.
But this is why we collect data on a large sample of clients, so that we can analyze and determine what the bigger picture is when it comes to a center’s success. Without such analyses, many are left to read online reviews, where let’s face it, similar to some reviews I have occasionally posted on Yelp after an unfortunate dining experience, it is not uncommon for the more disappointed clients to share their thoughts more often than those who experienced something more satisfying.
Of course, with online or even snail-mailed surveys, there is the possibility that one could receive feedback emanating from either the choir singing your song, or a group of unhappy customers. But this is why I recommended sending the survey to all of ATMC’s clients who met the requirements of the golden standard we were seeking with the study. The goal was to get responses from a large sample of uniquely different clients who experienced various outcomes.
A total of 103 clients responded with completed follow-up surveys, equating to a 34% response rate (more than twice that of typical industry standards). And after analyzing the data file, and documenting larger standard deviations than one might want to see (aka a limitation), it became very apparent that the sample consisted of clients with various experiences at ATMC. But as the final analysis documents, for the majority of clients who took part in this study, the outcomes were what they had hoped for: a better life.
For access to more details on the study’s outcomes, please click here to view a summary of the report.
Limitations of the ATMC Study
To be clear, it’s not always as cut and dry as the text and bullets above might suggest. For most clients I have worked with, ATMC included, too often the internal data historically or currently collected provides many limitations to quickly performing an analysis capable of producing results. The ATMC efforts ran into several challenges or limitations (e.g., adopting existing reliable and valid scales, recruiting enough participants). Also, depending on the severity of these challenges, it could equate to endless hours for an analyst to enter, clean, merge, code, recode and analyze the data file or files. And then when you are basically analyzing a sample of clients who typically seek such help due to extreme symptomology, analyzing for outliers also becomes a challenge; when basically your whole sample of clients might appear at times to all be outliers, setting up camp at both ends of the normal distribution bell curve.
Missing data is also a challenge when it comes to analysis. In order to use consistent measures across your sample, basically you need a complete data set. When medical records are riddled with missing data points, because the staff didn’t consistently collect information on specific aspects for all clients in the sample being studied or clients chose not to answer, it often forces an analyst to either remove these participants from the sample or use questionable data replacement procedures to estimate what a participant might have answered (something I would never do or recommend). And when you only have a portion of the questions being haphazardly answered, it can affect the scales’ mean scores for your outcome variables and thus the reliability and validity of the assessment tools used. In other words, you need to try your best to collect the same complete data set for every client whenever possible in order to perform the most robust analysis and produce the best results. In the case of ATMC’s study, due to either missing data not collected during treatment, or inadequate client answers on the follow-up surveys, approximately 20 clients were removed from the sample of surveys returned.
I think one of the other major limitations to be addressed that can help with future efforts is to make sure you also are documenting the treatment being provided. You need to document in more precise terms the specific type and level of “treatment” each client received. Because if you are a holistic or integrated practitioner, you most likely have a clientele which come to you with extreme but often quite different circumstances, past experiences, drug dependencies and levels of differing symptom severity. As a result, your “treatment program” for each of these clients is probably slightly different or uniquely designed for each client’s needs.
This means that if we really want to tweak out specific results six months after care, most probably need to do a better job documenting what type of specific therapy and care the client received and how much of the specific treatments they received (dosage). This is something ATMC is planning to do for future studies, so they can more precisely determine why clients who had great success, or not enough success, experienced such outcomes. It is not enough for analysis purposes to just show how many weeks they were in your care. It is not enough to just show what date they left care, and not provide a rationale as to why they left care. If you really want to know which specific efforts within your organization are leading to the best results, you need to document the protocol and specific treatment they received.
Summary
Let’s be honest, the talk about alternative drug treatment therapies not having an adequate body of research supporting their efforts is a legitimate critique. But an absence of evidence does not necessarily mean that such evidence does not exist. By all means, just as the doctor who unsuccessfully “treated” my friend for insomnia should have to show that the drugs they are prescribing and the approach they are taking are not only safe but also effective, it should be an expectation that every alternative therapy clinic has evidence supporting their specific treatment efforts. This is how legitimacy can be earned, clients can be confident in the selection they make, and the public can be aware if a treatment approach or program is or is not legitimately helping clients. With the existing collection of data at these clinics and the statistical software available today, however, there should be no reason for every center or clinic (offering highly effective services across our nation to help support clients seeking treatment for drug dependency) to not have the evidence to show they can help.
This effort was intended to illustrate what I found, and how other clinics and centers could be pursuing similar research agendas. I wanted to show how with so many currently collecting an abundance of data on their clients, finding evidence of their effectiveness is achievable. But as I have highlighted, there are numerous limitations and challenges to turning this information into data that can actually produce statistically significant results.
To some degree, it feels as if the pharmaceutical industry has developed a pill to mask the symptoms for every normal and possibly abnormal human experience or behavior a person can report. Furthermore, far too many doctors are prescribing such drugs with little concern for the side effects of addiction, suicidal ideation and so many other warnings we have become all too desensitized to hearing in the last half of those surreal 60 second commercials. And to make matters worse, many insurance companies won’t pay for therapy without a formal diagnosis code, and far too many clinicians still want to believe it is their ethical obligation to prescribe such dangerous drugs.
According to Consumer Reports, in the USA the use of prescribed pharmaceutical psychotropic drugs has never been higher than it is today. Yet as the drugs’ medication guides, research and many voices have shared for decades, psychotropic drug use can be quite dangerous and rarely holds any positive long-term benefit. And due to the many black box warnings these prescriptions come with, in countless cases even short-term advantages are questionable. As our nation wrestles with the rise of a full-blown opioid crisis, we are once again reminded that often pharmaceutical drugs (e.g., benzos, depression meds, ADHD stimulants) lead to dependency if not outright addiction. As a result, many are seeking help, and looking for more integrated physical, mental and spiritual — or might we say natural, holistic or ‘green’ mental health based — avenues for drug dependency treatment.
I hope this article has helped you conceptualize what an evidence-based practice should document. I also hope that you can now see how clients and treatment centers can benefit greatly by making such information readily available, as the Alternative to Meds Center has discovered.
One thing to consider:
In regards to measuring success, you say: “The analysis documents that ATMC’s holistic treatment process can significantly reduce a client’s dependency on pharmaceuticals, if not eliminate use entirely.”
However, I am familiar with three people who attended this ATMC and 2 of them have protracted withdrawal lasting many years (more than half a decade in each case) and the third is now not only reinstated on the benzo he went to ATMC to get off of, he is now a polypharmacy patient on many other psych meds bc the too-rapid ATMC benzo withdrawal left him in such bad shape he was then drugged with even more meds (gabapentin, Remeron).
I went to a detox center myself, much different than ATMC of course, but similar outcomes as I was also removed too-rapidly from psychiatric drugs, including benzodiazepines and others. On paper, I might look like a “success” statistic because I have remained psych med free for the last 8 years. In reality, like my friends who attended ATMC (the 2 I referenced above who remained med free), I am very unwell and have been since I walked out of the detox center’s doors (I am lucky I survived what happened after I left, actually – psychosis, suicidality, etc). I am unable to work or function in society, I live in crippling pain, and I suffer from a long list of other disabling symptoms that persist since that center negligently removed me from psychiatric drugs far too rapidly due to time and money restraints (and just bad overall practice guidelines in general).
I am certain I would’ve been far better served tapering at home, at my own pace, under the care of a taper-wise prescriber. I’d also have the many thousands I paid them to cripple me.
Unless someone was going to get free of psych meds with minimal issues to begin with, which we are currently unable to predict who will be who, I don’t see how, with the restraints these places have (limited time because of cost to patient), ATMC and similar places can do right by people who many of which would be better served tapering very slowly in an outpatient setting over the many months and often years it takes people to be free of these chemicals to spare themselves protracted injury.
In summary, psych medication free does not always equal success.
Report comment
I’m sorry to hear that you are suffering.
I’d just like to point out that one of the primary purposes of the science of statistics is to get beyond anecdotes. It aims to capture population level phenomena.
While we are all individuals of unique circumstance, stats provide us one of the best methods of assessment we can reach in our less than omniscient human condition.
Report comment
Of course the purpose of science is to get beyond anecdote. My point was if you’re only looking at who stayed off of psych meds, you are probably counting people who aren’t really “success” stories as successes. Lots of people can and do stay off psych meds after barbaric over-rapid tapers and CTs. They’re not functional though or a “success” by any means.
Report comment
Wow, an article on the esteemed site, Mad in America, touting positive statistics for that “psych drug withdrawal” center, Alternative to Meds, in Sedona, AZ? Are you kidding me? I too know someone who went there, and they, the staff, and their “medical director,” in my opinion handled her acute withdrawal symptoms in ways that ended up practically killing this person. This patient first was told, upon admission, that her depressed, suicidal, psychotic-like state, which was directly caused by a too rapid taper of an SSRI a few months previously, was “not due to psychiatric drug withdrawal.” But this was precisely the reason this person sought treatment there and which was why their “treatment team” agreed to admit her. This patient was someone with no significant history of mental illness. It is widely reported on sites like SurvivingAntidepressants.org that high doses of supplements, which ATMC uses, can exacerbate problems in withdrawal sufferers. So can exercise, which ATMC also uses as part of their treatment. From what I witnessed, their philosophy for using the protocols they use apparently is based on the bogus, widely debunked “chemical imbalance” theory. Mishandling acute or protracted withdrawal symptoms can be life threatening, leading to violent behavior, suicide, and other adverse outcomes. I agree with Cole, above, that yes, some people can manage to “stay off” psych meds after they cold-turkey or rapid taper, such as what occurs at a place like ATMC, but the fact that people “stayed off” their meds DOES NOT mean they were successfully “withdrawn” from their psych meds. Instead, people who go off too quickly, whether at ATMC or on their own, are at risk of ending up disabled, bed bound, and completely nonfunctional for weeks, months, and years–which is what happened to the person I know who went there. Fortunately, she has now fully recovered–no thanks whatsoever to ATMC. Yes, this case, like the ones cited by Cole, may be anecdotal, but it needs to be reported here in order to keep additional AD withdrawal sufferers who read this article from giving thousands of dollars, out of desperation, to a place like ATMC, which might only make their condition worse. From what I’ve seen so far, the only proven healer for psych drug withdrawal symptoms is 1) EXTREMELY slow tapering, and 2) the passage of time, which are not what ATMC offers at their highly expensive facility.
Report comment
All this data-collecting must have been a huge effort. But you didn’t answer the most basic question. What is it like there?
You can collect all the data you want on “outcome” on places like McLean but that won’t tell you the food is inedible, the staff are mostly incompetent and poorly trained, and the place is filthy. The units are locked and they coerce or force you into accepting “treatment” you otherwise would reject.
So the desired result? The patients are a lot quieter, aren’t they? What is the word used? Docile? The study even says so! Must be the best place on earth….
Report comment
I, too, know a woman who considers herself disabled after a months-long stay at ATMC. Sometimes, if something or some place sounds too good to be true (even with impressive statistics), it probably is.
Report comment
I rarely reply to the comments section in my blogs, but today I just wanted to share some reflection. I feel a treatment center willing to pursue research determining if their efforts are effective, and agreeing to allow the results to be shared publicly, deserves nothing less. But regardless, I want to start by saying thanks for replying. And for the most part, your concerns and theories are legitimate.
But to be clear, many of the critiques you put forward were answered in the blog and in the research report. The main point I was attempting to share, with today’s demand for evidence-based therapies, treatment centers similar to ATMC should try their best to produce evidence more objectively documenting the level of success they are experiencing or producing. Furthermore, relying on what I call “sample of one” anecdotal case studies, does not benefit an effort to improve such care nationally or provide enough evidence to discredit the findings of inferential statistical analysis utilizing more than one hundred cases.
Such larger scale research or statistical analysis efforts are essential so that one doesn’t wrongly assume that a few negative or even positive reviews accurately reflects the overall success of a treatment center. Additionally, such research is beneficial for continuous improvement efforts, discovering what is or is not working, and most of all, helping clients feel more confident in their personal chances for success.
But as I share in this blog, when working with individuals who have unfortunately entered this world of pharmaceutical reliance, there will always be the possibility of numerous clients not experiencing success, or for that matter, having a horrible experience. As I share in the blog, and as you share here, many of the challenges these folks face before entering treatment, will still be waiting for them once they leave the treatment center. But ATMC has treated more than 1,600 clients since opening their doors.
We reached out to 300 who had been treated in the last several years. About 100 replied with complete data capable of being included in the analysis. And with discovering that 87.5% were able to reduce or end dependency all together, that means there are probably 200 additional individuals out there that might feel the treatment at ATMC was not effective. But such percentages also suggest that there are possibly 1,400 individuals who would disagree and instead share success stories of how ATMC helped them.
Additionally, the blog explains that we did look at more than if they reduced the level of pharmaceutical drug use. More than 83.5% reported they also had experienced improved and sustained levels in the long lists of negative symptoms they were experiencing. More than 71.9% reported improved and sustained social relations and interactions. And more than 75% reported experiencing improved and sustained quality sleep patterns. So again while there might be some (16.5% to 28.1%) that did not experience such improvements beyond reducing their dependency on pharmaceutical drugs, 71.9% to 83.5% of the 1,600 would theoretically report good news connected to the help ATMC provided.
Additionally, the survey allowed for qualitative feedback from the past clients. So much so, that I didn’t have enough room in the limited word count to include such information in the blog. And while a few reported shortcomings they experienced as well as dissatisfaction with some aspects of ATMC’s efforts, they also politely shared ways that ATMC could improve their efforts. Many, however, did report great satisfaction and shared their gratitude for the help ATMC provided… aka sharing what it was like at ATMC. Here are a few of the dozens shared:
“Overall, I am grateful to ATMC for the care they provided and continue to provide.I have met a lifetime friend… and continue to Skype with her regularly since leaving ATMC. The love and care and so many aspects of the program are solid and extremely valuable.”
“I loved the program. I was there for only a month…”
“I would love to come visit! Maybe you folks could give me a job :)”
Again, this blog mainly was intended to help treatment centers look at how pursuing a research agenda can be much more reliable and valid, accurately informative, than a few negative online reviews. It was also intended for those seeking help. I wanted to illuminate how to better determine what evidence is needed to help them make an educated decision on where they could find the best support that has the best potential for addressing their specific needs.
In my experience working with ATMC, I have met numerous caring administrators and staff who just want to figure out how to improve what they are doing to increase their clients’ success. Is there room for improvement at ATMC? Of course there is. That is why they asked me to do the research.
I greatly respect and appreciate Mad in America and their loyal readers. But I also believe in science. And while I admit within the blog the limitations this research encountered during this initial research effort, I stand behind the statistical results shared.
Sincerely, I am sorry to hear about your friends, and can only hope they have found or will find the help they need. But critiquing a treatment center based solely upon a personal case you might be aware of, and dismissing the bigger picture findings reflecting what combining 100+ individuals actually provides, is not going to help the field move forward on finding the best way to help folks kick the meds, and more importantly enjoy a better life.
Many individuals cannot make such progress on their own. The great friend I lost was one of them. And please forgive me, but I believe we need to help the centers trying to do some good, do even better.
Report comment
Completely agreed! You definitely point out the critiques made in prior comments well in your original blog. I have experience first hand working with an another organization (in pediatrics)- the value of collecting data as the way you described…using standardized assessments, documenting intervention, choosing outcomes, follow-up, and so forth. It’s an ongoing process. It can help therapists advocate for clients by using this data to continue to improve and push for program development. It can also help with making referrals to other organizations and disciplines as needed for individuals who may need it. It’s nice to hear of the program and efforts of ATMC. Well done and thank you!
Report comment
Michael, I have a masters degree in creative writing with a concentration in memoir. You cannot just toss out stories told by people who are really there. To me, these stories mean far more than any statistics compiled by someone who was never on the treatment end of things there. I’m not saying the place is good or bad. I think MIA as a whole cares far too much about statistics and does not value stories enough. Storytelling is how we move people, how we reach them and evoke emotion and perhaps also inspire to action.
Report comment
the other thing about personal stories vs. statistics is whose truth is getting reflected. Firsthand accounts tell a more complete truth, the truth as defined by the storyteller. But in statistical research, it is only the truth as defined by the researchers. And how “success” gets defined by researchers, as we all know, looks very different from what the folks who’ve had to live through “treatments” would define as success. The devil is in the details, they say, and it is for sure in the operational definitions.
Report comment
Lavender Sage, You make a very good point regarding statistics vs personal narratives. Sure, it’s easy for some to dismiss personal stories as anecdotal (and soft, and feminine, perhaps?) while upholding statistics as “real” (solid, scientific, “numbers don’t lie” etc) but we’ve all learned, many of us the hard way, that studies can be skewed in all sorts of ways, intentionally or otherwise. I had been considering ATMC for treatment of the protracted withdrawal symptoms I’ve been experiencing, which brought me to this article. The “anecdotal” stories that were shared in the comments told me what I needed to know. For me, the comment sections on MIA carry a lot of weight – I pay particular attention to comments from fellow survivors.
Off the topic of treatment for psych drug withdrawal but still on the topic of statistics, after I received a borderline personality diagnosis and was referred to dialectical behavior therapy, I read over and over again that DBT was the first – and at that point, still the most – successful treatment for BPD. When I entered the program, I was told over and over again that the goal of DBT was to achieve a “life worth living” as defined by the person living the life (i.e. the patient). The clinicians would ask each patient to come up with a list of goals that, if achieved, would make them see their life as worth living. Everything the clinicians asked patients to do was purportedly in the service of achieving these goals, of realizing the “life worth living”.
At some point, after a few months in DBT, I did more reading about the studies that had shown DBT to be a successful treatment. To my great disappointment, I learned that the studies hadn’t been concerned with how many patients had achieved their life worth living. Instead, the success of the treatment was measured by the amount of time patients spent in the psych ward. So, if the patient had a reduction in emergency room visits and a reduction in the number of days they spent hospitalized, the treatment was deemed a success. After I learned this, I brought it up with my DBT clinicians, as I couldn’t help but feel a bit hoodwinked. Turns out they didn’t want to discuss it and eventually they began to label my attempts to discuss it as “treatment interfering behavior”. I had to start marking it on my diary card as a problem behavior every time I brought up the subject of the studies that showed DBT was “successful”. As you said, the truth of a study is defined by the researchers, not the subjects of the study.
Report comment
Personal stories can certainly affect others. That is not being disregarded. Not everyone, however, wants to share their personal story in a narrative format. Research, done ethically and responsibly, is a way to capture the critical information that is needed to make well informed decisions. It can help organizations be accountable, get funding for clients, improve professional education, and provide better service. The process is continuous. There is no claim that personal stories, positive or negative, should be “thrown out”. Rather, there is an emphasis of the importance of choosing meaningful outcome data and reaching out to as many people as possible as a way to provide rationale for current practices and future selected interventions. If an organization does this with the best intensions in mind, it can give more credibility to their practice and at the same time become more client focused and driven.
Report comment
Yes but I still don’t know what it’s like there. I don’t know what daily life is like. Where would I go to find out if wanted to go there? Statistics aren’t going to tell me anything. Any of these hospitals and rehabs have great statistics. Even McLean, which sucked very badly.
Report comment
Just as well, an organization can highight personal stories that reflect it in a good light for marketing. A variety of means should be available to illuminate the experience. Transparency is key. To your point, not many places provide this and it can take a lot of investigating. Research is just a piece of the overall picture but an important one in which not every organization partakes. Additionally, what is being measured and why it’s being measured is important. Perhaps I have been blessed with working within an organization that has the best interest in clients in mind when gathering data and making use of it- one reason why they remain successful and have a good reputation.
Report comment
So, ILNC, what is your organization? Are you going to be transparent about that? Interestingly, Massachusetts General Hospital has the best of the best reputations, yet while I was there I was deprived of water to the point where I would have died if I had not gotten out of there. Mass General runs McLean, which is a cesspool of bad care. Also, my highly esteemed MGH psychiatrist threatened to drug me until I could not write, telling me that her MGH administrators wanted to silence me. If MGH was so great, why didn’t they inform me that I my kidneys were failing from lithium? Why did they keep me in the dark when warning me might have saved me a lot of grief? Great reputation, sure. They manipulate the statistics in their favor, just like all of them. (It’s really easy to classify a wrongful death as something else, you know…)
Report comment
Julie, this is the ILNC:
http://wnyil.org/Independent-Living-Niagara
Report comment
Unfortunately you usually can’t tell much from a website. Looks like the person with username ILNC has now decided to bow out. I do not understand why he came here. This is a website that, I hope, does not validate any mental health “treatment,” and certainly would not endorse any treatment. Unfortunately, since a lot of so-called professionals want to get in on the act (after all, MIA loves to publish people with letters after their names!) then what has happened is that some professionals get to push their services, programs, etc, and I don’t even need to name names here. I just keep wondering why. We’re not going back. And that’s the whole point of all this. Never again.
Report comment
Right, Julie! Never again. Shout that loud and proud!
Report comment
I have never even heard of this organization. I have not and I am not using this platform to endorse any. It makes me sad that other people, in your opinion, have done so.
Report comment
ILNC, thanks for finally addressing the question. (Though you didn’t really.)
You did actually endorse the research methods and used your unnamed organization as another org that uses transparent methods. We guess (?) since we have no way or proving or disproving these statements.
I find it odd when someone speaks about their experience with an organization and then when asked which organization they’re speaking about, gets huffy and announces they’re leaving the conversation.
Report comment
Of course I’m not going to name it because that would be an advertisement. Clearly there are different interpretations and takeaways of this article. Yes, I said I finished commenting on it because I would be repeating my points already made.
Report comment
I’m sorry to hear about your personal experience. To further respond would be reiterating and the same as Dr. Corrigan’s, therefore, I am completely finished commenting on this article.
Report comment
Dr. Corrigan, please supply a full statement of your conflicts of interest regarding Alternative to Meds Center.
Here is the report you provided https://www.alternativetomeds.com/wp-content/uploads/2019/01/ATMC_OutcomeStudy2018.pdf as Vice President, Multi-Dimensional Education Inc.
More about your company here http://www.mdedinc.com/httpdocs/about_us
Did ATMC commission you to do this study? Thank you.
Report comment
“Please note, the Alternative to Meds Center has been in business since 2006, and has treated more than 1,600 clients since opening its doors. Assuming that not all of their clients experiencing serious psychological and dependency challenges had success in ending such reliance on the pharmaceutical industry, I felt compelled to advise them upfront (as I do with all clients) that there was a possibility the results might not statistically show they were making a significant positive difference.” (Taken from attached article)
This quote above was taken from the article that was trying to use statistical data to prove that treatment center (ATMC) does show significant difference to many of the programs and outcomes to other available programs. It is interesting to note that I was unable to share my personal comment.
Three years ago, Ryan entered ATMC, with very few or no other choices; suffering for years with mental health issues due to psychotropic drugs; very little hope for his future, and no other choice, but to put all he had into a holistic treatment program to give him hope to overcome this debilitating life he was living.
Ryan has been medication free for three years, has not had any psychiatric treatment since 2016, nor any sign of recidivism whatsoever. ATMC can help anyone that wants to live a better life, get off their medication protocol with a very conservative approach using “ortho molecular therapy, “the treatment of disease by varying the concentrations of substances normally present in the human body” through chelation of holistic and organic supplements with intense sauna treatment, is an understatement! No one doctor, no medication, and nothing could have saved our son from ending his life, we owe a lot to this center and will continue to sing our praises, refer others to ATMC, and promote their efforts for as long as they are in existence.
Report comment
Perhaps your perspective may be limited if you have not personally witnessed the integral structure of the ATMC protocol and program. Most data and statistics are based on an average and the data can be manipulated to produce the outcome you’re trying to obtain in your study and objectives. Knowing firsthand, first of all, ATMC does not typically take a patient right out of a hospital, on heavy medication, and, get on an airplane right after discharge from hospital. Medical data supports so many variables rather than cures, getting patients off medication, and to not be codependent on any treatment—therefore, I have to take exception here and that is what ATMC needs to remain the exception to medical and typical psychiatric protocol! Any individual that has the ultimate desire to be healthy and live health by paying the price (not financially) through hard work and dedication will have success at ATMC!
Report comment
I am not posturing myself as being any higher authority on drug tapering. I know firsthand because my son and about 15 others that were in the program at the time had similar outcomes. I cannot vouch that all of them are completely medication free. Furthermore, you obviously are involved with all the Big Pharma and Medical giants that are holding mental illness hostage to be afflicted with disabilities and side effects that make them worse than when they first were prescribed medication. Otherwise why would you be going after Mad in America, Robert Whittaker, unveiled the evidence on Big Pharma, I praise him and thank Gd for him everyday! The proof is out on all the culprits that for years have taken down innocent lives of those that were dependent on the pharmaceutical industry to get them well. Instead their sights were on making out like bandits with all their corruption and greed and billions of dollars of ineffective medications they sold! We’ll see soon they will all get theirs!
Report comment
You stated you achieved a 34% survey response rate- one that is “more than twice that of typical industry standard”. Of what industry do you speak? Where’s the citation? For publication in a SCIENTIFIC journal, as opposed to pseudoscience website, minimal response rates of 50%-60% are standard (Journal of Medical Toxicology and American Journal of Pharmaceutical Education, respectively.)
You published a “Summary Report” on the internet that totally lacks scientific rigor. No mention of the time frame in which these 134 individuals attended ATMC. During the time frame how many patients left treatment early because they found it lacking?? How did you factor that into your results? No mention of non-response bias … As a PhD scientist I could go on and on.
Report comment