In a new article, researcher Gerald Haeffel at the University of Notre Dame critiques psychology’s failure to follow the scientific method. Instead of rigorously testing theories and learning from the failures—a hallmark of the scientific method—“research” in psychology and psychiatry is often set up to confirm every pet theory. Yet, somehow, he writes, despite the complex and inscrutable nature of the mind and brain, psychological scientists “always win.”
Haeffel writes:
“Psychological science still does not embrace the scientific method of developing theories, conducting critical tests of those theories, detecting contradictory results, and revising (or disposing of) the theories accordingly.”
He writes that “nearly 100% of the published studies in psychology confirm the initial hypothesis.” This amazing winning streak, according to Haeffel, is not typical of the scientific method—it’s indicative of a scientific failure and the dishonest research practices pervasive in the psych field.
“The legitimacy of psychology’s winning streak has been called into question,” he writes. “Major replication projects show that only about half of psychological findings replicate. Further, there is evidence that psychology’s winning streak may be due to cheating. Similarly to how steroids fueled baseball’s home run chase in the United States (e.g., Bonds, McGuire, Sosa) and Lance Armstrong’s Tour De France streak was aided by doping, psychology’s winning streak may be the result of questionable research practices like p-hacking, HARKing, and piecemeal publication.”
Haeffel isn’t the first to make this criticism. In another recent article, researchers argued that psychology is “incompatible with hypothesis-driven theoretical science” and that researchers in the field typically explain away any result they find—even negative ones—as still being consistent with their theories. Thus, the field does not actually test these theories scientifically—even if it appears to—because no matter the results of their studies, researchers never reject or revise their theories.
According to Haeffel, the hallmark of good science is having a specific prediction that can be tested in order to find its failure points. This leads to revisions (which improve the theory’s explanatory power) or rejection of the theory (which allows other competing theories to step into the spotlight and be tested in turn). This pattern in the other fields has led to significant scientific discoveries (Einstein’s general relativity, for instance).
But psych researchers are not concerned with rigorously testing specific predictions. Instead, they have vague predictions, and any result they find—even a contradictory one—is used to promote their pet theories. This enables a culture of publication, grant money, academic hierarchy, and pharmaceutical industry dominance, but it does nothing to advance our understanding of the mind.
Haeffel asks, “If creating a theory is as simple as making a risky guess, then why are there so few good ones in psychology? One reason is that psychology has stopped using the scientific method.”
The other scientific fields approach research entirely differently. For example, in contrast to psychology, Haeffel writes that in other fields, “journal articles regularly include sections in which the scientists specifically describe the conditions in which the theory would be invalid and then describe how the experiments were designed to rule out these alternative explanations. The idea is that theories develop greater levels of verisimilitude over time as alternative explanations (and even entire classes of theories) are systematically eliminated.”
In other fields, he adds, researchers are excited to discover the areas in which the theory fails to predict something accurately—because it could be the “crack” that opens up into a far more comprehensive understanding. But in psychology, researchers are only concerned with confirming their pet theories over and over again—and it’s all designed so that there’s no way they can be wrong.
But this is the exact opposite of science, according to Haeffel.
“What differentiates science from non-science,” he writes, “is falsifiability. If you can be wrong, then it is scientific; if you cannot be wrong, then it is not scientific […] In science, refutations are more useful than confirmations.”
He adds, “It is not always easy to accept data that disagrees with our beliefs, but scientific progress depends on it.”
Proponents of psych research may argue that the field is still in its infancy, so its failure to predict or come up with specific theories accurately should be overlooked. But Haeffel contends that these failures lead to specific negative outcomes in a society—like ours—where psychiatry is afforded a pulpit to preach and considerable influence in policy-making.
For instance, he writes, “the recommendation for widespread mental health interventions […] could do more harm than good. Research indicates that intervening too early or with people not at risk of mental health problems can disrupt the normal recovery processes that create resilience. For example, there is some evidence that interventions such as grief counseling and critical incident stress debriefing can be iatrogenic. As noted by Bonanno, ‘many individuals exposed to violent or life-threatening events will show a genuine resilience that should not be interfered with or undermined by clinical intervention.’ Supporting this claim, at least one study reported that nearly 40% of the individuals receiving grief treatments got worse relative to no treatment.”
Haeffel suggests that the psych field is influencing policy and carrying out interventions based on untested assumptions. Even if done with the best intentions, these interventions can have iatrogenic effects—causing more harm than helping—and at a high cost in terms of effort and taxpayer money. For instance, the D.A.R.E. program, carried out in schools across the United States to reduce drug use in youth, has been reassessed as a complete failure—a waste of billions of dollars.
So what is the solution? Haeffel argues that the first incremental step involves fixing a culture that incentivizes only positive findings. He suggests that there is an alternative. It’s known as “registered reports,” in which journals accept studies for publication before knowing the results based on the question being tested and the study design.
While the current culture is focused on publishing splashy positive findings, no matter how poor the question and methods are, the registered report method incentivizes researchers to craft meaningful hypotheses and test them rigorously.
Haeffel writes:
“We contend the best option for fixing psychology’s ‘winning’ problem is the Registered Report format in which articles are accepted or rejected prior to knowing the results of the study. Scheel et al. compared results reported in published Registered Reports with those of standard publications; they found 44% positive results in Registered Reports and 96% positive results in standard publications.”
When registered reports were used, publication bias—in which only positive studies end up in journals—dropped by a huge amount.
“It is possible that making Registered Reports the default publishing option will eventually move psychologists towards a more problem-focused (rather than method-oriented) mindset,” Haeffel writes.
He concludes, “Psychologists are willing to be wrong as long as they can still get a publication.”
****
Haeffel, G. J. (2022). Psychology needs to get tired of winning. Royal Society Open Science, 9220099. Published online June 22, 2022. https://doi.org/10.1098/rsos.220099 (Link)
Psychological research may be poor, but the fact is other research is not that great either.
Most published research is false and most can’t be replicated. Journals tend to publish only positive research results and the pharmaceutical companies that sponsor most studies have a vested interest in massaging the data in order to come up with favorable results.
Report comment
Totally agree, particularly with respect to Pharma. Drugs these days are approved solely on the basis of Pharma-funded studies. And not only can Pharma simply not publish studies with negative findings, they have also become high adept at manipulating results to turn negatives into positives, and to hide side effects. Even RCTs are highly susceptible to these kinds of games. See David Healy’s masterpiece Pharmageddon for much more on this.
Report comment
Thank you for your honest reporting about the scientific fraud of the psychological industry, Peter. And I totally agree, psychologists do have a big problem in regards to both making incorrect assumptions about people, and outright lying to people. My experience was they either threaten and lie to people, and/or they gaslight people, which is “mental abuse,” not “mental health care.”
Report comment
Thank you for your article! I recently read twice a book ,referred to me by a colleague written by Sarah Fay,who I was unfamiliar with.In her book she blasts the often false diagnostic codes listed in the DSM,& how many of those labels ,handed out to patients were created to tailor match “new& approved “antidepress.etc.Big Pharma at its finest.In the 1980s there were 151 categories and in the current DSM 5 there are 550 + !! That alone is a wake up call.She began her journey & research after being given 6 diff diagnoses over 20 yrs !.She also shares of how she began to identify herself with her diff diagnoses,as if they were solid ,,factual & true!! Labels label & do not define & labels limit …her accts of the Drs etc.who write the DSM is a huge cause for concern .To validate her findings she has a former NIMH validate her story along with others,in the mental health industry.She also reveals indiv.& Non profits who rec.” Donations ” & were part of the DSM writing process .I myself began to realize ,for myself,of diff.diagnoses I was given, and 9 out of 10 after a first time 30-45 min.appt.Their source of ref.I am sure was the DSM.She also points out of the many depression ,anxiety etc online ‘self tests ‘dev by Big Pharma( not science ) & then their magic medication matches the diagnosis description.Made to order descriptions ,then wait …we have the answer to your diagnosis in our newest medication !The DSM lacks solid science ,and is often based on ” “opinions”,etc.The DSM is not a Bible of validity,& any Drs who solely rely on that as their Bible ,is always cause for concern ! I have no trust in in any Psychiatrists or Mental Health providers unless they Think Outside of the Box.I am of the inquisitive type by nature ,but it is so alarming ,when I hear indiv continue to share ” oh my Dr.gave me this ,then this & I have a diagnosis of ……..So few never ques their Drs ,their Gods of truth ,and why?? The DSM authors & Big Pharma are a disgrace,& this new book,again educated me.
Report comment
“But psych researchers are not concerned with rigorously testing specific predictions” –
Things of a psychological nature have too many variables to be specifically and rigorously tested, much less replicated. And there’s nothing wrong with that, except if you’re competing for “… publication, grant money, academic hierarchy, and pharmaceutical industry dominance”.
“Instead, they have vague predictions, and any result they find—even a contradictory one—is used to promote their pet theories”.
Perhaps they missed the lecture on confirmation bias.
Sounds like some psychology researchers haven’t faced their own demons: ego, pride, and greed.
Report comment
Wow! What a great article! Thanks for sharing!!1
Report comment
Here’s an interesting article: “Why psychology isn’t science”, by Alex B. Berezow
Report comment
Maybe psychological researchers should ask themselves why they’re obsessed with winning. Could it be ego, prestige, and money? That’s not very scientific.
Apparently, being a psychological researcher doesn’t guarantee you’ve more insight than the average person.
Too bad they can’t ditch their binary thinking. And when did science and competition become compatible?
Sounds like they need to check their egos.
Report comment
I’m reminded of Steven James Bartlett’s book: Normality Does Not Equal Mental Health: The Need to Look Elsewhere for Standards of Good Psychological Health. Says he,
“The fortitude and obduracy of systems of belief are their strength but also their downfall. Conservative thinking-adherence to and defense of conventions that are dominant at any particular time-therefore automatically brings with it a limited field of vision and a self-chosen myopia. If any blame can he laid for periods of slowed, nonexistent, or retrograde intellectual and scientific development, for periods of uncreative, sluggish, and at times imperceptible growth, that blame can be placed both on the natural human unwillingness to call into question beliefs that apparently have served well enough in the past and on the deeply entrenched disinclination to step outside of the preferred category set. Individuals who are willing to do these things tend to be few, and they should expect to meet correspondingly deeply rooted resistance, which of course indeed they have throughout the past. As a consequence of the psychology of belief, when we look at the history of science we see that its most basic concepts and presuppositions are often the least examined. They form the basis for all else in scientific thought, so that in their very mental activity scientists make habitual use of them. This results in it being all the more intellectually difficult and challenging to place them in the light of day. Since they serve as the fundamental, core conceptual vocabulary of scientific thought, they resist critical examination, because for a scientist to do this, he or she believes, often incorrectly, that those very concepts and presuppositions must be used”
Will somebody in MIA please write a book review on Dr Bartlett’s book? Better yet, a 3 or 4 part video interview would be the best way to get the word out on his body of work that we are in dire need of in this day and age.
Report comment
“….when we look at the history of science we see that it’s most basic concepts and presuppositions are often the least examined”.
And please include Dr. Bartlett’s book, “The Pathology of Man: A Study of Human Evil” –
Report comment
lol!…I got that book too!
The irony is that our mental health providers are suppose to help those with all manner of emotional distress, yet as they adhere -without question- to the gold standard of psychological normality, it inevitably makes things worse in their attempts to heal those in need. My dear niece is paying a high price for it as they misdiagnosed her BPD for decades and now she has to start therapy all over again which will take years to treat-even with the best therapist.
It’s frustrating as hell to see friends and family members seeking therapy with nothing much to show for it. It seems we must become our master and disciple in striving to be sane in an often insane world. Thank goodness for Dr. Bartlett and even MIA for calling out the sham we often see in mental health.
Report comment
You’re absolutely right about people having to become their own master and disciple in an insane world. There comes a point when you realize you have no choice but to do so. But you also realize it’s the choice you should have made in the first place.
I think the “mental health” system is the worst thing to come out of the twentieth century. And the worst thing about it is that it’s a “system”, meaning arbitrary standards and bureaucracy runs the show. Then add to that a therapist’s ego and financial incentive, and you’ve got one heck of a monster. But yes – thank goodness there’s MIA and books by Dr. Bartlett.
I hope things turn around for your niece. She’s lucky to have you in her family.
Report comment
What’s wrong with the psych research community?
It’s become a competitive culture fixated on winning rather than exploration, and it’s now overpopulated by too many closed minded, stiff necked, (and some would say cowardly) blowhards, trying to protect and enhance their own reputations and academic turf in order to procure tenure, status, and research dollars.
But the problem is fed by a population led to believe they need a “trained professional” to figure out their lives. But slowly, people are discovering that this isn’t the case at all – and that too often, those trained as “professionals” are the least able to help with their problems –
Report comment
What’s wrong with the psych research community?
The same thing that’s wrong with the rest of “higher education” – it’s gone from exploration to EXPLOITATION –
Report comment
removed for moderation.
Report comment
“….yet somehow psychology researchers always find a positive result” –
What does this mean?
It means they’re a bunch of egomaniacs, which is exactly what most psych professionals are.
Report comment
“….yet somehow psychology researchers always find a positive result” –
Maybe psych researchers need to research why they’re afraid of negative results. That way, they might learn something about themselves, and who knows?… maybe even about each other! After all, isn’t this what psychology is all about?
Report comment
It’s like they’re arguing about if they should throw the baby out with the bathwater without realizing they already killed the baby.
Psychiatric fraud has major implications. Modern vaccine resistance is rooted in mainstream medicine’s failure to respond to autism thus creating a desperate group of patients willing to accept an explanation of their condition with zero scientific basis. This is a great example of the ‘white wall of silence’ where doctors protect themselves at the expense of society.
Report comment
Hi Peter – I don’t know if I am too late coming to this, but I think I have reason for this apparent “failure” of psychology to emulate the scientific method. Wittgenstein wrote about it 70 years ago – see comment 371 at the end of the Investigations – where he says “problem and method pass one another by”. That it because psychology is more like mathematics than physics. The argument from maths is the number of statements which might or might not be provable is unlimited. In any large sample of papers by mathematicians the number of true statements far outweigh the number of untrue. If this was merely random you’d expect the opposite. Mathematicians seldom try to prove something unless they are convinced before hand of its truth. This is because mathematics is uncovering intuitive truths. Wittgenstein highlighted that this is the case in psychology too; and it should stop comparing itself with physics (physics envy) and start comparing itself with maths. Then we would be on the way to finding out our human nature; revealing it as it lies buried in cultural clutter.
I really wish they would actually notify me via e-mail if any one follows up on this.
Report comment
When I took a couple of psychology classes I was dismayed. The teacher and texts did the same thing: “Here is one theory. And here is a completely different one. Let’s move on.” I couldn’t believe no one was bothered by the lack of an answer.
Report comment
please, please, please either get an in depth interview with Professor of Social Work, William M Epstein or review his three key books relating to psychotherapy and its research base. The three key books are the illusion of psychotherapy, psychotherapy as religion and psychotherapy and the social clinic in the united states, soothing fictions. Across these books he takes the research apart on methodological grounds and what he demonstrates is fascinating – how this house of cards persists is yet more evidence of a self interested cultural disorder.
Report comment
The damage of this irresponsible and reckless disregard for true evidence based scientific research and study can be hortible. But the damage to our society as a whole has been nothing less than catastrophic.
I could list several areas. Just one theory comes to mind—the “super predator “ theory. This lead to the lifelong incarceration of 100s if not thousands of hound teenaged children back in the 1990s. That not only effected those individuals but their families, theist communities and our country.
And it was based on what? Science? Or an opinion of a “renowned” psychologist?
Report comment