Research Suggests that Forensic Psychological Examinations are Unreliable and Biased

Forensic psychology plays an important role in many court cases. However, concerns have been raised about inconsistent and unreliable results, which may lead to injustices in sentencing or even wrongful convictions.

In a new article published in Translational Issues in Psychological Science, researchers studied such forensic evaluations, and their results suggested that these examinations are unreliable and biased. Individual evaluators often disagree on the results, and the authors raise concerns about the lack of independence of forensic psychologists from the side retaining them.

Forensic psychological evaluation is used for numerous reasons. Psychologists may attempt to evaluate someone’s risk for violence, mental state at the time of a crime, or competency to stand trial, for instance. These results can be useful for determining everything from the appropriate sentencing for a crime, to the possibility of conditional release from prison, to child custody requirements. However, if these assessments are biased or unreliable, they could lead to “arbitrary or unjust legal outcomes [… and] diminish confidence in psychological expertise within the legal system.”

“Independent evaluators tend to disagree in approximately 25%-35% of sanity cases… and almost half (45%) of conditional release cases,” the authors write. “That is, in at least one out of every four cases (and possibly up to half the time), the result of a forensic evaluation is unreliable and lacks objectivity.”

There are numerous reasons for this problem. For one, there are hundreds of potential ways to evaluate psychological state, and many of those methods rely on an expert opinion on case history reports and interviews—which are methods at risk for bias and could be considered to lack objectivity. In fact, one large international study found that 26% of evaluators used clinical history without using a single objective measure.

Additionally, the authors identify that many states do not require special certification to perform these examinations—meaning that psychologists may lack the relevant training to objectively evaluate these issues.

“Studies assessing the thoroughness, relevance, and accuracy of the reports forensic clinicians submit to the court regularly find them deficient,” they write. “Competency evaluators’ reports in Utah failed to incorporate legally relevant aspects of competency and failed to adequately describe the reasoning underlying their final forensic opinion.”

Another critical issue is a lack of independence. The authors of the article cite numerous studies indicating that forensic evaluators are far more likely to find results in favor of the side retaining them.

For instance, the authors discuss one commonly used measure of an individual’s violent or dangerous tendencies, which requires that the evaluator selects a set of normative scores. The evaluator then compares the subject’s scores to the normative scores. However, in a study cited by the authors, researchers found that evaluators used very different normative scores based on whether they were retained by the prosecution or the defense.

In fact, 94% of the evaluations conducted by state-retained psychologists used norms “that makes the examinee seem more risky, as compared to routine sample norms,” while only 33% of psychologists retained by the subject themselves used this type of norm. That is, people who did not retain their own evaluator were far more likely to be graded harshly on this measure than people who hired their own psychological evaluator.

The authors suggest numerous improvements to counteract these sources of bias, such as:

• Mandating specific training requirements for forensic evaluators
• The use of several independent evaluators (Only 10 states require more than one)
• The use of standardized, structured measures whenever possible
• Requiring written justifications of reasoning and choice of evaluation measure
• The use of blinded expert evaluators (unaware of which side retained their services)

Additionally, the researchers argue that more research needs to be done to clarify which evaluation measures are most reliable and objective. In the meantime, the authors call for reforms to the legal system. They write, “Policy and practice changes are needed to improve the reliability of forensic opinions.”

 

****

Guarnera, L. A., Murrie, D. C., & Boccaccini, M. T. (2017). Why do forensic experts disagree? Sources of unreliability and bias in forensic psychology evaluations. Translational Issues in Psychological Science, 3(2), 143-152. http://dx.doi.org/10.1037/tps0000114 (Abstract)