Psychological Research Fails to Capture Human Diversity, Researchers Call for Action

New data published in the Journal of Experimental Child Psychology demonstrates that the vast majority of psychological research is conducted on “WEIRD” (people from western, educated, industrialized, rich, and democratic backgrounds) samples, and then incorrectly generalized to be representative of humans globally. This ongoing problem not only discounts the influence of culture on human development but shows little sign of improvement in recent years.

“Failure to confront the possibility that culturally specific findings are being misattributed as universal traits has broad implications for the construction of scientifically defensible theories and for the reliable public dissemination of study findings.”

Previous research has noted the dearth of psychological literature featuring data on the experiences of people outside of Western contexts. When only a small subset of people are consistently sampled, claims to provide evidence for the universality of those results are rendered incomplete, inaccurate, and potentially harmful. Yet, an earlier analysis of research published between the years 2003 and 2007, found that 96% of psychological samples came from countries representing only 12% of the world’s population.

As a way to call attention to the atypicality of samples behavioral scientists routinely draw from, researchers began referring to this population as WEIRD. This acronym is used to highlight the overreliance of sampling from Western, Educated, Industrialized, Rich, and Democratic societies, which are among the least representative of humanity overall. When used to theorize about the human experience broadly, these sampling methods contribute to the marginalization of those who deviate from an erroneous “norm” defined by WEIRD participant data.

“Where research efforts are focused on identifying core mechanisms or universal aspects of psychology, failure to acknowledge the possible impact of environment on the behavior of participants must be considered at best neglectful and at worst bad science.”

Nielsen and a team of researchers did a follow-up study to observe participant sampling in articles published in consistently ranked developmental psychology journals (Child Development, Developmental Psychology, and Developmental Science) between 2006 and 2010. Their findings were remarkably similar to earlier research demonstrating that the experiences of most people in the world are underrepresented in high impact developmental journals.

While Central and South Asia, the Middle East, Israel, Africa, and Asia contain about 85% of the world’s population, participants from these countries were only present in 112 out of the 1,582 articles reviewed in this study. Additionally, across all studies, 97% of participants were WEIRD. Only 4% of first authors for articles were located at institutions in Asia and Israel, 2 articles featured first authors located in Central or South America, and none were located in the Middle East or Africa.

“The critical point about author origin is that it emphasizes how developmental psychology, as a discipline, is characterized as one in which individuals in WEIRD institutions study WEIRD participants,” write the researchers.

The make-up of participant backgrounds in psychological research is therefore culture-specific and poses a risk when used to broadly inform policy and interventions. This is especially true considering how the evidence supports differences across cultures in areas related to parenting, children’s learning processes, and other tasks.

Perhaps most sobering are the stark similarities between the data found in previous studies and in the present one. This underscores the lack of progress to move away from sampling bias, despite increased promotion of multicultural competency in psychology.

“This does not present a picture of a discipline thoughtfully contemplating its limitations and readily embracing change.”

The research team addresses a number of reasons as to why WEIRD sampling may persist ranging from accessibility to the journal to language barriers and wester-centric publication criteria. They offer a number of suggestions to facilitate greater diversity in sampling including:

“(a) encouraging publication of studies that feature non-WEIRD participants, (b) encouraging replication in a new population of a previously established finding, and (c) encouraging theoretically motivated cross-cultural comparisons that examine how children’s cultural environments might affect their development.”

“We need to accept the challenge posed by diversity, provide the explanations it requires, and harness this information to build an improved set of encompassing theories about the development of the human mind.”

 

****

Nielsen, M., Haun, D., Kärtner, J., & Legare, C. H. (2017). The persistent sampling bias in developmental psychology: A call to action. Journal of Experimental Child Psychology, 162, 31-38. (Full Text)