How AI Risks Deepening Health Inequality

Artificial intelligence (AI) and large language models (LLMs) are increasingly shaping the future of healthcare. Advocates claim that these technologies have the potential to enhance accessibility, streamline medical decision-making, and address social determinants of health (SDOH).

However, a recent study published in Cell Reports Medicine highlights the risks that come with integrating AI into digital health systems. Researchers Ong, Seng, Law, Low, Kwa, Giacomini, and Ting warn that without intentional efforts to mitigate bias and increase digital accessibility, AI could exacerbate existing healthcare inequalities rather than resolve them.

“Despite rapid technological advancements, a concerted effort to address barriers to digital health… is still lacking. Low digital literacy, unequal access to digital health, and biased AI algorithms have raised mounting concerns over health equity. As AI applications and LLM models become pervasive, we seek to understand the potential pitfalls of AI in driving health inequalities and identify key opportunities for AI in SDOH from a global perspective.”

This study highlights the ways in which emerging technologies, often framed as neutral or universally beneficial, can, in fact, deepen existing power imbalances within healthcare. By exposing how AI and large language models risk reinforcing structural inequities—through biased data, digital exclusion, and the privatization of health information—it raises urgent questions about who controls the tools that shape mental and physical well-being. As the push toward AI-driven health solutions accelerates, this research underscores the need for critical scrutiny of how these technologies are designed, whose interests they serve, and whether they truly empower individuals or merely automate and obscure existing systems of control.

What Are Social Determinants of Health?

The World Health Organization (WHO) defines social determinants of health as non-medical factors that influence health outcomes. These include environmental, economic, and social conditions such as access to clean water, stable housing, education, and employment. A growing body of research has shown that these factors play a greater role in health outcomes than medical care itself.

The usual examples of SDOH are issues like lead in water lines, leading to increased lead-related conditions in children, and air pollution, causing increased percentages of children with asthma and asthma-related conditions. These examples, especially using children, highlight the conditions that are outside of a person’s control, often outweighing the individual’s ability to overcome these determinants through “good choices.” Another example that falls within the digital sphere is access to medical practitioners in underprivileged areas.

SDOH initiatives have led to policies aimed at reducing health disparities, such as the “Healthy People 2030” program in the U.S., which seeks to eliminate barriers to care. Digital health technologies, including telemedicine and AI-driven diagnostic tools, have been introduced as potential solutions to these disparities. However, the very people these innovations aim to help—those in low-income or rural areas, for example—often lack the digital access needed to benefit from them.

The Digital Divide in Healthcare

The expansion of digital health tools has highlighted the emerging concept of digital determinants of health (DDOH)—the ways in which digital access and literacy impact health outcomes. While digital health tools can improve access to care, they also introduce new barriers, particularly for those without reliable internet access, smartphones, or basic digital literacy.

“Digital health tools helped to address major global health challenges through improved access to healthcare services, strengthened health promotion and disease prevention, and enhanced care experiences for professionals and patients.”

Yet, despite their potential, these tools remain out of reach for many. A report from the UK found that 11 million people lack the skills to fully participate in the digital economy, including digital healthcare services. The elderly, disabled individuals and those in lower-income brackets are disproportionately affected. Telemedicine, for example, is often seen as a solution for those who lack access to in-person medical care, but it requires high-speed internet, digital literacy, and, in many cases, the ability to navigate complex healthcare portals—all of which can serve as barriers rather than solutions.

Digital platforms and tools continue to address the gaps identified in research on social determinants of health (SDOH) and digital determinants of health (DDOH), despite their inherent challenges. Many of these tools are now designed using artificial intelligence (AI) and large language models (LLMs), such as ChatGPT. They are also being integrated into electronic health records (EHRs) to facilitate the data collection necessary for the effective operation of AI and LLM programs. This combination of technologies presents both opportunities and challenges.

Opportunities and Barriers of AI in Social Determinants of Health

Ong et al. highlight the first social determinant of health (SDOH) factor related to the use of artificial intelligence (AI) and large language models (LLMs): high-income countries (HICs) have an inherent advantage over low- and middle-income countries (LMICs). HICs are the primary source of technology, giving them a competitive edge on a global scale.

Additionally, Ong et al. identify a significant challenge in the development of health-based AI: much of the data needed for these models will come from unstructured, uncoded qualitative sources, such as physician and practitioner notes. This understanding has spurred the creation of models capable of analyzing these notes to aggregate, synthesize, and utilize qualitative data effectively.

“[D]eep learning (DL) algorithms such as CNNs (convolutional neural networks) and BERT (bidirectional encoder representations from Transformers) have been applied to SDOH annotation from clinical unstructured.”

BERT models utilize contextual information by analyzing text and audio signals to identify patterns, which they then use to perform a variety of designated tasks. In this instance, they aim to represent the typical behaviors of physicians and other healthcare professionals.

AI and large language models (LLMs) can be particularly beneficial in addressing issues related to social determinants of health (SDOH), especially in the context of mental health and chronic health conditions. These AI systems can analyze individual data alongside predictive patterns to identify potential problems, such as those associated with diabetes. Furthermore, these models can provide an overview of the prevalence of mental health conditions and their related issues on a larger scale.

“[T]emporal evolution of SDOH factors, geospatial relationships, and their impact on health outcomes have also been poorly characterized by conventional statistical methods.”

AI and LLMs can predict the rise and shift in conditions related to SDOH better than current models. However, those predictions are based on existing data models, based on existing qualitative data, which itself may present a problem.

The Problems in AI-Driven Healthcare

Efforts to integrate artificial intelligence (AI) into healthcare face significant challenges, with experts pointing to the lack of standardization in data collection, storage, and modeling as one of the most pressing concerns. In a recent study, researchers highlight how disparities in electronic health records (EHR) and other data systems create major barriers to interoperability, making it difficult to effectively analyze and apply AI-driven insights.

“Data are often not harmonized nor interoperable across sectors, making data integration a challenge and subsequent analysis/modeling difficult,” the authors explain. “The lack of standardized terminologies and definitions will also limit semantic interoperability, posing a challenge to the implementation of algorithms.”

Some initiatives are attempting to address this issue. In the United States, for example, the AMA Integrated Health Model Initiative (IMHI) and the Gravity Project—spearheaded by the American Medical Association—aim to develop standardized protocols for collecting and transferring social determinants of health (SDOH) data. Their goal is to integrate these standards into medical coding systems such as Current Procedural Terminology (CPT) codes, potentially streamlining the way AI processes healthcare information.

Yet even if data standardization improves, AI still faces major hurdles related to accessibility and infrastructure. Physical platforms, telecommunications networks, and digital literacy all influence how effectively AI can be deployed in different regions. Researchers caution that disparities in internet access, language barriers, and technological skills could further entrench inequalities rather than alleviate them.

“Variances in internet connectivity, linguistic compatibility, operational skills, and hardware prerequisites may challenge access to benefits conferred by AI models and LLMs,” the study notes. “On the level of physical infrastructure, LMICs [low- and middle-income countries] and remote areas may not possess reliable telecommunication networks, stable internet connections, or sufficient equipment.”

Beyond logistical challenges, two larger concerns loom over the integration of AI into healthcare: bias and privacy.

Bias in AI: Reinforcing Systemic Inequalities

AI models are only as unbiased as the data they are trained on, and in healthcare, that data often reflects existing racial and socioeconomic disparities. Researchers warn that without intentional efforts to mitigate bias, AI could exacerbate health inequities rather than reduce them.

“Implementing algorithms trained using large datasets has been shown to be biased against individuals of different races and socioeconomic status,” the authors explain. “For instance, chest X-ray classifiers trained using datasets dominated by White patients performed poorly in Medicare patients, and a risk prediction algorithm trained using large datasets failed to triage African Americans for necessary care. This will remain a pervasive problem since more than half of the datasets used for clinical AI development.”

This raises a critical issue: If AI is designed to process SDOH data but does not account for the structural biases embedded within that data, it risks perpetuating the very disparities it aims to address. In essence, systemic racism and social prejudices are themselves determinants of health—failing to address them in AI development means codifying them into medical decision-making systems.

The Privacy Problem: Who Controls the Data?

Another unresolved issue is data privacy. With AI systems relying on vast amounts of patient information, questions remain about who is responsible for ensuring that data is handled ethically and securely.

“In the realm of data and model responsibility, the party that remains liable for the outcomes of AI and LLMs remains unclear,” the researchers note. “It is important to note that absence of autonomy and sentience renders a lack of moral agency in AI.”

This ambiguity raises difficult questions. If an AI system makes an incorrect diagnosis or fails to recommend necessary treatment, who is accountable? The healthcare provider relying on the AI’s recommendation? The developers who created the algorithm? Or is there a need for a new regulatory framework to oversee AI-driven healthcare decisions?

With AI development in healthcare advancing rapidly, these concerns remain unresolved. The technology is evolving faster than the ethical and legal frameworks designed to regulate it, leaving fundamental safety measures “up in the air.” Without clear policies addressing bias and accountability, AI risks becoming yet another tool that reinforces disparities rather than eliminating them.

Future Directions: Can AI Be Used Responsibly in Healthcare?

The study suggests that AI could still be a powerful tool for addressing social determinants of health—if it is developed and implemented responsibly. The authors argue that AI-driven health solutions should be designed with the most vulnerable populations in mind, rather than being developed in high-income countries and imposed on lower-income ones.

“Achieving health equity remains a challenge, and addressing SDOH is progressively becoming a global priority. In a post-pandemic world, the widening digital and health divide has thrown caution to the use of AI and digital health technologies. Significant challenges and barriers to equitable AI implementation remain, especially with regard to overcoming infrastructure limitations and algorithmic bias.”

As access to digital tools becomes increasingly central to healthcare, digital literacy and internet connectivity have emerged as “super” social determinants of health (SDOH). These factors influence nearly every aspect of modern healthcare delivery, from telemedicine to digital health records. However, disparities in digital access risk deepening existing health inequities rather than alleviating them.

“A key barrier to digital literacy lies in the digital divide, which afflicts disadvantaged populations more significantly,” the study notes. “For example, approximately one-sixth of low-income household families in the USA do not have broadband internet access.”

This digital gap highlights two key challenges for the equitable implementation of artificial intelligence (AI) in healthcare.

First, technology has the potential to either close or exacerbate existing disparities. If AI-driven healthcare tools are designed without considering who has access to them, they may reinforce inequalities rather than serve as a bridge to better care. For instance, AI-powered telehealth services may benefit those with stable broadband access while leaving behind those in rural or low-income communities who lack reliable internet.

Second, the people most affected by these digital disparities are often excluded from the development and implementation of AI-driven healthcare solutions. The researchers stress that those who have historically lacked access to digital health tools must be involved in shaping AI models to ensure these technologies address their needs rather than widen health gaps. Without inclusive design, AI risks reinforcing the very inequalities that social determinants research aims to dismantle.

The issue extends beyond national borders. Low- and middle-income countries (LMICs) face even greater obstacles, as high-income countries (HICs) dominate AI development. If global health technology continues to be shaped primarily by wealthier nations, it may create an additional SDOH barrier at an international level, further disadvantaging populations already struggling with access to care.

The authors emphasize that overcoming these challenges will require proactive efforts, including infrastructure improvements, investments in digital literacy programs, and the development of unbiased AI models. Without such measures, AI’s potential to transform healthcare could become another force entrenching disparities rather than alleviating them. However, with global collaboration and a commitment to equitable design, AI could serve as a powerful tool in reducing health inequalities—rather than reinforcing them.

****

CL., Seng, B. J. J., Law, J. Z. F., Low, L. L., Kwa, A. L. H., Giacomini, K. M., & Ting, D. S. W. (2024). Artificial intelligence, ChatGPT, and other large language models for social determinants of health: Current state and future directions. Cell Reports Medicine, 5(1). (Link)