Bias in ML (not to be confused with the bias term, denoted b or w₀) refers to the intentional or inadvertent typecasting, prejudice, or favoritism towards certain things, people, or groups over others which may affect 1.) the collection and interpretation of data, 2.) the design of a system, and 3.) how users interact with a system. Bias in ML may also refer to systematic error introduced by flawed sampling or reporting procedures.

Several known types of ML bias include automation bias, confirmation bias, experimenter’s bias, group attribution bias, implicit bias, in-group bias, out-group homogeneity bias, coverage bias, non-response bias, participation bias, reporting bias, sampling bias, and selection bias.

Watch this video from Google's News Initiative and review the accompanying guide.

In this brief video and accompanying guide from Google's News Initiative, we're introduced to the concept of human bias in ML via a simple directive: "Picture a shoe."

Though it may not be immediately obvious, each of us is biased toward one image of what a "shoe" looks like over the others.

A few key points:

• When training models informed by developers' perceptions, we expose them to bias

• Human biases often become part of the technology we create

• Decisions based on data are not always neutral

Review this mini-lecture offered by Margaret Mitchell from Google's Machine Learning Crash Course.

In this mini-lecture from Google's Machine Learning Crash Course, Margaret Mitchell talks about human bias mitigation and fairness in ML.

A few key points:

• When we store our ideas of the world, we store them based on what's typical or prototypical of objects (i.e., "bananas are yellow").

• When the conditions of an object cohere with our stored understanding, we tend not to mention those conditions (i.e., "bananas" vs "yellow bananas")

• Categorizing information using defining characteristics that may vary (such as color) impacts how we interact with examples that don't fit that mold. This is a form of a bias.

• How Do Biases Affect ML Systems?

  • • Reporting bias: The tendency for people to report things in a way that isn't a reflection of real-world frequencies, or the degree to which a property is characteristic of a class of individuals

    • Selection bias: Erroneous conclusions drawn from data sampled in ways that were non-adequately-representative of the population of interest

    • Overgeneralization: Conclusions drawn from information that is too limited or not specific enough

      • • This is closely related to out-group homogeneity bias: The tendency to assume that "out-group" members are somehow more similar to one another (i.e., more homogenous), and less nuanced compared to "in-group" members.

    • Confirmation bias: The tendency to search for, interpret, and favor information that confirms one's own preexisting beliefs and hypotheses

    • Automation bias: The preference toward suggestions and decisions from automated systems (ex. ML systems) as if they are somehow more objective than information sourced from humans

• What Strategies Can We Use to Design for Fairness?

  • • Consider the problem. What is being adequately represented in your dataset and what might be overlooked?

    • Solicit feedback from experts. They may provide guidance or suggest changes that may enable your project to have a more sustainable (and positive) impact.

    • Train models to account for bias. What do outliers look like in your dataset? How does your model handle outliers? What implicit assumptions does your system integrate and how might you model or mitigate those?

    • Interpret outcomes. Is the ML system overgeneralizing or overlooking dynamic social context? If a human being were to perform the system's task instead, what would appropriate social behavior look like? What interpersonal cues does the system not account for?

    • Publish with context and disclosure. What are the appropriate scenarios in which to apply this model? What are the model's limitations?

Review this paper written by Milena Gianfrancesco and colleagues.

This paper underscores the potential for ML models which use EHR data to inadvertently perpetuate biases in healthcare settings.

A few key points:

• While the use of ML methodologies in healthcare offers tremendous opportunity for health care efficacy and efficiency, concern exists around biases and deficiencies in data used by ML algorithms that may contribute to socioeconomic disparities in care.

• Key problems outlined include:

  • • Missing Data and Patients Not Identified by Algorithms

      • • Model misinterpretations may happen when key data are unavailable or represent metadata that are not included in clinical decision support models.

    • Sample Size and Underestimation

      • • Insufficient sample sizes may result in erroneous interpretations and predictions by ML models as small sample sizes or lack of healthcare service utilization may be misinterpreted as lower disease burden.

• Misclassification and Measurement Error

  • • Misclassification of disease and measurement errors are common sources of bias in observational studies and analyses based on EHR data. For example, quality of care may be affected by implicit biases if uninsured patients more frequently receive substandard care than those with insurance.

    • While the effects of measurement error and misclassification in regression models are well studied, these effects and mitigation strategies in the broader context of ML require further assessment.

• Recommendations:

• Review datasets to assess representation across demographic categories and care interruptions

• Optimize ML models for imbalanced data sets

• Work with interdisciplinary teams to choose appropriate questions and settings for machine learning use, interpret findings, and conduct follow-up studies.

• Test clinical decision support models for bias and discriminatory aspects throughout all stages of development to ensure that models are not misinterpreting exposure-disease associations, including associations based on sex, race/ethnicity, or insurance.

• Ensure that efforts to assess the utility of ML models does not solely focus on performance metrics, but also improvements in clinically relevant outcomes.

Cite as: Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern Med. 2018 Nov 1;178(11):1544-1547. doi: 10.1001/jamainternmed.2018.3763. PMID: 30128552; PMCID: PMC6347576.