We humans make sense of the world by looking for patterns, filtering them through what we think we already know, and making decisions accordingly. When we talk about handing off decisions to AI, we expect it to do the same, only better. And when AI models produce biased results, high-profile embarrassment often ensues. But rooting out and remediating bias isn’t always simple either. Case in point: Google’s spectacularly rocky rollout of its Gemini AI chatbot. The family of models kicked off its launch by generating historically inaccurate images such as Black Vikings and a female pope. Google quickly issued a statement saying Gemini had been trained to include a diverse set of people in its results but that it had not been trained to exclude certain people based on a date range. It paused image generation until this could be sorted out.

AI chatbots don’t always get it right

If Google can get it wrong so quickly, what hope does anyone else have of spotting and fixing bias?

Of course, AI and variants such as machine learning will endure despite episodes like the Gemini launch. AI does have the potential to be a tremendous force for good. Humans are hindered by both their unconscious assumptions and their simple inability to process huge amounts of information. AI, on the other hand, can be taught to filter irrelevancies out of the decision-making process, pluck the most suitable candidates from a haystack of résumés, and guide us based on what it calculates is objectively best rather than simply what we’ve done in the past.

In other words, AI has the potential to help us avoid bias in hiring, operations, and customer service as well as in the broader business and social communities—and doing so makes good business sense. For one thing, even the most unintentional discrimination can cost a company significantly, in both money and brand equity. The mere fact of having to defend against an accusation of bias can linger long after the issue itself is settled.

Beyond managing risk related to legal and regulatory issues, though, there’s a broader argument for tackling bias: in a relentlessly competitive and global economy, no organization can afford to shut itself off from broader input, more varied experiences, a wider range of talent, and larger potential markets.

The algorithms that drive AI don’t reveal pure, objective truth just because they’re mathematical. Humans must tell AI what they consider suitable, teach it which information is relevant, and indicate that the outcomes they consider best—ethically, legally, and financially—are free from bias, consciously or otherwise. That’s the only way AI can help us create systems that are fair, more productive, and ultimately better for both business and society.

Bias is bad business

When people talk about AI and machine learning, they usually mean algorithms that learn over time as they process large data sets. Organizations that have gathered vast amounts of data can use these algorithms to apply sophisticated mathematical modeling techniques to see if the results can predict future outcomes, such as fluctuations in the price of materials or traffic flows around a port facility. Computers are ideally suited to processing these massive data volumes to reveal patterns and interactions that might help organizations get ahead of their competitors. As we gather more types and sources of data with which to train increasingly complex algorithms, interest in AI will become even more intense.

But AI is optimized to predict rather than to understand. For now, at least, it is often not strong at taking context into account. So ferreting out bias working inside the AI system—or the flipside, using AI to detect human bias—is always going to be complicated.

AI systems may reflect cognitive biases, which are unconscious errors that affect individuals’ judgments and decisions, according to AIMultiple Research. Examples include normalcy bias (the tendency to avoid planning for an event that has never happened even if it’s likely to happen) and confirmation bias (interpreting new information in a way that agrees with preconceived ideas). These biases spring from a system of shorthand logic that helps us simplify processing information about the world—but they often produce wrong conclusions and stereotypes. The notion that women don’t make good engineers is a classic example.

Cognitive biases can seep into machine learning algorithms through designers unknowingly introducing them into the model or through a training data set that includes those biases, according to AIMultiple principal analyst Cem Dilmegani.

Another thorny problem: sparse data, which may include bias. For example, many large medical data sets lack data from underrepresented racial and ethnic groups, leading to inaccurate predictions.

The potential negative effects of AI bias are growing. Using AI for automated decision-making is becoming more common, at least for simple tasks, such as recommending additional products at the point of sale based on a customer’s current and past purchases. When the automated decisions are more complex, such as identifying past criminals who are most at risk of offending again, the risk of harmful bias is significant.

As AI takes on these increasingly complex decisions, it should seek to reduce conscious and unconscious bias. By exposing a bias, algorithms allow us to lessen the effect of that bias on our decisions and actions. They help us make decisions that reflect objective data instead of untested assumptions, reveal imbalances, and alert us to our cognitive blind spots so that we can make more accurate, unbiased decisions.

For example, a major company realizes that its past hiring practices were biased against women and that it would benefit from having more women in its management pipeline. AI can help the company analyze its past job postings for gender-biased language, which might have discouraged some applicants. Future postings could be more gender-neutral, increasing the number of female applicants who get past the initial screenings.

AI can also support people in making less biased decisions. For example, a company is considering two candidates, a man and a woman, for an influential management position. The final decision lies with a hiring manager who assumes that she would prefer a part-time schedule when they learn that the female candidate has a small child at home.

That assumption may be well intentioned, but it runs counter to the outcome the company is looking for. AI could apply corrective pressure by reminding the hiring manager that with all qualifications being equal, the female candidate is an objectively good choice who meets the company’s criteria. The hope is that the hiring manager will realize their unfounded assumption and remove it from the decision-making process.

But it works both ways. A human manager can rely on context to make a better decision than an automated system—for example, if someone did not perform well in an interview due to extenuating circumstances. The human manager might override the system so as not to exclude someone who is otherwise the best candidate.

Rooting out AI bias

The reason for these checks and balances is clear: the algorithms that drive AI are built by humans, and humans choose the data with which to shape and train the resulting models. Because humans are prone to bias, we have to be careful that we are neither confirming existing biases nor introducing new ones when we develop AI models and feed them data.

“From the perspective of a business leader who wants to do the right thing, it’s a design question,” says mathematician Cathy O’Neil, founder of a consulting firm that helps organizations manage and audit their algorithmic risks and author of the best-selling book Weapons of Math Destruction.

“You wouldn’t let your company design a car and send it out in the world without knowing whether it’s safe. You have to design it with safety standards in mind,” she says. “By the same token, algorithms have to be designed with fairness and legality in mind, with standards that are understandable to everyone, from the business leader to the people being scored.”

To eliminate bias, you must first make sure that the data you’re using to train the algorithm is itself free of bias or that the algorithm can recognize bias in that data and bring the bias to a human’s attention. For example, companies today know not to include language as overtly discriminatory as “no women need apply”—but, deliberately or otherwise, they still use phrases like “outspoken” and “aggressively pursuing opportunities,” which are proven to attract male job applicants and repel female applicants, and words like “caring” and “flexible,” which do the opposite.

Once humans categorize this language and feed it into an algorithm, AI can learn to flag words that imply bias and suggest gender-neutral alternatives. Unfortunately, this process currently requires too much human intervention to scale easily, but as the amount of available de-biased data grows, this will become far less of a limitation in developing AI for HR and other applications.

Using newer automated systems for bias detection could allow it to be done at scale. One potential solution: companies are now using AI to detect bias in other AI models, an extension of the practice of using AI to monitor AI. For instance, the Allen Institute for AI, a nonprofit created by the late Microsoft founder Paul Allen in 2014, offers an open-source model called AllenNLP and other training tools as well as bias mitigation algorithms.

A host of other for-profit organizations—from startups to tech giants—also offer AI audit tools. Cloud AI providers are beginning to build relevant capabilities into their services, including bias detection tools that can test and refine AI models before deployment and check for bias while models are in use.

Look beyond the obvious

AI could be invaluable in radically reducing deliberate and unconscious discrimination in the workplace. However, the more data a company analyzes, the more likely it is to deal with stereotypes, says O’Neil, who blogs about data science and founded the Lede Program for data journalism, an intensive certification program at Columbia University. If you’re looking for math professors, for example, and you load the hiring algorithm with all the data you can find about math professors, it may give a lower score to a Black female candidate simply because there are fewer Black female mathematicians in your data set. Here, the problem is incomplete data. But if that candidate has a PhD in math from Cornell and if you’ve trained your AI to prioritize those criteria, the algorithm will bump her up the list of candidates rather than summarily ruling out a potentially high-value hire on the spurious basis of a sparse data set.

To further improve the odds that AI will be useful, companies have to go beyond spotting relationships between data and the outcomes they care about. It doesn’t take sophisticated predictive modeling to determine, for example, that women are disproportionately likely to jump off the corporate ladder at the halfway point because they’re struggling with work-life balance. Many companies find it too easy to conclude that women simply aren’t qualified for middle management when they quit those roles. However, a human manager committed to smart talent management will instead ask what it is about these positions that makes them incompatible with women’s lives. They will then explore what the company can change so that it doesn’t lose talent and institutional knowledge that will cost the organization far more to replace than to retain.

That company may even apply a second layer of machine learning that looks at its own suggestions and makes further recommendations: “It looks like you’re trying to achieve X outcome, so consider taking Y approach.” In this scenario, X might be promoting more women, making the workforce more diverse, or improving retention statistics; Y might be redefining job responsibilities with greater flexibility, hosting recruiting events in communities of color, or redesigning benefits packages based on what similar companies offer.

Constantly hunting bias—a critical part of responsible AI

Even though AI learns—and maybe because it learns—it can never be considered “set it and forget it” technology. To remain both accurate and relevant, it has to be continuously trained to account for changes in the market, your company’s needs, and the data itself.

Sources for language analysis, for example, tend to be biased toward standard American English. If you’re building models to analyze social media posts or conversational language input, you have to make a deliberate effort to include and correct for slang and nonstandard dialects, says Jason Baldridge, a former associate professor of computational linguistics at the University of Texas at Austin and now a research scientist at Google, where he works on natural language understanding. Standard English applies the word sick to someone having health problems, but it’s also a popular slang term for something good or impressive. Confusing the two could lead to an awkward outcome, and correcting for that or adding more rules to the algorithm, such as “the word sick appears in proximity to positive emoji,” requires human oversight.

Today, AI excels at making unconscious bias data obvious, but that isn’t the same as eliminating it. It’s up to human beings to pay attention to bias and enlist AI to help avoid it. That goes beyond simply implementing AI to insist that it meet benchmarks for positive results; this step is a critical part of a program to use AI responsibly to generate fair outcomes.

O’Neil is attempting to create an entirely new industry dedicated to auditing and monitoring algorithms to ensure that they not only reveal bias but actively eliminate it. She proposes the formation of groups of data scientists who evaluate supply chains for signs of forced labor, connect children at risk of abuse with resources to support their families, or alert people through a smartphone app when their credit scores are used to evaluate eligibility for something other than a loan.

As we begin to entrust AI with more complex and consequential decisions, organizations may also want to be proactive about ensuring that their algorithms do good—so that their companies can use AI to do well.