AI Bias Favors Western Names in Corporate Promotion

Imagine a company where every meeting is recorded, transcribed, and read by a machine. This is the reality for many firms, including Amazon and Meta, which use tools like Granola, Read.ai, Fireflies, and Avoma to track talk time and participation. These tools are meant to make contribution more measurable and fair. However, a recent study has found that AI can be biased against non-Western names. It's a problem that doesn't just affect one or two companies - it's widespread.

Many companies don't realize they're using biased AI tools.

The study, conducted by a professor at Cornell and INSEAD, found that AI heard Western names about five times more often than non-Western names. This is because the training data for the AI models skews American and native, resulting in higher error rates for non-native and accented speakers. The professor didn't just stop at finding the problem - they also compared the AI's judgment against their own participation ratings. They found a significant discrepancy. For Western names, the correlation between the AI's ratings and the professor's ratings was 0.53.

For non-Western names, it was 0.27. This discrepancy is a concern, as it shows that the AI isn't rating everyone equally.

This bias isn't limited to speech models. Large language models used for screening resumes have also been found to favor white-associated names 85% of the time. This is a concern, as AI is increasingly being used to make decisions about promotions and hiring. If the AI is biased, it could lead to unfair outcomes for individuals with non-Western names. They won't get a fair shot at getting hired or promoted. It's not just about hiring - it's about creating a fair workplace.

The professor notes that this is a data science problem and can be fixed. By reweighting the training set to give more signal to underrepresented groups, transforming features to break their correlation with sensitive attributes, and adding fairness constraints during training, the bias can be reduced. It's also important to audit predictions against tests like demographic parity and equalized odds. The professor says it's a problem that can be solved with the right tools and techniques.

The good news is that this is a data science problem, and data science has tools for it.

The implications of this study are significant. As AI becomes more ubiquitous in the workplace, it's essential to ensure that it's fair and unbiased. This requires a clear-eyed acceptance of the potential biases in AI tools and a commitment to testing and validating their results. The professor's study is a step in the right direction, highlighting the need for more research and development in this area. It's not going to be easy - but it's necessary.

We can't just ignore the problem and hope it goes away.

The study's findings also have implications for diversity and inclusion initiatives in the workplace. If AI is biased against non-Western names, it could perpetuate existing inequalities. Companies need to be aware of these biases and take steps to mitigate them. This could involve using multiple evaluation methods, including human assessment, to ensure that all employees are given a fair chance to contribute and advance. They shouldn't rely solely on AI - they should use a combination of methods.

It's the best way to ensure that everyone gets a fair shot.

The study highlights the importance of ensuring that AI tools are fair and unbiased. By acknowledging the potential biases in these tools and taking steps to address them, we can create a more inclusive and equitable workplace. It won't happen overnight - but it's possible. We just need to be committed to making it happen. It's a goal that's worth working towards.