Many companies are now using AI tools to sort through applications, with the goal of saving precious time for human resources teams. But how it’s used is as important as the fact that it’s used. For some employers, their use of AI is resulting in some major liabilities. While we have yet to see litigation around AI implementation in Canada, in the U.S. there have already been some major cases.
Caselaw examples
Last July, CVS Health Corp. and CVS Pharmacy settled a class action lawsuit against them over the use of AI during job interviews. The plaintiff in that case, Brendan Baker, alleged that his facial expressions during his job interview were used by an AI system to analyze such qualities as his “willingness to learn”, “conscientiousness & responsibility”, and “personal stability”, which formed part of an “employability score”. He had applied unsuccessfully for a job and only found out afterwards about the AI analysis. The lawsuit claimed that such an AI analysis amounted to a “lie detector” test on job applicants, which is illegal under U.S. law. Since the case was settled, we do not yet have a judge’s decision on whether the use of the AI tool in this situation was illegal or problematic.
Also last July, a California court allowed a class action to proceed against Workday, which provides human resources management systems to medium and large businesses. Part of Workday’s offerings are AI tools for screening job applications. The plaintiff in that lawsuit, Derek Mobley, claims he was rejected for more than 100 jobs he applied for between 2017 and 2024 with companies that use Workday’s AI hiring tools because the platform discriminates based on race, age, and disability. Mobley is Black, over 40 years of age, and has anxiety and depression.
Considerations When Implementing an AI System
Both of these above cases raise a number of helpful questions to consider when deciding on an AI-powered system:
- What problem are we trying to solve?
- What are the pros/cons of the AI support? For example, what are the benefits and drawbacks of having a computer program read an applicant’s facial expressions?
- What is the impact on trust between the applicant and the company, if the applicant finds out their interview is being analyzed by AI? Should communications be made to the candidate to lessen this impact on trust?
- Might the known use of AI systems deter some people from applying? What are steps to consider to encourage good applicants, even in the face of AI hesitancy?
AI-Powered Recruitment & Discrimination
In some situations, AI can accomplish a task that would be impossible for a human, such as sorting through hundreds of job applications in a very short amount of time. But whether this produces desirable results depends on how the AI system accomplishes that task.
It’s already widely known that AI can produce discriminatory results if it’s not used properly. To understand this, we have to go back to a computer science principle called “Garbage In, Garbage Out”. Simply put, it means that the quality of the outputs that an AI system produces will depend on the quality of the data it is trained on.
You might remember that Amazon once scrapped an algorithm that was being tested as a recruitment tool because it preferred male candidates. That discriminatory preference was because the algorithm had been trained on data from mostly male applicants. Therefore, it “thought” that men were better applicants compared to women.
Is this what happened to Derek Mobley, but with race, age, and disability instead? Unfortunately, we don’t know yet. If that case proceeds to a trial, there’s a real chance we might find out more about Workday’s algorithm and how it works. But if it doesn’t, we might never learn those details.
Take-Away
The important lesson for companies here is that they should never blindly trust AI systems to do a job “properly”. Companies should always ask questions about how an AI system is being trained, and monitor the outputs to ensure that the results are desirable. With proper human monitoring and intervention, AI systems will increasingly be core tools that save enormous amounts of time – but without the right safeguards, they can be liabilities that produce trouble behind the scenes.
In future posts, we’ll review some of the ways that companies can use AI to get the results they’re looking for. Stay tuned. In the meantime, if you’re considering implementing AI in your business or have any concerns about its current use, don’t hesitate to reach out.
