.png)
Published: October 4, 2021
McKinsey estimates that up to 30% of workers will be displaced due to automation by 2030. Many of those workers will be thrown into a job market that’s also become automated by AI that rejects resumes for small errors or superficial assumptions that block candidates with the right skills for the job.
The danger of AI bias has been widely documented in areas like voice recognition, facial recognition, and in job matching, too. In the landscape of automated hiring tools, the four-fifths rule recommended by the Equal Employment Opportunity Commission (EEOC) provides job seekers with little protection simply because companies are not required to publish their hiring data.
At Talk Hiring, we’re fighting automation with automation. Our mission is to create the hiring technology that supports populations that are often left behind by technology. To combat AI bias in our own machine learning algorithms, we are committed to transparency and going beyond the EEOC’s data sharing requirements.
This summer, we sent out a survey to Talk Hiring users (excluding high school and college students due to FERPA) and 132 participants opted to share their demographic information with us. While the sample size is small, we’ve taken this opportunity to do an initial data analysis with the goal of identifying potential bias in our AI-powered interview feedback system.
For each Talk Hiring user that filled out the demographic survey, we pulled the average mock interview score and compared performance across demographics. For every dimension, we made a box & whisker plot to see the median score and how the data was distributed. While some variability is expected, we looked for outliers that match up with the white/male bias most often found in AI algorithms.
Ethnicity
We compared the median scores of the four largest ethnicity groups that responded to our survey, which were Black (70.92), Asian (75.2), Hispanic/Latinx (76.26), and White (70.40). White participants had the lowest median score while Hispanic/Latinx participants had the highest.
Gender
For gender, we compared participants that identified as male and female (leaving out other gender identifications due to the small sample size). Female participants, with a median score of 74.28, slightly outperformed male participants with a median score of 73.50.
Age
We saw the greatest variability across demographics in the category of age. Our lowest performing age group was the youngest with a median score of 67.53. This makes sense since young adults have had less job interviewing experience. The highest performing age group was 26-30 year olds with a median score 78.15.
In summary, our first data analysis shows demographics slightly outperforming others that go against traditional stereotypes about white/male dominance in AI algorithms. While the current sample size is too small to run statistical significance tests, we plan to do so on our performance data in the future to ensure our algorithms remain non-discriminatory.