Your browser is not supported. please upgrade to the latest version of Google Chrome, Mozilla Firefox, Apple Safari or Microsoft Edge.

Research: To Reduce Gender Bias, Anonymize Job Applications

Stefanie K. Johnson and Jessica F. Kirk

03/05/2020

As the existence of gender, race, and other biases becomes more widely acknowledged, many organizations are “blinding” their talent selection systems. Whether in a hiring process or an application for funding or other opportunities, there is some suggestions that anonymizing details about the applicant — removing their name, for example — leads to the selection of more candidates from underrepresented groups.

But the evidence supporting this approach hasn’t motivated the science community, where women represent only 28% of the science and engineering workforce, are paid less, receive less funding, and are cited less often than their male counterparts. Though institutions such as the National Institutes of Health have discussed anonymizing applications to reduce gender bias for at least a decade, there has been no movement towards implementing this process.

Our recently published research confirms that anonymizing can mitigate gender bias in the review of scientific research applications. Specifically, we found that when indications of candidates’ gender (such as their first name) were removed from applications for time on the Hubble Space Telescope, women were selected at a higher rate than when their gender was obvious.

One of the largest implementations of anonymous evaluations occurred among symphony orchestras in the United States. In the 1970s several U.S. symphony orchestras began changing their audition processes so that musicians auditioned from behind a screen. (Think of the popular TV show “The Voice.”) One analysis of the data showed that the percentage of women in the top five U.S. orchestras increased from 5% in 1970 to 25% by the 1990s. Anonymizing the audition in the preliminary round alone increased the probability a woman would advance to the next round by 50%.

The results from this study were compelling, and we wondered whether they would translate to other domains. The research on the symphonies also never examined whether male or female evaluators were more affected by anonymization. We wanted to test those two questions, so we turned to data from 15,545 applications spanning 16 years of applications for research time on the Hubble Space Telescope from 2001 to 2018.

This data was ideal for analysis because the Hubble Space Telescope Time Allocation Committee changed their application review process fairly significantly over that period. In 2014, evidence emerged that there was a statistical gender bias against women in the application process: A study revealed that female applicants’ success rate was about 19%, even though nearly 23% of the applications came from women. The Hubble Space Telescope Time Allocation Committee decided something had to be done and embarked on a process of obscuring applicants’ gender. Initially the first name of the applicant was simply removed from the front cover of the application. In 2018, all personally identifying information was removed and evaluators were instructed not to discuss characteristics of the scientists, but to only evaluate the merit of the science.

When we compared application success rates over these years, we found that female applicants were significantly more likely to have their proposals accepted when their gender was obscured in the application process. The most effective of all of the changes was to completely remove all names and instruct the reviewers not to discuss characteristics of the scientists. Before any anonymization, men outperformed women by about 5%. After just the removal of the names, that number dropped to less than 3%. When the applications were fully anonymized, women outperformed men by 1%.

Women are still vastly underrepresented, since they represented only 23% of applicants on average across this time period. But their relative success has been steadily growing: While women represented 18% of the accepted applications before anonymization, that number rose to 23% after first names were removed and to 30% when fully anonymized.

In addition, we conducted analyses comparing the year prior to anonymization to the three most recent years to examine whether rater gender affected the results. We found that the change in women’s success rates appears to be driven primarily by changes in ratings from male reviewers. This is somewhat surprising in that we all hold unconscious biases, but it seems that men were more likely to let those biases affect their ratings. We believe this might be due in part to the fact that the female reviewers — astrophysicists themselves — likely have more woman scientists in their own networks and might therefore have less-gendered unconscious biases. The female reviewers are a bit younger than the men; this would mean they went through school at a time when there were more women in higher education and science, thus again potentially making them less prone to unconscious gender bias.

What can organizations — STEM and otherwise — do with this information? We recommend that they implement anonymized recruitment where possible, especially at the early stages of applicant screening. When you have evidence that gender, race, age, or other differences are affecting your selection process, despite their not being relevant selection criteria, you have error in your processIn other words, extraneous information about one’s identity is causing you to make less accurate decisions. Given that we all want to make good (and legal!) selection decisions, the focus should be on defining relevant criteria beforehand and ensuring that extraneous information is not erroneously influencing your decisions.

Read More

    Company Culture
    Gender Equity/Diversity
    ​Unconscious Bias

Load older comments...

Loading comments...

Add comment

21

October 2020

Is The Sky Blue? Amy Coney Barrett Still Won't Answer Really Obvious Questions.

13

February 2019

Silicon Valley Is Using Trade Secrets to Hide Its Race Problem

18

April 2019

California Bill Would Expand Definition of Race to Include Hairstyle

10

August 2020

The Fortune 500 now has a record number of female CEOs: A whopping 38

17

December 2018

It’s time to rethink traditional mentoring. Here’s a guide to alternatives

You've Been Timed Out

Please login to continue