Women and non-white researchers offer science more original concepts but their contributions are less likely to be adopted by others or lead to academic positions, a new US study has found.

A ‘diversity paradox’ has been noted before: diversity breeds innovation, yet under-represented groups that diversify organisations have less successful careers within them. Now researchers from Stanford University show for the first time that the paradox holds for scientists as well.

A team led by Bas Hofstra and Daniel McFarland amassed data on around 1.2 million PhD recipients in the US between 1977 and 2015, following their careers into publishing and faculty positions. They used census and social security data to infer students’ gender and whether they belonged to minorities under-represented in science.

To measure novelty and innovation, the team used language processing techniques, such as machine learning and text analysis, to identify terms in PhD theses that represented ‘meaningful concepts’. Next, they identified when pairs of concepts were first related to one another. By adding up the number of these original, related ideas within each thesis, they developed a measure of novelty. The next stage was to calculate how much of this novel thinking had any impact. They measured how often a thesis’s new linked ideas were taken up in other documents.

The researchers believe this approach works better than using citation metrics. ‘We studied PhD students [who] often don’t have many citations or publications to measure novelty,’ explains Hofstra. ‘Therefore, we resorted to studying their texts and the novelty footprints they leave in their dissertation abstracts.’

The team found that historically underrepresented groups in science, such as women and racial minorities, have more novel ideas, but other researchers are less likely to follow them up so they have less impact. White men have the lowest rate of novelty but the highest levels of ‘impactful novelty’.

Then Hofstra and McFarland looked at those students who obtained a university research position and those who carried on with research in another way, whether in industry or other non-tenured roles, shown by their publishing records in the Web of Science, a large publication database. They found that, compared with men, women are 5% less likely, and under-represented races 25% less likely, to gain tenured research positions. Women were also 6% less likely, and under-represented races 10% less likely, to continue research in another way.

The study reveals ‘troublesome patterns’, says Hofstra. There appears to be a ‘stratified system’ where underrepresented groups have to innovate at higher levels to have similar careers to their white peers. Universities must continue to address biases in hiring researchers, research evaluation and publication practices, he adds.

‘To my knowledge, this is the first time that diversity and innovation have been looked at in this way,’ comments Kimberly Griffin, an associate professor of student affairs at the University of Maryland. ‘I am familiar with the literature that links diversity in teams, particularly in science and engineering, with innovation and impactful work, but this work is unique in that it tries to pinpoint who is innovating and whether all innovation is viewed in the same positive regard and translates to positive career outcomes.’

Griffin says the work is important because it quantifies what many have suspected for a long time. ‘It confirmed [suspicions] about whether the creativity and innovations of women and men of colour are recognised in the same ways as their white male peers. Decades of research suggests that the skills and talents of women and men of colour are often doubted and scrutinised, which can lead to their potential and contributions being overlooked.’

The study highlights the need to examine the roles that racism and sexism play in assessing the value of science produced by women and men of colour, she continues. Stereotypes, along with implicit and explicit biases, affect whose work is given more attention. ‘Institutions must continue to acknowledge these dynamics, and push for policies and training for faculty and institutional leaders that foster deeper understanding of bias and how to mitigate it.’