Assessing the value of researchers’ work is hard, but there are some easy ideas to avoid

An image showing papers vs scientists

Source: © Matt Herring/Ikon Images/Getty Images

The diversity and unpredictability of research work means that performance is hard to assess, and some review methods make the problem worse, not better

I have, by now, been through enough cycles of performance review in the biopharma industry that I feel confident in saying that there is no good way to do it. Not for researchers, at least. The jobs before us are wildly variable and can change without warning, and on top of that, evaluating how we’re doing at them is next to impossible. Companies can select from several unsuitable systems with varying degrees of utility, ranging from ‘not that helpful’ down to ‘frankly destructive’. A look at the latter type might be helpful, so that it can be recognised from a distance.

Many companies use or have used some sort of stack-ranking system. This takes the total list of employees in a given unit and at a given job level and ranks them in order from best to worst. Typically, this is coupled to a set of criteria that separate people into categories with names like ‘exceeds expectations’, ‘meets expectations’ and lower levels with ominously euphemistic names, rather like the government agencies in George Orwell’s 1984. ‘Improving’ may be a hopeful adjective when describing the weather, but it is not a hopeful category in a performance review system.

How does someone who makes a dozen thoughtful and difficult compounds get ranked next to someone who cranks out a much larger list of easier, near-random analogues

This system has its problems – a fundamental one being that it can be hard to evaluate researchers even in broad terms, and even less so by precise ranking. How does someone who makes a dozen thoughtful and difficult compounds get ranked next to someone who cranks out a much larger list of easier, near-random analogues? What if none of that first person’s compounds were active, but their work validated an important hypothesis? And what if the second one made the likely clinical candidate, but it was just another compound on the list and it happened to work out? Who gets a better review?

That last question can get rather fraught in a stack-ranking organisation. Often enough, the employees are competing for a zero-sum pool of money for payrises and rewards, so if one person’s amount is increased, someone else’s must go down. Many companies force a distribution, often a rough Gaussian bell curve, ignoring the fact that this would only apply if they hired their employees randomly with no regard to their abilities. This can have the effect of making each category zero-sum as well: if you want to add someone to the ‘exceeds’ band, someone else has to be pushed back to ‘meets’.

You run a terrible risk of punishing people because they had the bravery and optimism to take on a hard project, or because they encountered sudden misfortune

The most vicious form is ‘rank and yank’, popularised by former business deity Jack Welch of General Electric and adopted at one time or another by a long list of companies. The premise is to simply fire the bottom 10% or so from your ranked employees.

This approach is supposed to sharpen everyone’s focus and improve the effectiveness of the employee population with every review cycle. Real-world experience, though, has shown that it can turn a workplace into a Hobbesian war of all against all and bring on all sorts of counterproductive behaviour. Everything is done in fear of inadvertently ending up in the bottom 10%. Managers may deliberately avoid hiring the best candidates for open positions and people will avoid joining new teams or projects that already have too many good or productive people on them.

Rank and yank is thus brutally inappropriate for a team of scientists. Everyone doing R&D is at the mercy of the physical world, when you get down to it, which means that you run a terrible risk of punishing people because they had the bravery and optimism to take on a hard project, or because they encountered sudden misfortune during one that had been going well. And against that background, constantly fighting not to end up on the bottom of the pile brings on a culture of cautious, transactional game-players who evaluate every move by how likely it might be to lead to trouble. It brings to life Gore Vidal’s infamous quote: ‘It’s not enough that I succeed – other people have to fail’.

I can imagine nothing more toxic to the open, shared risk-taking of a productive research organisation. You may have no choice but to navigate through a stack-ranking environment, but if you see the scythe coming out for the folks at the far end, my advice is to start seeking employment elsewhere. Trouble is coming – in fact it has already arrived.