The number of filed patents is a misleading metric for assessing academic research

For some, the basic yardstick of innovation is the patent – and in Europe, this is apparently in high gear. In March, the European Patent Office’s annual report revealed that it granted a record 95,940 patents in 2016, a 40% increase on the previous year that was largely due to efficiency reforms at the office. Some 23,353 of those patents were in chemistry, where patent filings are dominated by the behemoths of the chemical industry: BASF, Dow Chemical and the rest.

Patent/University/Innovation

Source: (c) Mutello/iStock

The report notes that just 6% of patent applications came from universities or public research centres. Many would like to see that number grow. Universities were once free to focus on research and teaching, but they are increasingly expected to be innovators as well, spinning basic research into commercial gold. Innovation offers a way to get a return on public investment in research, and it can also bolster university coffers against funding constraints.

Patents offer a deceptively simple way to measure that innovation. Many technology transfer offices judge their success on the number of patents granted and licensed; universities trumpet these numbers in annual reports, to burnish their innovation credentials. In the US, the Federal RePORTER system includes patents (among other indicators) to assess the impact of federal research grants, while a growing number of international rankings use patents to identify the most innovative universities.

And in the UK, at least, there are concerns about whether universities are patenting enough (witness the near-annual handwringing about the country’s dearth of graphene patents). But are patents really an effective way to measure the innovation that springs from universities?

Pitfalls pending

Last year, Reuters published its ranking of the top 100 most innovative universities in the world. This used a variety of metrics, including the number of patent applications, how often those were granted, and how frequently they were cited. Perhaps unsurprisingly, the top three slots were taken by the US’ Stanford University, Massachusetts Institute of Technology and Harvard University.

But patents capture only a tiny amount of the innovation at these institutions. A study published in early April 2017 showed that simply counting patents filed by university researchers ignores a huge swath of innovation impact. It surveyed 365,380 grants made by the US National Institutes of Health (NIH) from 1980 to 2007, and found that while 8.4% of NIH grants generated a patent, more than 30% generated academic papers that were subsequently cited by patents.1 By looking only at direct patenting by academics, ‘you miss a lot of the potential applied value of the work’, says Danielle Li at Harvard Business School, who co-authored the study.

There’s no doubt that some inventions developed in universities have an immediate potential to be extremely lucrative and are worth patenting: hence the epic fight over the Crispr-Cas9 gene editing technology. But the vast majority of universities’ patents go nowhere. Most are never licensed; some are even sold to patent brokers who may be more interested in suing other companies for infringement than developing products.2 In many cases, the commercial potential of the science would be better realised by publishing the work in academic journals and using it as a foundation to develop genuine partnerships with industry.

Patents are also a very poor global indicator of innovation. Different patent offices operate under their own rules, and the costs of filing and protecting a patent vary considerably. Meanwhile, government policies can hugely distort patenting behaviour. I remember visiting the Tsinghua-Foxconn Nanotechnology Research Center in Beijing in 2009, where I was proudly shown two adjacent floors in the pristine facility. The lower floor was given over to research on nanoelectronics; the upper floor devoted the same area to a business and legal team, poised to file patents on any novelty arising from the lab beneath. Three years later, the facility had applied for more than 3000 domestic and international patents. This is not because China’s scientists are inherently more innovative, it is the result of a national strategy: the government funds universities to file patents and then rewards them on their output, so universities naturally encourage researchers to generate as many patents as possible.

Partnerships, not patents

There are plenty of alternatives to patent counting. Last year, a study3 led by Robert Tijssen of Leiden University in the Netherlands suggested that a basket of other metrics can provide a more realistic assessment of a university’s innovation capacity, such as how many academics hold dual appointments in industry, the scale and productivity of shared R&D facilities, and so on.

In January, Clarivate Analytics ranked innovative universities by looking at how many papers their researchers had co-authored with the world’s most innovative companies, and then assessed their citation impact. This attempts to capture whether universities are forming the right partnerships to turn basic research into collaborative research and development.

And a third option, favoured by Alex Usher, president of Higher Education Strategy Associates, a Toronto-based consultancy, is to survey the opinions of expert panels about the strength of academic-industry innovation partnerships in universities. This is a more intensive effort and it is qualitative, but it may ultimately give a more rounded picture of innovation activities.

Any numerical metric based on patents will always fail to capture the true level of innovation in a university. As such, policymakers and university leaders should not make strategic decisions, or allocate resources, based on patent counts.