Made to measure

The familiar ways of reaching consensus about measurements are leading chemists into troubled waters. Mike Sargent looks for some clear solutions

I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind. Lord Kelvin (1824 - 1907)

Measuring is arguably the most fundamental activity in science. Determining whether something exists is only half the battle: as Lord Kelvin pointed out, it is only when you measure something that you can truly begin to understand it. 

Crucially, those measurements must be reproducible in laboratories around the world. The traditional way to achieve this was laid out in the Convention of the Metre in 1875, a diplomatic treaty that created the International Bureau of Weights and Measures (BIPM) and now includes all the major industrialised countries. It proposed that a single object was constructed to define each physical quantity, such as a kilogram for the standard of mass, and carefully stored in Paris. Copies of the object could be hosted by each nation and periodically checked against the master, thus creating a traceable hierarchy of calibration. 

Although most physical measurements now rely on fundamental principles rather than a real artefact kept in Paris, the basic idea of traceability to recognised standards remains the same - in physics, at least. The use of measurement standards in this way depends on a complex infrastructure of national measurement institutes, the International Measurement System. This provides a global framework for collaboration to resolve measurement differences and improve techniques.  

But as analytical scientists study ever more complex mixtures of molecules, chemistry has moved away from this hierarchical model, which historically underpinned classical analysis. It has become impossible to compare meaningfully a pure standard with the messy samples of real life, even using modern instrumentation. Instead, consensus about chemical measurements is increasingly achieved not by comparison with a ’gold standard’, but by comparisons between a group of laboratories, or by reaching a consensus about the best analysis method to use for a particular task. 

However, the familiar ways of reaching this consensus are coming under increasing pressure from expansion of trade, which brings more countries and more laboratories into the system, and the expanding requirement by regulators and industry for proven comparability of measurements. 

Chemists also need to contend with the problem of measuring arbitrary quantities - such as the amount of ’fibre’ in a cereal - where quantification can depend entirely on the measurement method. More recently, the enormous expansion of molecular biology has introduced a whole new range of calibration issues with techniques such as microarrays or, for many forensic applications, the polymerase chain reaction (PCR). Here we need not only standards for calibration but also quality control materials to validate the entire measurement process - a nightmare when dealing with complex and unstable biological samples. 

Troubled waters 

The analytical chemists’ pragmatic approach - relying more on agreed methods and laboratory comparisons rather than traceable physical standards - may now be leading them into troubled waters. As regulators demand ever-more stringent tests, analysts must strive to develop new and reliable methods. But this has always been achieved by voluntary collaboration between scientists, which requires time and money. In today’s laboratories both can often be in short supply, leaving analysts floundering as science overtakes their ability to measure accurately complex entities in inconceivably small amounts.  

This problem has been highlighted by certain EU directives, such as those controlling mycotoxins [85/591/EC] and veterinary residues [96/23/EC] in foods. These set measurement specifications that were simply unattainable using methods available at the time. 

Bodies such as the RSC’s Analytical Methods Committee have previously coordinated extensive experimental refinement and testing, giving a reliable and robust method which could be readily applied in many other laboratories with minimal further validation. But such committees now find it difficult to involve practicing analysts, and often almost impossible to find enough laboratories willing to develop and test new methods. 

In the US, government bodies such as the Environmental Protection Agency (EPA) and the Food and Drug Administration (FDA) have taken on the role of defining these methods and providing funding to support development and testing. Doing this in their own laboratories, as well as through collaborative efforts, has alleviated the problem of keeping up with the demand for new standard methods.  

Messy samples 

The situation is worse in Europe where no central bodies have been prepared to consistently take a similar lead, or to fund the work. Instead, EU regulators let each laboratory use their own procedure as long as it exceeds a basic threshold of quality, set out as analytical specifications incorporated into new legislation. This has advantages in terms of flexibility and speed, as there is no need to await a new standard method from a centralised organisation. But it can also mean that essential new methods are simply not developed in a coherent and timely fashion. 

Regardless of the preferences in the US and Europe for standard method or analytical specification, the problem of those messy samples remains. Calibrating a method using physical or chemical standards is, on its own, insufficient to get reliable results. 

One approach to improve this situation is to develop certified reference materials (CRMs), a unique type of chemical standard that incorporates the element or molecule under investigation into a matrix that mimics the real substances that land on an analyst’s bench.  

Faced with assessing the amount of lead in a kilogram of road dust, for example, a CRM that approximates this chemical cocktail can prove to be far more useful for validating the method than a pristine, aqueous solution of lead. 

But the need for a wide variety of application-specific CRMs has meant that there are few links between materials produced for different applications, or by different organisations. There are thousands of CRMs in use, but many required for critical applications are unavailable. In addition, production costs are high and it is difficult or impossible to manufacture sufficiently stable CRMs for some applications.  

Given limited sales of most CRMs, production of essential materials is often possible only with government funding. As with any other product, laboratories also need some way to assess which of the many CRMs are reliable and which are best left alone. 

The National Institute of Standards and Technology (NIST) in the US has a major programme of CRM production which has served analysts around the world for many years. The NIST approach to certification involves measurements using their own reference methods. These provide rigorous, traceable results but are too time-consuming for routine testing. The alternative is ’round robin’ certification, based on results from a large number of routine testing laboratories using their regular methods. Such programmes have been popular in Europe, including the Euronorm consortium for metals, the former Community Bureau of Reference (BCR?) programmes of the European commission, and the UK’s Valid Analytical Measurement (VAM) Programme at LGC.  

Three government-supported laboratories in Europe - LGC, BAM and IRMM (see ’Setting the standard’) - have recently formed a consortium called European Reference Materials (ERM). This aims to help consolidate CRM production and achieve some of the benefits of scale enjoyed by NIST. The consortium’s strategy involves both approaches to certification, and selects the best combination on merit for each new material. This is an effective compromise for the current economic and regulatory climate. 

Inter-laboratory comparisons are another important way to bridge the gap between pure calibration standards and the real-life samples that confront analysts. They originated as informal round robin studies, but are now mostly formal proficiency testing (PT) or external quality assurance (EQA) schemes, often made mandatory by regulators. Like CRMs, however, these schemes have limitations which are becoming increasingly apparent.  

Measurement comparability usually extends only to the immediate participants in a single comparison because different schemes, or even repeat comparisons, are rarely linked. It is also impractical to organise comparisons for every application or to involve all relevant laboratories worldwide.  

So what can be done to ensure that CRMs and PT schemes can be trusted - and that all will lead the analytical scientist to the same ’right’ answer? 

Clear solutions 

The first strategy is to adopt traditional quality assurance principles. The International Standards Organisation (ISO) has a Reference Materials Committee (REMCO) which has produced an extensive range of guides over many years. These provide advice on all aspects of reference materials including their production, certification and use. Its ISO Guide 34 has been used as the basis of a new accreditation scheme for producers. The United Kingdom Accreditation Service (UKAS) recently completed a pilot project which accredited the first five UK producers, and a similar accreditation system for PT providers has been in place for several years.  

These efforts are now being complemented by integrating the traditional approaches for chemical and biological measurements into the International Measurement System.  

Since 1993, the International Committee for Weights and Measures (CIPM), which is the governing body of international metrology, has aimed to provide a unified international infrastructure for chemical and biological measurements through its Consultative Committee for Amount of Substance (CCQM) programme.  

National laboratories, including LGC, are working through the CCQM to confirm the equivalence of their reference methods, materials and standards by comparison with their peers. Their services and products, which generally use more stringent (and time-consuming) calibration techniques than field laboratories, can ultimately be used to check the results from other laboratories and reference material producers. The UK Department of Trade and Industry’s (DTI’s) Valid Analytical Measurement (VAM) Programme has produced a guide to applying the concept of traceability to routine methods. 

The ultimate goal is to provide traceable chemical reference values or calibrations, just like the well-known physical parameters such as mass or volume. Either way, the traceable certificate helps laboratories to decide whether the value is trustworthy and suitable as the basis for their results, because it provides a provenance for the data. Applying a single scheme of traceable measurements across all areas of chemistry and biology will achieve the same ambition that brought so many scientists to Paris in 1875: to ensure the equivalence of all measurements, no matter who makes them. 

These schemes are no magic bullet, because reliable measurements depend on qualified scientists being given the right tools for the job, not merely on building infrastructures and quality systems. The current economic climate, and limited government funding, is a serious issue for the future of chemical and biological measurement in Europe. In the UK, DTI’s VAM programme at LGC has done an excellent job since the late 1980s, but is now under pressure.  

Many other UK government departments provide only limited funding for new analytical methods and reference materials, relying on ’commercial pressure’ to solve their problems. Our major universities have never been ready to offer the support for analytical science which it merits, and the number of top tier research groups is now miniscule.  

Most important of all, the willingness of companies to support collaborative developments has declined dramatically since the heydays of the 1950s and 1960s. We need clear vision and firm leadership if we are to reverse these trends and ensure the future of analytical science.  

Mike Sargent is chief chemical metrologist at LGC. The views expressed are those of the author.

Further Reading

Meeting the traceability requirements of ISO17025: An analyst’s guide, September 2005. Third edition, ISBN: 0948926236, LGC, Teddington, UK.