Scientists in Canada have developed a platform for visualising the pathology of tissue in real time, producing a superimposed colour-coded image over a video feed. Their augmented reality platform could be combined with other technologies to deliver molecular-guided surgery.

Typically, a surgeon will determine the disease state of tissue by removing a sample, sending it to be analysed with staining techniques and then receiving the results from a pathologist. However, recently developed mass spectrometry probes are now able to provide that information within 5–10 seconds by analysing a tissue’s molecular signature pattern and comparing it with profiles from histologically classified tissue types.

‘There are all sorts of probes out there that perform the same function of being able to deliver very rapid pathology assessments,’ explains Arash Zarrine-Afsar, whose team at the University of Toronto carried out the research. ‘What we’ve thought about is how we can use that pathology assessment information and make an impact in surgery.’ Their probe, which the surgeon would hold just above the tissue, vaporises small molecule metabolites and lipids with a 300 picosecond laser pulse to generate an aerosol plume. This plume is collected through a tube held in the surgeon’s other hand and evaluated with time of flight mass spectrometry.

An augmented reality display superimposes coloured pixels onto the surgical video feed to portray a diagnosis of the tissue within 5 seconds. These pixels are coloured red or blue according to whether the probe has detected healthy tissue or tumour. An optical navigation system completes the set-up, in which a dual infrared light camera tracks IR sensors attached to the probe to find its position. The camera providing the video feed of the surgical area also has IR sensors attached to it. This tracking system allows the scientists to correlate the position of the mass spectrometry probe with the video feed to map the colour-coded pixels onto the display.

‘This is an absolutely fine, state of the art technology being used, with much future application. However, we have to be careful with statistical analysis and data sampling,’ comments Martin Grootveld of De Montfort University in the UK, a researcher in bioanalytical chemistry and chemical pathology.

The scientists tested their system on a section of normal mouse brain tissue next to a piece of tumour. This created a clear border between the two, as an example of a homogeneous, non-infiltrating tumour. However, in reality tumours can be infiltrating and spread into healthy tissue. ‘The signal we get from our mass spectrometry is really a population average. So if you have one cancer cell and 999 healthy cells, you would likely see something that looks more like healthy cells than cancer,’ says Zarrine-Afsar. ‘I want the community to run with this and … make this research better by incorporating real-use scenarios and understanding.’

The team say their augmented reality platform could be adapted to fit various surgical applications by combining a video display with a variety of mass spectrometry probes with different functions, for example, those with the potential to make incisions and remove tumours. ‘I think the big challenge is still can we improve outcome with these technologies?’ says Zarrine-Afsar. ‘That’s still very much an open question.’