Scientists must be trained to work without AI, as well as working with it

In the last few months, concerns around AI have made the headlines as announcements of the technology’s ever-growing capabilities continue to emerge. There have even been warnings that AI could pose an existential threat to humanity, with companies behind these technologies asking to put the brakes on amid reports of agents going rogue and hacking websites and even government agencies. At the same time, AI’s impressive achievements continue – such as OpenAI’s (disputed) claim of a solution to the Navier–Stokes equation. In that respect, scientists and engineers might feel that the more immediate threat from AI is that it is taking on more of the work involved in research.

In chemistry and materials science, for example, agentic AIs linked to self-driving labs are creating closed-loop research systems. Increasing the scale of these autonomous labs should enable us to explore greater expanses of chemical space, but every new, exciting example of more capable self-driving labs fills me with dread. As a statistician, I worry that this could just increase the amount of wasteful experimentation, and as a scientist I worry about the loss of agency. With each story we need to separate the real capability improvements from the hype and reflect on how this changes the way that we educate the next generation of scientists.

AI is becoming a capable chemist

A recent collaboration between OpenAI and Gingko Bioworks, for example, used an AI-guided autonomous lab to optimise a protein synthesis route. The experiments were designed by the GPT-5 large language model in San Francisco and then executed on Gingko’s high-throughput laboratory in Boston. Over many iterations and a total of 36,000 experimental runs, the final recipe (covering at least 25 ingredients and reagent variables) resulted in an impressive 40% reduction in cost of production.

36,000 runs though! While this is a great example of AI-enabled autonomous science, it’s hard to argue it’s a productivity breakthrough. Maybe efficiency was not the focus here: it’s not in the interests of these companies to optimise learning per token or experimental run. Yet with over 40 (presumably human) authors of the paper from Gingko alone, even the autonomy claim seems a stretch.

We should teach and test the fundamentals of chemistry as we always have done, without AI

Another example from a recent preprint relates how the AI agent framework, La Agente Óptima, developed by Alán Aspuru-Guzik and his team, made reasoning calls similar to a human scientist. In optimising a reaction, Óptima picked up on an unusual 0% yield and analysed the NMR data and acquisition metadata to correctly conclude that the 0% result likely reflected genuine chemistry rather than an instrument failure. Perhaps most impressively, Óptima derived rules to flag repeated zero-yield results and halt the campaign if a pattern suggested an instrument problem. Nevertheless, a human-directed campaign for the same problem found a higher yield, albeit in more than twice as many runs.

With or without AI? Both is good

These examples both represent impressive advances. Yet they also highlight the need for human reasoning. We don’t want the future of science to be vast, resource-intensive lights-out labs controlled by a few well-funded tech companies. Like data centers now, they could easily become unpopular physical manifestations of AI and the focus of opposition to the race to build more at all costs. And while chemists can benefit from these new tools, handing more experimental agency to AI risks not only risk losing control, but also losing the basic skills and understanding we need in order to add value as humans-in-the-loop.

We should teach and test both methods

This is a version of the general concern that humans are becoming dumber as we let AI do more of the hard thinking for us. Our ability to ask questions and interrogate the world to better understand it – the very practice of science – is being altered as evaluations, inferences and decision-making steps are moved from human minds into machines. As a recent editorial in Science notes – being able to track decision making and weighing alternatives essential for the contestability of claims that is a cornerstone of science. This doesn’t mean we must ringfence human decision-making, but rather sustain the education and training that ensures human reasoning retains its value.

When the pocket calculator was introduced in the 1970s, educators were concerned that students were not learning basic mathematical skills. The UK government-commissioned Cockcroft report found a simple solution to arrest this decline: teach and test both methods – with and without calculators. We should teach and test the fundamentals of chemistry as we always have done without AI, while also introducing them to AI-enabled solutions for route-selection, for example.

Manual laboratory practicals will give students knowledge and experience that they can use for success with automated laboratory hardware, and for those reactions and processes that can’t easily be automated. And the basics of statistical design and analysis of experiments can be taught and examined with pen and paper to ensure chemists have a conceptual grasp of data-driven experimentation for effective agency in AI-enabled exploration.