Trusting our intuition too strongly highlights how little we know about chemical activity
What is chemical intuition? One example would be the ability, won through experience, of a chemist to look over a set of compounds and sort them (however roughly) into the ones that are more or less likely to be of interest for some purpose. I work in drug discovery, so the examples that I see tend to be questions such as: ‘What compounds should be in our screening libraries?’ and: ‘Which of these hits from the assays should we spend more time on?’ But the same process applies to people in materials science, polymers, coatings, energy applications and all parts of chemistry. We’re all faced with more options than we can possibly follow up on, and we need to work on the ones that are more likely to lead to something useful. Over time, we work on successful projects and, inevitably, not so successful ones, and we naturally take note of those compounds and classes that have been useful and the ones that have led us astray.
As we should! Experience is indeed worth something (it had better be, since we tend to get paid more as we accumulate it). But there’s danger in there as well. It’s tempting for us to overvalue ourselves, and there’s empirical evidence that we do. A rather infamous paper some years ago presented lists of compounds to chemists across a drug discovery organisation, asking them to rate the structures by their desirability in the screening collection. The agreement between individuals was not high, unfortunately. You would have hoped for a bit more consensus, but the overriding factor seemed to have been what sorts of compounds people had had personal good or bad experiences with. One person’s ‘funny-looking’ compound was another’s old friend, and one person’s acceptable structure would set off alarm bells with someone else, since they’d been burned by something similar and had sworn never to let that happen again.
Experience is worth something; it had better be, since we tend to get paid more as we accumulate it
The really embarrassing part came when the same chemists saw the same compounds again, after a few months, without being told that they were seeing the same lists. Their agreement with their own past preferences was not robust, either. I suspect that similar experiments would show similar results in other fields. The problem is that we overestimate our abilities for the task we’ve set. For drug discovery chemists, humility is close at hand: all you have to do is look through the structures of the top 100 compounds by sales or prescription number. Here’s the important part: pretend that these are assay hits and not successful drugs. If you’re honest with yourself, you will soon encounter things that you would cross off a list (or at least move into the ‘only if we don’t find something more appealing’ category).
Consider metformin, an incredibly useful and successful diabetes medication. I would not wish to guess how many patients it has helped, nor how many billions of dollars it has produced in revenue (it’s been a generic drug for years now). But metformin is also a ridiculous-looking molecule, at least to a medicinal chemist. It’s very small and it has more nitrogen atoms than carbons. Honestly, it looks more like something you would use as an additive in marine paints to keep barnacles from adhering, or some kind of urea alternative for a fertiliser mix. I can tell you without hesitation that if I had seen this molecule early in my career, as a screening hit, I would have tossed it aside immediately. Can’t be right. Not a drug.
But when I was younger, I (thought I) knew a lot more about what I was doing. I am now experienced enough – I hope – to know more about how much I don’t know, and I have the example of metformin as a constant rebuke to my ability to pick out billion-dollar drugs from a list. There are still compounds that I don’t want to work on (polyphenols, in case you’re wondering, just to pick one class). But I need very clear reasons and examples to kill an odd-looking structure now, as opposed to ‘that just looks wrong’. I am not smart enough to tell you that – but fortunately, I am (just barely) smart enough now to recommend running a few experiments before I go around laying down the law. Progress!