Holes in the ‘holey graphyne’ story

At one of the first meetings of my brand-new research group, a student was enthusiastically talking about a groundbreaking paper he had just read. A group of researchers had selectively broken down a 1,2,3-triazole, the product of one of the most important click reactions, with ultrasound.¹ Click chemistry was the subject of my PhD thesis, and our group used the method extensively. I was delighted with my student’s choice of paper, until I noticed something amiss: that infrared spectrum looked very wrong!

A lively discussion ensued. The more I looked at the data, the less I believed the authors’ claims. And my student could not believe that I would dare to question a paper published in Science. We agreed that the data were inconsistent but could not pinpoint the mistake. Four years later, the paper was retracted following an institutional finding of research misconduct.

My mentors did their best to teach me how to design experiments, analyse data and think critically. While the handling of scientific errors was a part of my training, the emphasis was on intellectual honesty and comprehensive controls. I assumed that every scientist always did their best and honest work. The realisation that someone would consciously cherry-pick the data, or engage in outright fabrication, came as a shock.

The key indicators of achievement in modern academia are citation metrics and the number of publications. The relentless pursuit of ‘excellence’ through bibliometrics has skewed incentives and promoted questionable research practices. These range from relatively harmless exaggerations of findings, to Feynmanian ‘cargo cult science’, to egregious offences like data fabrication and medical studies conducted without ethical approvals.

Despite the common belief that science self-corrects, most flawed papers evade retraction

This metric-driven enterprise generates a torrent of problematic papers, which leave an indelible mark on the global body of knowledge and create serious consequences for science and society. Peer-reviewed studies, regarded as credible and authoritative, inform clinical treatments, policy decisions and funding allocation. Despite the common belief that science self-corrects, most flawed papers evade retraction, as even senior academics often hesitate to criticise their potential reviewers. For the same reason, open discussions of questionable research practices are a very recent development.

In response to this emerging situation, several forums and media outlets focused on research integrity emerged. A number of ‘sleuths’ have been remarkably effective in holding authors, institutions and journals accountable, especially in biomedicine where many of the key data are pictorial. While image manipulations like duplicated regions are detectable by skilled individuals or AI, the problem certainly extends beyond the visible. In fields such as chemistry and physics, where data take different forms, identifying problematic papers at a glance is challenging. And it is virtually impossible to get one retracted.

Recently, the journal Matter published a paper describing a novel form of carbon.² This purported allotrope, ‘holey graphyne’, is comprised mainly of cyclooctadiyne rings. Moreover, the synthesis was supposedly accomplished using a simple copper catalyst. Typically, the C–C bond-forming reactions of the kind claimed in the Matter paper require expensive palladium.

The unusual structure and the unprecedented chemistry should have raised eyebrows during peer review, and ideally before this review commenced. Eight-membered rings with even a single triple bond are highly reactive (harnessing that reactivity has brought a Nobel prize to Carolyn Bertozzi). A material containing that many two-triple-bond rings would be more energetic than TNT, and likely quite prone to rapid unscheduled disassembly. Contrary to these expectations, the paper asserted perfect stability of the ‘holey’ material up to 700°C. While the reported spectroscopy was demonstrably mismatched with the claimed structure, the authors simply declared that everything fits.

The holey graphyne paper would have been easy to ignore. Surely, anyone with even a basic knowledge of chemistry will discount the grotesque structure! However, it was brought up by one of the reviewers for our tangentially related manuscript,³ requiring a lengthy and unnecessary explanation.

I got in touch with the editor of Matter and proposed a replication study. After all, my group had plenty of the hard-to-make intermediate! No holey graphyne formed under the claimed conditions. The corresponding author of the original paper refused to supply raw data or answer questions, and tacitly assigned all the blame for irreproducibility to a former group member.

Our replication is now published.⁴ We are grateful to the editor of Matter, as I know from experience that not every editor would even acknowledge the problem.

Regrettably, this story doesn’t qualify as a decisive win for post-publication review. Following their policy, Matter allowed the authors to publish a response, which doubled down on the original dubious conclusions. For now, the ‘holey’ paper remains unretracted, though I remain hopeful.

You may have come across equally troubling papers in your field. Don’t remain silent. Share your concerns on platforms like PubPeer or social media, reach out to journal editors and inform research compliance offices. This may end up being some of your most important work.