The “phantom reference:” How a made-up article got almost 400 citations

By Victoria Stern

Here’s a mystery: How did a nonexistent paper rack up hundreds of citations?

Pieter Kroonenberg, an emeritus professor of statistics at Leiden University in The Netherlands, was puzzled when he tried to locate a paper about academic writing and discovered the article didn’t exist. In fact, the journal—Journal of Science Communications—also didn’t exist.

Perhaps Kroonenberg’s most bizarre discovery was that this made-up paper, “The art of writing a scientific article,” had somehow been cited almost 400 times, according to Clarivate Analytics’ Web of Science.

Anne-Wil Harzing, a professor of International Management at at Middlesex University in London, who recounted Kroonenberg’s discovery in her blog, wrote:

To cut a long story short, the article appeared to be completely made up and did not in fact exist. It was a “phantom reference” that had been created merely to illustrate Elsevier’s desired reference format.

Here’s the reference from Elsevier’s reference style section, part of its author guidelines (we’ve seen examples that cite the paper as from 2000 as well):

Van der Geer, J., Hanraads, J.A.J., Lupton, R.A., 2010. The art of writing a scientific article. J Sci. Commun. 163 (2) 51-59.

Puzzled, Harzing set out to understand how so many authors could cite this paper.

Harzing found that nearly 90% of the citations were for conference proceedings papers, and nearly two-thirds of these appeared in Procedia conference volumes, which are published by Elsevier.

When examining some of the papers more closely, Harzing found “most citations to the phantom reference occurred in fairly low-quality conference papers,” and were written by authors with poor English. She said she suspects that some authors may not have understood that they were supposed to replace the template text with their own or may have mistakenly left in the Van der Geer reference while using the template to write their paper. There may be minimal quality control for these conference papers, says Harzing; still, she found that the phantom reference did appear in about 40 papers from established journals.

Harzing concluded that the mystery of the phantom reference “ultimately had a very simple explanation: sloppy writing and sloppy quality control.”

We contacted several researchers who cited the phantom reference; all attributed it to some kind of mistake. One said he believes two similar references were somehow confused, and the “Van de Geer” replaced the correct one; another author said he has contacted the publisher to fix the error.

Although 400 citations sounds significant, Harzing put the number in context: Out of nearly 85,000 Procedia conference papers, the phantom reference appeared in less than 0.5% of articles:

Whilst unfortunate, one might consider this to be an acceptable ‘margin of error’.

She added:

In a way we can be glad that our phantom reference IS a phantom reference. If this had been an existing publication, the mistakes might have had far more serious consequences.

The take-home, Harzing says, is “If something looks fishy, it probably IS fishy!”

Update, 1500 UTC, 11/15/17: The researcher who requested the correction told us the publisher has agreed.

Hat tip: Research Whisperer

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here. If you have comments or feedback, you can reach us at