Dr Gene E Likens is a pretty important American scientist. He was one of a small group of researchers who first brought acid rain to public attention back in the 1970s, and since then has won a slew of accolades, including a National Medal of Science.
These days, though, Dr Likens is less concerned with science than he is with the anonymous editors who present science to the world on Wikipedia, the world’s biggest online encyclopedia. Since 2003, Likens has closely (you might even say obsessively) monitored the Wikipedia page for acid rain. He noticed that there were “near daily edits” on the page, which, he claims, sometimes amounted to “egregious errors” and “distortion” of scientific fact.
Now, along with co-author Adam M Wilson, Likens has released a paper which analyses a decade’s worth of edits to seven scientific Wikipedia pages. The paper claims to show that politically controversial topics like evolution and climate change are subject to far more changes than other, less hot-button issues. The global warming page, for example, was edited around twice a day on average by around 100 words, while a page on particle physics was edited around 0.2 times a day, by around 10 words.
The authors conclude that this “volatility” of the entries is dangerous and misleading, since it relies on other editors spotting the mistakes and changing them:
The high rate of change observed in these pages makes it difficult for experts to monitor accuracy and contribute time-consuming corrections, to the possible detriment of scientific accuracy. As our society turns to Wikipedia as a primary source of scientific information, it is vital we read it critically and with the understanding that the content is dynamic and vulnerable to vandalism and other shenanigans.
In the paper, the pair present a catalogue of the changes made to the acid rain page over the course of just a few days in November 2011, and their annoyance is palpable:
…an anonymous editor removed the introductory paragraph which defined acid rain and replaced it with a statement calling acid rain “a load of bullshit.” This change was quickly reverted, but the next day the paragraph was again deleted and replaced by “Acid rain is a popular term referring to the deposition of wet poo and cats.” Five minutes later this edit was reverted and repeated again, and then reverted again…
Another sentence was changed from “During the 1990s, research continued.” to “During the 1990s, research on elfs continued,” which remained for over seven hours. Later that day the sentence “AciD Rain [sic] killed bugs bunny” was briefly added. Fifteen minutes later the section title “Chemistry in cloud droplets” was changed to “Blowjobs.”
Yet while the insertion of elves and Bugs Bunny into your life’s work is no doubt irritating, the paper fails to prove that any more serious edits, like long-term substitution of facts for believeable falsehoods, is actually taking place on Wikipedia. One commenter on the journal article notes that “the authors seem to falsely correlate the number of changes to an article and the veracity of those changes” – the researchers assume that all changes are, by their nature, inaccurate and negative. But the pages with more changes are also areas of constant research. It’s understandable that as the field of climate change develops, its Wikipedia page would too.
Wikipedia allows anyone, regardless of their training, to submit pages and changes, but this is both a weakness and strength, as any change you make will be seen by thousands of other readers within a few days, and the site’s editing policy is structured to make sure that the “better” information, linked to a reliable source, should usually win out. The policy also notes that “libel, nonsense, hoaxes, and vandalism should be completely removed”, which should cover every edit to the acid rain page quoted in Likens and Wilson’s paper. Meanwhile, more contentious edits – like a bias in the article, or changes to major facts – result in the page being marked as “under discussion”, so it’s clear to readers that the content may not be totally reliable. Likens and Wilson’s focus is on the edits, not on the fact that within a day or two, all were removed.
The authors’ division of topics into “political” and “non political” is also worth examining: the pages with more changes are also viewed by a lot more people than those which are changed less often. Global warming, for example, has around 15,000 views and an average of 1.9 edits per day. Heliocentrism (the model which claims that planets orbit the sun) has around 1,000 views and 0.2 edits a day. This correlation isn’t examined in the paper, despite the fact it appeared in a peer-reviewed scientific journal. One gets the feeling that on Wikipedia – which is, in essence, an enormous peer-reviewed resource – other scientists or statisticians would have noticed this oversight and explored it further over time.
The Wikimedia Foundation (the nonprofit behind Wikipedia) has released a statement on the study on its blog. This is an excerpt:
It didn’t surprise us to learn that articles considered to be controversial are frequently edited. The nature of controversy, after all, is that it generates discussion and public attention. For example, while the properties of water (H2O) have been well established, the causes of the Arctic sea ice decline are the subject of ongoing scientific inquiry and political debate.
Unfortunately, the study also jumped to conclusions about what this means for Wikipedia’s reliability, overstating findings and inferring facts not in evidence. Much of the press about the study has repeated the assertion that controversial articles are also more likely to be inaccurate, despite a lack of strong supporting evidence: the study only references a handful of anecdotal examples of inaccuracies. Instead, the study simply seems to confirm that the articles chosen as controversial are, in fact, controversial and thus frequently edited. “
The post also notes that other studies have found a correlation between the number of edits and the quality of the article, which suggests that, contrary to Likens and Wilson’s findings, more edits lead to a more accurate piece.
As a result, while pages on Wikipedia might be less wholly accurate in the moment, they evolve towards accuracy in a way that even prestigious scientific studies cannot. Wikipedia’s editing policy notes that “even the best articles should never be considered complete”. Perhaps scientists should acknowledge the same about their own fields of study.
Content from our partners
Related
