Wikipedia in the Newsroom, Part 2: Putting It to the Test

There's no unanimity about Wikipedia among academic experts, who have engaged in vigorous debates about the online encyclopedia. While many professors refuse to allow students to cite it, it has attracted some prominent defenders, including historians and scientists who have analyzed its content.

"If a journalist were to find something surprising on Wikipedia and the journalistic instincts suggested it was correct, the journalist might add that as an unsubstantiated Wiki-fact and invite comment," says Cathy Davidson, a professor at Duke University and cofounder of HASTAC (Humanities, Arts, Science, and Technology Advanced Collaboratory), a network of researchers developing new ways to collect and share information via technology. "Perhaps an online version of the printed piece, for example, might include a blog inviting people to comment on the Wiki-fact. It may be that there would be Wikifacts online that were not in the printed piece. In other words, why not use the new technologies available to expand knowledge in all kinds of ways?"

Changing Opinions

Journalists also should consider, Davidson says, whether some of the sources they deem reliable have their own inadequacies. For example, when she recently researched the origins of calculus, she found that standard Western histories generally credited England's Isaac Newton and Germany's Gottfried Wilhelm Leibniz.

But Wikipedia went much further, tracing the discovery of basic calculus functions back to the Egyptians in 1800 BC, and then to China, India and Mesopotamia -- hundreds of years before the Europeans.

So while journalists should be cautious no matter what resources they use, "What Wikipedia does reveal to those in the Euro-American world is knowledge which most of our sources, even the most scholarly, have, in the past, neglected because it did not fit in our intellectual genealogies, in our history of ideas," Davidson says.

Clinical Trial

In December 2005, the science journal Nature published a survey of several experts about the content of comparable Wikipedia and online Encyclopedia Britannica entries. In a conclusion hotly disputed by Britannica, Nature said that Wikipedia "comes close to Britannica in terms of the accuracy of its science entries," in that the average Wikipedia article contained four errors to Britannica's three. Britannica's 20-page response said that "almost everything about the journal's investigation ... was wrong and misleading ... the study was so poorly carried out and its findings so error-laden that it was completely without merit."

The company further asserted that Nature had misrepresented its own data -- its numbers, after all, showed that Wikipedia had a third more inaccuracies than Britannica and asked for "a full and public retraction of the article." Nature stood by its story.

The Nature piece profoundly undermined the authority upon which Britannica depends," says Gregory Crane, editor in chief of the Perseus Digital Library at Tufts University. He is a recent convert to the pro-Wikipedia camp, calling it "the most important intellectual phenomenon of the early 21st century."

Winning Converts

He recognizes its faults, especially when Wikipedians write about controversial topics. So "people have to do some critical thinking," Crane says, by evaluating their sources, "whether it's Wikipedia or The New York Times."

In an article he wrote in 2005, Crane acknowledged that Wikipedia "is an extreme case whose success so far has shocked skeptical scholars." But he noted as well that other, more mainstream reference works had similar foundations -- for example, the Oxford English Dictionary was written over a period of 70 years by thousands of people, including "an inmate at an asylum for the criminally insane."

A 2006 analysis by another scholar and Wikipedia fan, George Mason University historian Roy Rosenzweig, found some inaccuracies, omissions, uneven writing and even plagiarism in selected entries. But his comparison of several Wikipedia biographies against comparable entries in two other encyclopedias found that Wikipedia "roughly matches" Microsoft's (Nasdaq: MSFT) Encarta in accuracy while still falling short of the Oxford University Press' American National Biography Online. This general conclusion is supported by studies comparing Wikipedia to other major encyclopedias, wrote Rosenzweig, who was director of the university's Center for History and New Media until his death last year.

Holding Fast

Still, many if not most in the academic community think that Wikipedia, if used at all, should be no more than a secondary source, and they frequently tell their students as much. For Cornell University professor Ross Brann, that position was reinforced in early 2007, after the outing of a salaried Wikipedia employee and editor who called himself "Essjay" and claimed to be a tenured professor with doctorates in theology and canon law. Turns out he had seriously padded his résumé: The New Yorker discovered after interviewing Essjay that he was actually a 24-year-old community college dropout. To Brann, a professor of Judeo-Islamic studies and director of graduate studies for the Department of Near Eastern Studies, the incident confirmed that Wikipedia could not be trusted as a primary source.

"I just tell students, 'Do not use Wikipedia, do not cite it, do not go there for my classes.' We're trying to teach them how to use sources, how to evaluate different sources, and I think that in general, although obviously a wonderful resource, for a student who just uses a search engine and they use the first thing that pops up ... this undermines the kind of thing we're trying to teach them," Brann says.

Brann notes that Wikipedia's popularity probably has a lot to do with the fact that its entries so frequently pop up first, because that's the nature of search engines. "Many of them just work by the multiplicity of uses, others by virtue of ad arrangements -- somebody is deciding for you what you're going to look at," he says.

Next-Generation Journalists

And what about college journalists, a group that has never known life without computers? A news database search suggests that they are just as reluctant to cite Wikipedia as their professional colleagues. In August, for example, the University of Iowa newspaper, the Daily Iowan, used the WikiScanner database to determine that thousands of Wikipedia entries had been made or modified by people using the campus computer network. Some involved obvious but harmless enough vandalism: "Hawkeyes Rule" was inserted into text about the college's football stadium; less generously, a former university president was called an "eater of monkey brains," according to the paper's story.

Jason Brummond, editor in chief of the Daily Iowan, says he considers Wikipedia a good initial source, "but you go from there to find what most people would consider a more reputable source." Reporters in his newsroom generally understand that, he adds.

Brummond thinks the age of the journalist doesn't necessarily have that much to do with accepting Wikipedia: "It's more a personal awareness of how Wikipedia works."

In September, the University of Kansas student newspaper ran an editorial calling upon Wikipedia to do a better job of restoring "adulterated pages," noting that "despite a thousand recitations by our professors that Wikipedia is not a genuine source, students trust the site to give them accurate information." Nevertheless, Erick Schmidt, editor of the University Daily Kansan, says he doesn't rely much on Wikipedia, in part because his reporters write mostly about college and community issues. Plus, "we're taught to be cautious of things and skeptical," he says.

Schmidt rejects the notion that college students uncritically accept Wikipedia because they are infatuated with all things Internet . "We don't want to move things to technology because we think it's cool or paper is lame," he says. "But honestly, we are pressed for time, and if technology speeds things up... that's why we're being drawn to it."

For his part, Wales maintains that the more people use Wikipedia, the more they'll come to understand and accept it. His conclusion, he says, "comes from people who have used the site for a long time and know, 'I have to be careful' ... which is what good reporting is supposed to be about anyway."

But whatever the verdict on Wikipedia, one thing should not change, says The New York Times' Johnston: "No matter who your sources are, when you sign your name, you are responsible for every word, every thought, every concept."

Wikipedia in the Newsroom, Part 1: Slowly Gaining Acceptance