Wikipedia:External peer review

[[:|WP:EPR]]

This is a list of peer reviews conducted by external agencies. It aims to be both a record of the findings of these reviews and a way to highlight any problems (or indeed positive aspects) that were encountered.

Marking articles externally peer reviewed

A talk page template along the lines of {{external peer review}} has been proposed for marking articles that are the subject of external peer reviews – please comment on the discussion page.

The Guardian

Source: The Guardian
Date: October 24, 2005
Title: Can you trust Wikipedia?
URL: http://www.guardian.co.uk/g2/story/0,,1599116,00.html

Findings

Experts review 7 articles and rated them on a 0 to 10 scale. Controversially Robert McHenry (former Editor-in-Chief of Britannica) reviewed the encyclopedia article. The scores ranged from 0/10 (Haute couture) to 8/10 (Bob Dylan).

Responses

See Wikipedia talk:Response to the Guardian

Mail & Guardian

Source: Mail & Guardian
Date: 7 November, 2005
Title: Can you trust Wikipedia?
URL: http://www.mg.co.za/articlePage.aspx?articleid=255920&area=/insight/insight__national/
Signpost article: Wikipedia:Wikipedia Signpost/2005-11-14/Article evaluations

Findings

Reviewed 8 articles relating to South Africa and gave them scores ranging from 2/10 (Media in South Africa) to 10/10 (South Africa national rugby league team).

Responses

Many of the articles were quickly improved and the Mail & Guardian published a second article "Wikipedia springs into action after M&G Online article" which said, in part, "...most of the entries have been edited and improved."

Elf-friend rewrote the Media in South Africa article.
Inaccuracies were removed from African National Congress

Nature

Source: Nature
Date: 14 December 2005
Title: Internet encyclopaedias go head to head
URL: http://www.nature.com/news/2005/051212/full/438900a.html; List of articles reviewed, Nature blog

Nature compared Wikipedia and Britannica articles and sent them to experts in the field. The number of "factual errors, critical omissions and misleading statements" were recorded.

Findings

Of the 42 articles reviewed, 38 were found to have at least one error – Britannica had 40 articles with at least one error.

The following articles had the highest number of errors:

The following articles had no errors highlighted:

Responses

Dmitry Mendeleev has been tagged with {{accuracy}} by BrokenSegue - 20:48, 14 December 2005
- All of them are now tagged. —Steven G. Johnson 02:31, 15 December 2005 (UTC)[reply]
  - And in your zest, you tagged at least one article (Cambrian explosion) that had already been rewritten. Dragons flight 02:48, 15 December 2005 (UTC)[reply]
    - I know that it had been rewritten, but we have no way whatsoever to know whether the rewritten version corrects the alleged problems. Unless we get a specific list of the criticisms from Nature, of course, we may have no choice but to remove the tags after giving the article our best look. In the short term, however, it is best to acknowledge that we take seriously reviews by such a reputable publisher. Moreover, realize that a lot of people are going to be visiting the articles that are listed in Nature right now, and it will reflect well on Wikipedia if we have some admission of the potential problems. —Steven G. Johnson 03:05, 15 December 2005 (UTC)[reply]
      - Bleh, I don't like having the "disputed" template on an article when A) you don't know what's disputed and B) you don't know if it has already been fixed. Instead I created {{NatureDispute}} and put that on Cambrian explosion, which I think serves your purpose in a friendlier way. Use it on other articles if you wish. Dragons flight 03:41, 15 December 2005 (UTC)[reply]
        They are all retagged now with {{NatureDispute}}, which can itself be edited should anyone want to make a refinement. - Nunh-huh 04:57, 15 December 2005 (UTC)[reply]

Should we ask Nature to send us a copy of the errors found? If they have such details, that is. violet/riga (t) 21:57, 14 December 2005 (UTC)[reply]

I have emailed Nature and hope they respond by publishing the results. Unfortunately, I suspect they will only be available to paying subscribers to the magazine. — Ambush Commander^(Talk) 21:59, 14 December 2005 (UTC)[reply]

That's not a problem. I am a paying subscriber (well, through my university employer), as I'm sure are many other Wikipedians, and would be happy to pass along any errors that they publish. However, I doubt this is the case. I'm more worried that the review was done through the usual refereeing process, which is traditionally protective of the anonymity of the referees—they may be reluctant to simply publish the referee reports, although they might give us a summary. Or they might be persuaded to give them to a Wikipedia representative under some conditions of privacy. —Steven G. Johnson 02:30, 15 December 2005 (UTC)[reply]

Our four correct articles should be marked somehow. Could these be our first {{stable}} articles? ‣ᓛᖁ♀ᑐ 23:41, 14 December 2005 (UTC)[reply]

no errors != done (they could have poor grammar or could be incomplete) Broken S 23:44, 14 December 2005 (UTC)[reply]

Makes it easier though :-) Ta bu shi da yu 02:13, 15 December 2005 (UTC)[reply]

They counted "critical omissions", so at least what we have is fairly complete. All that should be left is checking the structure... should we see what Wikipedia:Peer review thinks? ‣ᓛᖁ♀ᑐ 03:36, 15 December 2005 (UTC)[reply]

Well, the articles that are accurate should be marked somehow once article validation is turned on, don't you think? Perhaps a "externally peer-reviewed" marker? And yes, send them to Peer review, these should be FAs soon. Tito xd^{(?!? - did you read this?)} 04:17, 15 December 2005 (UTC)[reply]

Errors/word comparisons

Please post below a table of errors/word statistics, based upon the Nature article and the word counts in the corresponding articles, so that we can see a more controlled comparison of error rates. —Steven G. Johnson 02:53, 15 December 2005 (UTC)[reply]

Britannica displays the word count for each article, doesn't it? At least that part shouldn't take long. ‣ᓛᖁ♀ᑐ 03:29, 15 December 2005 (UTC)[reply]

You'd think that not all words are created equal. — Ambush Commander^(Talk) 03:43, 15 December 2005 (UTC)[reply]

No doubt, but this seems the quickest reasonable statistic to gather. —Steven G. Johnson

Note that, for a fair comparison, we shouldn't include tables of contents, external links, "see also", or references — most Britannica articles do not include these, and the Nature review did not consider referencing quality.

Article name	Britannica			Wikipedia
Article name	Word count	Errors	Errors/word	Word count	Errors	Errors/word
Acheulean industry	?	1	?	420	Template:Bg-red \| 7	0.0167
Agent Orange	252	2	0.00793	1270	2	0.0015748
Aldol reaction	130	4	0.030769	660	3	0.0045455
Archimedes' principle	350	2	0.0057143	607	2	0.0032949
Australopithecus africanus	235	1	0.0042553	496	1	0.0020161
Bethe, Hans	658	1	0.0015198	1823	2	0.0010971
Cambrian explosion	519	Template:Bg-red \| 10	0.019268	702 (13 Dec.)	Template:Bg-red \| 11	0.0157
Cavity magnetron	394	2	0.0050761	1121	2	0.0017841
Chandrasekhar, Subrahmanyan	365	4	0.010959	417	Template:Bg-green \| 0	Template:Bg-green \| 0
CJD	591	2	0.0033841	1373	Template:Bg-red \| 5	0.0036417
Cloud	641	3	0.0046802	1689	Template:Bg-red \| 5	0.0029603
Colloid	561	3	0.0053476	896	Template:Bg-red \| 6	0.0066964
Dirac, Paul	837	Template:Bg-red \| 10	0.011947	1044	Template:Bg-red \| 9	0.0086207
Dolly	113	1	0.0088496	807	4	0.0049566
Epitaxy	178	Template:Bg-red \| 5	0.028090	235	2	0.0085106
Ethanol	315	3	0.0095238	2631	Template:Bg-red \| 5	0.0019004
Field effect transistor	588	3	0.0051020	933	3	0.00322
Haber process	241	1	0.0041494	531	2	0.0037665
Kinetic isotope effect	210	1	0.0047619	569	2	0.0035149
Kin selection	923	3	0.0032503	404	3	0.0074257
Lipid	349	3	0.0085960	676	Template:Bg-green \| 0	Template:Bg-green \| 0
Lomborg, Bjorn	518	1	0.0019305	1501	1	0.00066622
Lymphocyte	479	1	0.0020877	351	2	0.0056980
Mayr, Ernst	357	Template:Bg-green \| 0	Template:Bg-green \| 0	753	3	0.0039841
Meliaceae	?	1	?	?	3	?
Mendeleev, Dmitry	?	Template:Bg-red \| 8	?	?	Template:Bg-red \| 19	?
Mutation	?	Template:Bg-red \| 8	?	?	Template:Bg-red \| 6	?
Neural network	?	2	?	?	Template:Bg-red \| 7	?
Nobel prize	?	4	?	?	Template:Bg-red \| 5	?
Pheromone	?	3	?	?	2	?
Prion	?	3	?	?	Template:Bg-red \| 7	?
Punctuated equilibrium	?	1	?	?	Template:Bg-green \| 0	Template:Bg-green \| 0
Pythagoras' theorem	?	1	?	?	1	?
Quark	?	Template:Bg-red \| 5	?	?	Template:Bg-green \| 0	Template:Bg-green \| 0
Royal Greenwich Observatory	?	3	?	?	Template:Bg-red \| 5	?
Royal Society	?	Template:Bg-red \| 6	?	?	2	?
Synchrotron	?	2	?	?	2	?
Thyroid	?	4	?	?	Template:Bg-red \| 7	?
Vesalius, Andreas	?	2	?	?	4	?
West Nile Virus	?	1	?	?	Template:Bg-red \| 5	?
Wolfram, Stephen	?	2	?	?	2	?
Woodward, Robert Burns	?	Template:Bg-green \| 0	Template:Bg-green \| 0	?	3	?
Total		123			162