Wikipedia:Perl Mediation/Archive

Intro

Hi. I'm the mediator taking Wikipedia:Mediation Cabal/Cases/2006-05-23 Perl. A little background on neutrality.

1. I've known been programming for about 26 years and professionally involved in the computer field (though not as a developer) for about 13 years.
2. I know Perl and have used it over the last 10 years for a variety of projects.
3. I've noticed Randal is participating on one issue. I should disclose that I did try and hire Randal's company about 7 years ago and was unsuccessful (he was focussed on training and I needed a custom module written). So we have had minor business dealings, OTOH he doesn't seem to be a major participant.
4. I know a wide variety of other languages and I'd say my favorite right now is Haskell.
5. By in large I favor Python for enterprise use and Perl for personal use

So I hope that is neutral enough. Now.... I started this page because the mediation page had turned into a mess very quickly, as is the talk page. I want to keep this page reasonably clean and to the point. So the first rule we have is no discussion of personal bias. Wikipedia editors are by and large motivated by passion to write, since they are unpaid. At the same time we attempt institutionally to be dispassionate the way we do this is via. WP:V, WP:NOR, WP:NPOV. I am perfectly OK with why a particular passage is biased I don't want to hear about how an editor is biased.

So now lets address 2 subtopics. Please jump in below jbolden1517^Talk 15:38, 27 May 2006 (UTC)

Sorry ... why should anyone care? This is one person, -Barry-, against consensus. This is a waste of time, and I implore everyone to refuse to participate. Pudge 22:06, 27 May 2006 (UTC)

That's fine. So I just indicate refused to participate and take you off the mediation effort? Remember the request was made by User:Revragnarok not Barry. jbolden1517^Talk 00:27, 28 May 2006 (UTC)

I'd prefer that we quickly demonstrate that there is consensus and that the edits were made with valid reasons and not vandalism as has been claimed. Steve p 23:05, 27 May 2006 (UTC)

That's better. If you can demonstrate that there is a consensus and that he refused to act in good faith I'll be happy to indicate that in the report and from there you could go for an rfc regarding Barry rather than regarding the issue. But that's going to take some time. This is a process. jbolden1517^Talk 00:27, 28 May 2006 (UTC)

The fall in popularity of Perl

This seems to be one of the key issues under debate. Basically has Perl fallen off in terms of popularity as measured by usage, webhits, book sales...? Now what I'd like to determine is (and you all should answer these questions):

1. Am I accurately describing the debate?
2. Now do the facts point to which of the following two statements: For example Cobol is obviously a language in tremendous decline but it is still heavily used, while Java is a language which is increasing usage much more slowly than it was 5 years ago.
   1. Perl's usage is falling off
   2. The increase in Perl's usage has fallen off
3. Does this deserve a paragraph a section or a subsection?

jbolden1517^Talk 15:38, 27 May 2006 (UTC)

I don't think you're accurately describing the debate. My issues are with verifiability, not the particular issue at hand. We have no way to measure the statement, and any single measure is certainly to leave out something. Additionally, different measures will show opposite conclusions. This clearly shows that we can't verify the conclusion. There are no facts to support any statement about Perl's popularity either way, so the submissions so far do not meet Wikipedia's standards. I don't think that the popularity of anything, Perl included, is encyclopedic. It certainly doesn't change the identity or characteristic of the subject. Should we add a section on popularity of the topic to every page and let people express loosely evidenced opinions on current books, music, political philosphies? Scarpia 17:19, 27 May 2006 (UTC)

OK fair enough. So let me try again. You are arguing that:

1. We don't have good numbers on Perl's popularity
2. Even if we did popularity figures they are not important because we don't generally consider such things?

Is that correct? jbolden1517^Talk 20:30, 27 May 2006 (UTC)

Correct, there is no verifiability, and whether Perl's popularity has fallen or not is not criticism of Perl. Even if Perl's popularity has fallen, using it as a reason for criticising Perl is a case of bandwagon fallicy. If there is a valid or verifiable criticism of Perl, it should stay. Criticisms based on specious reasoning are not appropriate, and a consensus of the editors have agreed. Steve p 21:42, 27 May 2006 (UTC)

I'm not sure you are correct here. Things like availability of hires, availability of libraries, 3rd party support... are all functions of popularity. Popularity is of a great deal of importance when choosing a language from a practical standpoint. Right now Perl has publicly available libraries collection second only to Java. On the other hand were usage to fall off by say 90% or so its utility would be diminished substantially. Another 90% from there and directly hiring for the skill sets becomes very difficult. That most certainly does effect language choice. If there genuinely was a consensus on the Perl page not to address issues of popularity that would be very curious since its popularity (and its effects i.e. CPAN and the "everyone knows Perl") more so than any other feature that makes Perl a "safer" choice than Ruby or Python. Purely on the merits its a much more complex choice which comes down to subtle opinions.

So again I'm not here to overturn a consensus but I'd fine it very odd. Perl is one of the most well known languages and is used freely in the way that C++, Java, Cobol, are used not the way Ruby, Lisp and Ada are used. The bandwagon fallacy doesn't apply when an increase in popularity has meaningful effect. Quite simply in choosing a programming in a professional environment popularity matters. How much is an area of debate jbolden1517^Talk 01:13, 28 May 2006 (UTC)

I think you missed my point. My point is not that popularity doesn't matter when selecting a language. Popularity, however, is not a valid criticism of a language. For example, "Language A is more popular than language B; therefore, language A is better". The popularity arguments have nothing to do with the criticism of Perl, the language. My personal opinion is that the RFCs for Perl 6 are a much more valid criticism of Perl. Those are the real gripes that Perl programmers have had with the language. Steve p 02:01, 28 May 2006 (UTC)

We don't have a way to measure popularity. The Tiobe results, besides being a flawed methodology, only measure what people talk about. Out of all the people that do anything, no matter what it is, very few of them talk about it on the net. It's not been my personal experience that popularity matters when choosing a programming language. People care about it getting the job done. Indeed, that's how Perl, Ruby, etc, got to be where they are. They didn't spring out of nothingness into instant popularity.

This is the page I'm getting the popularity ratings from.

OK that's a piece of data. Its being questioned. Do you have any other independent data which supports the relative positions? Also you didn't answer my questions above, could you answer those? jbolden1517^Talk 01:13, 28 May 2006 (UTC)

[defense of Tiobe numbers snipped] Barry, before we get into either attack or defending those numbers I want to address the more general questions. Popularity of programming languages is not an obscure topic we have a variety of ways of measuring it. Right now the question is about whether popularity figures should be used at all. jbolden1517^Talk 01:13, 28 May 2006 (UTC)

Randal L. Schwartz, co-author of several Perl books published by O'Reilly, has said that OSCON's organizers are openly hostile to Perl, and that Perl isn't interesting to O'Reilly anymore.

Another piece of evidence. But do remember this is an off the record comment. Its substantially weaker than an official statement but still definitely something to show falling interest. jbolden1517^Talk 01:13, 28 May 2006 (UTC)

In the article Comparison of programming languages, at this time, the Tiobe data is there (last two columns of the top chart) under the headings Serp Rank and Serp Rank Change. I'd like it to remain there. (I'll post in the Benchmarking section later)-Barry- 00:14, 28 May 2006 (UTC)

Those seem like reasonable facts for that chart. I'll see if other's object. jbolden1517^Talk 01:13, 28 May 2006 (UTC)

Ok, in response to your three questions:

1. I don't think the popularity part of the debate is about whether Perl usage, webhits, or book sales have declined. I think the debate is about whether enough people consider the popularity change of Perl a "con" to justify mentioning it in the Con subsection of the Opinion section. But maybe some others think that there should be good evidence to support a decline before putting it in the Con section. Since the Con section is a subsection of the Opinion section, I think we need to determine whether certain opinions belong there by trying to determine whether they're common enough opinions. But I'd have no objection to including some relevant facts as well.

2. I basically presented all I know about whether there's an actual decline in Perl's popularity in my first post in this section. It's based on the Tiobe chart and Randal's comments. I've seen various other comments supporting it, but I don't have references.

3. I think the popularity issue belongs in the Con subsection.

-Barry- 02:47, 28 May 2006 (UTC)

Would you have any problem with presenting those two things in a pro section for Perl. I.E. one of the things that's unique about Perl (relative to other dynamic languages) is its popularity. CPAN is gigantic (on par Java's library), many many programmers know Perl, .... You could write a paragraph about how popular Perl is and then mention that there is anecdotal evidence that its popularity is declining. Give your two points in the reference section right after. Does that sound reasonable? jbolden1517^Talk 03:03, 28 May 2006 (UTC)

I'm not conviced of using popularity as a Pro or Con, but if this can put an end to the issues on the Perl page, then I'll go along with it. Steve p 03:21, 28 May 2006 (UTC)

No problem at all. Tiobe's data shows Perl to be more popular than Python and a bunch of other languages. It's one of the reasons I chose to learn Perl. I saw so many free Perl scripts around the web that I ignored the part in Learning Perl about Perl not being the right language if you'll be using it less often than (forgot how often). (Hmmm...something else for the con section). And yes, CPAN is huge. Not sure that's about popularity, but maybe you can link it to popularity somehow. If it doesn't logically fit in a certain section, that's not as important to me as making sure nothing is left out. -Barry- 03:37, 28 May 2006 (UTC)

Excellent. We have a compromise then on the popularity issue. So go ahead and do that paragraph. jbolden1517^Talk 03:39, 28 May 2006 (UTC)

Before I propose some wording for the pro section, I want to point out that Steve p said he'll go along with this compromise if it puts an end to the issues on the Perl page. It won't. I intend to bring several other issues to mediation, myself this time, or to arbitration of there's refusal to mediate. -Barry- 05:09, 29 May 2006 (UTC)

Barry you are a long way from arbitration. The arb committee doesn't take content disputes they take questions of policy and disciplinary matters; and POV pushing is not disciplinary matter. They rely on the dispute resolution process, what is happening here and further steps, for questions of content. It would only becomes an arb committee issue if:for example, I determine that somebody is not acting in good faith and am unable to change their attitudes, this judgement i upheld by an administrator, its then upheld by the rfc process and mild sanction is ineffective. Or alternately it can become an arb committee issue if I were to determine this issue is too complicated and kicked this over to the mediation committee and they made a determination of bad faith issue sanction and those sanctions are ignored. If you want to add issues go ahead and add one of them to the bottom of this page in a new section and we'll get started on it. In the meanwhile I want to wrap up this issue. So please write the paragraph discussed above. jbolden1517^Talk 11:15, 29 May 2006 (UTC)

You suggested mentioning that one of the unique things about Perl relative to other dynamic languages is its popularity, but it looks like PHP is a more popular dynamic language. It's not as powerful as Perl though, so maybe I could qualify the statement somehow. I don't like the sound of "full featured language." Not sure. Then there's Visual Basic, which is also more popular than Perl, but there's room for qualification there too because it's a static/dynamic hybrid. I don't know if that's a meaningful distinction from dynamic though. I tried looking up these terms, but it has to do with variable types and I don't know what they are either.

I wouldn't have a problem praising Perl's popularity without being specific, and I can add the large library praise too, but I think all I have in me is one sentence for each. Maybe a third if I mention Tiobe's anecdotal evidence of Perl's popularity in a seperate sentence. I'll see what I could come up with if nobody else suggests anything. -Barry- 16:47, 29 May 2006 (UTC)

Ok, there was already stuff about Perl being popular and about CPAN, but I added what I could: http://en.wikipedia.org/w/index.php?title=Perl&diff=55820166&oldid=55748002 -Barry- 23:06, 29 May 2006 (UTC)

So, Could I add the following now (between the lines)?

--------------

There's also criticism of a less technical nature that may be no less important to some. Some people believe that Perl's popularity has declined. As of May, 2006, the [http://www.tiobe.com/tpci.htm TCPI Long Term Trends chart] of the ten most popular programming languages, based on the results of search engine queries, shows that Perl's popularity is at its lowest since before June, 2001 (the earliest date plotted), and has dropped more than any other language over the past year.

There are also signs of a decline in Perl's popularity from book publisher O'Reilly. OSCON — the open source convention sponsored by O'Reilly — is much less Perl-oriented than it used to be. [[Randal L. Schwartz]], co-author of several Perl books published by O'Reilly, has said that OSCON's organizers are openly hostile to Perl, and that Perl isn't interesting to O'Reilly anymore.<ref>Paraphrased from merlyn's posts (Randall Schwartz's IRC handle) on [[Freenode]]'s #perl6 irc channel, logged [http://colabti.de/irclogger/irclogger_log/perl6?date=2006-05-10,Wed&sel=86#l151 here] and [http://colabti.de/irclogger/irclogger_log/perl6?date=2006-05-10,Wed&sel=93#l161 here].</ref>

--------------

-Barry- 23:13, 29 May 2006 (UTC)

No too much detail and too weasel words. My first impression is to go for this

''There is evidence that Perl's popularity has declined to its lowest levels since June, 2001 <ref> long terms trend chart [http://www.tiobe.com/tpci.htm] </ref> <ref>[Randal L. Schwartz]], co-author of several Perl books published by O'Reilly, has said that OSCON's organizers are openly hostile to Perl, and that Perl isn't interesting to O'Reilly anymore [http://colabti.de/irclogger/irclogger_log/perl6?date=2006-05-10,Wed&sel=86#l151] [http://colabti.de/irclogger/irclogger_log/perl6?date=2006-05-10,Wed&sel=93#l161]''

See the difference. One sentence not 2 paragraphs with the same information handled in footnotes. jbolden1517^Talk 00:28, 30 May 2006 (UTC)

I don't think the popularity stuff belongs in the Perl#Opinion section, either Perl#Pro or Perl#Con. As discussed in Talk:Perl#Opinion is not advocacy, the purpose of the Opinion section is to present and summarize opinions that are held of Perl. Data indicating the prevalance of various opinions isn't really to the point. Compare the American politics page, which has separate sections titled American politics#Political culture (outlining the political views of Americans) and American politics#Political parties and elections (tabulating recent election results).

If we moved Popularity data to its own section, then we could start to analyze and document it in a orderly fashion. Questions include

- how many people know Perl?

- how many people like Perl?

- how many people use Perl?

- how many people are paid to use Perl?

- how many companies use Perl?

- how many Perl programs are there?

- how many Perl programs are in production (i.e. run regularly)?

- how much programming is done in Perl?

-- programs/year?

-- KLOC/year?

- what is the availability of Perl books/tutorials/documentation/training resources?

- what is the availability of Perl interpreters?

- what is the availability of Perl code libraries?

- what is the availability of Perl development environments?

- what is the availability of Perl programmers?

- what is the availability of Perl consultants?

Swmcd 01:42, 30 May 2006 (UTC)

I think that's great. If you have access to good cites for any reasonable subset of that data that's far better then a line or two. Yes if we know where to get that data then go for it. jbolden1517^Talk 01:59, 30 May 2006 (UTC)

Indicating the prevalence of opinion is related to opinion and belongs in the opinion section. It's not just a statistic. It helps people determine what language considered good by others (and which they should learn) and it tells people how hard it would be to find programmers if a project is begun in a certain language.

We don't have all of the information you mention ready to add to the article, so a separate section isn't a real option now, but I agree that you should go for it if you write all that up. For now, we just have some Tiobe statistics. If we did have all of what you mentioned written up, then I think we'd still need to put some of that in the pro or con section, and I might want it to be repeated in a separate section too.

I think it should be pointed out in the Pro section that when a language is popular, it's easier to find good programmers when you need them. Using your strict interpretation of "opinion," you'd probably word it as "some employers prefer popular languages because it makes it easier to find programmers" and then you might even want me to show you some statistics to prove it. We shouldn't worry much about either the wording or the proof in this case. If it could be worded like a fact and it is one, then it wouldn't bother me to word it like a fact rather than like some people's opinion. I wouldn't consider that advocacy. And if it is a fact, we can assume it's some people's opinion without a Gallup poll. -Barry- 02:44, 30 May 2006 (UTC)

Benchmarking

Benchmarking any language is very complicated. However discussions about problems with benchmarking that are not specific to Perl belong in Benchmark (computing) not in a Perl article. Obviously we need to include some benchmarks for Perl and some discussion of how to benchmark Perl. Perl has been optimized far more than most "interpreted" languages, it has been studied on this issue for decade or more. So what I want is a strategy for addressing this issue. I'd like the participants in the benchmarking debate to propose one. jbolden1517^Talk 15:38, 27 May 2006 (UTC)

The Wikipedia article on a programming language is not its documentation. We don't need to explain how to do anything. Wikipedia is not a HOWTO. I don't see that its obvious that we need to include benchmarking information, or what purpose it would serve. We can include discussion about design trade-offs that gives insight into the architecture and inner-workings of the language, but simply throwing numbers around does not inform the reader. Scarpia 17:24, 27 May 2006 (UTC)

Again. Let me just make sure I understand. You are arguing that knowing how fast a language is at various tasks is not important to be people considering / evaluating a language? jbolden1517^Talk 20:32, 27 May 2006 (UTC)

A language itself doesn't do anything at any speed, just as the english language doesn't tell a story or make a poem. People can write programs to do tasks, and we can time those programs. That's only timing the programs that somebody wrote, and the speed only measures that particular implementation. Another programmer (e.g. one with more skill) may write a program in the same language to do the same thing faster (and I've been on both sides of that many times, as have most practicing programmers probably). The execution speed, even if we could measure it, is only a tiny portion of the utility and reason to use a language, and to focus on it gives it undue importance. If everyone wanted really fast programs (i.e. cared about execution speed), they'd be programming in machine language. Since there are concerns more important to them (time to market, how much time they have, return on investment, total programmer time, portability, and so on), elevating a single concern to stand above all others adds bias, even if the numbers favor Perl.

It's not of particular importance to Perl specifically, but may be appropriate in an article about dynamic languages in general. I addressed some of these concerns in my last edits to the "Comparative Performance" section by relating the claims to the design of Perl and linking to two significant discussions of the impact of Perl's implementation to its performance.

Knowing how to benchmark something is certainly interesting to users, but as I said before, Wikipedia is not a HOWTO. Scarpia 22:12, 27 May 2006 (UTC)

Benchmarking is an appropriate thing to consider when choosing a language. It is not appropriate, however, to provide benchmark results as they may not be at all relevant as a criticism to a programming language in all cases. There are too many variables to consider (differences in CPU, memory, file systems, bus speed, etc.) that make a single set of benchmark results unusable when not comparing identical system. Benchmark results involving Perl and other interpreted languages can also be slanted through differences in C compilers used to compile the interpreter, optimization levels, and other compiler options. Using benchmark comparisons between languages is also often a slanted argument. I'm sure most programmers will concede that Assembly Language is faster in almost every task than any other language. However, this would certainly be specious reasoning if you are using this alone to select a programming language or using it as a criticism against another language. I am highly suspect of adding benchmark information for any computer language as it can provide an editor a way to add a positive or negative point of view.

As for how to benchmark Perl, its not to much different than any other statistical experiment. Set up a base case with a measured level of expected error. Experiment with and measure an opposing case. If you cannot prove the opposing case is statically significant from the base, you cannot say that the opposing case is any different from the base. Knowing how to benchmark has little to do with Perl. It requires a basic understanding of statistical inference and an ability to be dispassionate about the results. Steve p 22:52, 27 May 2006 (UTC)

There's currently benchmark data for Perl here, and I think it would fit nicely in the Perl article. It's a fairly compact chart (three rows of data with no sideways scrolling on a 1024 x 768 screen). It compares Perl to six popular languages. About 16 tests are used for each value of speed, memory, and program size shown in the chart. Actually, 32 tests because two counts of tests won are included in each table cell, one for the Debian OS and one for Gentoo, separated by a slash. I think it's small enough to show here:

The following data comes from Debian : AMD™ Sempron™ benchmarks from from May 7, 2006 and Gentoo : Intel® Pentium® 4 benchmarks from May 10, 2006. The Debian and Gentoo tests used equivalent benchmarks, but on Gentoo, some benchmarks had a higher workload, most language implementations were built from source, and Size tests measured GZip bytes instead of lines of code.

Number of tests won (Debian : AMD™ Sempron™ / Gentoo : Intel® Pentium® 4)

UNIQ269d0b953609861e-HTMLCommentStrip21d3b4836f2a6cb00000001

Speed

Memory

Size

Perl	C (gcc)
1/1	12/15
0/1	13/15
11/14	2/2

Perl	C++ (g++)
0/2	14/12
0/0	14/14
10/14	4/0

Perl	Java JDK Server
3/3	13/13
12/12	4/4
13/16	2/0

Perl	PHP
9/8	4/6
10/10	3/5
10/11	3/4

Perl	Python
5/7	11/9
8/8	8/8
6/3	9/13

Perl	Ruby
14/14	2/2
10/9	6/7
8/2	6/14

Include all the disclaimers you want, but include the chart because it's meaningful. It's not available in such an easy to use form anywhere else. I created it from data taken from the source (mentioned above) especially for Perl comparisons. -Barry- 03:19, 28 May 2006 (UTC)

____

We are getting really off track

1. Scapia, benchmarks are as tandard features of language evaluation, how speed is measured. This is not some weird topic. If you want to argue why benchmarks shouldn't be used pick a different article (like benchmarks). Your opinion is well into the minority (and for good reason IMHO). I have no objection to a link like [1] next to the benchmark paragraph. I do have objections to pretending there are no speed differences between languages. People use languages like C specifically to get performance.

I'm not saying it's a weird topic. I'm saying that you don't benchmark languages. You benchmark programs. I'm not saying that there are no speed differences between languages, but I do know that people can easily write a slow programs or fast programs in any language. The Shootout link explicitly says this on it's first page. You can't use their data and throw away their disclaimer. Scarpia 04:41, 29 May 2006 (UTC)

1. Benchmarks are more relevant to Perl than other dynamic languages since Perl has been optimized for performance, particularly in areas like Regexes
2. There is no evidence that Barry is acting in bad faith in including benchmarks
3. Barry - The particular chart you are choosing is not very good. Your source includes good relative comparisons win lose #s are nowhere near as good.
4. I think summary data is more important than that chart. We want information not data. So I would want a paragraph like "+/- 50% Perl performs on par with PHP on CGI. It on generally slower than Java for math computation by a factor of 2-4, though faster than other dynamic languages not specifically designed for math by abut 30%. In terms of Regex it outperforms or equals just about any engine out there. In terms of ..." or whatever. Obviously that's harder to get agreement on but that's information not data. We can link to the data. So no raw facts without interpretation

jbolden1517^Talk 03:59, 28 May 2006 (UTC)

Unfortunately, the results from those benchmarks are not verifiable. Interpreted languages are run by an interpreter that typically is a C program. Like all C programs, their performance can be greatly influenced by the optimizer flags used at compile time. On my 2.0 Ghz Celeron, there are significant differences in performance depending on the flags passed to Perl's Configure script prior to compiling. For my test, I ran the nsieve program. Since it appears that they run the tests once without averaging, I'm doing that as well.

1. Flags: -des -Doptimize="-g" Runtime: 0m36.757s
2. Flags: -des Runtime: 0m30.629s
3. Flags: -des -Doptimize="-O3" Runtime: 0m29.426s
4. Flags: -des -Doptimize="-O3 -ffast-math -funroll-loops" Runtime: 0m28.116s
5. Flags: -des -Dcc=/opt/intel/cc/9.0/bin/icc -Aoptimize="-xW" Runtime: 0m27.639s

Better still, Randal Schwartz wrote an article and provides an optimized version of the nsieve, which I adapted to provide additional runtime improvements.

1. Flags: -des -Dcc=/opt/intel/cc/9.0/bin/icc -Aoptimize="-xW" Runtime: 0m18.756s

So, unfortunately, without the information on the optimization flags for the various interpreted languages, the results are not verifiable or repeatible. In addition, there is no way to tell if the results are slanted in favor of one language over another either accidentially or not. As it appears that the gcc C programs are being compiled with various optimization flags, this may be the case. This threatens to add a POV to the article, and, as such, I am very much against any of it being added. Steve p 07:04, 28 May 2006 (UTC)

Steve that is great data. And Perl specific. Put it somewhere else on the web and lets cite it. You may want to mention which of the above is the default i.e. the one likely to be encountered in practice). I'd also do things like average....

Now in terms of your argument, you are talking worst case about a factor of 2. Excluding the extreme cases you are talking a factor of 10%. Some of the differences in terms of comparison between languages are much larger than 2 and 10% is well within the ranges to make meaningful comparisons. I'm sure that's true of C programs as well, which means we can benchmark Perl modulo the problems we have benchmarking any program. Again I'm the mediator. I can't really defend benchmarks, since I'm not here to argue. You have to show something that is specifically true of benchmarking Perl scripts relative to scripts in other languages that's not true of any other pair otherwise you are arguing against benchmarks. jbolden1517^Talk 12:36, 28 May 2006 (UTC)

I probably won't put them on the web or cite it on the Perl page. Per your request, I reran the nsieve benchmark with varying Configure flags. The benchmark program was run ten times and an average runtime and standard deviation were calculated.

1. -des -Doptimize="-g" - Average 38.7713s Std Dev. 2.48432222145196
2. -des - Average 29.5436s Std. Dev. 2.15090137177675
3. -des -Doptimize="-O3" - Average 28.2735s Std. Dev. 1.40536236924463
4. -des -Doptimize="-O3 -fomit-frame-pointer -ffast-math -funroll-loops" - Average 26.6352 Std. Dev. 0.112951120204961

The follow conclusions can be made with a 95% degree of certainty:

• Using -g as the optimize flag does not produce an outling result. Its average result could not have been produced by any of the other perl's compiled for this comparison.
• Using -O3 did not produce significantly different results from using -O2 (the default). Using -O3, however, did produce more consistant results.
• It cannot be ruled out that a perl compiled with the optimize flag with -O2 or -O3 can produce as good of results as compiling with "-O3 -fomit-frame-pointer -ffast-math -funroll-loops". However, it can also be concluded that a perl compiled with "-O3 -fomit-frame-pointer -ffast-math -funroll-loops" will produce better results than the average results of a perl compiled with -O2 or -O3.
• The above show that differing compiler optimization flags do have significant impact on the performance of the compiled perl executable.

Now, as I've demonstrated, the benchmark results that User:-Barry- wants to link to are fundementally flawed. Without providing average results, standard deviations, or optimization flags used to compile the various interpreted languages, there is no way to verify the results or make meaningful comparisons. As a consensus of the editors have agreed, these results do not belong on the Perl page, and I would question their inclusion elsewhere. Steve p 21:00, 28 May 2006 (UTC)

Oh, you wanted to know the flags used to compile the interpreter. I don't know, but considering the number of different tests performed and the community and bug report system that's set up for the shootout site, and the site's popularity and lack of any other data remotely comparable, their results should definitely be linked to. I think my overall win/lose count is good too, but I won't fight over my particular chart much. If it's not used on Wikipedia, that will mean there won't be a duplicate content penalty when I post it elsewhere.

With the community for that shootout site, there's much more peer review than we could get from the behind-the-scenes original research you're doing, but if you have valid concerns, you can include them in the article. The Perl article simply can't ignore a such a major source of benchmark data that's accessible with a click. And it's linked to in four other articles.

Steve p: Also, when someone asks for advice on what language to use for a program that needs to be run fast, would you suggest that nobody answer unless they have all the details about the not-yet-written program and how the interpreter was compiled, etc? At least you wouldn't be wrong if you didn't answer, but the person with the question would just go to a different forum and get a much less accurate answer than the shootout site provides, and it would probably be a useful answer.-Barry- 01:34, 29 May 2006 (UTC)

I would tell them they have more important things to worry about when selecting a language. Also, peer-review is irrelevant if what is allowed on the website can be controlled by a small group of individuals. The value of their peer-review is dubious when their C programs fail to compile. Finally, yes, there are four other languages mention the shootout. The Clean editors agree with the consensus of the Perl editors, when they wrote "Clean performs pretty well in the Computer Language Shootout Benchmarks - though the value of such benchmarks can be debated." Obviously, this is not a minority position. Steve p 15:21, 29 May 2006 (UTC)

It's much more a question of accuracy than POV, but "You can see the build commands and runtime commands on each program page in the build & benchmark results section." For example, the build commands for ackermann.gcc-3.gcc are here. -Barry- 11:45, 28 May 2006 (UTC)

Without being verifiable, there is no way say whether the benchmark is neutral or not. Steve p 14:36, 28 May 2006 (UTC)

Steve please feel free to cite an alternate source for benchmarks. Without a cite your criticisms fail WP:V#Verifiability.2C_not_truth. Most programmers believe in benchmarks, the shootout results are fairly representative. Your comments are very good research and if you post them to another cite I'll agree to link to them (as I've said) but I'm ruling that Barry is acting within WP:V with the concepts of benchmarks. The specific he wants is terrible but he's already agreed to write the summary paragraph. I know you believe the benchmarks are bad, but that would need to be the overwhelming position not a minority position for you to win this. An easier argument is to find a better site than shootout. Do you have one? jbolden1517^Talk 02:26, 29 May 2006 (UTC)

Without a seconday cite, the shootout benchmarks would seem to fail WP:V#Self-published sources. Also, I don't see how it is a minority position when the creators of the shootout agree with what Scarpia said above. The shootout has entertainment value, but it is of limited value determining whether a specific language should be chosen over another. That is why the majority of editors and even yourself agreed what had been added was not appropriate for the Perl entry. Also, can you please explain why the Perl entry should be the how to benchmarking? Steve p 15:21, 29 May 2006 (UTC)

A self published source would be something like the Larry Wall putting out a benchmark. Shootout is 3rd party information. As for Perl and the how of benchmarking I think I was unclear since both you and Scarpia misunderstood me. In brief:

• Information about benchmarking in general or problems with benchmarking in general belong in the benchmarking article
• Information about benchmarking Perl that does not apply to other languages can be in the Perl article (or a separate "benchmarking Perl" article)
• Information about benchmarking dynamic interpreted languages can be in the benchmarking article (or a separate article)
• Specific details for Perl regarding information about benchmarking dynamic interpreted languages can be in the Perl article or a separate article.

And following the same chain of reasoning:

• Any argument against Perl benchmarks that would apply to benchmarks of any dynamic interpreted language are invalid. This is the wrong level to deal with them. Even worse if the arguments apply to benchmarking of languages or benchmarking in general.

As for what was added (the chart) Barry has agreed not to include it. That part of mediation is done. I'm offering Barry or anyone else who wants to include benchmarking information the opportunity to include useful information. That failing nothing goes in. jbolden1517^Talk 00:14, 30 May 2006 (UTC)</ref>