DMOZ: Difference between revisions
No edit summary |
Informatox (talk | contribs) Undid revision 1285077100 by Internet4everywhere (talk) Tags: Undo Mobile edit Mobile web edit Advanced mobile edit |
||
Line 1: | Line 1: | ||
{{Short description|Open content directory of Web links}} |
|||
The '''Open Directory Project''' ('''ODP'''), also known as '''DMoz''' (for Directory.Mozilla), is a multilingual [[open content]] [[Web directory|directory]] of [[World Wide Web]] links run by [[Netscape Corporation]] (owned by [[AOL Time Warner]]) that is constructed and maintained by a [[Virtual community|community]] of [[volunteer]] editors. |
|||
{{Multiple issues| |
|||
{{Lead extra info|date=June 2019|reason=e.g., Curlie is mentioned here, but no where else in the article}} |
|||
{{More citations needed|date=July 2023}} |
|||
{{Update|date=February 2025|reason=DMOZ was wholly replaced by Curlie and according to our article, they have been treated as synonymous since 2018, so this article should really be at [[Curlie.org]] and be about it, and its prior history as the former DMOZ. Either that, or we need a separate Curlie.org article}} |
|||
}} |
|||
{{Use mdy dates|date=October 2012}} |
|||
{{Infobox website |
|||
| name = DMOZ |
|||
| former_name = {{ubl|GnuHoo (1998)|NewHoo (1998)|Open Directory Project (1998–1999)}} |
|||
| logo = DMOZ logo.svg |
|||
| logo_caption = |
|||
| logo_alt = "dmoz" in white on a green background with each letter in a separate square |
|||
| screenshot = Dmoz - Open Directory Project.PNG |
|||
| caption = |
|||
| url = <nowiki>www.dmoz.org</nowiki> ({{webarchive| url = https://web.archive.org/web/20180119172207/http://www.dmoz.org/ }}) <!-- { {URL|http://dmoz.org/} } Forbidden, access denied! --> |
|||
| commercial = No |
|||
| collapsible = |
|||
| collapsetext = |
|||
| type = [[Web directory]] |
|||
| registration = Optional |
|||
| language = 90 languages, including English |
|||
| num_users = 90,000 |
|||
| parent = [[AOL]] |
|||
| launch_date = {{Start date and age|1998|6|5}} |
|||
| current_status = Closed |
|||
| content_license = [[Creative Commons Attribution|Creative Commons Attribution 3.0 Unported]], Open Directory License |
|||
| dissolved = {{end date and age|2017|3|17}} |
|||
| successor = Curlie.org |
|||
}} |
|||
'''DMOZ''' or '''DMoz''' (stylized '''dmoz''' in its logo; from ''directory.mozilla.org'', an earlier [[domain name]]) was a multilingual [[open content|open-content]] directory of [[World Wide Web]] links. The site and community who maintained it were also known as the '''Open Directory Project''' ('''ODP'''). It was owned by [[AOL]] (now a part of [[Verizon Media|Yahoo! Inc]]) but constructed and maintained by a [[virtual community|community]] of volunteer editors. |
|||
== History == |
|||
DMOZ used a hierarchical [[Ontology (information science)|ontology]] scheme for organizing site listings. Listings on a similar topic were grouped into categories which then included smaller categories. |
|||
=== Motivation and Founders === |
|||
ODP was founded as '''Gnuhoo''' by [[Rich Skrenta]] and [[Bob Truel]] in [[1998]]. At the time, Skrenta and Truel were working as engineers for [[Sun Microsystems]]. [[Chris Tolles]], who worked at Sun Microsystems as the head of marketing for network security products, also signed on in 1998 as a co-founder of Gnuhoo along with co-founders [[Bryn Dole]] and [[Jeremy Wenokur]]. Skrenta was already well known for his role in developing [[TASS]], an ancestor of [[TIN]], the popular threaded [[Usenet]] newsreader for [[Unix]] systems. Coincidentally, the original category structure of the Gnuhoo directory was based loosely on the structure of Usenet newsgroups then in existence. |
|||
DMOZ closed on March 17, 2017, because AOL no longer wished to support the project.<ref>{{Cite news |last=Sullivan |first=Danny |date=March 17, 2017 |title=DMOZ has officially closed after nearly 19 years of humans trying to organize the web |url=http://searchengineland.com/dmoz-has-officially-closed-271530 |url-status=live |archive-url=https://web.archive.org/web/20170717103526/http://searchengineland.com/dmoz-has-officially-closed-271530 |archive-date=July 17, 2017 |access-date=July 17, 2017 |work=[[Search Engine Land]]}}</ref><ref name="resource-zone">{{Cite news |date=April 16, 2017 |title=Why Dmoz Was Closed ? |url=https://www.resource-zone.com/forum/t/why-dmoz-was-closed.53529/ |url-status=live |archive-url=https://web.archive.org/web/20201028022237/https://www.resource-zone.com/forum/t/why-dmoz-was-closed.53529/ |archive-date=October 28, 2020 |access-date=April 29, 2017 |work=Resource-Zone.com}}</ref> The website became a single landing page on that day, with links to a static archive of DMOZ, and to the DMOZ discussion forum, where plans to rebrand and relaunch the [[Web directory|directory]] were being discussed.<ref name="resource-zone" /> |
|||
=== Gnuhoo to Newhoo to the Open Directory Project === |
|||
The Gnuhoo directory went live on [[June 5]], 1998, and was renamed '''Newhoo''' after a [[Slashdot]] article was posted wherein Gnuhoo's detractors claimed that Gnuhoo had nothing in common with the spirit of [[free software]] for which the [[GNU]] project was known and was simply a commercial enterprise seeking to construct an alternative to [[Yahoo!]] using volunteer labor.[http://slashdot.org/article.pl?sid=98/06/23/0849239] Newhoo became ODP after it was acquired by [[Netscape Communications Corporation]] for the sum of $1 million in [[October]] of 1998 and the content was released under an [[open content]] [[license]]. Netscape was acquired by AOL shortly thereafter, and ODP was one of the assets included in the acquisition. AOL later merged with [[Time-Warner]]. |
|||
{{As of|2017|9}}, a non-editable mirror remained available at dmoztools.net,<ref>{{Cite web |title=The Directory of the Web – This site includes information formerly made available via DMOZ |url=https://dmoztools.net/ |url-status=live |archive-url=https://web.archive.org/web/20170823003912/http://dmoztools.net/ |archive-date=August 23, 2017 |access-date=August 21, 2017 |website=dmoztools.net}}</ref> and it was announced that while the DMOZ URL would not return, a successor version of the directory named '''Curlie''' would be provided.<ref>{{Cite web |date=May 5, 2017 |title=New dmoz |url=https://www.resource-zone.com/forum/t/new-dmoz.53614/ |url-status=live |archive-url=https://web.archive.org/web/20170822014148/https://www.resource-zone.com/forum/t/new-dmoz.53614/ |archive-date=August 22, 2017 |access-date=August 21, 2017 |website=Resource-Zone.com}}</ref><ref>{{Cite web |date=2017-03-29 |title=Curlie: Present |url=https://curlie.org/top/present.html |url-status=dead |archive-url=https://web.archive.org/web/20170918065934/http://curlie.org/top/present.html |archive-date=September 18, 2017 |access-date=2017-09-17 |website=curlie.org}}</ref> DMOZ, ODP, and Curlie were considered synonymous by 2018.<ref>{{Cite conference |last=Han |first=Miyoung |last2=Wuillemin |first2=Pierre-Henri |last3=Senellart |first3=Pierre |date=2018 |editor-last=Mikkonen |editor-first=Tommi |title=Focused Crawling Through Reinforcement Learning |series=Lecture Notes in Computer Science Information Systems and Applications, incl. Internet/Web, and HCI |publisher=Springer |volume=10845 |page=272 |isbn=9783319916620 |journal=Web Engineering: 18th International Conference, ICWE 2018, Cáceres, Spain, June 5–8, 2018 |editor-first2=Ralf |editor-last2=Klamma |editor-first3=Juan |editor-last3=Hernández}}</ref> Curlie was well established by 2022, using the hierarchy from DMOZ.<ref>{{Cite book |last=Wu |first=Dan |title=Cross-device Web Search |last2=Dong |first2=Jing |last3=Liang |first3=Shaobo |date=2022 |publisher=Taylor & Francis |isbn=9780429510342 |pages=32–33}}</ref> |
|||
=== Directory Growth and Maturation === |
|||
By the time Netscape assumed stewardship, the Open Directory Project had about 100,000 URLs indexed with contributions from about 4500 editors. On [[October 5]], [[1999]], the number of URLs indexed by ODP reached one million. According to an [http://www.geniac.net/odp/ unofficial estimate], the number of URLs in the Open Directory surpassed the number of URLs in the Yahoo! directory in [[April]] [[2000]] with about 1.6 million URLs. ODP achieved the milestone of indexing two million URLs on [[August 14]], 2000, and the milestone of three million listings was reached on [[November 18]], [[2001]]. As of [[July]] [[2003]], ODP has about 3.8 million listings organized into over 460,000 categories and claimed to be derived from the contributions of some 57,000 editors. At that time, the number of active editor accounts (i.e. those editor logins which had not been removed, voluntarily resigned, or timed out due to inactivity of four months) was over 9,000. |
|||
==History== |
|||
=== Competing and Spinoff Projects === |
|||
{{more citations needed|section|date=April 2019}} |
|||
ODP has inspired the formation of at least two other Web directories edited by volunteers and sponsored by [[public company|public companies]]: The now defunct [[Go directory]] (formerly owned by [[The Walt Disney Company]]), and [[Zeal]] (acquired by [[LookSmart]]). However, neither of these Web directories have licensed their content for [[open content]] distribution, a strategy which ensured ODP's success in a highly competitive market. |
|||
DMOZ was founded in the United States as '''GnuHoo''' by [[Rich Skrenta]] and [[Bob Truel]] in 1998 while they were both working as engineers for [[Sun Microsystems]]. Chris Tolles, who worked at Sun Microsystems as the head of marketing for network security products, also signed on in 1998 as a co-founder of Gnuhoo along with co-founders Bryn Dole and Jeremy Wenokur. Skrenta had developed [[TASS (software)|TASS]], an ancestor of [[Tin (newsreader)|tin]], the popular threaded [[Usenet]] newsreader for [[Unix]] systems. The original category structure of the Gnuhoo directory was based loosely on the structure of Usenet newsgroups then in existence. |
|||
The Gnuhoo directory went live on June 5, 1998. After [[Richard Stallman]] and the [[Free Software Foundation]] objected to the use of ''Gnu'' in the name, ''GnuHoo'' was changed to ''NewHoo''.<ref name="SlashdotGnuhoo">{{cite web|url=http://slashdot.org/article.pl?sid=98/06/23/0849239|access-date=April 27, 2007|work=[[Slashdot]]|title=The GnuHoo BooBoo|date=June 23, 1998 |archive-date=July 27, 2009|archive-url=https://web.archive.org/web/20090727050616/http://slashdot.org/article.pl?sid=98%2F06%2F23%2F0849239|url-status=live}}</ref> [[Yahoo!]] then objected to the use of ''Hoo'' in the name, prompting a proposed name change to ''ZURL''.<ref>[http://blog.topix.com/archives/2004_05.html "Zurl Directory"] {{webarchive|url=https://web.archive.org/web/20071223080027/http://blog.topix.com/archives/2004_05.html |date=December 23, 2007 }}</ref> Prior to switching to ''ZURL'', NewHoo was acquired by [[Netscape|Netscape Communications Corporation]] in October 1998 and became the Open Directory Project. Netscape released Open Directory data under the Open Directory License. Netscape was acquired by [[AOL]] shortly thereafter and DMOZ was one of the assets included in the acquisition. |
|||
ODP has inspired the formation of a number of proprietary Web directories, some of which are edited by volunteers, such as ''Zeal'' and the multilingual [[MavicaNet]], and others which are edited by independent contractors, such as [[Hotrate]]. Moreover, the concept of using a large-scale community of editors to compile online content has been successfully applied to other types of projects such as [[Wikipedia]], a freely licensed online encyclopedia. |
|||
[[File:Odp sitecount top.png|thumb|400px|DMOZ size by date, 1998 to 2015]] |
|||
Three open content volunteer projects have been inspired by ODP's editing model: an open content restaurant directory known as [[ChefMoz]] (launched by ODP management), an open content music directory known as [[MusicMoz]], and an encyclopedia known as [[Open Site]]. However, none of the three have yet to achieve success at the level of ODP. |
|||
By the time Netscape assumed stewardship, the Open Directory Project had about 100,000 [[Uniform Resource Locator|URLs]] indexed with contributions from about 4500 editors. On October 5, 1999, the number of URLs indexed by DMOZ reached one million. According to an unofficial estimate, the URLs in DMOZ numbered 1.6 million in April 2000, surpassing those in the [[Yahoo! Directory]].<ref>[http://www.geniac.net/odp/ ODP and Yahoo Size Charts] {{Webarchive|url=https://web.archive.org/web/20071024115013/http://www.geniac.net/odp/ |date=October 24, 2007 }} by ODP editor geniac</ref> DMOZ achieved the milestones of indexing two million URLs on August 14, 2000, three million listings on November 18, 2001, and four million on December 3, 2003. As of April 2013 there were 5,169,995 sites listed in over 1,017,500 categories. On October 31, 2015, there were 3,996,412 sites listed in 1,026,706 categories. |
|||
In January 2006, DMOZ began publishing online reports to inform the public about the development of the project. The first report covered the year 2005. Monthly reports were issued subsequently until September 2006.<ref name="odpreports">[http://freenet-homepage.de/miscellanea/odp_reports/ ODP reports] {{Webarchive|url=https://web.archive.org/web/20080310214513/http://freenet-homepage.de/miscellanea/odp_reports/ |date=March 10, 2008 }} by ODP volunteer administrator chris2001</ref> These reports gave greater insight into the functioning of the directory than the simplified statistics provided on the front page of the directory. The number of listings and categories cited on the front page included "Test" and "Bookmarks" categories but these were not included in the RDF dump offered to users. There were about 7330 active editors during August 2006.<ref name="odpreports"/> 75,151 editors had contributed to the directory as of March 31, 2007.<ref name=frontpage>[http://www.dmoz.org/ ODP Front Page] {{Webarchive|url=https://web.archive.org/web/20170501142910/http://www.dmoz.org/ |date=May 1, 2017 }}. Retrieved August 15, 2006</ref> As of April 2013, the number of contributing editors had increased to 97,584.<ref name=frontpage /> |
|||
== ODP Content == |
|||
===System failure and editing outage, October to December 2006=== |
|||
===Organization and Scope of Content=== |
|||
On October 20, 2006, DMOZ's main server suffered a [[catastrophic failure]]<ref>{{cite news|url=http://dmozgrunt.blogspot.com/2006/10/dmozs-catastrophic-serverhardware.html|title=Dmoz's Catastrophic Server/Hardware Failure|work=dmozgrunt.blogspot.com|date=October 27, 2006|access-date=July 17, 2017|archive-date=July 30, 2017|archive-url=https://web.archive.org/web/20170730153521/http://dmozgrunt.blogspot.com/2006/10/dmozs-catastrophic-serverhardware.html|url-status=live}}</ref> that prevented editors from working on the directory until December 18, 2006. During that period, an older build of the directory was visible to the public. On January 13, 2007, the Site Suggestion and Update Listings forms were again made available.<ref>[https://archive.today/20070108033810/http://www.resource-zone.com/forum/showthread.php?t=45325 dmoz.org technical problems] archive.li Retrieved July 17, 2017.</ref> On January 26, 2007, weekly publication of RDF dumps resumed. To avoid future outages, the system resided on a redundant configuration of two Intel-based servers from then on.<ref>[http://www.miscellanea.de/newsletter/2006Winter/new_servers.html ''The Hamsters' New Home''] {{Webarchive|url=https://web.archive.org/web/20070427033438/http://www.miscellanea.de/newsletter/2006Winter/new_servers.html |date=April 27, 2007 }}, in: Open Directory newsletter issue Winter 2006. Retrieved December 26, 2006.</ref> |
|||
ODP uses a hierarchical ontology scheme for organizing site listings. Listings on a similar topic are grouped into categories, which can then include smaller categories. |
|||
The site's interface was given an upgrade in 2016, branded "DMOZ 3.0", but AOL took it offline the following year. |
|||
Gnuhoo borrowed its initial [[ontology]] from [[Usenet]]. For example, the topic covered by the comp.ai.alife newsgroup was represented by the category Computers/AI/Artificial_Life. The original divisions were for ''Adult'', ''Arts'', ''Business'', ''Computers'', ''Games'', ''Health'', ''Home'', ''News'', ''Recreation'', ''Reference'', ''Regional'', ''Science'', ''Shopping'', ''Society'', and ''Sports''. While these fifteen ''top-level'' categories have remained intact, the ontology of second- and lower-level categories has undergone a gradual evolution; significant changes are initiated by discussion among editors, and then implemented when consensus has been reached. |
|||
===Competing and spinoff projects=== |
|||
In [[July]] [[1998]], the directory became multilingual with the addition of the ''World'' top-level category. The remainder of the directory for English language sites only. By July [[2003]], sixty-seven languages were represented. While the English component of the directory held almost 75% of the sites, the growth rate of the non-English components of the directory was greater than the English component through [[2002]] and 2003. Ontology in non-English categories generally mirrors that of the English directory, although exceptions which reflect language differences are quite common. |
|||
As DMOZ became more widely known, two other major [[Web directory|web directories]] edited by volunteers and sponsored by [[Go.com]] and [[Zeal (web)|Zeal]] emerged, both now defunct. These directories did not license their content for [[open content]] distribution.<ref>[http://www.zeal.com/about/terms_of_use.jhtml "Terms of Use"] {{webarchive|url=https://web.archive.org/web/20020202210117/http://www.zeal.com/about/terms_of_use.jhtml|date=February 2, 2002}}</ref><ref>[http://info.go.com/doc/policy/terms.html "GO Network Terms of Service and Conditions of Use"] {{webarchive|url=https://web.archive.org/web/20000510161046/http://info.go.com/doc/policy/terms.html |date=May 10, 2000 }}</ref> |
|||
The concept of using a large-scale community of editors to compile online content has been successfully applied to other types of projects. DMOZ's editing model directly inspired at least three other open content volunteer projects: music site MusicMoz, an open content restaurant directory known as ChefMoz<ref>[http://www.dmoz.org/newsletter/2003Autumn/chefmoz-column.html ''ChefMoz Fine Dining Menu''] {{Webarchive|url=https://web.archive.org/web/20080513120425/http://www.dmoz.org/newsletter/2003Autumn/chefmoz-column.html |date=May 13, 2008 }}, in: Open Directory newsletter issue Autumn 2003</ref> and an encyclopedia known as [[Open Site]].<ref>[http://open-site.org/help/ help] {{webarchive|url=https://web.archive.org/web/20080629225234/http://open-site.org/help |date=June 29, 2008 }} on open-site.org</ref> Finally, according to [[Larry Sanger]], DMOZ was part of the inspiration for the [[Nupedia]] project, out of which [[Wikipedia]] grew.<ref>{{cite news|url=https://www.theguardian.com/technology/2006/jul/13/media.newmedia|title=This time, it'll be a Wikipedia written by experts|work=[[The Guardian]]|last=Moody|first=Glyn|date=July 13, 2006|access-date=July 17, 2017|archive-date=December 14, 2013|archive-url=https://web.archive.org/web/20131214111418/http://www.theguardian.com/technology/2006/jul/13/media.newmedia|url-status=live}}</ref> |
|||
Several of the top-level categories have unique characteristics. The ''Adult'' category is not present on the directory homepage, but it is fully available in the RDF dump that ODP provides. While the bulk of the directory is categorized primarily by topic, the Regional category is categorized primarily by region. This has led many to view ODP as two parallel directories: ''Regional'' and ''Topical''. |
|||
===Logo history=== |
|||
On [[November 14]], [[2000]], a special directory within the Open Directory |
|||
<gallery> |
|||
was created ''for'' people under 18 years of age. Key factors |
|||
GnuHoo logo.png|Original logo as from June 1998 |
|||
distinguishing this "Kids and Teens" [http://dmoz.org/Kids_and_Teens/] area from the main directory are: |
|||
NewHoo logo.png|Logo after rebranding as NewHoo |
|||
* Stricter guidelines which limit the listing of sites to those which are targeted or appropriate for people under 18 years of age.[http://dmoz.org/kguidelines.html] |
|||
Open Directory Project logo.png|Logo after acquisition by Netscape |
|||
* Category names as well as site descriptions use vocabulary which is age appropriate. |
|||
DMOZ Open Directory Project logo.png|Logo from 1999 to 2014 with the name DMOZ |
|||
* ''Age tags'' on each listing distinguish content appropriate for kids (age 12 and under), teens (13 to 15 years old) and mature teens (16 to 18 years old). |
|||
DMOZ logo.png|Logo from March 2014 |
|||
* Kids and Teens content is available as a separate RDF dump. |
|||
DMOZ logo.svg|Logo from June 2016 until the closing in March 2017 |
|||
* Editing permissions are such that the community is parallel to that of the Open Directory. |
|||
Curlie-logo.png|Curlie logo (2019) |
|||
</gallery> |
|||
==Content== |
|||
By [[July]] [[2003]], this portion of the Open Directory included over 22,000 |
|||
Gnuhoo borrowed the basic outline for its initial ontology from [[Usenet]]. In 1998, Rich Skrenta said, "I took a long list of groups and hand-edited them into a hierarchy."<ref>{{cite news|url=https://searchenginewatch.com/sew/study/2065949/the-search-engine-report-july-1998-number#newhoo|title=NewHoo: Yahoo Built By The Masses|work=[[Search Engine Watch]]|last=Sullivan|first=Danny|date=July 1, 1998|access-date=July 17, 2017|archive-date=April 29, 2018|archive-url=https://web.archive.org/web/20180429160310/https://searchenginewatch.com/sew/study/2065949/the-search-engine-report-july-1998-number#newhoo|url-status=live}}</ref> For example, the topic covered by the comp.ai.alife newsgroup was represented by the category Computers/AI/Artificial_Life. The original divisions were for ''Adult'', ''Arts'', ''Business'', ''Computers'', ''Games'', ''Health'', ''Home'', ''News'', ''Recreation'', ''Reference'', ''Regional'', ''Science'', ''Shopping'', ''Society'', ''Sports'' and "World". While these sixteen ''top-level'' categories have remained intact, the ontology of second- and lower-level categories has undergone a gradual evolution; significant changes are initiated by discussion among editors and then implemented when consensus had been reached. |
|||
site listings. |
|||
In July 1998, the directory became [[multilingualism|multilingual]] with the addition of the ''World'' top-level category. The remainder of the directory lists only English language sites. By May 2005, seventy-five languages were represented. The growth rate of the non-English components of the directory had been greater than the English component since 2002. While the English component of the directory held almost 75% of the sites in 2003, the ''World'' level grew to over 1.5 million sites as of May 2005, forming roughly one-third of the directory. The ontology in non-English categories generally mirrors that of the English directory, although exceptions which reflect language differences are quite common. |
|||
===Directory Maintenance=== |
|||
Directory listings are maintained by editors on a daily basis. While some editors focus on the addition of new listings, others focus on maintaining the existing listings. This includes tasks such as the editing of individual listings to correcting spelling and/or grammar errors, as well as monitoring the status of sites which are linked to. |
|||
Several of the top-level categories have unique characteristics. The ''Adult'' category is not present on the directory homepage but it is fully available in the RDF dump that DMOZ provides. While the bulk of the directory is categorized primarily by topic, the Regional category is categorized primarily by region. This has led many to view DMOZ as two parallel directories: ''Regional'' and ''Topical''. |
|||
[[Robozilla]] is a [[web crawler]] written to check the status of all sites listed in ODP. Periodically, Robozilla will flag sites which appear to have moved or disappeared, and editors follow up to check the sites and take action. This process is critical for the directory in striving to achieve one of its founding goals: to reduce the [[link rot]] in web directories. |
|||
On November 14, 2000, a special directory within DMOZ was created for people under 18 years of age.<ref>[http://www.dmoz.org/newsletter/2000Nov/press.html Kids and Teens Launches!] {{Webarchive|url=https://web.archive.org/web/20080221200521/http://www.dmoz.org/newsletter/2000Nov/press.html |date=February 21, 2008 }} Open Directory Project Newsletter, November/December 2000</ref> Key factors distinguishing this "Kids and Teens" area from the main directory are: |
|||
Another challenge for ODP is how to deal with site listings which point to domains which have changed ownership and no longer offer the content upon which the listing was based (i.e. [[domain hijacking]]). Due to the popularity of the Open Directory and its resulting impact on search engine rankings, listed domains which have lapsed registration are particularly attractive. Editors consistently attempt to locate and deal appropriately with these listed domains. |
|||
* stricter guidelines which limit the listing of sites to those which are targeted or "appropriate" for people under 18 years of age;<ref>{{cite web |url=http://www.dmoz.org/guidelines/kguidelines/ |title=Kids&Teens Guidelines |publisher=Dmoz.org |access-date=June 18, 2014 |archive-url=https://web.archive.org/web/20140716060749/http://www.dmoz.org/guidelines/kguidelines/ |archive-date=July 16, 2014 |url-status=dead |df=mdy-all }}</ref> |
|||
* category names as well as site descriptions use vocabulary which is "[[age appropriate]]"; |
|||
* ''age tags'' on each listing distinguish content appropriate for kids (age 12 and under), teens (13 to 15 years old) and mature teens (16 to 18 years old); |
|||
* Kids and Teens content is available as a separate RDF dump; |
|||
* editing permissions are such that the community is parallel to that of DMOZ. |
|||
By May 2005, this portion of DMOZ included over 32,000 site listings. |
|||
===License and Requirements=== |
|||
ODP data is made available for open content distribution under the terms of the [[Open Directory License]] which requires a specific ODP attribution table on virtually every Web page that uses the data. However, not all data is included: a large number of categories, some "Test" categories, and private editor notes and forums are excluded. Furthermore, the RDF dumps have gained a reputation in some circles for being frequently corrupted because they are based on the draft standard and work on making it standard compliant progresses slowly. |
|||
From early 2004, the whole site was in [[UTF-8]] encoding. Prior to this, the encoding had been [[ISO 8859-1]] for English language categories and a language-dependent character set for other languages. The RDF dumps were encoded in UTF-8 from early 2000. |
|||
===Users of ODP content=== |
|||
At the height of ODP's popularity, ODP data powered the core directory services for many of the Web's largest search engines and portals, including [[Netscape Search]], [[AOL Search]], [[Google]], [[Lycos]], [[HotBot]], and [[DirectHit]]. However, most of these search engines have stopped updating their ODP data, and some smaller sites stopped using RDF dumps, as they grew increasingly large, choosing to query live data directly from the ODP website. Moreover, many noteworthy Web portals that once embraced ODP's free data co-branding have since abandoned ODP's free data model for the revenue sharing model of ODP's major competitor LookSmart. |
|||
===Maintenance=== |
|||
== ODP Policies and Procedures == |
|||
Directory listings were maintained by editors. While some editors focused on the addition of new listings, others focused on maintaining the existing listings, and some did both. This included tasks such as the editing of individual listings to correct spelling and/or grammatical errors, as well as monitoring the status of linked sites. Still others went through site submissions to remove spam and duplicate submissions. |
|||
Robozilla was a [[Web crawler]] written to check the status of all sites listed in DMOZ. Periodically, Robozilla would flag sites which appeared to have moved or disappeared and editors follow up to check the sites and take action. This process was critical for the directory in striving to achieve one of its founding goals: to reduce the [[link rot]] in web directories. Shortly after each run, the sites marked with errors were automatically moved to the unreviewed queue where editors may investigate them when time permits. |
|||
=== Becoming an editor === |
|||
There are restrictions imposed on who can become an ODP editor. The primary gatekeeping mechanism is an editor application process, presided over by ODP's meta editors, wherein editor candidates are required to demonstrate their editing abilities and disclose any and all website affiliations that might pose a conflict of interest. Approximately 90% of these applications are rejected, but re-application is the norm. |
|||
Due to the popularity of DMOZ and its resulting impact on [[Web search engine|search engine]] rankings (See [[PageRank]]), domains with lapsed registration that were listed on DMOZ attracted [[domain hijacking]], an issue that was addressed by regularly removing expired domains from the directory. |
|||
=== Editing model === |
|||
ODP's editing model is a hierarchical one. Upon becoming an editor, an individual will generally have editing permissions in only a small category. Once they have demonstrated basic editing skill in compliance with the Editing Guidelines, they are welcome to apply for additional editing privileges, in either a broader category, or in a category elsewhere in the directory. Mentorship relationships between editors are encouraged, and internal forums provide a vehicle for new editors to ask questions. |
|||
While corporate funding and staff for DMOZ diminished over time, volunteers created editing tools such as linkcheckers to supplement Robozilla, category crawlers, spellcheckers, search tools that directly sift a recent RDF dump, [[bookmarklet]]s to help automate some editing functions, mozilla based add-ons,<ref>[https://addons.mozilla.org/firefox/addon/176740 "ODPExtension"]{{dead link|date=February 2018 |bot=Ost316 |fix-attempted=yes }} Mozilla based add-on, ODP Magic. formerly known as ODP Extension</ref> and tools that helped work through unreviewed queues. |
|||
Over time, senior editors may be granted additional privileges which reflect their editing experience and leadership within the editing community. The most straightforward are ''editall'' privileges, which allow an editor to access all categories in the directory. ''Meta'' privileges additionally allow editors to perform tasks such as reviewing editor applications, setting category features, and handling external and internal abuse reports. ''Cateditall'' privileges are similar to ''editall'', but only for a single directory category; similarly, ''catmod'' privileges are similar to ''meta'', but only for a single directory category. ''Catmv'' privileges allow editors to make changes to directory ontology by moving or renaming categories. All of these privileges are granted by staff, usually after discussion with ''meta'' editors. |
|||
===License and requirements=== |
|||
=== Editing Guidelines === |
|||
DMOZ data was previously made available under the terms of the Open Directory License, which required a specific DMOZ attribution table on every Web page that uses the data. |
|||
All ODP editors are expected to abide by ODP's Editing Guidelines.[http://dmoz.org/guidelines/] These guidelines describe: what types of sites may be listed and which may not; how site listings should be titled and described in a loosely consistent manner; conventions for the naming and building of categories; conflict of interest limitations on the editing of sites which the editor may own or otherwise be affiliated with; and a code of conduct within the community. Editors who are found to have violated these guidelines may be contacted by staff or senior editors, have their editing permissions cut back, or, as a last resort, lose their editing privileges entirely. ODP Guidelines are periodically revised after discussion in editor forums. |
|||
The Open Directory License also included a requirement that users of the data continually check DMOZ site for updates and discontinue use and distribution of the data or works derived from the data once an update occurs. This restriction prompted the [[Free Software Foundation]] to refer to the Open Directory License as a non-free documentation license, citing the right to redistribute a given version not being permanent and the requirement to check for changes to the license. |
|||
=== Site submissions === |
|||
The original motivation for forming Gnuhoo/Newhoo/ODP was the frustration that many people experienced in getting their sites listed on Yahoo!. However, Yahoo! has since addressed this problem by implementing a paid service for timely consideration of site submissions, making free site submissions the primary advantage of ODP. In striking contrast, ODP now has approximately one million unreviewed site submissions, in large part due to spam and incorrectly submitted sites, making the average processing time for a site properly submitted to ODP approximately six months. Moreover, because of concerns about abusive e-mail, ODP's volunteer editors are discouraged from communicating with site submitters, leaving many submitters to wonder whether and when their site has been considered and rejected for inclusion in ODP. In response to ongoing criticism of this policy, ODP editors have set up a public forum where queries about site submission status can be posted.[http://resource-zone.com/] |
|||
In 2011, DMOZ silently changed its license to a [[Creative Commons]] Attribution license{{citation needed|date=May 2016}}, which is a free license (and GPL compatible).<ref>GNU Project: [https://www.gnu.org/licenses/license-list.html#ccby on the Creative Commons Attribution license] {{Webarchive|url=https://web.archive.org/web/20090716201618/http://www.gnu.org/licenses/license-list.html#ccby |date=July 16, 2009 }}</ref> |
|||
== Controversy and Criticism == |
|||
===RDF dumps=== |
|||
=== Allegations of abusive editing practices === |
|||
DMOZ data is made available through an RDF-like dump that is published on a download server, older versions are also archived there.<ref>{{cite web |url=http://rdf.dmoz.org/ |title=Open Directory RDF Dump |publisher=Rdf.dmoz.org |access-date=June 18, 2014 |url-status=dead |archive-url=https://web.archive.org/web/20140626235229/http://rdf.dmoz.org/ |archive-date=June 26, 2014 |df=mdy-all }}</ref> New versions are usually generated weekly. A DMOZ editor has catalogued a number of bugs that are encountered in the DMOZ RDF dump, most importantly that the file format is not RDF.<ref>{{cite news|url=http://www.steevithak.com/sw/odp/rdflist.html|title=ODP/dmoz Data Dump ToDo List|work=steevithak.com|access-date=July 17, 2017|archive-date=June 22, 2017|archive-url=https://web.archive.org/web/20170622054836/http://www.steevithak.com/sw/odp/rdflist.html|url-status=live}}</ref> So while today the so-called RDF dump is valid [[XML]], it is not valid [[Resource Description Framework|RDF]] and as such, software to process the DMOZ RDF dump needs to be specifically written for DMOZ data. |
|||
There have long been allegations that volunteer ODP editors give favorable treatment to their own websites while concomitantly thwarting the good faith efforts of their competition, altough little evidence of this has been shown. Such allegations are fielded by ODP's staff and meta editors, who have the authority to take disciplinary action against volunteer editors who are suspected of engaging in abusive editing practices. In 2003, ODP introduced a new ''Public Abuse Report System'' that allows members of the general public to report and track allegations of abusive editor conduct using an online form. [http://inelegant.org/report-abuse/] |
|||
===Content users=== |
|||
In a widely publicized federal lawsuit which is still ongoing in the [[United States]], a prominent [[tax law]] firm known as J.K. Harris obtained a [[temporary restraining order]] and then a [[preliminary injunction]] against a volunteer ODP editor. The plaintiff alleged, in part, that the editor in question had engaged in abusive editing practices that violated both state and federal laws restricting unfair competition. The Honorable Claudia Wilken, United States District [[Judge]] for the Northern District of California, found that this allegation did not justify injunctive relief, but granted injunctive relief on other grounds.[http://www.eff.org/IP/20030328_taxes-com_amended_prelim_injunc.pdf] |
|||
DMOZ data powers the core [[directory service]]s for many of the Web's largest search engines and portals, including Netscape Search, AOL Search, and [[Alexa Internet|Alexa]]. [[Google Directory]] used DMOZ information, until being shuttered in July 2011.<ref>{{cite news|title=Google Streamlining: Say Goodbye to the Google Directory and Labs!|url=http://www.pandia.com/sew/3963-google-streamlining-say-goodbye-to-the-google-directory-and-labs.html|access-date=July 25, 2011|newspaper=Pandia Search Engine News|date=July 21, 2011|archive-date=July 25, 2011|archive-url=https://web.archive.org/web/20110725012548/http://www.pandia.com/sew/3963-google-streamlining-say-goodbye-to-the-google-directory-and-labs.html|url-status=live}}</ref> |
|||
Other uses are also made of DMOZ data. For example, in the spring of 2004 [[Yahoo! Search Marketing|Overture]] announced a search service for third parties combining Yahoo! Directory search results with DMOZ titles, descriptions and category metadata. The search engine Gigablast announced on May 12, 2005, its searchable copy of DMOZ. The technology permits search of websites listed in specific categories, "in effect, instantly creating over 500,000 vertical search engines".<ref>{{cite news|url=http://www.gigablast.com/prdir.html|title=Gigablast Launches 500,000 Vertical Search Engines|work=[[Gigablast]]|date=May 12, 2005|access-date=July 17, 2017|archive-date=May 25, 2017|archive-url=https://web.archive.org/web/20170525200317/http://www.gigablast.com/prdir.html|url-status=dead}}</ref> |
|||
Allegations of unfair competition at ODP and unethical ''quid pro quo'' also arose when ODP's paid staff gave the paid employees of professional content providers such as [[AOL Time Warner|AOL]] and [[Rolling Stone magazine]] high level editing access at ODP in an effor to boost the number of links. Many volunteer editors perceived this to be a sellout of the grass roots principles on which ODP was based. |
|||
{{as of|2006|9|8}}, DMOZ listed 313 English-language Web sites that use DMOZ data as well as 238 sites in other languages.<ref>[http://www.dmoz.org/Computers/Internet/Searching/Directories/Open_Directory_Project/Sites_Using_ODP_Data/ Category: Sites Using ODP Data] {{Webarchive|url=https://web.archive.org/web/20090316003426/http://www.dmoz.org/Computers/Internet/Searching/Directories/Open_Directory_Project/Sites_Using_ODP_Data/ |date=March 16, 2009 }} on www.dmoz.org. Retrieved September 8, 2006.</ref> However, these figures do not reflect the full picture of use, as those sites that use DMOZ data without following the terms of the DMOZ license are not listed. |
|||
=== Ownership and Management of ODP === |
|||
Underlying much of the controversy surrounding ODP is its ownership and management. Many of the original GnuHoo volunteers felt that they had been deceived into joining a commercial enterprise, but most of the initial controversy died down when the project was renamed NewHoo. Moreover, when Netscape acquired the project, renamed it ODP, and released ODP's content under an open content license, ODP's critics all but disappeared. However, as the community of editors at ODP began to grow, and ODP's content became widely used by most major search engines and Web directories, the issue of ODP's ownership and management resurfaced. |
|||
==Policies and procedures== |
|||
At ODP's inception, there was little thought given to the idea of how ODP should be managed, and there were no official forums, guidelines, or FAQs. In essence, ODP began as a free for all. Even after ODP set up its internal editor forums, many editors remained blissfully unaware that these forums existed until they were directed to the forums by one of their fellow editors. Moreover, given that ODP had no official guidelines at first, ODP editors simply hashed out some sort of consensus among themselves and published unofficial FAQs. |
|||
[[File:Rich Skrenta.jpg|alt=|thumb|DMOZ co-founder [[Rich Skrenta]] in 2009]] |
|||
Restrictions are imposed on who can become an DMOZ editor. The primary gatekeeping mechanism is an editor application process wherein editor candidates demonstrate their editing abilities, disclose affiliations that might pose a conflict of interest{{citation needed|date=July 2019}}, and otherwise give a sense of how the applicant would likely mesh with the DMOZ culture and mission.<ref>{{cite web |url=http://www.dmoz.org/cgi-bin/apply.cgi |title=Become an Editor at the Open Directory Project |publisher=Dmoz.org |access-date=June 18, 2014 |archive-url=https://web.archive.org/web/20140827045332/http://www.dmoz.org/cgi-bin/apply.cgi |archive-date=August 27, 2014 |url-status=dead |df=mdy-all }}</ref> A majority of applications are rejected but reapplying is allowed and sometimes encouraged. The same standards apply to editors of all categories and subcategories.{{citation needed|date=December 2016}} |
|||
DMOZ's editing model is a [[hierarchical]] one. Upon becoming editors, individuals will generally have editing permissions in only a small category. Once they have demonstrated basic editing skills in compliance with the Editing Guidelines, they are welcome to apply for additional editing privileges in either a broader category or else another category in the directory. Mentorship relationships between editors are encouraged, and internal forums provide a vehicle for new editors to ask questions.{{citation needed|date=December 2016}} |
|||
As time went on, the ODP Editor Forums became the ''de facto'' ODP parliament, and when one of ODP's staff members would post an opinion in the forums, it would be deferred to as an official ruling. (I.e., "Staff has spoken.") There was also a short-lived attempt at moderation of the ODP Editor Forums, but it was abandoned as being the antithesis of the egalitarian principles on which the ODP community was supposed to be based. Even so, ODP staff began to give trusted senior editors additional editing privileges, including the ability to approve new editor applications, which eventually led to a stratified hierarchy of duties and privileges among ODP editors, with ODP's paid staff having the final say regarding ODP's policies and procedures. |
|||
DMOZ has its own internal forums, the contents of which are intended only for editors to communicate with each other primarily about editing topics. Access to the forums requires an editor account and editors are expected to keep the contents of these forums private.<ref name="commguide" /> |
|||
=== Allegations that ODP editors are removed for criticizing ODP's policies === |
|||
ODP's paid staff has imposed controversial policies from time to time, and volunteer editors who openly dissent often find their editing privileges removed, an ongoing situation which has been chronicled at the XODP Yahoo! eGroup since [[May]] of [[2000]]. The first noteworthy expose was ''Life After the Open Directory Project'', a [[June 1]], 2000 guest column written for Traffick.com by David F. Prenatt, Jr. (former ODP editor 'netesq'), who founded the XODP Yahoo! eGroup after losing his ODP editing privileges.[http://www.traffick.com/story/06-2000-xodp.asp] Another noteworthy example was the volunteer editor known by the alias [[The Cunctator]], who was banned from the ODP soon after submitting an article to Slashdot on [[October 24]], 2000, which criticized changes in ODP's [[copyright]] policies.[http://slashdot.org/articles/00/10/24/1252232.shtml] In light of the apparent risks of expressing dissent openly, at least one ODP insider has expressed his or her dissent in ''AOL Meddling in ODP Causes Shift in Balance of Editorial Power'', an article published at Traffick.com on [[September 4]], [[2001]] under the pseudonym of Julian McCreary.[http://www.traffick.com/story/portals/200108_aolodp.asp] |
|||
Over time, senior editors can be granted additional privileges which reflect their editing experience and leadership within the editing community. The most straightforward is ''edit all'' privileges, which allow an editor to access all categories in the directory. ''Meta'' privileges additionally allow editors to perform tasks such as reviewing editor applications, setting category features, and handling external and internal abuse reports. ''Cateditall'' privileges are similar to ''edit all'', but only for a single directory category. Similarly, ''catmod'' privileges are similar to ''meta'', but only for a single directory category. ''Catmv'' privileges allow editors to make changes to directory ontology by moving or renaming categories. All of these privileges are granted by admins and staff, usually after discussion with ''meta'' editors.{{citation needed|date=December 2016}} |
|||
=== Editor Removal Procedures === |
|||
ODP's editor removal procedures, which are overseen by ODP's staff and meta editors, are very controversial. According to ODP's official editorial guidelines, editors are removed for abusive editing practices. However, discussions that may result in disciplinary action against volunteer editors take place in a private forum which can only be accessed by ODP's staff and meta editors, and volunteer editors who are at risk of losing their editing privileges may not be given any notice that such proceedings are taking place, much less notice of an adverse decision. The rationale that is publicly asserted for this policy is that volunteer editors are assumed to know when they are violating ODP guidelines.[http://dmoz.org/newsletter/2000Sep/removal.html] |
|||
In August 2004, a new level of privileges called ''admin'' was introduced. Administrator status was granted to a number of long serving metas by staff. Administrators have the ability to grant editall+ privileges to other editors and to approve new directory-wide policies, powers which had previously only been available to root (staff) editors.<ref name="adminguide">{{cite web |url=http://www.dmoz.org/guidelines/admin/ |title=Open Directory Project Administrator Guidelines |publisher=Dmoz.org |access-date=June 18, 2014 |archive-url=https://web.archive.org/web/20140722205313/http://www.dmoz.org/guidelines/admin/ |archive-date=July 22, 2014 |url-status=dead |df=mdy-all }}</ref> |
|||
ODP has a standing policy that prohibits any current ODP editors in a position to know anything from discussing the reasons for specific editor removals. In the past, this has led to claims that many ODP editors are left to wonder why they cannot login at ODP to perform their editing work. However, ODP is now set up in such a way that when someone attempts to login at ODP using a deactivated editor login, a generic web page is displayed that informs a removed editor that a final decision has been made regarding the deactivation of his or her login and providing a list of possible reasons as to why such a decision might have been made. At the same time, a software glitch can result in the same page being displayed when an editor login has simply timed out and the editor in question is in fact eligible for reinstatement. Consequently, the ambiguity typically associated with editor removal procedures remains an ongoing issue. |
|||
All DMOZ editors are expected to abide by DMOZ's Editing Guidelines. These guidelines describe editing basics: which types of sites may be listed and which may not; how site listings should be titled and described in a loosely consistent manner; conventions for the naming and building of categories; conflict of interest limitations on the editing of sites which the editor may own or otherwise be affiliated with; and a code of conduct within the community.<ref>{{cite web |url=http://www.dmoz.org/guidelines/ |title=ODP Directory Editorial Guidelines |publisher=Dmoz.org |access-date=June 18, 2014 |archive-url=https://web.archive.org/web/20140701150829/http://www.dmoz.org/guidelines/ |archive-date=July 1, 2014 |url-status=dead |df=mdy-all }}</ref> Editors who are found to have violated these guidelines may be contacted by staff or senior editors, have their editing permissions cut back, or lose their editing privileges entirely. DMOZ Guidelines are periodically revised after discussion in editor forums.{{citation needed|date=December 2016}} |
|||
ODPSS is an [[acronym]] for ''Open Directory Project Suicide Syndrome'' and is a term applied to editors at Open Directory Project who are abusive, rude, and/or insulting in forum threads, something which eventually leads to them either resigning or being removed. |
|||
==Controversy and criticism== |
|||
=== Size of the directory and number of editors === |
|||
There have long been allegations that volunteer DMOZ editors give favorable treatment to their own websites while concomitantly thwarting the good faith efforts of their competition.<ref>{{cite news|url=https://www.webmasterworld.com/forum17/94.htm|title=How To: ODP Editor Is Competitor|work=webmasterworld.com|date=November 4, 2000|access-date=July 17, 2017|archive-date=July 30, 2017|archive-url=https://web.archive.org/web/20170730151811/https://www.webmasterworld.com/forum17/94.htm|url-status=live}}</ref> Such allegations are fielded by ODP's staff and meta editors, who have the authority to take disciplinary action against volunteer editors who are suspected of engaging in abusive editing practices.<ref>[http://www.dmoz.org/guidelines/meta/abuse.html ODP Meta Guidelines: Editor Abuse and Removal] {{Webarchive|url=https://web.archive.org/web/20090123062946/http://www.dmoz.org/guidelines/meta/abuse.html |date=January 23, 2009 }}. Retrieved October 9, 2008.</ref> In 2003, DMOZ introduced a new ''Public Abuse Report System'' that allows members of the general public to report and track allegations of abusive editor conduct using an online form.<ref>{{cite web |url=http://report-abuse.dmoz.org/ |title=Open Directory Project: Public Abuse Report System |publisher=Report-abuse.dmoz.org |access-date=June 18, 2014 |url-status=dead |archive-url=https://web.archive.org/web/20100807152404/http://report-abuse.dmoz.org/ |archive-date=August 7, 2010 |df=mdy-all }}</ref> Uninhibited discussion of DMOZ's purported shortcomings has become more common on mainstream [[webmaster]] discussion forums. Although site policies suggest that an individual site should be submitted to only one category,<ref>{{cite web |url=http://www.dmoz.org/add.html |title=How to suggest a site to the Open Directory |publisher=Dmoz.org |access-date=June 18, 2014 |archive-url=https://web.archive.org/web/20140616153445/http://www.dmoz.org/add.html |archive-date=June 16, 2014 |url-status=dead |df=mdy-all }}</ref> as of October 2007, Topix.com, a news aggregation site operated by DMOZ founder Rich Skrenta, had more than 17,000 listings.<ref>[http://search.dmoz.org/cgi-bin/search?search=topix Open Directory Project Search: "topix"]{{dead link|date=July 2017 |bot=Robertgombos |fix-attempted=yes }} (Retrieved October 18, 2007)</ref> |
|||
As of July [[2003]], ODP claimed to have over 3.8 million entries in 460,000 categories, built by 57,238 editors. However, the according to critics number of editors contributing to ODP is exaggerated by a ratio of at least 5 to 1, due to the fact that ODP tracks the total number of editor logins ever created rather than the number of currently active editors. After an inactive period of four months, a login times out, but can be reactivated if the editor wants so. Other logins that are included in the overall tally represent the logins of former editors who have had their editing privileges removed, either for abusive editing practices as by consensus of ODP's staff and meta editors. Moreover, when ODP editor logins are intentionally deactivated, some former editors may simply reapply under an assumed identity. |
|||
Early in the history of DMOZ, its staff gave representatives of selected companies, such as ''[[Rolling Stone]]'' or [[CNN]], editing access in order to list individual pages from their websites.<ref>{{cite news|url=https://www.webmasterworld.com/forum17/1330.htm|title=Multiple URL's in DMOZ|work=webmasterworld.com|date=January 30, 2003|access-date=July 17, 2017|archive-date=May 25, 2017|archive-url=https://web.archive.org/web/20170525200325/https://www.webmasterworld.com/forum17/1330.htm|url-status=live}}</ref> Links to individual CNN articles were added until 2004, but were entirely removed from the directory in January 2008<ref>{{cite web|url=http://www.dmoz.org/News/|title= DMOZ news|access-date=June 7, 2009|archive-date=June 6, 2009|archive-url=https://web.archive.org/web/20090606003048/http://www.dmoz.org/News/|url-status=dead}}</ref> due to the content being outdated and not considered worth the effort to maintain. There have been no similar experiments with the editing policy since then. |
|||
=== Private editor forums === |
|||
Critics of ODP have questioned the propriety of ODP having private editor forums in what is purportedly an open project, as both ''Go'' and ''Zeal'' have always allowed public access to their forum discussions. However, the defenders of ODP's private forums assert that such forums are necessary for the discussion of matters that are internal to ODP. Even so, some of these private forum discussions have been published outside of ODP after being forwarded to ODP's critics by anonymous ODP insiders. [http://donotgo.com/oded.htm] |
|||
=== |
===Ownership and management=== |
||
Underlying some controversy surrounding DMOZ is its ownership and management. Some of the original GnuHoo volunteers felt that they had been deceived into joining a commercial enterprise.<ref name="SlashdotGnuhoo" /> To varying degrees, those complaints have continued up until the present. |
|||
Critics of ODP, most notable among them the aforementioned GNU project, also point to the fact that ODP's editing software is not [[free software]] and that ODP's content license restricts the freedom of licensees unnecessarily. That ODP's software isn't open sourced is because it relies on third party software that isn't open source. As such, there have been some efforts to provide alternatives to ODP (see below). These alternatives would allow communities of like-minded editors to set up and maintain their own open source/open content Web directories. However, no noteworthy open source/open content alternative to ODP has yet emerged. |
|||
At DMOZ's inception, there was little thought given to the idea of how DMOZ should be managed and there were no official [[Internet forum|forums]], guidelines or [[FAQ]]s.<ref>{{cite news|url=http://www.laisha.com/zine/odphistory.html|title=The Open Directory Project: The Spirit of the Web|work=laisha.com|access-date=July 17, 2017|archive-date=May 16, 2008|archive-url=https://web.archive.org/web/20080516025327/http://www.laisha.com/zine/odphistory.html|url-status=live}}</ref> |
|||
==References== |
|||
As time went on, the ODP editor forums became the de facto DMOZ parliament, and when one of DMOZ's staff members would post an opinion in the forums, it would be considered an official ruling.<ref name="commguide">{{cite web |url=http://www.dmoz.org/guidelines/communication.html |title=ODP Communication Guidelines |publisher=Dmoz.org |access-date=June 18, 2014 |archive-url=https://web.archive.org/web/20140722214551/http://www.dmoz.org/guidelines/communication.html |archive-date=July 22, 2014 |url-status=dead |df=mdy-all }}</ref> Even so, DMOZ staff began to give trusted senior editors additional editing privileges, including the ability to approve new editor applications, which eventually led to a stratified hierarchy of duties and privileges among DMOZ editors, with DMOZ's paid staff having the final say regarding DMOZ's policies and procedures.<ref name="adminguide" /><ref name="metaguide" /> |
|||
*[1] - Slashdot | ''The GnuHoo BooBoo'' | Posted by CmdrTaco (Tuesday June 23, 1998) |
|||
*[2] - Open Directory Project - Kids and Teens Directory |
|||
*[3] - Open Directory Project - Kids and Teens Directory Editing Guidelines |
|||
*[4] - Open Directory Project Editing Guidelines |
|||
*[5] - Open Directory Project Public Forum |
|||
*[6] - Open Directory Project Public Abuse Report System |
|||
*[7] - EFF | J.K. Harris v. Steven Kassel. (Preliminary Injunction [PDF].) |
|||
*[8] - Traffick.com | ''Life After the Open Directory Project'' | Guest Column by David F. Prenatt, Jr. (June 1, 2000) |
|||
*[9] - Slashdot | ''Dmoz (aka AOL) Changing Guidelines In Sketchy Way'' | Posted by CmdrTaco (Tuesday October 24, 2000) |
|||
*[10] - Traffick.com | ''AOL Meddling in ODP Causes Shift in Balance of Editorial Power'' | By Julian McCreary (September 4, 2001) |
|||
*[11] - DMOZ Newsletter | ''Editor Removal Explained'' by Arlarson (September 2000) |
|||
*[12] - Donotgo.com | Dumb-oz (Reproduction of Internal ODP Editor Forum) |
|||
Robert Keating, a principal of Touchstone Consulting Group in Washington, D.C., worked as AOL's program manager for DMOZ beginning in 2004. He started working for AOL in 1999 as senior editor for AOL Search, then as managing editor for AOL's DMOZ program, and then as media ecosystem manager of AOL Product Marketing.<ref>[http://blog.dmoz.org/2009/01/08/meet-aols-dmoz-staff-team/ Meet AOL's DMOZ Staff Team] {{webarchive|url=https://web.archive.org/web/20120101091318/http://blog.dmoz.org/2009/01/08/meet-aols-dmoz-staff-team |date=January 1, 2012 }}, DMOZ Blog, January 8, 2009</ref><ref>{{citation |url=http://www.linkedin.com/pub/robert-keating/2/457/424 |title=Robert Keating |publisher=LinkedIn |access-date=December 17, 2011 |archive-date=December 24, 2010 |archive-url=https://web.archive.org/web/20101224010602/http://www.linkedin.com/pub/robert-keating/2/457/424 |url-status=live }}</ref>{{Failed verification|date=October 2024|reason=Sources show Robert Keating did hold the position of AOL program manager at DMOZ, but cited sources do not specify he began with DMOZ in 2004, or give any other starting date.}} |
|||
==External links== |
|||
=== |
===Editor removal procedures=== |
||
DMOZ's editor removal procedures are overseen by DMOZ's staff and meta-editors. According to DMOZ's official editorial guidelines, editors are removed for abusive editing practices or uncivil behaviour. Discussions that may result in disciplinary action against volunteer editors take place in a private forum which can only be accessed by DMOZ's staff and meta editors. Volunteer editors who are being discussed are not given notice that such proceedings are taking place.<ref name="metaguide">{{cite web |url=http://www.dmoz.org/guidelines/meta/ |title=Open Directory Project Meta Guidelines |website=dmoz.org |url-status=dead |archive-url=https://web.archive.org/web/20170304211520/http://www.dmoz.org/guidelines/meta/ |archive-date=March 4, 2017 |date=December 31, 2013 |access-date=July 21, 2017 |df=mdy-all }}</ref> Some people find this arrangement distasteful, wanting instead a discussion modeled more like a trial held in the U.S. judicial system.<ref name="yahooxodp">{{cite news|url=https://groups.yahoo.com/neo/groups/xodp/conversations/topics/1|title=Life After ODP|work=[[Yahoo! Groups]]|last=Prenatt|first=David|date=May 29, 2000|access-date=July 17, 2017|archive-date=May 25, 2017|archive-url=https://web.archive.org/web/20170525201231/https://groups.yahoo.com/neo/groups/xodp/conversations/topics/1|url-status=dead}}</ref> |
|||
* [http://dmoz.org/ The Open Directory Project Home Page] |
|||
* [http://dmoz.org/about.html About the ODP] |
|||
* [http://dmoz.org/Home/Consumer_Information/Computers_and_Internet/Internet/Directories/Open_Directory_Project/ Consumer Reviews of ODP] |
|||
* [http://rdf.dmoz.org/rdf/ Open Directory RDF Dump] |
|||
* [http://dmoz.org/Computers/Internet/Searching/Directories/Volunteer-Edited/ Community-built Web Directories] |
|||
In the article "Editor Removal Explained", DMOZ meta editor Arlarson states that "a great deal of confusion about the removal of editors from DMOZ results from false or misleading statements by former editors".<ref>Arlarson, [http://www.dmoz.org/newsletter/2000Sep/removal.html Editor Removal Explained] {{Webarchive|url=https://web.archive.org/web/20081202125613/http://www.dmoz.org/newsletter/2000Sep/removal.html |date=December 2, 2008 }}, Open Directory Project Newsletter (September 2000).</ref> |
|||
===Alternatives to ODP=== |
|||
* [http://savannah.nongnu.org/projects/freemoz/ FreeMoz] |
|||
The DMOZ's confidentiality guidelines prohibit any current DMOZ editors in a position to know anything from discussing the reasons for specific editor removals,<ref name="metaguide" /> however a list of potential reasons was provided in the guidelines.<ref>{{cite web |url=http://www.dmoz.org/guidelines/accounts.html#removal |title=Guidelines: Account Removal |website=dmoz.org |archive-url=https://web.archive.org/web/20170312193338/http://www.dmoz.org/guidelines/accounts.html#removal |archive-date=March 12, 2017 |date=July 3, 2016 |access-date=July 21, 2017 }} [http://dmoztools.net/docs/en/guidelines/accounts.html#removal Alt URL] {{Webarchive|url=https://web.archive.org/web/20170526083156/http://dmoztools.net/docs/en/guidelines/accounts.html#removal |date=May 26, 2017 }}</ref> In the past, this has led to removed DMOZ editors wondering why they cannot log in at DMOZ to perform their editing work.<ref>{{cite news|url=https://www.resource-zone.com/forum/t/editor-account-expired.18061/|title=Editor account expired|work=resource-zone.com|date=July 27, 2004|access-date=July 17, 2017|archive-date=September 29, 2017|archive-url=https://web.archive.org/web/20170929044226/https://www.resource-zone.com/forum/t/editor-account-expired.18061/|url-status=live}}</ref><ref>[http://www.resource-zone.com/forum/showthread.php?t=3655 Thread: Can't Login] {{webarchive|url=https://web.archive.org/web/20081128044638/http://www.resource-zone.com/forum/showthread.php?t=3655 |date=November 28, 2008 }} on Resource-Zone</ref> |
|||
* [http://xodp.sourceforge.net/ XODP @ SourceForge.Net] |
|||
* [http://www.erational.org/software/wiklink/ WikLink] |
|||
David F. Prenatt Jr., former DMOZ editor ''netesq'', and another former editor known by the alias ''The Cunctator'', both claim to have been removed for disagreeing with staff about changes to policies, particularly DMOZ's [[copyright]] policies. According to their claims, staff use the excuse of uncivil behaviour as a means to remove bothersome editors.<ref name="yahooxodp" /><ref>{{cite news|url=http://www.traffick.com/story/06-2000-xodp.asp|title=Life After the Open Directory Project|work=traffick.com|last=Prenatt|first=David F. Jr.|date=June 1, 2000|access-date=July 17, 2017|url-status=dead|archive-url=https://web.archive.org/web/20170525200320/http://www.traffick.com/story/06-2000-xodp.asp|archive-date=May 25, 2017|df=mdy-all}}</ref><ref>{{cite web |author=CmdrTaco |url=http://slashdot.org/articles/00/10/24/1252232.shtml |title=Dmoz (aka AOL) Changing Guidelines In Sketchy Way |work=[[Slashdot]] |date=October 24, 2000 |access-date=July 17, 2017 |archive-date=February 6, 2010 |archive-url=https://web.archive.org/web/20100206093457/http://slashdot.org/articles/00/10/24/1252232.shtml |url-status=live }}</ref> |
|||
===Blacklisting allegations=== |
|||
Senior DMOZ editors have the ability to attach "warning" or "do not list" notes to individual domains but no editor has the unilateral ability to block certain sites from being listed. Sites with these notes might still be listed and at times notes are removed after some discussion.<ref>[http://www.dmoz.org/urlnote.html Add Note to URL Feature] {{Webarchive|url=https://web.archive.org/web/20080513105131/http://www.dmoz.org/urlnote.html |date=May 13, 2008 }}, in ODP Documentation</ref> |
|||
===Hierarchical structure=== |
|||
Criticism of DMOZ's [[hierarchical]] structure emerged by around 2005. Many believe hierarchical directories are too complicated. With the emergence of [[Web 2.0]], [[folksonomy|folksonomies]] began to appear, and some editors proposed that folksonomies, [[network (mathematics)|networks]] and [[directed graph]]s are more "natural" and easier to manage than hierarchies.<ref>{{cite web|last=Hriţcu|first=C.|url=http://hritcu.wordpress.com/2005/04/08/folksonomies-vs-ontologies/|title=Folksonomies vs. Ontologies|work=hritcu.wordpress.com|date=April 8, 2005|access-date=July 17, 2017|archive-date=July 31, 2017|archive-url=https://web.archive.org/web/20170731044815/https://hritcu.wordpress.com/2005/04/08/folksonomies-vs-ontologies/|url-status=live}}</ref><ref>[http://itc.conversationsnetwork.org/shows/detail470.html "Ontology is Overrated"] {{webarchive|url=https://web.archive.org/web/20130729231420/http://itc.conversationsnetwork.org/shows/detail470.html |date=July 29, 2013 }}</ref><ref>{{cite web |author=Hammond, Tony |author2=Hannay, Timo |author3=Lund, Ben |author4=Scott, Joanna |url=http://www.dlib.org/dlib/april05/hammond/04hammond.html |title=Social Bookmarking Tools (I) |work=[[D-Lib Magazine]] |date=April 2005 |access-date=July 17, 2017 |archive-date=June 2, 2018 |archive-url=https://web.archive.org/web/20180602115558/http://www.dlib.org/dlib/april05/hammond/04hammond.html |url-status=live }}</ref> |
|||
==Software== |
|||
===Search=== |
|||
The ODPSearch software is a derivative version of [[Isearch]] which is [[open-source software|open-source]], licensed under the [[Mozilla Public License]].<ref>{{cite web |url=http://www.dmoz.org/docs/en/searchguide.html |title=Open Directory Search Guide |publisher=Dmoz.org |access-date=June 18, 2014 |archive-url=https://web.archive.org/web/20140616140811/http://www.dmoz.org/docs/en/searchguide.html |archive-date=June 16, 2014 |url-status=dead |df=mdy-all }}</ref> |
|||
===Editor forums=== |
|||
The ODP Editor Forums were originally run on software that was based on the proprietary [[Ultimate Bulletin Board]] system. In June 2003, they switched to the open source [[phpBB]] system. As of 2007, these forums were powered by a modified version of phpBB. |
|||
===Bug tracking=== |
|||
The bug tracking software used by the ODP is [[Bugzilla]] and the web server [[Apache HTTP Server|Apache]]. [[Squid cache|Squid]] web proxy server was also used but it was removed in August 2007 when the storage servers were reorganized. All these applications are open source. |
|||
===Interface=== |
|||
The DMOZ database/editing software is closed source (although Richard Skrenta has said in June 1998 that he was considering licensing it under the [[GNU General Public License]]). This has led to criticism from the aforementioned GNU project, many of whom also criticized the DMOZ content license. The content was later released under a Creative Commons license, which is compatible with the GNU license.<ref>{{cite news|url=https://www.gnu.org/licenses/license-list.html#OtherLicenses|title=Licenses for Works of Practical Use besides Software and Documentation|work=[[GNU Project]]|access-date=July 17, 2017|archive-date=July 16, 2009|archive-url=https://web.archive.org/web/20090716201618/http://www.gnu.org/licenses/license-list.html#OtherLicenses|url-status=live}}</ref> |
|||
As such, there have been some efforts to provide alternatives to DMOZ. These alternatives would allow communities of like-minded editors to set up and maintain their own open source/open content Web directories. |
|||
==ChefMoz== |
|||
'''Chef Moz''', an offshoot of DMOZ, was an [[open content]] [[Web directory|directory]] of [[World Wide Web]] links of restaurants. The website was constructed and maintained by a [[virtual community|community]] of [[Volunteering|volunteer]] editors, and owned by [[Netscape]]. |
|||
Chef Moz, similar to its parent DMOZ, used a hierarchical [[Ontology (computer science)|ontology]] scheme for organizing site listings. Listings on a similar topic were grouped into categories, which can then include smaller categories. |
|||
On February 17, 2011, DMOZ administrator "lisagirl" confirmed that Chef Moz was dead.<ref>{{Cite web |url=http://www.resource-zone.com/forum/index.php?showtopic=51013&hl=chefmoz |title=Announcement of Chef Moz's Death |access-date=May 19, 2021 |archive-date=August 7, 2011 |archive-url=https://web.archive.org/web/20110807153602/http://www.resource-zone.com/forum/index.php?showtopic=51013&hl=chefmoz |url-status=live }}</ref> |
|||
===Growth=== |
|||
From its beginning in 2000 to November 2009 (when it became impossible for editors to log into the site), ChefMoz had grown to become the largest global directory of restaurants on the Internet. The total number of restaurants indexed since 2000 is recorded in the following table:<ref name="growth">Statistics from the [https://web.archive.org/web/*/http://chefmoz.org Wayback Machine]</ref> |
|||
{| class="wikitable" |
|||
|- |
|||
! Year |
|||
! Number of restaurants |
|||
|- |
|||
| 2000 |
|||
| 48,000 |
|||
|- |
|||
| 2001 |
|||
| 75,000 |
|||
|- |
|||
| 2002 |
|||
| 179,000 |
|||
|- |
|||
| 2003 |
|||
| 208,000 |
|||
|- |
|||
| 2004 |
|||
| 258,000 |
|||
|- |
|||
| 2005 |
|||
| 266,000 |
|||
|- |
|||
| 2006 |
|||
| 275,000 |
|||
|- |
|||
| 2007 |
|||
| 315,000 |
|||
|- |
|||
| 2008 |
|||
| 325,000 |
|||
|} |
|||
==See also== |
|||
{{Portal|Internet}} |
|||
* [[List of web directories]] |
|||
==References== |
|||
{{reflist|25em}} |
|||
==External links== |
|||
{{sister project links|commons=Category:DMOZ |species=no |m=no |mw=no |v=DMOZ |s=no |wikt=no |b=no |n=no |d=Q41226 |q=no}} |
|||
{{wikidata property|P998}} |
|||
* {{webarchive |url=https://web.archive.org/web/20170314015905/http://www.dmoz.org/ |date=March 14, 2017 |title=dmoz.org — official website}} |
|||
{{Netscape}} |
|||
===Miscellaneous links=== |
|||
{{AOL Inc.}} |
|||
* [http://www.usemod.com/cgi-bin/mb.pl?OpenDirectoryProject MeatballWiki: OpenDirectoryProject] |
|||
* [http://www.gnu.org/licenses/license-list.html#NonFreeDocumentationLicenses GNU Project - Non-free Documentation Licenses - The Open Directory License (aka The dmoz.org License)] |
|||
* [http://groups.yahoo.com/group/xodp XODP Yahoo! eGroup] |
|||
[[Category:DMOZ| ]] |
|||
<!-- Interlanguage links --> |
|||
[[Category:1998 establishments in the United States]] |
|||
[[de:Open Directory Project]] |
|||
[[Category:AOL]] |
|||
[[eo:Projekto por Malferma Katalogo]] |
|||
[[Category:Collaboration]] |
|||
[[ia:Open Directory Project]] |
|||
[[Category:Internet properties established in 1998]] |
|||
[[sv:Open Directory Project]] |
|||
[[Category:Mozilla]] |
|||
[[Category:Netscape]] |
|||
[[Category:Open content projects]] |
|||
[[Category:Internet properties disestablished in 2017]] |
|||
[[Category:Multilingual websites]] |
|||
[[Category:Web directories]] |
Latest revision as of 12:02, 22 April 2025
![]() | This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)
|
![]() | |
Formerly |
|
---|---|
Type of site | Web directory |
Available in | 90 languages, including English |
Dissolved | March 17, 2017 |
Successor(s) | Curlie.org |
Parent | AOL |
URL | www.dmoz.org (Archived 2018-01-19 at the Wayback Machine) |
Commercial | No |
Registration | Optional |
Users | 90,000 |
Launched | June 5, 1998 |
Current status | Closed |
Content license | Creative Commons Attribution 3.0 Unported, Open Directory License |
DMOZ or DMoz (stylized dmoz in its logo; from directory.mozilla.org, an earlier domain name) was a multilingual open-content directory of World Wide Web links. The site and community who maintained it were also known as the Open Directory Project (ODP). It was owned by AOL (now a part of Yahoo! Inc) but constructed and maintained by a community of volunteer editors.
DMOZ used a hierarchical ontology scheme for organizing site listings. Listings on a similar topic were grouped into categories which then included smaller categories.
DMOZ closed on March 17, 2017, because AOL no longer wished to support the project.[1][2] The website became a single landing page on that day, with links to a static archive of DMOZ, and to the DMOZ discussion forum, where plans to rebrand and relaunch the directory were being discussed.[2]
As of September 2017[update], a non-editable mirror remained available at dmoztools.net,[3] and it was announced that while the DMOZ URL would not return, a successor version of the directory named Curlie would be provided.[4][5] DMOZ, ODP, and Curlie were considered synonymous by 2018.[6] Curlie was well established by 2022, using the hierarchy from DMOZ.[7]
History
[edit]This section needs additional citations for verification. (April 2019) |
DMOZ was founded in the United States as GnuHoo by Rich Skrenta and Bob Truel in 1998 while they were both working as engineers for Sun Microsystems. Chris Tolles, who worked at Sun Microsystems as the head of marketing for network security products, also signed on in 1998 as a co-founder of Gnuhoo along with co-founders Bryn Dole and Jeremy Wenokur. Skrenta had developed TASS, an ancestor of tin, the popular threaded Usenet newsreader for Unix systems. The original category structure of the Gnuhoo directory was based loosely on the structure of Usenet newsgroups then in existence.
The Gnuhoo directory went live on June 5, 1998. After Richard Stallman and the Free Software Foundation objected to the use of Gnu in the name, GnuHoo was changed to NewHoo.[8] Yahoo! then objected to the use of Hoo in the name, prompting a proposed name change to ZURL.[9] Prior to switching to ZURL, NewHoo was acquired by Netscape Communications Corporation in October 1998 and became the Open Directory Project. Netscape released Open Directory data under the Open Directory License. Netscape was acquired by AOL shortly thereafter and DMOZ was one of the assets included in the acquisition.

By the time Netscape assumed stewardship, the Open Directory Project had about 100,000 URLs indexed with contributions from about 4500 editors. On October 5, 1999, the number of URLs indexed by DMOZ reached one million. According to an unofficial estimate, the URLs in DMOZ numbered 1.6 million in April 2000, surpassing those in the Yahoo! Directory.[10] DMOZ achieved the milestones of indexing two million URLs on August 14, 2000, three million listings on November 18, 2001, and four million on December 3, 2003. As of April 2013 there were 5,169,995 sites listed in over 1,017,500 categories. On October 31, 2015, there were 3,996,412 sites listed in 1,026,706 categories.
In January 2006, DMOZ began publishing online reports to inform the public about the development of the project. The first report covered the year 2005. Monthly reports were issued subsequently until September 2006.[11] These reports gave greater insight into the functioning of the directory than the simplified statistics provided on the front page of the directory. The number of listings and categories cited on the front page included "Test" and "Bookmarks" categories but these were not included in the RDF dump offered to users. There were about 7330 active editors during August 2006.[11] 75,151 editors had contributed to the directory as of March 31, 2007.[12] As of April 2013, the number of contributing editors had increased to 97,584.[12]
System failure and editing outage, October to December 2006
[edit]On October 20, 2006, DMOZ's main server suffered a catastrophic failure[13] that prevented editors from working on the directory until December 18, 2006. During that period, an older build of the directory was visible to the public. On January 13, 2007, the Site Suggestion and Update Listings forms were again made available.[14] On January 26, 2007, weekly publication of RDF dumps resumed. To avoid future outages, the system resided on a redundant configuration of two Intel-based servers from then on.[15]
The site's interface was given an upgrade in 2016, branded "DMOZ 3.0", but AOL took it offline the following year.
Competing and spinoff projects
[edit]As DMOZ became more widely known, two other major web directories edited by volunteers and sponsored by Go.com and Zeal emerged, both now defunct. These directories did not license their content for open content distribution.[16][17]
The concept of using a large-scale community of editors to compile online content has been successfully applied to other types of projects. DMOZ's editing model directly inspired at least three other open content volunteer projects: music site MusicMoz, an open content restaurant directory known as ChefMoz[18] and an encyclopedia known as Open Site.[19] Finally, according to Larry Sanger, DMOZ was part of the inspiration for the Nupedia project, out of which Wikipedia grew.[20]
Logo history
[edit]-
Original logo as from June 1998
-
Logo after rebranding as NewHoo
-
Logo after acquisition by Netscape
-
Logo from 1999 to 2014 with the name DMOZ
-
Logo from March 2014
-
Logo from June 2016 until the closing in March 2017
-
Curlie logo (2019)
Content
[edit]Gnuhoo borrowed the basic outline for its initial ontology from Usenet. In 1998, Rich Skrenta said, "I took a long list of groups and hand-edited them into a hierarchy."[21] For example, the topic covered by the comp.ai.alife newsgroup was represented by the category Computers/AI/Artificial_Life. The original divisions were for Adult, Arts, Business, Computers, Games, Health, Home, News, Recreation, Reference, Regional, Science, Shopping, Society, Sports and "World". While these sixteen top-level categories have remained intact, the ontology of second- and lower-level categories has undergone a gradual evolution; significant changes are initiated by discussion among editors and then implemented when consensus had been reached.
In July 1998, the directory became multilingual with the addition of the World top-level category. The remainder of the directory lists only English language sites. By May 2005, seventy-five languages were represented. The growth rate of the non-English components of the directory had been greater than the English component since 2002. While the English component of the directory held almost 75% of the sites in 2003, the World level grew to over 1.5 million sites as of May 2005, forming roughly one-third of the directory. The ontology in non-English categories generally mirrors that of the English directory, although exceptions which reflect language differences are quite common.
Several of the top-level categories have unique characteristics. The Adult category is not present on the directory homepage but it is fully available in the RDF dump that DMOZ provides. While the bulk of the directory is categorized primarily by topic, the Regional category is categorized primarily by region. This has led many to view DMOZ as two parallel directories: Regional and Topical.
On November 14, 2000, a special directory within DMOZ was created for people under 18 years of age.[22] Key factors distinguishing this "Kids and Teens" area from the main directory are:
- stricter guidelines which limit the listing of sites to those which are targeted or "appropriate" for people under 18 years of age;[23]
- category names as well as site descriptions use vocabulary which is "age appropriate";
- age tags on each listing distinguish content appropriate for kids (age 12 and under), teens (13 to 15 years old) and mature teens (16 to 18 years old);
- Kids and Teens content is available as a separate RDF dump;
- editing permissions are such that the community is parallel to that of DMOZ.
By May 2005, this portion of DMOZ included over 32,000 site listings.
From early 2004, the whole site was in UTF-8 encoding. Prior to this, the encoding had been ISO 8859-1 for English language categories and a language-dependent character set for other languages. The RDF dumps were encoded in UTF-8 from early 2000.
Maintenance
[edit]Directory listings were maintained by editors. While some editors focused on the addition of new listings, others focused on maintaining the existing listings, and some did both. This included tasks such as the editing of individual listings to correct spelling and/or grammatical errors, as well as monitoring the status of linked sites. Still others went through site submissions to remove spam and duplicate submissions.
Robozilla was a Web crawler written to check the status of all sites listed in DMOZ. Periodically, Robozilla would flag sites which appeared to have moved or disappeared and editors follow up to check the sites and take action. This process was critical for the directory in striving to achieve one of its founding goals: to reduce the link rot in web directories. Shortly after each run, the sites marked with errors were automatically moved to the unreviewed queue where editors may investigate them when time permits.
Due to the popularity of DMOZ and its resulting impact on search engine rankings (See PageRank), domains with lapsed registration that were listed on DMOZ attracted domain hijacking, an issue that was addressed by regularly removing expired domains from the directory.
While corporate funding and staff for DMOZ diminished over time, volunteers created editing tools such as linkcheckers to supplement Robozilla, category crawlers, spellcheckers, search tools that directly sift a recent RDF dump, bookmarklets to help automate some editing functions, mozilla based add-ons,[24] and tools that helped work through unreviewed queues.
License and requirements
[edit]DMOZ data was previously made available under the terms of the Open Directory License, which required a specific DMOZ attribution table on every Web page that uses the data.
The Open Directory License also included a requirement that users of the data continually check DMOZ site for updates and discontinue use and distribution of the data or works derived from the data once an update occurs. This restriction prompted the Free Software Foundation to refer to the Open Directory License as a non-free documentation license, citing the right to redistribute a given version not being permanent and the requirement to check for changes to the license.
In 2011, DMOZ silently changed its license to a Creative Commons Attribution license[citation needed], which is a free license (and GPL compatible).[25]
RDF dumps
[edit]DMOZ data is made available through an RDF-like dump that is published on a download server, older versions are also archived there.[26] New versions are usually generated weekly. A DMOZ editor has catalogued a number of bugs that are encountered in the DMOZ RDF dump, most importantly that the file format is not RDF.[27] So while today the so-called RDF dump is valid XML, it is not valid RDF and as such, software to process the DMOZ RDF dump needs to be specifically written for DMOZ data.
Content users
[edit]DMOZ data powers the core directory services for many of the Web's largest search engines and portals, including Netscape Search, AOL Search, and Alexa. Google Directory used DMOZ information, until being shuttered in July 2011.[28]
Other uses are also made of DMOZ data. For example, in the spring of 2004 Overture announced a search service for third parties combining Yahoo! Directory search results with DMOZ titles, descriptions and category metadata. The search engine Gigablast announced on May 12, 2005, its searchable copy of DMOZ. The technology permits search of websites listed in specific categories, "in effect, instantly creating over 500,000 vertical search engines".[29]
As of 8 September 2006[update], DMOZ listed 313 English-language Web sites that use DMOZ data as well as 238 sites in other languages.[30] However, these figures do not reflect the full picture of use, as those sites that use DMOZ data without following the terms of the DMOZ license are not listed.
Policies and procedures
[edit]
Restrictions are imposed on who can become an DMOZ editor. The primary gatekeeping mechanism is an editor application process wherein editor candidates demonstrate their editing abilities, disclose affiliations that might pose a conflict of interest[citation needed], and otherwise give a sense of how the applicant would likely mesh with the DMOZ culture and mission.[31] A majority of applications are rejected but reapplying is allowed and sometimes encouraged. The same standards apply to editors of all categories and subcategories.[citation needed]
DMOZ's editing model is a hierarchical one. Upon becoming editors, individuals will generally have editing permissions in only a small category. Once they have demonstrated basic editing skills in compliance with the Editing Guidelines, they are welcome to apply for additional editing privileges in either a broader category or else another category in the directory. Mentorship relationships between editors are encouraged, and internal forums provide a vehicle for new editors to ask questions.[citation needed]
DMOZ has its own internal forums, the contents of which are intended only for editors to communicate with each other primarily about editing topics. Access to the forums requires an editor account and editors are expected to keep the contents of these forums private.[32]
Over time, senior editors can be granted additional privileges which reflect their editing experience and leadership within the editing community. The most straightforward is edit all privileges, which allow an editor to access all categories in the directory. Meta privileges additionally allow editors to perform tasks such as reviewing editor applications, setting category features, and handling external and internal abuse reports. Cateditall privileges are similar to edit all, but only for a single directory category. Similarly, catmod privileges are similar to meta, but only for a single directory category. Catmv privileges allow editors to make changes to directory ontology by moving or renaming categories. All of these privileges are granted by admins and staff, usually after discussion with meta editors.[citation needed]
In August 2004, a new level of privileges called admin was introduced. Administrator status was granted to a number of long serving metas by staff. Administrators have the ability to grant editall+ privileges to other editors and to approve new directory-wide policies, powers which had previously only been available to root (staff) editors.[33]
All DMOZ editors are expected to abide by DMOZ's Editing Guidelines. These guidelines describe editing basics: which types of sites may be listed and which may not; how site listings should be titled and described in a loosely consistent manner; conventions for the naming and building of categories; conflict of interest limitations on the editing of sites which the editor may own or otherwise be affiliated with; and a code of conduct within the community.[34] Editors who are found to have violated these guidelines may be contacted by staff or senior editors, have their editing permissions cut back, or lose their editing privileges entirely. DMOZ Guidelines are periodically revised after discussion in editor forums.[citation needed]
Controversy and criticism
[edit]There have long been allegations that volunteer DMOZ editors give favorable treatment to their own websites while concomitantly thwarting the good faith efforts of their competition.[35] Such allegations are fielded by ODP's staff and meta editors, who have the authority to take disciplinary action against volunteer editors who are suspected of engaging in abusive editing practices.[36] In 2003, DMOZ introduced a new Public Abuse Report System that allows members of the general public to report and track allegations of abusive editor conduct using an online form.[37] Uninhibited discussion of DMOZ's purported shortcomings has become more common on mainstream webmaster discussion forums. Although site policies suggest that an individual site should be submitted to only one category,[38] as of October 2007, Topix.com, a news aggregation site operated by DMOZ founder Rich Skrenta, had more than 17,000 listings.[39]
Early in the history of DMOZ, its staff gave representatives of selected companies, such as Rolling Stone or CNN, editing access in order to list individual pages from their websites.[40] Links to individual CNN articles were added until 2004, but were entirely removed from the directory in January 2008[41] due to the content being outdated and not considered worth the effort to maintain. There have been no similar experiments with the editing policy since then.
Ownership and management
[edit]Underlying some controversy surrounding DMOZ is its ownership and management. Some of the original GnuHoo volunteers felt that they had been deceived into joining a commercial enterprise.[8] To varying degrees, those complaints have continued up until the present.
At DMOZ's inception, there was little thought given to the idea of how DMOZ should be managed and there were no official forums, guidelines or FAQs.[42]
As time went on, the ODP editor forums became the de facto DMOZ parliament, and when one of DMOZ's staff members would post an opinion in the forums, it would be considered an official ruling.[32] Even so, DMOZ staff began to give trusted senior editors additional editing privileges, including the ability to approve new editor applications, which eventually led to a stratified hierarchy of duties and privileges among DMOZ editors, with DMOZ's paid staff having the final say regarding DMOZ's policies and procedures.[33][43]
Robert Keating, a principal of Touchstone Consulting Group in Washington, D.C., worked as AOL's program manager for DMOZ beginning in 2004. He started working for AOL in 1999 as senior editor for AOL Search, then as managing editor for AOL's DMOZ program, and then as media ecosystem manager of AOL Product Marketing.[44][45][failed verification]
Editor removal procedures
[edit]DMOZ's editor removal procedures are overseen by DMOZ's staff and meta-editors. According to DMOZ's official editorial guidelines, editors are removed for abusive editing practices or uncivil behaviour. Discussions that may result in disciplinary action against volunteer editors take place in a private forum which can only be accessed by DMOZ's staff and meta editors. Volunteer editors who are being discussed are not given notice that such proceedings are taking place.[43] Some people find this arrangement distasteful, wanting instead a discussion modeled more like a trial held in the U.S. judicial system.[46]
In the article "Editor Removal Explained", DMOZ meta editor Arlarson states that "a great deal of confusion about the removal of editors from DMOZ results from false or misleading statements by former editors".[47]
The DMOZ's confidentiality guidelines prohibit any current DMOZ editors in a position to know anything from discussing the reasons for specific editor removals,[43] however a list of potential reasons was provided in the guidelines.[48] In the past, this has led to removed DMOZ editors wondering why they cannot log in at DMOZ to perform their editing work.[49][50]
David F. Prenatt Jr., former DMOZ editor netesq, and another former editor known by the alias The Cunctator, both claim to have been removed for disagreeing with staff about changes to policies, particularly DMOZ's copyright policies. According to their claims, staff use the excuse of uncivil behaviour as a means to remove bothersome editors.[46][51][52]
Blacklisting allegations
[edit]Senior DMOZ editors have the ability to attach "warning" or "do not list" notes to individual domains but no editor has the unilateral ability to block certain sites from being listed. Sites with these notes might still be listed and at times notes are removed after some discussion.[53]
Hierarchical structure
[edit]Criticism of DMOZ's hierarchical structure emerged by around 2005. Many believe hierarchical directories are too complicated. With the emergence of Web 2.0, folksonomies began to appear, and some editors proposed that folksonomies, networks and directed graphs are more "natural" and easier to manage than hierarchies.[54][55][56]
Software
[edit]Search
[edit]The ODPSearch software is a derivative version of Isearch which is open-source, licensed under the Mozilla Public License.[57]
Editor forums
[edit]The ODP Editor Forums were originally run on software that was based on the proprietary Ultimate Bulletin Board system. In June 2003, they switched to the open source phpBB system. As of 2007, these forums were powered by a modified version of phpBB.
Bug tracking
[edit]The bug tracking software used by the ODP is Bugzilla and the web server Apache. Squid web proxy server was also used but it was removed in August 2007 when the storage servers were reorganized. All these applications are open source.
Interface
[edit]The DMOZ database/editing software is closed source (although Richard Skrenta has said in June 1998 that he was considering licensing it under the GNU General Public License). This has led to criticism from the aforementioned GNU project, many of whom also criticized the DMOZ content license. The content was later released under a Creative Commons license, which is compatible with the GNU license.[58]
As such, there have been some efforts to provide alternatives to DMOZ. These alternatives would allow communities of like-minded editors to set up and maintain their own open source/open content Web directories.
ChefMoz
[edit]Chef Moz, an offshoot of DMOZ, was an open content directory of World Wide Web links of restaurants. The website was constructed and maintained by a community of volunteer editors, and owned by Netscape.
Chef Moz, similar to its parent DMOZ, used a hierarchical ontology scheme for organizing site listings. Listings on a similar topic were grouped into categories, which can then include smaller categories.
On February 17, 2011, DMOZ administrator "lisagirl" confirmed that Chef Moz was dead.[59]
Growth
[edit]From its beginning in 2000 to November 2009 (when it became impossible for editors to log into the site), ChefMoz had grown to become the largest global directory of restaurants on the Internet. The total number of restaurants indexed since 2000 is recorded in the following table:[60]
Year | Number of restaurants |
---|---|
2000 | 48,000 |
2001 | 75,000 |
2002 | 179,000 |
2003 | 208,000 |
2004 | 258,000 |
2005 | 266,000 |
2006 | 275,000 |
2007 | 315,000 |
2008 | 325,000 |
See also
[edit]References
[edit]- ^ Sullivan, Danny (March 17, 2017). "DMOZ has officially closed after nearly 19 years of humans trying to organize the web". Search Engine Land. Archived from the original on July 17, 2017. Retrieved July 17, 2017.
- ^ a b "Why Dmoz Was Closed ?". Resource-Zone.com. April 16, 2017. Archived from the original on October 28, 2020. Retrieved April 29, 2017.
- ^ "The Directory of the Web – This site includes information formerly made available via DMOZ". dmoztools.net. Archived from the original on August 23, 2017. Retrieved August 21, 2017.
- ^ "New dmoz". Resource-Zone.com. May 5, 2017. Archived from the original on August 22, 2017. Retrieved August 21, 2017.
- ^ "Curlie: Present". curlie.org. March 29, 2017. Archived from the original on September 18, 2017. Retrieved September 17, 2017.
- ^ Han, Miyoung; Wuillemin, Pierre-Henri; Senellart, Pierre (2018). Mikkonen, Tommi; Klamma, Ralf; Hernández, Juan (eds.). Focused Crawling Through Reinforcement Learning. Web Engineering: 18th International Conference, ICWE 2018, Cáceres, Spain, June 5–8, 2018. Lecture Notes in Computer Science Information Systems and Applications, incl. Internet/Web, and HCI. Vol. 10845. Springer. p. 272. ISBN 9783319916620.
- ^ Wu, Dan; Dong, Jing; Liang, Shaobo (2022). Cross-device Web Search. Taylor & Francis. pp. 32–33. ISBN 9780429510342.
- ^ a b "The GnuHoo BooBoo". Slashdot. June 23, 1998. Archived from the original on July 27, 2009. Retrieved April 27, 2007.
- ^ "Zurl Directory" Archived December 23, 2007, at the Wayback Machine
- ^ ODP and Yahoo Size Charts Archived October 24, 2007, at the Wayback Machine by ODP editor geniac
- ^ a b ODP reports Archived March 10, 2008, at the Wayback Machine by ODP volunteer administrator chris2001
- ^ a b ODP Front Page Archived May 1, 2017, at the Wayback Machine. Retrieved August 15, 2006
- ^ "Dmoz's Catastrophic Server/Hardware Failure". dmozgrunt.blogspot.com. October 27, 2006. Archived from the original on July 30, 2017. Retrieved July 17, 2017.
- ^ dmoz.org technical problems archive.li Retrieved July 17, 2017.
- ^ The Hamsters' New Home Archived April 27, 2007, at the Wayback Machine, in: Open Directory newsletter issue Winter 2006. Retrieved December 26, 2006.
- ^ "Terms of Use" Archived February 2, 2002, at the Wayback Machine
- ^ "GO Network Terms of Service and Conditions of Use" Archived May 10, 2000, at the Wayback Machine
- ^ ChefMoz Fine Dining Menu Archived May 13, 2008, at the Wayback Machine, in: Open Directory newsletter issue Autumn 2003
- ^ help Archived June 29, 2008, at the Wayback Machine on open-site.org
- ^ Moody, Glyn (July 13, 2006). "This time, it'll be a Wikipedia written by experts". The Guardian. Archived from the original on December 14, 2013. Retrieved July 17, 2017.
- ^ Sullivan, Danny (July 1, 1998). "NewHoo: Yahoo Built By The Masses". Search Engine Watch. Archived from the original on April 29, 2018. Retrieved July 17, 2017.
- ^ Kids and Teens Launches! Archived February 21, 2008, at the Wayback Machine Open Directory Project Newsletter, November/December 2000
- ^ "Kids&Teens Guidelines". Dmoz.org. Archived from the original on July 16, 2014. Retrieved June 18, 2014.
- ^ "ODPExtension"[permanent dead link] Mozilla based add-on, ODP Magic. formerly known as ODP Extension
- ^ GNU Project: on the Creative Commons Attribution license Archived July 16, 2009, at the Wayback Machine
- ^ "Open Directory RDF Dump". Rdf.dmoz.org. Archived from the original on June 26, 2014. Retrieved June 18, 2014.
- ^ "ODP/dmoz Data Dump ToDo List". steevithak.com. Archived from the original on June 22, 2017. Retrieved July 17, 2017.
- ^ "Google Streamlining: Say Goodbye to the Google Directory and Labs!". Pandia Search Engine News. July 21, 2011. Archived from the original on July 25, 2011. Retrieved July 25, 2011.
- ^ "Gigablast Launches 500,000 Vertical Search Engines". Gigablast. May 12, 2005. Archived from the original on May 25, 2017. Retrieved July 17, 2017.
- ^ Category: Sites Using ODP Data Archived March 16, 2009, at the Wayback Machine on www.dmoz.org. Retrieved September 8, 2006.
- ^ "Become an Editor at the Open Directory Project". Dmoz.org. Archived from the original on August 27, 2014. Retrieved June 18, 2014.
- ^ a b "ODP Communication Guidelines". Dmoz.org. Archived from the original on July 22, 2014. Retrieved June 18, 2014.
- ^ a b "Open Directory Project Administrator Guidelines". Dmoz.org. Archived from the original on July 22, 2014. Retrieved June 18, 2014.
- ^ "ODP Directory Editorial Guidelines". Dmoz.org. Archived from the original on July 1, 2014. Retrieved June 18, 2014.
- ^ "How To: ODP Editor Is Competitor". webmasterworld.com. November 4, 2000. Archived from the original on July 30, 2017. Retrieved July 17, 2017.
- ^ ODP Meta Guidelines: Editor Abuse and Removal Archived January 23, 2009, at the Wayback Machine. Retrieved October 9, 2008.
- ^ "Open Directory Project: Public Abuse Report System". Report-abuse.dmoz.org. Archived from the original on August 7, 2010. Retrieved June 18, 2014.
- ^ "How to suggest a site to the Open Directory". Dmoz.org. Archived from the original on June 16, 2014. Retrieved June 18, 2014.
- ^ Open Directory Project Search: "topix"[permanent dead link] (Retrieved October 18, 2007)
- ^ "Multiple URL's in DMOZ". webmasterworld.com. January 30, 2003. Archived from the original on May 25, 2017. Retrieved July 17, 2017.
- ^ "DMOZ news". Archived from the original on June 6, 2009. Retrieved June 7, 2009.
- ^ "The Open Directory Project: The Spirit of the Web". laisha.com. Archived from the original on May 16, 2008. Retrieved July 17, 2017.
- ^ a b c "Open Directory Project Meta Guidelines". dmoz.org. December 31, 2013. Archived from the original on March 4, 2017. Retrieved July 21, 2017.
- ^ Meet AOL's DMOZ Staff Team Archived January 1, 2012, at the Wayback Machine, DMOZ Blog, January 8, 2009
- ^ Robert Keating, LinkedIn, archived from the original on December 24, 2010, retrieved December 17, 2011
- ^ a b Prenatt, David (May 29, 2000). "Life After ODP". Yahoo! Groups. Archived from the original on May 25, 2017. Retrieved July 17, 2017.
- ^ Arlarson, Editor Removal Explained Archived December 2, 2008, at the Wayback Machine, Open Directory Project Newsletter (September 2000).
- ^ "Guidelines: Account Removal". dmoz.org. July 3, 2016. Archived from the original on March 12, 2017. Retrieved July 21, 2017. Alt URL Archived May 26, 2017, at the Wayback Machine
- ^ "Editor account expired". resource-zone.com. July 27, 2004. Archived from the original on September 29, 2017. Retrieved July 17, 2017.
- ^ Thread: Can't Login Archived November 28, 2008, at the Wayback Machine on Resource-Zone
- ^ Prenatt, David F. Jr. (June 1, 2000). "Life After the Open Directory Project". traffick.com. Archived from the original on May 25, 2017. Retrieved July 17, 2017.
- ^ CmdrTaco (October 24, 2000). "Dmoz (aka AOL) Changing Guidelines In Sketchy Way". Slashdot. Archived from the original on February 6, 2010. Retrieved July 17, 2017.
- ^ Add Note to URL Feature Archived May 13, 2008, at the Wayback Machine, in ODP Documentation
- ^ Hriţcu, C. (April 8, 2005). "Folksonomies vs. Ontologies". hritcu.wordpress.com. Archived from the original on July 31, 2017. Retrieved July 17, 2017.
- ^ "Ontology is Overrated" Archived July 29, 2013, at the Wayback Machine
- ^ Hammond, Tony; Hannay, Timo; Lund, Ben; Scott, Joanna (April 2005). "Social Bookmarking Tools (I)". D-Lib Magazine. Archived from the original on June 2, 2018. Retrieved July 17, 2017.
- ^ "Open Directory Search Guide". Dmoz.org. Archived from the original on June 16, 2014. Retrieved June 18, 2014.
- ^ "Licenses for Works of Practical Use besides Software and Documentation". GNU Project. Archived from the original on July 16, 2009. Retrieved July 17, 2017.
- ^ "Announcement of Chef Moz's Death". Archived from the original on August 7, 2011. Retrieved May 19, 2021.
- ^ Statistics from the Wayback Machine
External links
[edit]Curlie ID (P998) (see uses)
- dmoz.org — official website at the Wayback Machine (archived March 14, 2017)