Guidelines for Developing Reference Label Generation Rulesets (LGRs) for the Second Level

Purpose: To facilitate and improve consistency of testing and stability of registry operations of new gTLDs, ICANN is developing reference Internationalized Domain Name (IDN) tables in machine readable format, called Label Generation Rulesets (LGRs) for the second level, for use in Pre-Delegation Testing (PDT) and the Registry Service Evaluation Process (RSEP). Based on the process finalized after public comment, as the first step ICANN is releasing the Guidelines for Developing Reference LGRs for the Second Level [PDF, 544 KB]. The community is requested to review and provide feedback on the effectiveness of these guidelines for the development of the reference second level LGRs, and propose any further improvements.

Current Status: To address immediate concerns of PDT The Internet Foundation of Sweden (IIS), who perform the PDT of New gTLDs for ICANN, has developed a set of baseline IDN tables for several languages. IIS has made these tables, and the guidelines used to develop them, available at their GITHUB webpage with an open license. The community can use these tables as a baseline for their own development work or as reference documents for PDT and on-going registry operations.

Next Steps: Based on the Guidelines finalized, ICANN will continue to develop the IDN tables and post these for public comment. The tables will be published in two batches.

The LGRs are to be developed for the languages given below, organized in two batches and prioritized on the basis of complexity and demand. Division in batches allows Batch 1 languages to be released sooner and halves the number of LGRs to review at a time by the community. Additional languages will be added later, as needed.

1. Batch 1: Japanese, Korean, Chinese, Danish, Norwegian, Latvian, Lithuanian, Russian, Arabic, Ukrainian, Belarusian, Bulgarian, Macedonian, Bosnian (in both Cyrillic and Latin scripts), Serbian, Hebrew
2. Batch 2: English, Spanish, French, German, Portuguese, Polish, Swedish, Italian, Hungarian, Icelandic, Finnish, Montenegrin

The following steps provide the process to undertake the work, as finalized after the public comment:

1. Development of Overall Guidelines
   1. As the first step, the provider will create a detailed set of guidelines and process [PDF, 544 KB] to undertake the work.
   2. Once the guidelines and process are signed off with ICANN after public comments, the provider will proceed with the creation and verification of the reference IDN tables. It is recommended that the work done by IIS already should be used as the baseline.
2. Analysis and Documentation
   1. For each language, the authoritative sources will be gathered and analyzed. These include national and international standards, published dictionaries and other sources identified in the guidelines.
   2. Based on the analysis of these authoritative sources, review of other data (e.g. IDN tables published by IANA and IIS, informational RFCs, etc.), the reference LGRs suitable for second level will be created in the machine readable format defined in https://tools.ietf.org/html/draft-davies-idntables (the XML Format). If authoritative sources are not available, a more rigorous creation process would need to be instituted, to be specified in the guidelines.
   3. A concise document is created along with each language LGR, listing the authoritative and other sources used, process followed (especially if authoritative sources are not available) and summarizing the analysis and conclusions. The document should either confirm that no deviation is needed or justify any suggested variance from the authoritative sources. Any allowable variations based on contexts, if any (e.g. difference in code points in a language across different regions), will also be documented.
3. Expert Review
   1. As an independent follow up step, the reference LGR and the associated documentation are reviewed by linguistic expert(s) of that language and script, who confirm that the documentation and contents of the LGR for the second level are adequate and complete. A separate linguistic expert review report for each LGR is created at this stage.
   2. As an independent follow up step, the reference LGR and the associated documentation are reviewed by relevant technical DNS and IDN expert(s), who confirm that the proposed documentation and contents of the LGR provided for the second level adequately addresses any security and stability concerns. A separate technical expert review report for each LGR is created at this stage.
4. Public Comments
   1. The reference language based LGRs, the associated documentation and the related language and security and stability expert reports for each language will be released for public comments by ICANN, separately for the two batches.
   2. The public comments will be considered and the reference LGRs and their associated documentation will be updated based on the feedback received.
5. Finalization and Publication
   1. The final set of LGRs and associated documents will be released by ICANN as a reference for PDT and RSEP with open license for use by the community.
   2. Once published, community may report issues in these tables by emailing to IDNProgram@icann.org. These issues will be collated and reviewed and the LGRs will be updated as needed.

The registries are generally encouraged to collaborate in defining common language based or script based tables to allow for consistency for end users. There are multiple formats to submit IDN tables, and applicants can also arbitrarily use their own format. The IDN tables used by each gTLD and some ccTLDs are posted at the IANA Repository for IDN Practices. During the new gTLD program's Pre-Delegation Testing (PDT), ICANN has noted a large number of IDN table submissions.

• The IANA Repository for IDN Practices: https://www.iana.org/domains/idn-tables
• The machine readable format for Label Generation Rulesets: https://tools.ietf.org/html/draft-davies-idntables
• IDN tables and associated information made available by IIS: https://github.com/dotse/IDN-ref-tables
• Public comments on the process being used to develop these tables: https://www.icann.org/public-comments/lgr-second-level-2015-03-10-en

• Report