Initial Report on Data & Metrics for Policy Making

Purpose: This public comment proceeding seeks to obtain community input on the Initial Report from the GNSO's Working Group (WG) on possible recommendations for the use of Data and Metrics for Policy Making.  The WG's intent is to develop solutions that will improve the way in which consensus policies are developed and help ensure that the most critical registrant, registry, and registrar issues are addressed by evolving the culture to better informed, fact-based policy development and decision-making.

Next Steps: After close of the public comment period, the WG will review the submitted comments and update the report as agreed upon, thus producing the Final Report for submission to the GNSO Council.

The Data & Metrics for Policy Making Working Group (WG) is exploring opportunities to review standard methodologies of reporting and metrics that could better inform fact-based policy development and decision making; including how the community can collaborate with Contracted Parties and other service providers in the sharing of metrics and data. 

The WG's charter contains eight specific questions for the WG, derived from the Final Issue Report that was published in March 2013. The Charter questions cover issues including:

• The question “which comes first, policy-process or definitive data describing the problem?” along with suggestions as to how data can be gathered when it has not yet been included in the reporting process.
• How processes can be continuously improved, simplified and made more consistent for people wishing to either report a problem or learn about their options when their problem falls outside ICANN policy;
• Principles that enhance metrics and data available to better inform the GNSO policy development process;
• Improved understanding of the limits of ICANN policies regarding data measurement and tracking and other options to pursue if an issue is not covered by policies that gather data
• Mechanisms whereby GNSO working groups can request information (both internal to ICANN or external, including GNSO contracted parties) which support fact-based policy-making;
• Mechanisms to ensure appropriate safeguards with regard to the confidentiality of certain types of information;
• A framework for distributing information to the GNSO policy-making community with the intent of both informing those groups and providing the ongoing basis for identifying and correcting problem-reporting and data-collection problems;
• Any changes needed to incorporate the processes described above into the ongoing Policy Development Process.

Opportunities discovered by the group include:

• Improve the way in which Consensus Policies are developed and help ensure that the most critical registrant, registry, and registrar issues are addressed
• Evolve culture to better informed, fact-based policy development and decision making
• Base deliberations and decisions on tangible evidence as opposed to “gut feeling” or anecdotal evidence
• Instill notion of continuous improvement to the policy process and effectiveness consensus policy implementations

Key Definitions:

• Data: Individual facts, a set of values, statistics, or items of information
• Metrics: A set of measurements that help quantify results, which allows for better determination of the level of success against a set of goals

The WG’s preliminary recommendations include:

• Initiation of a pilot effort whereby the GNSO community at the early stages of the policy process (Issue Identification or Working Groups) can submit tactical sized requests for data and metrics to assist in validation of issues or to better inform policy deliberations
• Creation of WG work product templates (Charter, Final Report) including updates to the GNSO’s Operating Procedures and Working Group Guidelines (WGG)
• Update to the Charter template to assist drafting teams in determining key metrics to be used to meet the goals of resolving issues
• Updates to the Final Report template with a standard but modifiable recommendation forcing a review of the policies implemented at a future date to determine if it met the stated intent
• Creation of a Metrics Request Decision Tree and Request Form to formalize the request process; including inclusion into the WGG.

Following analysis of all public comments received, the WG will finalize its recommendations and prepare a Final Report for delivery to the GNSO Council, for its review and action.

The 2010 Registration Abuse Policies Working Group (RAPWG) identified the Meta Issue: Uniformity of Reporting which it described as "need for more uniformity in the mechanisms to initiate, track, and analyze policy-violation reports."  The RAPWG recommended in its Final Report that "the GNSO and the larger ICANN community in general, create and support uniform problem-reporting and report-tracking processes."

The GNSO Council recommended the creation of an Issue Report to further research metrics and reporting needs in hopes to improve the policy development process.  The report created by ICANN staff outlined accomplishments regarding reporting and metrics by the Contractual Compliance function and it also reviewed other reporting sources that may be of relevance.  The GNSO Council subsequently adopted the recommendation to form this non-PDP Working Group tasked with exploring opportunities for developing reporting and metrics processes and/or appropriate standardized methodologies that could better inform fact-based policy development and decision making.

The WG has developed preliminary recommendations for a number of the charter questions it was tasked to address, which are described in its Initial Report [PDF, 395 KB].  Those recommendations that contain suggested edits to the GNSO's Operating Procedures are also included in a redline version [PDF, 768 KB] should the WG's recommendations be adopted by the GNSO Council.

• Working Group Initial Report: https://www.icann.org/en/system/files/files/report-gnso-data-metrics-policy-making-initial-29jul15-en.pdf
• Redline of GNSO Operating Procedures: https://www.icann.org/en/system/files/files/gnso-operating-procedures-redline-29jul15-en.pdf
• Working Group Charter: http://gnso.icann.org/en/issues/dmpm-charter-23jan14-en.pdf
• DMPM Project Page: http://gnso.icann.org/en/group-activities/active/dmpm
• DMPM Wiki: https://community.icann.org/display/marwg

Note that a few of the WG’s recommendations suggest updates to the GNSO’s Operating Guidelines; specifically Annex 1 "GNSO Working Group Guidelines" and Annex 2 "Policy Development Process Manual”. The recommendations within the Initial Report include the proposed changes or creation of new material. A redline version of GNSO’s Operating Guidelines are included to highlight the possible updates to this document should the GNSO Council adopt the WG’s recommendations.

• Report