The WHOIS Pilot Study Report published today for public comment describes the results of a pilot accuracy study conducted by NORC at the University of Chicago. The Report illustrates the findings and methodology to be deployed in the WHOIS Accuracy Reporting System (ARS) under development by ICANN. By publishing it for public comment, ICANN is seeking feedback on the approach, methodology, and reporting formats used in the Pilot Study to help shape the final design of the ARS.
Section I: Description and Explanation
The WHOIS Pilot Study Report published today for public comment describes the results of a pilot accuracy study conducted by NORC at the University of Chicago in collaboration with private and public sector validation specialists. The Report illustrates the findings and methodology to be deployed in the WHOIS Accuracy Reporting System (ARS) under development by ICANN.
The Pilot Accuracy Study was a precursor to the ARS, designed with the intention of isolating and testing key components of the system. Results from the Pilot Study will be used to inform the design of the ARS.
The Pilot Study included an in-depth examination of postal addresses, email addresses and telephone numbers. Postal address statistics were developed with guidance from the Universal Postal Union (Switzerland), a specialized agency of the UN that coordinates postal policies worldwide for its member countries. Other validation expertise was provided by leading commercial firms, including StrikeIron (USA), utilizing its proprietary email validation systems, DigiCert (USA), a provider of digital certificates and telephone validation services, and aided by a unique data parsing service by Whibse (USA).
The sample sizes used in the Pilot Study is described in the chart below:
|Data Element||Syntactic Validation||Operational Validation|
The Pilot study examined accuracy rates from multiple perspectives to give a realistic picture of today's WHOIS. Some notable findings are:
- Operationally, registrars under the 2013 RAA have more accuracy for email addresses than registrars under the 2009 RAA. Registrars under the 2009 RAA have more operational accuracy in telephone numbers while the two groups are similar on postal addresses.
- Email and telephone syntactical accuracy is not statistically different for prior and new gTLDs while syntactical accuracy is better for Prior gTLDs on postal addresses.
- The operational numbers for email addresses indicate that New gTLDs have slightly better accuracy than Prior gTLDs. Prior gTLDs do have more operational accuracy on telephone numbers, but the two groups are equal on operational postal address accuracy.
- The differences in syntactical accuracy between registrars under the 2013 RAA and registrars under the 2009 RAA for email, telephone numbers and postal addresses are not statistically significant.
Specific Feedback Sought
The Pilot Report [PDF, 1.5 MB] published today reflects the input and suggestions received on the Preliminary Findings Paper [PDF, 668 KB] at the Los Angeles Meeting and from consultations with the Registrar Stakeholder Group.
The purpose of the Pilot Study is to test assumptions using real data. The methodology can be adjusted based upon the public comments received on the Pilot Report [PDF, 1.5 MB]. During the Public Comment, ICANN is seeking input on:
- Sample Design & Methodology
- Types of Accuracy Reports to be published through the ARS
- Whether ICANN should conduct Identity Validation in subsequent phases of the ARS Development
- Whether the methodology should treat registrations under privacy or proxy services differently, and if so, how
- Any other aspect of the ARS
Process for Correcting Inaccurate Records: ARS Implementation Advisory Group
A key function of the ARS will be to forward records identified as potentially inaccurate to registrars for follow-up to confirm their accuracy. The ARS is being designed to track and report on the progress of these records.
Engagement with registrars and other interested stakeholders is necessary to define an efficient process for transmitting, reviewing, and updating, as appropriate, the identified WHOIS records. ICANN plans to work with registrars and the broader Community in the months ahead in order to develop this process.
Compliance Pilot Initiated
ICANN will kick-off a Compliance Pilot in January 2015, to examine the results of the Pilot Study to determine if a compliance response is appropriate for the WHOIS records that have been categorized as inaccurate from a syntactical perspective. ICANN's Contractual Compliance Department is in the process of auditing the results of the Pilot Study as part of a compliance pilot to determine whether a compliance follow-up is needed.
Section II: Background
In response to the recommendations of the WHOIS Review Team, the ICANN Board approved a series of improvements on the manner in which ICANN carries out its oversight of the WHOIS program. The Review Team's Final Report called for ICANN to publish ongoing statistics on WHOIS accuracy. As part of this effort, ICANN committed to proactively identify inaccurate WHOIS records and forward them to registrars for investigation and follow-up.
In addition, the Government Advisory Committee (GAC) has issued a series of correspondences calling for ICANN to adopt specific safeguards related to WHOIS as part of the New gTLD Program. The GAC's Los Angeles Communiqué [PDF, 127 KB] has asked for an assessment of the feasibility, costs and benefits of conducting identity validation as part of the development of the ARS.
ICANN initiated the development of the WHOIS Accuracy Reporting System (ARS) – a framework for conducting repeatable assessments of WHOIS accuracy over time and publicly reporting the findings. ICANN turned to NORC at the University of Chicago to design, work with validation providers, and conduct the analysis necessary to produce and deliver this analysis.
Section III: Relevant Resources
Section IV: Additional Information
- NORC WHOIS Accuracy Pilot Report [PDF, 1.5 MB] is available here
- Next Steps for the WHOIS Accuracy Reporting System – Appendix B [PDF, 196 KB].
- Additional Information about the latest WHOIS developments are available in the Annual Report on WHOIS Improvements [PDF, 1.11 MB]
- For more details on the services sought by ICANN, please see the WHOIS Accuracy Reporting System Request For Proposal (RFP) Announcement.
- For additional background information, please see the Draft Implementation Plan for the WHOIS Accuracy Reporting System Announcement.
- The GAC's Communiqués and correspondence related to WHOIS are posted here.
- To learn about WHOIS, visit the WHOIS website.