Assessment of ICANN Organization Evaluation of Information Acquisition Mechanisms for ICANN org Performance and Accountability to Stakeholders Implementation of Work Stream 2, Recommendation 7.2.1

ICANN org Complaints Office

The Complaints Office is a function within the ICANN org that:

• Provides a centralized location to submit complaints related to the ICANN org.
• Receives complaints, researches them, collects facts, reviews, analyzes, and resolves issues as openly as possible.
• Helps the ICANN org build on its effectiveness, and contributes to increased transparency from the Org.
• Aggregates the data from complaints to identify and solve for operational trends that should be improved.

The Complaints Office handles complaints regarding the ICANN org that don't fall into an existing complaints mechanism, such as Contractual Compliance, Request for Reconsideration and the Ombudsman. This may include complaints about how a request has been handled, a process that appears to be broken, insufficient handling of an issue, or something that may be an indication of a systemic issue, among other things. To learn more about the kinds of complaints the Complaints Office handles, please visit the Complaints Office Report page.

The Complaints Office reviews verifiable information to ensure recommendations and resolutions are based in fact. It strives to be open and transparent, responsive and accountable to all parties, and to make recommendations that are constructive and actionable. Above all else, the Complaints Office acts with the utmost integrity in service of ICANN org's mission.

Anyone wishing to submit a complaint related to the ICANN org may do so by completing a web-form. Out of scope submissions are forwarded and responded to by ICANN org's Global Support Center team who has expertise in navigating the entire ICANN org and will get the submitter the information and/or assistance they are looking for. Submissions that fall within the scope of the Complaints Office, will be handled in the following stages:

1. Receive, Acknowledge and Publish
2. Evaluate and Consider
3. Response Drafting
4. Response Issued and Published

The Complaints Officer provides the complaint with the response and the response is published on the Complaints Office Report Page.

Internet Assigned Numbers Authority (IANA) Functions Customer Engagement Survey. (Supported by ICANN org's IANA function).

The annual engagement survey is conducted to evaluate IANA's performance in engaging with customers and stakeholders. The results help ICANN org understand if it needs to change the way it engages with our stakeholders, improve the tools it uses to collaborate with customers, and identify topics of interest for future engagement strategies. The team discusses the results with the groups in a segmented way, focusing on trends and areas of concern so it can do a deep dive on outliers. Survey results are also used as input to our strategic planning efforts. In addition, having a survey is a contractual mandate in the IANA Naming Functions Contract, the Service Level Agreement for the IANA Numbering Services.

This survey is conducted on an annual basis with many stakeholders, both representations of community leadership, as well as direct customers of the IANA functions. Using the 2021 survey as an example, respondents were asked to rate the IANA functions' performance according to 22 key statements relating to credibility, transparency, attentiveness, fairness, timeliness, and accountability. Each statement was rated on a 1-5 scale. The survey is anonymous and administered by a third party vendor. It has followed this methodology since 2019. Prior to 2019 (2013-2018), the survey was focused on customer satisfaction with our request processing. The approach was changed once the post-Interaction survey was launched. The findings of the survey are published. Here is an example of The 2021 IANA Survey.

"How Did We Do?" post-interaction survey. (Supported by ICANN org's IANA function).

This survey is conducted to measure customer perception of ICANN org's request processing standards and evaluate satisfaction with ICANN org's customer service delivery. The team uses the feedback from these surveys to evaluate current practices and, when needed, change existing processes and plan for system enhancements. In addition, having a survey is a contractual clause in the IANA Naming Functions Contract, the Service Level Agreement for the IANA Numbering Services.

Once a request is processed and resolved, the requestor receives a one question survey: How did we do? The choices are "satisfied" or "dissatisfied" with an option to describe their reasons. The tool was developed in-house and it is linked to our ticketing system. IANA's continuous improvement team manages the responses, reports and tracks against our internal Service Level Agreement. Satisfaction rates are published on the iana.org website and also included in the CEO Quarterly report. Everyone who submits a request using our ticketing system receives a survey at the completion of their request (the same requestor email will not get more than one survey within 60 days).

The high-level monthly aggregate satisfaction and response rate from this survey is published, see example here. The specific answers and feedback from the community is circulated amongst staff for individual attention.

Periodic Meetings with Oversight groups (Customer Standing Committee (CSC), Internet Engineering Task Force (IETF) Leadership, Trusted Community Representatives (TCR)). (Supported by ICANN org's IANA function).

These periodic meetings with oversight groups provide the opportunity for members of the community to assess performance results against ICANN org's contractual obligations, Service Level Agreements (SLAs), and ask questions on exceptions. The information is collected for continuous Improvement purposes and to ensure compliance with contractual obligations.

During these meetings, feedback is gathered from these community groups on priorities that need to be set when developing Strategic and Operating Plan & Budgets. During these meetings, status reports on key projects of interest are also provided to each group. CSC meetings happen monthly, IETF-IANA leadership meetings happen three times per year. TCR meetings happen after each Root Key Signing Key (KSK) Ceremony and the goal is for them to debrief on the ceremony, provide feedback to the team and discuss the root cause of any documented exception.

IANA Naming Functions Review. (Supported by ICANN org's IANA function).

The IANA Naming Function Review is one of the new accountability mechanisms created as part of the IANA stewardship transition to ensure that ICANN org via its affiliate Public Technical Identifiers (PTI) meets the needs and expectations of its naming customers.

The IANA Naming Function Review (IFR) is conducted by a team comprised of members from the global stakeholder community. In this way, the review represents an information acquisition mechanism for gathering input from the community because the review team itself is the community. Beyond a survey or questionnaire, the review provides a deep level of analysis and feedback from the community on the Public Technical Identifiers' (PTIs') performance of the IANA naming function against the contractual requirements in the IANA Naming Function Contract and the IANA Naming Function Statement of Work (SOW). To learn more about the different methodologies used by the community in conducting this important review, please see the 2021 IFR Final Report (p. 12, Scope and Review findings).

IANA Numbering Services Review Committee (IANA RC). (Supported by ICANN's IANA function).

The IANA Numbering Services Review Committees (RC) function is to advise and assist the Number Resource Organization (NRO) EC in its periodic review of the service level of the IANA Numbering Services provided to the Internet Number Community. The IANA Numbering Services Review Committee serves as a tool for the Internet Number Community to evaluate and review the performance of the IANA Numbering Services provided. The Review Committee ensures community involvement and supports and enhances the multistakeholder model in a transparent, open, and bottom up process to ensure that the number resources component of the IANA operations meets the needs and expectations of its customers, namely the Internet numbers community.

The IANA Numbering Services RC is composed of qualified representatives from each RIR region (see below) and each region has equal representation. The IANA RC publishes the IANA Performance Matrix Summary Report during the first quarter of each year. The report pertains to the numbering services that IANA provided to the Numbers community during the previous year. When the IANA Performance Matrix Summary Report is published, a 30-day comment period begins and the five Regional Internet Registry (RIR) communities are invited to provide input. At the end of the comment period, the IANA RC reviews and summarizes any input received and compiles its final report. Please find an example of the IANA RC Report 2021.

ICANN Strategic Planning Process (Supported by ICANN org's Planning function).

The purpose of strategic planning is to set the long-term strategic objectives for the organization, reflecting ICANN's mission and vision. Strategic planning is a core element of ICANN's three-fold planning process cycle, namely the Strategic Plan, 5-year Operating Plan, and the Annual Operating Plan and Budget process. Strategic planning helps answer two very important questions: where are we now? And where do we want to be?

Throughout the strategic planning process, there are mechanisms to gather input from the community to allow the ICANN org to better ascertain its overall performance and accountability to relevant stakeholders. As a first step, the community, Board, and ICANN org participate in an annual strategic outlook program through trend identification sessions to discuss emerging trends that could affect ICANN. The session output is analyzed and used to develop the annual trend impact assessment, a process to evaluate each proposed trend against the targeted outcomes and risks in the 5-year Strategic Plan, to determine if any action should be taken based on the trend's impact to ICANN. Significant shifts in trends could result in appropriate adjustments to the strategic plan, following the strategic planning process. The strategic plan is drafted and finalized every five years unless a significant new trend or significant shift in an existing trend requires an amendment. The ICANN org revises the draft strategic plan based on community input and the ICANN Board finalizes and adopts the new strategic plan.

Information is shared via webinars, public sessions at ICANN meetings, and public comment opportunities to provide input into the strategic planning process.

The information gathered from the community through the strategic outlook trend identification sessions, webinars, public sessions at ICANN meetings, and public comment opportunities is integrated into each planning cycle and process.

Please see this example of Public Comments for the FY23–27 Operating & Financial Plan and FY23 Operating Plan & Budget

ICANN org Finance and Planning community input is collected via public comments, webinars, and face-to-face meetings. (Supported by ICANN org's Finance and Planning function).

ICANN org Finance and Planning collects input from the community on its performance via public comments, webinars, and face-to-face meetings.This information is used to continuously improve how ICANN org reports on its finance and planning work, and at times, how financial and operating plans are developed.

ICANN org's Finance and Planning functions meet with various SOs and ACs at each ICANN meeting (as a general cadence). In this effort ICANN org Finance and Planning functions also seek to hold Public Comment Periods on an annual basis, and they participate within some working groups within the SOs and ACs.

Survey of members of Specific Review teams. (Supported by ICANN org's Reviews Support and Accountability function).

This survey is conducted with community members following the completion of a Specific Review to gather feedback from the community on ICANN org's performance in supporting community-led Specific Review Teams. This feedback will be considered as part of the Lifecycle of Reviews, to inform further improvements to how specific reviews are conducted. Moving forward this survey will be held at the conclusion of each Specific Review.

This information is acquired through the use of a survey that uses a combination of quantitative and qualitative survey questions to receive input from the community in order to gauge the effectiveness of each review as well as ICANN org's performance, and opportunities for overall improvement. Please see this example of the survey results following the Competition, Consumer Trust, and Consumer Choice Review (CCT).

Survey of community Working Party members supporting the work of Organizational Reviews. Supported by ICANN org's Reviews Support and Accountability function).

The survey is conducted with the community to gather feedback after the Review work has concluded. Feedback from the community is incorporated by ICANN org to improve the way it supports reviews and community working parties. Additionally, this feedback will be considered as part of the Lifecycle of Reviews, to inform further improvements to how organizational reviews are conducted, and how they eventually evolve into a continuous improvement program (recommended by the ATRT3).

This information is acquired through the use of a survey that uses a combination of quantitative and qualitative survey questions to receive input from the community in order to gauge the effectiveness of each review as well as ICANN org's performance, and opportunities for overall improvement. Please see this example of the survey results following the GNSO2 Organizational Review.

Closure Surveys that occur after Global Support Inquiry. Supported by ICANN org's Global Support function).

ICANN org uses closure surveys after Global Support inquiries to gather community input on their perception of the performance that Global Support provides. In some cases, information gained can indicate areas for improvement such as the content of a response.

ICANN org sends a survey to all individuals who receive assistance from Global Support. The number of individuals who receive assistance varies greatly over time as does the rate at which individuals respond to the survey.

Contracted Parties Annual Survey. (Supported by ICANN org's Global Accounts and Services function).

ICANN org uses the Annual Contracted Parties Survey to solicit feedback regarding its performance in serving Contracted Parties and to uncover opportunities for improvement.

ICANN org conducts this Annual Survey with Contracted Parties. The 2021 Survey, for example, was made up of 9 questions and requested just one response from each contracted party's organization. This survey started in 2017 and was quite extensive. For several years staff attempted to compare results year-over-year. After a lower participant rate in 2020, the survey was shortened to just 9 questions which resulted in a higher participation rate. Please see this example of the 2021 Contracted Parties Survey Results.

GDD Industry Summit Post Event Survey. (Supported by ICANN org's Global Accounts and Services function).

ICANN org conducts a survey following the GDD Industry Summit to collect feedback from event attendees to understand what worked well and what could be improved at the event. The survey seeks input from the community on, for example, session topic, length, content, and event location and facilities. The survey also seeks input from contracted parties that did not attend the event to inquire why they did not participate. ICANN org uses the results from this survey to analyze the success of the GDD Industry Summit in order to identify opportunities for improvement. This survey has been a very successful tool for ICANN org to receive input from the community and it will be sent again once the next GDD Industry Summit is held.

ICANN org sends this survey to Contracted Parties on the closing day of the GDD Summit. Please see this example of findings from the 2019 GDD Summit.

Post ICANN Meeting Survey. (Supported by ICANN's Meetings function).

The objective of the survey is to gather feedback after an ICANN meeting has concluded. Feedback from the community is incorporated by ICANN org to improve the way it conducts meetings, or modifies the participants' experience. Additionally, this feedback is considered as part of the decision making process for big changes, to inform further improvements to meetings and events. This process also includes consultation with ICANN Community Leaders during monthly ICANN Meeting planning calls.

This information is acquired through the use of a survey that uses a combination of quantitative and qualitative survey questions. This survey generally rates satisfaction with ICANN org's performance, and opportunities for overall improvement. Currently, surveys are only conducted when ICANN org is actively seeking feedback on ICANN meetings and events. For many years, ICANN org conducted surveys after every ICANN meeting, but discontinued because response rates were very low. ICANN org modified its strategy to be more targeted with its surveys by conducting them when the org is actively seeking community feedback on targeted topics. This change has improved community feedback drastically, and made it more clear in understanding what the community would like to see updated or changed. Please see this example of the "ICANN68 and By the Numbers and Survey Report".

Regular engagement (meetings and surveys) between community leadership and support staff. (Supported by ICANN org's Policy function).

ICANN org facilitates staff-community leadership meetings where feedback from the community on ICANN org's performance is often immediate and orally provided or exchanged via email. This allows for timely remediation (if appropriate) or escalation to management (again if appropriate) as well as for open and frank dialogue with the community leaders, to identify the source and reasons for any concerns or issues. The Policy function also conducts many surveys, within the context of progressing policy development and advice work – these include surveys of Council members, group leaders, policy development process (PDP) participants, the community as a whole, etc.

ICANN org has support teams in place for every SO, AC, RALO, SG, constituency and formal community structure. These support teams meet regularly (often weekly) with various leadership teams in each structure, including individual working groups and committees. While there is no formal survey or fixed format or timing, these prep/debrief meetings provide regular information acquisition mechanisms for the community to alert staff about actual or potential issues or concerns on a regular basis.

Post briefing survey. (Supported by ICANN org Government Engagement function).

ICANN org uses surveys to gather input from the community on the performance of the Governmental Advisory Committee (GAC) briefings it provides. The input that ICANN org receives from the GAC community is used to improve briefings and workshops.

Representatives who attended briefings provided by ICANN org may receive a follow-up survey to help ICANN org understand their assessment of the briefing, whether they would be interested in further such briefings and the level of interest in the topics covered.

Pre- and post GAC Capacity Building workshop surveys. (Supported by ICANN org's Government Engagement function).

Pre and post-GAC capacity-building workshop surveys are used to invite GAC members to identify topics for the workshops. Post-workshop surveys are used to invite GAC members to assess effectiveness of the workshops. The GAC capacity building workshops are demand-driven bottom-up engagement. GAC members choose the topics they are most interested in and post workshop surveys are to help ICANN org improve the utility of the sessions and allow participants to identify future topics on which they wish to be briefed.

ICANN org uses pre and post-workshop surveys to gather input from the community on GAC capacity building sessions.

Survey Results on Transition to In-Person ICANN org Public Meetings. (Supported by ICANN org's Global Stakeholder Engagement function).

ICANN org conducted a survey with the community to better understand its performance in conducting ICANN meetings virtually and to apply findings into considering which lessons that were learned from virtual meetings to carry over into our Public Meetings.

ICANN org asked community members to complete a survey to gauge their general comfort level and the desired conditions for a return to in-person ICANN Public Meetings. Please find this link to the results of this survey. This survey was a one-time occurence.

Domain Name System Security Extensions (DNSSEC) Survey for the Africa region. (Supported by ICANN org's Global Stakeholder Engagement function).

The Domain Name System Security Extensions (DNSSEC) Survey for the Africa region aimed to provide ICANN org with a better understanding on the participants' DNSSEC implementation status, and DNSSEC deployment plans. The results will help the org redesign any future DNSSEC roadshow activity in Africa, to help increase internet penetration and deployment of DNSSEC.

This survey was open to any organization running a Domain Name System (DNS) service in the region, such as country code top-level domains registries (ccTLDs), and generic Top-Level Domain (gTLD) registries and registrars, and Internet Service Providers (ISPs). This survey was a one-time-occurance.

Community Satisfaction Survey on Middle East Forum. (Supported by ICANN org's Global Stakeholder Engagement function).

The purpose of the survey was to assess whether the forum had achieved its objectives, and to understand what is next for this regional platform.

When the Middle East DNS Forum reached its five-year milestone, a community survey was sent to previous participants, as well as the broader Middle East community, on 23 May 2018. Please find this link to the results of the survey. This survey was a one-time occurance.

Asia Pacific (APAC) Space Survey. (Supported by ICANN org's Global Stakeholder Engagement function).

The APAC Space is a platform for ICANN community members from the APAC region to raise issues, share views, and coordinate activities. APAC Space takes place bi-monthly, either face-to-face at ICANN meetings or via web conference. Discussions are community-led, with a focus on facilitating APAC community members to participate in ICANN's policy development work.

The APAC Space Survey was intended to gauge APAC community interests and priorities regarding current ICANN-related policy issues that can be discussed at APAC Space. It provided valuable feedback to make APAC Space more relevant to APAC community members and encourage discussion on targeted topics.

The APAC Space survey was conducted from 2017-2019. The survey was then discontinued and replaced with direct engagement where ICANN org asks for community input directly through its mailing list discuss@apacspace.asia. This occurs in the form of (i) open questions asking for topic proposals from the community; and/or (ii) where ICANN org proposes the topic and asks for feedback.

Community-Wide Survey on Gender and Participation. (Supported by ICANN org's public responsibility support function).

ICANN org is committed to promoting greater diversity and supporting broad, inclusive participation.To gather meaningful data to support ongoing community discussions on diversity, the findings offer insights into current perceptions of gender diversity in the community, potential and perceived barriers to participation, and the community's support for initiatives to enhance gender diversity.

The ICANN org conducted the Gender Diversity and Participation Survey with the ICANN community from 9 June – 8 July 2017. This survey was a one-time-occurrence and open to the community. Please use this link to see the results of the survey.

Community-Wide Survey on Age Diversity and Participation. (Supported by ICANN org's Public Responsibility Support function).

The ICANN org is committed to promoting greater diversity and supporting broad, inclusive participation. The survey report offered insights into the community's perceptions of age diversity, potential and perceived barriers to participation, and its support for initiatives to enhance age diversity. The Age Diversity and Participation Survey report provided useful data and findings to guide these future discussions.

The ICANN org conducted the Community-Wide Survey on Age Diversity and Participation from 10. This survey was a one-time-occurrence and was conducted with the community March - April 2019. Please use this link to see the results of the survey.

Fellowship Post-Meeting Survey. (Supported by ICANN org's Public Responsibility Support function).

The Fellowship Post-Meeting Survey gathers input on the performance of ICANN org Fellowship program based on feedback of each fellowship class. Data is used to measure satisfaction with the program. Feedback is used as part of the program's continuous improvement efforts.

After an ICANN meeting, Fellows participate in a survey with quantitative and qualitative questions. This survey occurs every three years. Please use this link to see the and examples of the results of the survey.

Fellowship Alumni Survey. (Supported by ICANN org's Public Responsibility Support function).

The Fellowship Alumni Survey helps ICANN org assess alumni engagement at ICANN after the Fellowship Program (and therefore the success or performance of this program). The survey also collects suggestions on how ICANN org can enhance the program. Feedback has helped inform improvements to the program such as the New Fellowship Program Approach.

Fellowship alumni participate in surveys with quantitative and qualitative questions. This survey occurs every three to five years. Please use this link to see an example of the results of the survey.

NextGen@ICANN Post-Meeting Survey. (Supported by ICANN org's Public Responsibility Support function).

The NextGen@ICANN Post-Meeting Survey gathers feedback on the performance of the NextGen program that ICANN org supports after the completion of each NextGen class. Data is used to measure performance of the program. Feedback is used as part of the program's continuous improvement efforts.

After an ICANN Meeting, NextGen program participants take a survey with quantitative and qualitative questions. The survey is conducted three times a year. Please use this link to view an example of the results of the survey.

NextGen@ICANN Alumni Survey. (Supported by ICANN org's Public Responsibility Support function).

The NextGen@ICANN Alumni Survey assesses the performance of this program that ICANN org supports including its impact on participants and areas of opportunity for improvement in the future. The survey also collects suggestions on how ICANN org can enhance the program. Feedback has helped inform improvements to the program.

NextGen Program alumni participate in a survey with quantitative and qualitative questions. This survey occurs every three to five. Please use this link to view an example of the results of the survey.

Leadership Program Survey. (Supported by ICANN org's Public Responsibility Support function).

The Leadership Program survey gathers feedback from community participants after the completion of each Leadership Program class. Data is used to measure satisfaction with the program. Feedback is used as part of the program's continuous improvement efforts.

Participants of the Leadership Program take a survey with quantitative and qualitative questions. Survey is conducted once a year. (Currently on hold).

Chairing Skills Program Survey. (Supported by ICANN org's Public Responsibility Support function).

The Chairing Skills Program survey gathers feedback from community participants after the completion of each Chairing Skills Program class. Data is used to measure satisfaction with the program. Feedback is used as part of the program's continuous improvement efforts.

Participants of the Chairing Skills Program take a survey with quantitative and qualitative questions. Survey is conducted once a year. (Currently on hold).

ICANN org Community Child Care Pilot Program Survey. (Supported by ICANN org's public responsibility support function).

The ICANN org Community Child Care Pilot Program survey gathers input from the community on ICANN org's program operations and impact performance. Data is used to measure satisfaction with the program. Feedback is used as part of the program's continuous improvement efforts.

After each ICANN Meeting where the Child Care Pilot Program is run, participants take a survey with quantitative and qualitative questions. (Currently on hold).

Public Comment. (Supported by ICANN org's's Policy Support function).

Public Comment provides an information acquisition mechanism for stakeholders to have their opinions and input formally and publicly documented. It is an opportunity for the ICANN community to effect change and improve policies and operations.

The ICANN Board, Supporting Organizations, Advisory Committees, or ICANN org can open a Public Comment proceeding, outlining specific issues or questions for feedback. Stakeholders provide opinions and recommendations, and their submissions are public and archived. ICANN org summarizes the submissions and identifies common themes in a summary report. The group that opened the proceeding, reviews the submissions and summary report, addresses the input, and proposes next steps.

Recent examples where the community has provided input on ICANN org's performance include:

- Third Accountability and Transparency Review Team (ATRT3) Draft Report and Final Report

- Middle East and Adjoining Countries (MEAC) Strategy 2021-2025

- ICANN Africa Regional Plan for Fiscal Years 2021-2025

- Latin America and Caribbean (LAC) Regional Strategic Plan for FY 2021-2025

- Revised Community Travel Support Guidelines

Please see this example: Third Accountability and Transparency Review Team (ATRT3) Draft Report and Final Report

ICANN Executive Q&A sessions take place multiple times each year during ICANN meetings.

(Supported by ICANN's Meetings function).

The ICANN Executive Q&A session provides the members of the global stakeholder community with the ability to address the ICANN executive team and share their input on ICANN performance in meeting its commitments to the community.

The ICANN Executive Q&A is the community's opportunity to make comments and ask questions on the main topics directly to the executive team at ICANN meetings in front of the rest of the community. For transparency, a scribe feed is provided both in the room and via remote participation in real-time to allow all comments and questions raised during the session to be viewed by everyone. Remote participants have the opportunity to raise questions and/or comments can occur via email.

ICANN Specific Reviews (Supported by ICANN org's Reviews Support and Accountability function).

Specific Reviews are led by community members to assess ICANN org's performance in meeting its commitments. Please click on the links below to learn more about the four types of Specific Reviews:

• The Accountability and Transparency Review Review (ATRT)
• The Security, Stability and Resiliency Review
• The Registration Directory Service Review (RDS) formerly the WHOIS Review
• The Competition, Consumer Trust and Consumer Choice Review CCT

Specific Reviews are led by teams of community members. All review teams are a way for the community to provide input to ICANN; further, each review is required to consider further public input in the development of their recommendations. Beyond a survey or questionnaire, Specific Reviews give the community the opportunity to provide analysis and feedback on ICANN org's performance in meeting its commitments to the global stakeholder community.