Please be advised that the third party review of the Ombudsman – Client Survey is complete. It will be posted at: http://www.icann.org/ombudsman/program.html
This review, and reporting, was completed by an independent third party. This party was appointed by the Audit Committee of the Board of Directors. The executive summary of the independent reviewer's report is attached:
Executive Summary
The Internet Corporation for Assigned Names and Numbers (ICANN) Office of the Ombudsman (the Office) is executing a first-in-class assessment process. One assessment step (please see the other completed assessment components at www.icann.org/ombudsman/program.html) was a Client Survey, which was codified into a report (the Report). This Commentary verifies the claims made in the Report, offers alternative analysis and provides recommendations intended to inspire future-oriented action impacting Office of the Ombudsman operations.
In the Report, Frank Fowlie, ICANN ombudsman posits that Respondents to the Client Survey (with limited, notable exceptions) were “generally satisfied“with services provided by the Office. Upon review of the Survey results and the Report this indeed appears true. This finding is basically positive, but there are concerns, among these: The Survey return rate was limited; Narrative responses tended to negative; and The majority of responses were from people with issues outside the mandate of the Office. This last point is the most important.
The Report, and especially the Survey results themselves offer nuanced and valuable information about both the Office and ICANN as an institution. Readers are strongly encouraged to examine each of these source documents and especially consider the narrative responses from the Survey.
The reasonably constructed survey instrument asked 17 questions of differing types, with multiple sub-elements for many of the questions. The Survey queried far more than satisfaction. Perhaps better thought of as a “user experience survey,” the tool also asked: How well Users actually knew the Office; How they accessed the Office; and How they rate the Office’s website.
The Survey process, in spite of any shortcomings should definitely be repeated at least annually, if not more frequent intervals. Twenty-two percent of the 387 invitees for a total of 85 responses, answered all or portions of the Survey. A commercial third party hosted the Survey, to assure Respondent anonymity. The Report clearly conveys the actual answers provided by Respondents and offers interpretations of the responses.
Summarizing, the Survey engaged sound methods and the Report offered accurate findings, as well as many reasonable interpretations. It was greatly affected by the large number of Survey Respondents with issues outside the design intent and mandate of the Office. This Commentary discusses certain considerations likely to result in Office enhancement (improving the quality of referrals and follow-up on those referrals) as well as survey design and process improvement (repeat invitations, make survey questions clearer, and ask about fairness). Irrespective of these possible improvements, the ICANN Office of the Ombudsman was found to be generally satisfactory to Survey Respondents, especially those with issues firmly within the mandate of the Office. Further, the Office provides assistance to those with issues outside its jurisdiction, by helping them identify other potential resources. Therefore, the Office of the Ombudsman is likely serving, at a more than acceptable level and has once again proven itself to be potentially the single most completely assessed Ombuds Program in existence.