Pre-Delegation Testing Provider for New gTLDs - RFP Questions & Answers
Respondents are requested to respond to the Request For Proposals by replying to firstname.lastname@example.org. The period to submit questions about the RFP is now closed. The final response to the RFP is due on 20 November 2012.
The following questions were submitted between 30 October and 7 November 2012. The full set of questions and responses are being posted on ICANN's website for the benefit of all the respondents.
Questions are grouped together where similar questions were received, or where the same answer is relevant to multiple questions.
The text below are based on questions received from prospective RFP respondents, and has been rewritten to be vendor neutral.
Question 1: Q.16: What does ICANN specifically define as the "three major phases?"
These are the three distinct parts defined in section 2.2 ("Scope").
Question 2: Is it expected that web interfaces within each registry system will need to be tested as well?
Whois interface should be tested as described in 184.108.40.206. The AGB states:
"ICANN will verify that Whois data is accessible over IPv4 and IPv6 via both TCP port 43 and via a web interface and review self-certification documentation regarding Whois transaction capacity. Response format according to Specification 4 of the registry agreement and access to Whois (both port 43 and via web) will be tested by ICANN remotely from various points on the Internet over both IPv4 and IPv6."
Question 3: Will there be a dedicated ICANN resource for the testing provider to consult during the design and implementation phases should any questions arise?
Question 4: The RFP indicates that the process will award a contract to "one or more" providers, which could affect pricing. Can ICANN provide any additional guidance in regards to this?
As requested in Q.24, Q.25 and Q.26, vendors should supply independent pricing for each of these items listed.
Question 5: Is testing required to be performed per Back-End Registry Service Provider, or per applicant?
The pre-delegation testing provider shall verify that each applicant has met its commitment to establish registry operations in accordance with the gTLD Applicant Guidebook (AGB).
Question 6: Given that there are only about 1400 unique strings being applied for, is it still expected that there will be approximately 2000 tests at the rate of 20 tests per week?
We envision 1400 applicants to be tested at an initial rate of 20 tests per week. Note however that we are also requesting the provider to be able to scale up to 100 tests per week, if requested by ICANN.
Question 7: Can ICANN provide a definitive statement of the anticipated minimum number of pre-delegation tests?
The total number of applied-for strings that will be approved is unknown at this point. Therefore, we cannot assert the minimum number of registries to be tested but only the maximum: 1,400. We expect that the actual numbers of registries tested should be close to the maximum.
Question 8: ICANN required that the Pre-Delegation Testing Software should be made available to ICANN in source code format, and may be published by ICANN as Open Source Software. Please clarify this requirement.
Any limitations in the delivery of the software or the license provided should be clearly stated in the bid. At a minimum, ICANN is requiring to be granted a non-exclusive, perpetual royalty-free license to use the provided software for the purpose of pre-delegation testing.
Question 9: Does "the Pre-Delegation Testing Software" refer to all software used within the Pre-Delegation Testing System or only the software used to perform the technical tests?
All software required to provide the Pre-Delegation Testing Service.
Question 10: The RFP requires the Pre-Delegation Testing Provider to review the "self-certification" documents provided by the Registry Operator being tested. Would ICANN confirm that this review is meant to ensure completeness and does not reflect additional tests to verify the self-attestations?
The Pre-delegation Testing Provider should review the self-certification documents to verify completeness, plausibility and compliance with specific assertions made in the gTLD application.
Question 11: Concerning the reports that will be generated through the Pre-delegation Testing System, does the Pre-Delegation Testing Provider have to provide a report for each applicant, or only a set of uniform reports for each back-end registry service provider’s technical platform?
Yes, for each gTLD tested the providers is expected to provide a report reflecting the status of the Registry Operator compliance with the Pre-Delegation Testing requirements.
Question 12: Will proposals from gTLD applicants or Back-End Registry Service Providers and/or DNS Service Providers, who have been contracted by new gTLD applicants, be disqualified or otherwise penalized in this RFP process?
ICANN will accept proposals from new gTLD applicants and/or new gTLD Back-End Registry Service Providers. As required per Q.3 and Q.6, all conflicts of interest should be clearly declared by the respondent. A conflict of interest does not automatically disqualify or penalize any respondents.
Question 13: Will ICANN consider EBEROs as Pre-Delegation Testing Providers?
Question 14: If the Pre-Delegation Testing Provider candidate is the Back-End Registry Service Provider for an applicant itself, can it verify or test its own technical specifications according to the guidelines in the RFP or are there other methods to deal with such situations? If there are, please describe in details.
A Pre-Delegation Service Provider cannot test its own specifications or registry services provided by itself for an applicant as this constitutes a conflict of interests situation.
Question 15: Would ICANN allow a two-vendor approach for the Pre-Delegation Testing engagement (i.e. one vendor develops the Pre-Delegation Testing Software and hosts the Pre-Delegation Testing System, and the other vendor would provide the Pre-Delegation Testing Services)?
Question 16: Would ICANN consider a contract award to a single supplier to allow for a) economies of scale and b) a consistent testing model for all?
Having a single supplier is the preferred option for ICANN. However, ICANN is willing to consider more than one provider if that is a better alternative.
Question 17: Is it possible to execute a single test per Back-End Registry Service Provider (as this would limit the requirement to about 80 such tests)?
The AGB specifies one Pre-Delegation Test per Registry Operator (contracted party to ICANN).
Question 18: Will ICANN consider proposals that address a subset of the Pre-Delegation Testing? More specifically: Given its more technical nature and importance, would ICANN consider proposals that solely address the DNS and DNSSEC components of the RFP?
Question 19: Does ICANN have any concerns regarding conflict of interests with other application evaluation services provided to ICANN as part of the New gTLD Program?
Question 20: Q.26: About the onsite audit, how many people are required to accommodate one on-site audit?
The Pre-Delegation Testing Provider should assemble a team which holds the required competence for an on-site audit. It is anticipated that only a small number of these tests would be required.
Question 21: Can ICANN please clarify audit elements to be examined at an onsite audit?
The Pre-Delegation Testing Provider should verify compliance with the requirements in section 5.2 of the AGB and the related assertions made in the gTLD application
Question 22: Q.26: Does ICANN expect the number of tests to increase significantly throughout the contract period?
It is anticipated that Pre-Delegation Testing will be carried out at an initial rate of 20 tests per week, with the possibility of increasing the rate to up to 100 per week. As described in Q.26(a), pricing should be provided per test.
Question 23: R.6: Is it acceptable to not implement mutual authentication of Pre-Delegation Testing agents if the Pre-Delegation Testing Provider operates in an internal/private network environment?
No, the Pre-Delegation Testing System should be designed for operation in an open environment (e.g. over the public Internet) – mutual authentication and encryption is a requirement.
Question 24: Is IPv6 a requirement for "tests are to be performed over IPv4 and IPv6"?
Tests are to be performed over IPv6 when applicable. Note that some services (e.g. DNS and Whois) must be provided over IPv6 (as required by The New gTLD Agreement Specification 6).
Question 25: R.18: What is the "gTLD Contract"?
The gTLD contract is the signed "New gTLD Agreement".
Question 26: R.19: What are the contents of the absent footnote 1?
Question 27: Q.26(b) is not properly rendered
"A data Escrow Agent release process test. It is expected that ICANN will require in the order of 20 tests throughout the contract period."
Question 28: R.25: Will ICANN provide a root server for DNSSEC trust anchor testing?
Question 29: R.26: Can ICANN provide more details in the standards of a DPS?
Per section 1.3 of Specification 6 of the base agreement, the DPS should conform to the forthcoming RFC describing a DPS framework (currently in draft https://datatracker.ietf.org/doc/draft-ietf-dnsop-dnssec-dps-framework/), and that it is describing critical security controls and procedures for e.g. key material storage, access and usage of its own keys and secure acceptance of registrants' public-key material.
Question 30: Questions about limits
- Is there a time limit for an application to pass the test? What is the timeframe required for an application to complete its technical tests before gTLD delegation to the zone?
- If the application fails the technical test for the first time, how many extra chances do they still have before the abortion of the pre-delegation testing procedure?
- Or, if there are additional tests involved, can the Pre-Delegation Service Provider charge additional fees to ICANN?
Applicants should be given reasonable time (to be agreed between the selected provider and ICANN) for correction of technical issues or to complement the submitted information. If the applicant is unable to provide the evidence required within this time frame, the test should be regarded as failed.
Question 31: Q.5: Are there any specific information required regarding listed subcontractors.
As described in Q.15(a), respondents should provide a conflict of interest assessment based on the current list of new gTLD applicants.
Question 32: Should TLD Operators fail a component of the Pre-Delegation Testing, can ICANN clarify the degree of support that is expected to be offered by the Pre-Delegation Testing Provider to the TLD Operator, other than re-testing?
It is expected that the Pre-Delegation Testing Provider will supply a descriptive response for each failed test to the applicant, and allow the applicant to take corrections within a reasonable time frame (to be agreed between the selected provider and ICANN) before conclusively failing the test.
Question 33: Given the requirement to list names and IPv4/IPv6 unicast addresses of anycast nodes, what actions does ICANN wish to take if the TLD operator does not, in their normal operations, make individual servers in their anycast sets reachable by any other public IP address other than the anycast address?
If the TLD operator cannot allow queries from the Pre-Delegation Testing Provider to the unicast addresses of all anycast nodes, the applicant will be required to present a compensating reachability assertion for all nodes not testable.
Question 34: Can ICANN please clarify audit elements to be considered when reviewing self-certified load tests?
The Pre-Delegation Testing Provider should review the self-certification documents to verify completeness, plausibility and compliance with relevant assertions made in the gTLD application.
Question 35: Is the Data Escrow Release process test centered around a release of an existing data escrow deposit from the data escrow agent to ICANN or to the Pre-Delegation Testing Provider?
The release test is focused in the release of deposits from the gTLD Data Escrow Agent to the Pre-Delegation Testing Provider.
Question 36: Concerning the data escrow in 220.127.116.11; is it a requirement to check if the data escrow is in compliance with the requirements with a specific software?
No, the Pre-Delegation Testing Provider should validate the data escrow profile and the data escrow format of one full and one incremental data escrow deposit. Hence, this test is not dependent on a specific software.
Question 37: R.24: Does “verify that data can be released within 24 hours” refers to a manual review of the SLA between the Data Escrow Service Provider and the applicant (as provided by the applicant), or does this refer to an actual test that must be performed by the Pre-Delegation Testing Provider to ensure that data is released and returned in a manner compliant with the Data Escrow requirements?
For most cases ICANN only expects the review of the SLA, but in some cases ICANN may requests that the actual release capability of each Data Escrow Service Provider should be verified. As indicated by Q.26, ICANN expects in the order of 20 such tests throughout the contract period.
Question 38: Would it be acceptable to test full Data Escrow deposits only?
No, the AGB requires both full and incremental deposits, implicating both have to be tested.
Question 39: R.19 & R.20: Please clarify the requirements for IDN table verification
The primary objective is to verify that the applicants IDN capabilities are compliant with those described in the supplied IDN tables. Performing such tests is vastly simplified if the IDN tables are supplied in a machine-readable format that can be read programmatically, but ICANN recognizes that there might be IDN tables in other formats, e.g. given more complex variant generation algorithms.
Question 40: R.13: Is the EPP interface only to be tested for the listed EPP commands?
No, the EPP interface should be tested for standards compliance with the requirements described in the gTLD Applicant Guidebook section 5.2, i.e. "verify conformance to appropriate RFCs (including EPP extensions for DNSSEC)".
Question 41: R.18: Are there any technical tests required for reviewing the applicant's EPP extensions?
Question 42: 18.104.22.168: Is there a requirement to test the Whois web interface over HTTPS?
If the Whois web interface is provided over HTTPS, it should tested.
Question 43: Q.26: Does "audit of load test" refer to a manual review of the applicant's self-certification documentation, or does it refer to actual tests that the Pre-Delegation Testing Provider must run independently in order to validate the assertions made by the applicant.
Q.26(c,d,e) refers to actual technical load tests performed by the Pre-Delegation Testing Provider. It is anticipated that only a small number of these tests would be required.