Domain Metrica FAQs

What is the Domain Metrica Project?

ICANN Domain Metrica is a measurement platform that aims to provide various sets of information about domain names, domain name registration, usage and behavior. Initially limited to covering domain name abuse, it will evolve and expand to cover any aspect of the domain name ecosystem which might be of interest to the community.

Why did the ICANN organization develop Domain Metrica?

The Domain Abuse Activity Reporting (DAAR) project has been around since 2017 and OCTO saw the need to update DAAR based on evolving needs. Over time, new requirements and use cases became clear. Since DAAR was originally built to serve one purpose only, it was not possible to improve it based on the recommendations from our community or to expand it. In part Domain Metrica will initially serve as a replacement for DAAR, and over time it is envisioned that it will grow to be much more than DAAR.

What is the purpose of Domain Metrica?

The overarching purpose of Domain Metrica is to improve the metrics and data that we can share with the ICANN community, which can then use the data to make informed decisions. While DAAR was focussed solely on abuse related reports, Domain Metrica has a broader remit in that we will consider any data, metadata, metric, or information about a domain as in scope. That said, the initial iterations will be concentrating on listed domain abuse reports.

Module one is about meeting the basic requirements currently being served by DAAR, that is collecting and displaying reported DNS abuse data, but adding the ability to aggregate data to the registrar level where DAAR is limited to top-level domains (TLDs). This should immediately add an interesting viewpoint on where abuse is currently more prevalent. We are also negotiating license agreements which allow ICANN to share more detailed information with Domain Metrica users, including registries and registrars. Being able to share lists of reported domain names (within certain limitations) will aid us in meeting commonly requested ICANN community requirements for the system.

What types of DNS abuse does Domain Metrica observe?

Domain Metrica identifies and tracks reported domain names associated with four kinds of Domain Name System (DNS) abuse:

• Phishing: Domain names that support web pages that disguise as a trustworthy entity such as a bank, known brand, online merchant, or government agency.

• Malware: Domain names that facilitate the hosting and/or spreading of hostile or intrusive software that is installed on end systems, potentially without the permission of the user.

• Botnet command-and-control: Domain names that are used to identify hosts that control botnets. Botnets are collections of malware-infected computers that can be used to perpetrate various abusive activities like launching denial of service attacks, and sending spam email or phishing campaigns, among others.

• Spam: Domains that are advertised in unsolicited bulk email or used to name spam mail exchange systems. The term spam no longer describes only unsolicited bulk email but has become a major means of delivery for identifiers (domain names, hyperlinks, or addresses) used to support the above-listed DNS abuse.

What data does the Domain Metrica system use?

We believe that it is beneficial to collect the same DNS abuse data that is reported to industry and Internet users. Security systems such as anti-spam or anti-malware gateways or firewalls that protect billions of users incorporate these data into their DNS abuse mitigation measures. Domain Metrica thus reflects how the users and network operations communities see the domain name ecosystem through the lens of reported DNS abuse data.

Domain Metrica incorporates a large number of reputation block list (RBL) feeds. You can review the list of feeds and providers at the end of this FAQ for the feeds in use as of the date of this writing. Collectively, these feeds provide multiple sources for the DNS abuse reports that Domain Metrica can measure or analyze.

Domain Metrica is designed to be extensible beyond just abuse data to ensure the fullest possible coverage of potentially useful information. Therefore, we will not limit the data intake to just the reputation block lists (RBLs) used for DAAR, but will incorporate other data including DNS records, popularity rankings, and others.

What kind of data do the RBLs contain?

Reputation block lists (RBLs) typically contain domain names, URLs and/or IP addresses associated with entities that are "reported" to be participating or distributing malicious activity or attacks and not all of them end up causing any security incidents. These lists are either reported by a vetted community, or are gathered by RBL providers themselves through sandboxes, sinkholes and honeypots.

How does Domain Metrica compile reported DNS abuse data?

Domain Metrica does not work in isolation. The system does not generate DNS abuse reports. It relies on open or commercial reputation data to identify and classify the four types of DNS abuse mentioned above. The reputation feed providers of the data Domain Metrica uses have different accuracy, coverage, industry adoption, and the feed's ability to classify events into the DNS abuse classes that Domain Metrica tracks.

If a domain is listed for two or more types of DNS abuse, that domain will be counted in each relevant category. However, only unique domains are counted for the total reported DNS abuse counts in the TLD or registrar portfolio.

How reliable is Domain Metrica's data?

Domain Metrica uses many categories of data: zone data, registration data (available for gTLD domains only), reputation data, and so on.

Domain Metrica collects TLD zone data daily, using ICANN's Centralized Zone Data Service and/or provided directly through agreements with TLD operators. Zone data, the availability of which is contractually mandated for generic top-level domains (gTLDs) and volunteered by country code top-level domains (ccTLDs), are generally provided once a day. As such, Domain Metrica will not observe zone changes after the daily release of zone data until the following day.

Domain Metrica collects reputation data from providers that were selected based on our RBL characterisation process documented in OCTO-37. The providers must demonstrate the value they add to the overall dataset, and we periodically reassess our providers along with new sources to maintain confidence that we have good coverage.

Other data sets collected are similarly tested for their reliability, availability, and usefulness.

Some metrics that we want to share will be computed from other data. Where this process is probabilistic in some way (for example, where it comes from a machine learning model) this will be clearly indicated and a measurement accuracy will be available.

How up-to-date is the data?

The data used by Domain Metrica is updated each day. The domain counts are collected daily from fresh TLD zone files. The reputation providers continually add domains to their lists. Each provider also has a procedure for removing domains from their lists, and these removals are tracked and accounted for within Domain Metrica. Generally, each provider lists a domain name for as long as the provider believes the domain constitutes a problem, after which the domain is removed. A domain name may be listed only for minutes, or for months, depending on the provider's policies and criteria.

There are a few lists that do not track abuse status and therefore do not provide "removal" flags. One example is the Anti-Phishing Working Group (APWG) phishing feed. This is a list of newly confirmed phishing identifiers, however the APWG does not then track to see which sites are online and which are down. To be conservative, when a domain is listed in the APWG feed, we only count that domain as "listed" or "active" for one day.

Note also that while we try to present data as a "point in time" measurement, different sources update at differing frequencies and on different schedules. Therefore, metrics should be regarded as reflecting our understanding of the situation at any moment. Where possible we provide timestamps for data to be clear about when we collected it.

Does Domain Metrica find all instances of abuse in the DNS?

No. The Domain Metrica system uses DNS abuse domains that are listed by multiple reputation service providers. However, these providers do not claim to see or list all abusive activity happening on the Internet. Note also that Domain Metrica does not have access to all sources of DNS abuse reports. Therefore, it will also not count all reported abuse. We therefore note that Domain Metrica provides a baseline indication of where abuse is concentrated, and that the amount of DNS abuse associated with domain names is larger than this system catalogs. Users of Domain Metrica data should assume that the data it presents are a subset of the DNS abuse problem, and that domains without any reports are not guaranteed to be non-malicious.

Does Domain Metrica list only domains that were registered by malicious parties?

In general, most reputation providers do not differentiate between maliciously registered domains and legitimate but compromised domains. Thus, initially Domain Metrica is not likely to know the motives for registering domain names.

We are however working on developing methodologies to be able to distinguish between the two sets of domains. This feature will be added to Domain Metrica once it is mature enough to be considered reliable.

How does Domain Metrica deal with false positives in reputation data?

Domain Metrica does not modify the data received from reputation feeds, so if the feeds include false positives, those false positives are reflected in Domain Metrica's output. However, since numerous parties rely on these reputation feeds – for example, email service providers, Internet service providers, and resolver operators – any false positives will affect the domain name ecosystem regardless of how Domain Metrica reports them. As such, efforts within the Domain Metrica system to further reduce false positives would result in conflicting or false information from the perspective of impact of reported DNS abuse to those parties. ICANN's SSR Team is continuously monitoring the quality of the data. Therefore, feeds can be added or removed based on quality assessment performed by members of the SSR Team or based on the community's feedback.

That said, one potential source of false positives is malicious domains that have already been mitigated yet remain listed on abuse feeds. We are looking at ways in which we can determine domains in this state.

Do Reports equate to Abuse Incidents?

We define DNS abuse incidents as reported domains on RBLs (presumably attacks) that are materialized. That is, if they caused tangible losses (e.g., money and resources) and intangible losses (e.g., reputation and credibility).

Therefore, reported domains refers simply to the presence of a domain on one or more of the RBLs we have access to and is NOT necessary an abuse incident. While we do remove some entries, for example if they are not present in the current zone file for the TLD in question, we do not gather sufficient extra robust evidence to present a call to action. Over time we will add more context to each report, but this is intended to assist any investigation into a domain, it will not replace an investigation.

Who will have access to Domain Metrica?

We have two user roles defined.

1. Standard User:
   Anyone with an ICANN account will be able to log in to Domain Metrica and see statistical data for TLDs, registrars, and search for individual domains. There will also be a community Application Programming Interface (community API) to allow programmatic data retrieval.

2. Registry/Registrar (Ry/Rr) User:
   Accounts with NSp credentials linked to registries and registrars will have further access to domains under their control. This access will be limited by our license agreements in some cases, but as far as possible we will share all the data we can.

Registry and registrar access can be both through the web user interface and also via Monitoring System API (MoSAPI).

How does Domain Metrica fall into ICANN's remit?

For ICANN to help ensure the security and stability of the top-level of the Internet's system of unique identifiers that it coordinates according to its mission, both the ICANN organization and community must be aware of threats, trends, and features of that system. In keeping with ICANN's requirements for openness and transparency, the organization must, as much as possible, make the data we collect available to the community. Finally, the role of the ICANN organization in general, and the Office of the Chief Technology Officer in particular, is to provide neutral, unbiased data and analyses to facilitate policy discussions and development.

I have found a bug in Domain Metrica, how should I report it?

Please contact ICANN global support at [email protected] to report your issue. Please provide as much information as you can about the issue such as the URL and the type and version of the browser you used to access Domain Metrica. Please also provide a screenshot of the issue, if this is appropriate to the bug being reported.

Can I download the underlying data?

Yes, the charts displayed in Domain Metrica have a download icon which gives the opportunity to download the data in either csv or JSON format. An API will be available soon which will provides JSON structured data.

Data access limits

To protect the service and comply with licensing agreements, certain limits are imposed on data access:

• Daily Search Quota: User Interface and Community API searches share a daily limit of 50 requests per type:

  • 50 TLD searches

  • 50 registrar searches

  • 50 domain searches

These limits also apply to adding entities into comparisons on the charts.

• Domain List Downloads: Ry/Rr users additionally have access to lists of abuse reports for TLDs and IANA Registrar IDs linked to their accounts.

  • Only "fresh" reports are included, that is reports first seen in the current day's data.

  • Downloads via the UI have a 1,000 report limit

  • Downloads via MOSAPI do not have this restriction

  • Some data sources impose additional limits:

  • Commercial Sources: Some datasets may be capped at 25% of their full data or a maximum of 100 records.

  • Spamhaus: Data is limited to a fixed cap of 500 domains.

  • Other Sources: Licensing agreements may change these limits; details may vary depending on source agreements.

List of Reputation Data Providers and data feeds:

As of July 2024, Domain Metrica incorporates the following blocklists.

• Spamhaus Domain Blocklist (DBL): Domains advertised in spam, domains used for phishing, and domains used to support malware.

• SURBL: Domains advertised in spam, domains used for phishing, and domains used to support malware.

• WMC Global PhishFeed: Phishing data specializing on the mobile ecosystem.

• APWG: URL blocklist feed featuring domains used for phishing.

• Phishtank: Domains used for phishing.

• URLHaus: URLhaus is a project from abuse.ch with the goal of sharing malicious URLs that are being used for malware distribution.

We are also looking at a number of other sources which we will add to the system if we see a benefit from the data and can agree appropriate licensing terms with the providers.

How can ccTLDs join the Domain Metrica Project?

Our initial release covers gTLDs only. Future releases plan to add the ability for interested ccTLDs to participate in Domain Metrica. The process for doing so will be communicated nearer the time that it will be available.

Can you explain some of the terminology used?

"Total number of domains": This is how many domains we see in our CZDS data. This covers the gTLDs only.

"Total number of gTLDs": This is a count of how many gTLD zone files we have access to for the current report.

"Total number of registrars seen": This is a count of how many unique registrars we see in our data.