The Dark Web: The Land of Hidden Services
According to Internet Live Stats, the World Wide Web passed the one billion website benchmark in 2014 and is still hovering around that figure. The publishers of these billion websites compete for search engine relevance and the attention of nearly 3.6 billion Internet users. There is another part of the Web, however, where publishers and visitors want to navigate websites and conduct business transactions in secret. This is the Dark Web, a land of hidden services, where leaving no tracks and preserving anonymity are valued over search engine rankings and web experience personalization.
The Dark Web
The Dark Web is an important part of the Internet ecosystem. It allows for the publication of websites and the dissemination of information without revealing the publisher's identity or location. The Dark Web is only accessible through services such as Tor. Many users use Tor and similar services as a means to provide freedom of expression and association, access to information, and the right to privacy.
The Deep Web
The Deep Web is the collection of all websites that are not indexed by search engines. Some Deep Websites are unconventional marketplaces that offer a disturbing range of products or services. You can buy or broker illegal drugs, weapons, counterfeit goods, stolen credit cards or breached data, digital currencies, malware, national identity cards or passports. You can contract digital or criminal services, ranging from spam campaigns to distributed denial-of-service (DDoS) attacks. Novices can even purchase eBooks that explain how to attack websites, steal identities or otherwise profit from illegal activities.
But you can also use the Deep Web to anonymously share information with media outlets such as the New York Times, the Washington Post, The Intercept and others, as well use search engines without giving up your privacy, or engage in legitimate e-commerce network such as OpenBazaar.
Leave No Trace: Encryption and Evasion for the Dark Web
Many Internet users use encryption – for example, Virtual Private Networks (VPNs) – to keep Internet activities private. VPN connections typically abide by the conventional behavior of Internet routing for (1) the determination of an end-to-end path from a user's computer to a server that hosts content that the user wants to access, and (2) the bidirectional transmission of requests and response traffic along this path. Conventional routing, however, is susceptible to traffic analysis, a surveillance technique that can reveal traffic origins, destinations and times of transmission to third parties. Traffic analysis is related to metadata collection, a topic we've covered in an earlier post.
Tor networks are popular solutions for maintaining anonymity and privacy and for defeating traffic analysis. Who uses Tor? Journalists, whistleblowers, dissidents, or generally any Internet users who do not want third parties to track their behavior or interests. Tor serves many good purposes, but also attracts Dark Web users wanting to keep their activities or marketplaces secret and untraceable.
Like VPNs, Tor networks use virtual tunnels, but unlike VPNs, these tunnels don't connect clients directly to servers. Instead, Tor clients create circuits through relay points in the Tor network. Tor circuits have three important properties.
- No relay point knows the entire path between circuit endpoints.
- Each connection between relays is uniquely encrypted.
- All connections are short-lived to prevent observation of behavior over time.
Constructed using these properties, these Tor private network pathways defeat traffic analysis and support the ability to publish content without revealing identity or location.
Names for Dark Websites
Unlike the human-readable domain names that we are accustomed to using when we navigate the web, Dark Websites use names of Tor hidden services. These are always 16-character values prepended to the .onion top-level domain. Any computer that runs Tor software can host a hidden (e.g., web) service. Dark Web users often find names out of band, for example, from pastebin or Dark Web market lists.
Tor software operating on a Tor host will create a local file directory, assign a port number for the service, and generate a public-private key pair when it configures a hidden service. Tor software creates a 16-character hostname by first computing a hash of the public key of that key pair and then converting the first 80 bits of this hash from a binary value to ASCII to make the resulting 16 characters conform to the "letter digit hyphen" requirement for the Domain Name System (DNS) protocol.
Dark Web visitors do not use the public DNS to resolve .onion names to Internet Protocol (IP) addresses – instead, resolution occurs using the entirely separate Tor hidden service protocol. This protocol helps services make their existences known and helps clients find services, while preserving the anonymity and the location (IP address) of both client and service. Both the client and the hidden service host have active roles in this process.
First, a Tor host "advertises" a hidden service by creating and publishing a service descriptor to a distributed directory service. This descriptor contains the hidden service public key and a list of Tor nodes that will serve as introduction points, trusted intermediaries for the hidden service. Next, the Tor host creates connections to the introduction points it has listed. Any Tor client that wants to connect to the hidden service can now do so through these introduction points.
To connect to a hidden service, a Tor client queries the directory service for the service descriptor. It randomly chooses an introduction point from the list in the service descriptor. The Tor client then randomly chooses a rendezvous point in the Tor network, anonymously connects to the chosen introduction point through the rendezvous point, and transmits a message to the hidden service via the introduction point. This message contains the identity of the rendezvous point, encrypted using the hidden service's public key, and material needed to begin a cryptographic "handshake." The hidden service also creates a connection back to this chosen rendezvous point and sends a message that completes the cryptographic handshake. At this point, the client and hidden service have set up a private network pathway that is resistant to surveillance – and they can exchange data anonymously and confidentially.
Why Are All Dark Websites in the .onion Top-Level Domain ?
The .onion top-level domain is reserved for hidden service names. Contrary to popular misconception, ICANN did not delegate .onion from the public root of the DNS. The Internet Engineering Task Force (IETF) designated .onion as a special-use top-level domain (see RFC 7686) to be used in implementing an anonymous service with strong confidentiality characteristics, deemed to be "desired new functionality" (see RFC 6761).
Can I Visit the Dark Web? Should I?
You may want to use Tor to avail yourself of some of the Dark Web's services. Even though you might benefit from increased anonymity on the dark web, this is never a reason to engage in illegal activities.
In my next post, I'll explain how to prepare to navigate the Dark Web. We'll consider the risks you might face and discuss measures you must take to protect yourself.