Skip to main content

The Dark Web: The Land of Hidden Services

Dark web 225x175

According to Internet Live Stats, the World Wide Web passed the one billion website benchmark in 2014 and is still hovering around that figure. The publishers of these billion websites compete for search engine relevance and the attention of nearly 3.6 billion Internet users. There is another part of the Web, however, where publishers and visitors want to navigate websites and conduct business transactions in secret. This is the Dark Web, a land of hidden services, where leaving no tracks and preserving anonymity are valued over search engine rankings and web experience personalization.

The Dark Web

The Dark Web is an important part of the Internet ecosystem. It allows for the publication of websites and the dissemination of information without revealing the publisher's identity or location. The Dark Web is only accessible through services such as Tor. Many users use Tor and similar services as a means to provide freedom of expression and association, access to information, and the right to privacy.

The Deep Web

The Deep Web is the collection of all websites that are not indexed by search engines. Some Deep Websites are unconventional marketplaces that offer a disturbing range of products or services. You can buy or broker illegal drugs, weapons, counterfeit goods, stolen credit cards or breached data, digital currencies, malware, national identity cards or passports. You can contract digital or criminal services, ranging from spam campaigns to distributed denial-of-service (DDoS) attacks. Novices can even purchase eBooks that explain how to attack websites, steal identities or otherwise profit from illegal activities.

But you can also use the Deep Web to anonymously share information with media outlets such as the New York Times, the Washington Post, The Intercept and others, as well use search engines without giving up your privacy, or engage in legitimate e-commerce network such as OpenBazaar.

Leave No Trace: Encryption and Evasion for the Dark Web

Many Internet users use encryption – for example, Virtual Private Networks (VPNs) – to keep Internet activities private. VPN connections typically abide by the conventional behavior of Internet routing for (1) the determination of an end-to-end path from a user's computer to a server that hosts content that the user wants to access, and (2) the bidirectional transmission of requests and response traffic along this path. Conventional routing, however, is susceptible to traffic analysis, a surveillance technique that can reveal traffic origins, destinations and times of transmission to third parties. Traffic analysis is related to metadata collection, a topic we've covered in an earlier post.

Tor networks are popular solutions for maintaining anonymity and privacy and for defeating traffic analysis. Who uses Tor? Journalists, whistleblowers, dissidents, or generally any Internet users who do not want third parties to track their behavior or interests. Tor serves many good purposes, but also attracts Dark Web users wanting to keep their activities or marketplaces secret and untraceable.

Like VPNs, Tor networks use virtual tunnels, but unlike VPNs, these tunnels don't connect clients directly to servers. Instead, Tor clients create circuits through relay points in the Tor network. Tor circuits have three important properties.

  • No relay point knows the entire path between circuit endpoints.
  • Each connection between relays is uniquely encrypted.
  • All connections are short-lived to prevent observation of behavior over time.

Constructed using these properties, these Tor private network pathways defeat traffic analysis and support the ability to publish content without revealing identity or location.

Names for Dark Websites

Unlike the human-readable domain names that we are accustomed to using when we navigate the web, Dark Websites use names of Tor hidden services. These are always 16-character values prepended to the .onion top-level domain. Any computer that runs Tor software can host a hidden (e.g., web) service. Dark Web users often find names out of band, for example, from pastebin or Dark Web market lists.

Tor software operating on a Tor host will create a local file directory, assign a port number for the service, and generate a public-private key pair when it configures a hidden service. Tor software creates a 16-character hostname by first computing a hash of the public key of that key pair and then converting the first 80 bits of this hash from a binary value to ASCII to make the resulting 16 characters conform to the "letter digit hyphen" requirement for the Domain Name System (DNS) protocol.

Dark Web visitors do not use the public DNS to resolve .onion names to Internet Protocol (IP) addresses – instead, resolution occurs using the entirely separate Tor hidden service protocol. This protocol helps services make their existences known and helps clients find services, while preserving the anonymity and the location (IP address) of both client and service. Both the client and the hidden service host have active roles in this process.

First, a Tor host "advertises" a hidden service by creating and publishing a service descriptor to a distributed directory service. This descriptor contains the hidden service public key and a list of Tor nodes that will serve as introduction points, trusted intermediaries for the hidden service. Next, the Tor host creates connections to the introduction points it has listed. Any Tor client that wants to connect to the hidden service can now do so through these introduction points.

To connect to a hidden service, a Tor client queries the directory service for the service descriptor. It randomly chooses an introduction point from the list in the service descriptor. The Tor client then randomly chooses a rendezvous point in the Tor network, anonymously connects to the chosen introduction point through the rendezvous point, and transmits a message to the hidden service via the introduction point. This message contains the identity of the rendezvous point, encrypted using the hidden service's public key, and material needed to begin a cryptographic "handshake." The hidden service also creates a connection back to this chosen rendezvous point and sends a message that completes the cryptographic handshake. At this point, the client and hidden service have set up a private network pathway that is resistant to surveillance – and they can exchange data anonymously and confidentially.

Why Are All Dark Websites in the .onion Top-Level Domain ?

The .onion top-level domain is reserved for hidden service names. Contrary to popular misconception, ICANN did not delegate .onion from the public root of the DNS. The Internet Engineering Task Force (IETF) designated .onion as a special-use top-level domain (see RFC 7686) to be used in implementing an anonymous service with strong confidentiality characteristics, deemed to be "desired new functionality" (see RFC 6761).

Can I Visit the Dark Web? Should I?

You may want to use Tor to avail yourself of some of the Dark Web's services. Even though you might benefit from increased anonymity on the dark web, this is never a reason to engage in illegal activities.

In my next post, I'll explain how to prepare to navigate the Dark Web. We'll consider the risks you might face and discuss measures you must take to protect yourself.

Comments

    Michael Wick  22:13 UTC on 05 July 2017

    A quick note on VPNs. If you're going to use one, make sure the service doesn't keep logs. I've been using ExpressVPN for years and have had no issues. It's also one of the few services (if not the only) to host their own Tor servers.

    David A. Holland Zoney  12:44 UTC on 22 July 2017

    fEEDBACK APPRECIATED AS FOR TRANSPARENCY AND PROBLEM SOLVING. twitterdot com/daveholland1964/status/888719699080564737 twitter dot com/daveholland1964/status/888784803176349696

    pascalccbb  09:36 UTC on 04 August 2017

    thx for this content

Domain Name System
Internationalized Domain Name ,IDN,"IDNs are domain names that include characters used in the local representation of languages that are not written with the twenty-six letters of the basic Latin alphabet ""a-z"". An IDN can contain Latin letters with diacritical marks, as required by many European languages, or may consist of characters from non-Latin scripts such as Arabic or Chinese. Many languages also use other types of digits than the European ""0-9"". The basic Latin alphabet together with the European-Arabic digits are, for the purpose of domain names, termed ""ASCII characters"" (ASCII = American Standard Code for Information Interchange). These are also included in the broader range of ""Unicode characters"" that provides the basis for IDNs. The ""hostname rule"" requires that all domain names of the type under consideration here are stored in the DNS using only the ASCII characters listed above, with the one further addition of the hyphen ""-"". The Unicode form of an IDN therefore requires special encoding before it is entered into the DNS. The following terminology is used when distinguishing between these forms: A domain name consists of a series of ""labels"" (separated by ""dots""). The ASCII form of an IDN label is termed an ""A-label"". All operations defined in the DNS protocol use A-labels exclusively. The Unicode form, which a user expects to be displayed, is termed a ""U-label"". The difference may be illustrated with the Hindi word for ""test"" — परीका — appearing here as a U-label would (in the Devanagari script). A special form of ""ASCII compatible encoding"" (abbreviated ACE) is applied to this to produce the corresponding A-label: xn--11b5bs1di. A domain name that only includes ASCII letters, digits, and hyphens is termed an ""LDH label"". Although the definitions of A-labels and LDH-labels overlap, a name consisting exclusively of LDH labels, such as""icann.org"" is not an IDN."