Hiding on the Dark Web just got a whole lot easier

Only ten to twenty percent of the web can be searched and indexed by search engines, and it’s this reason why the remainder of the web gets referred to as the Dark Web, or the Darknet. While many people think of the darknet as being the home of criminals and miscreants the fact is that it’s also home to organisations and universities who still want the benefit of inter-connectedness but without their sites and information being visible to the public. They want to keep them private.

The darknet operates under what many people would regard as a privacy paradox, while anyone who knows a dark web site’s address can visit it, no one can figure out who hosts that site or where it operates from. It hides in plain sight. But now, changes in the anonymity tools underlying the darknet promise to make a new kind of privacy possible. Soon anyone will be able to create their own corner of the internet that’s not just anonymous and untraceable, but entirely undiscoverable without an invite.

Over the coming months, the non profit Tor (short for The Onion Router) project will upgrade the security and privacy of the service’s “onion services,” or “hidden services,” that are increasingly becoming the foundation of the darknet’s anonymity. While the majority of people who run the Tor project’s software use it to browse the web anonymously, and circumvent censorship in countries like Iran and China, the group also maintains code that allows anyone to host an anonymous website or server - the basis for the darknet.

That code is now getting a revamp, set to go live sometime later this year, designed to both strengthen its encryption and to let administrators easily create fully secret darknet sites that can only be discovered by those who know a long string of impossible to guess characters. And those software tweaks, says Tor Project co-founder Nick Mathewson, could not only allow tighter privacy on the darknet, but also help serve as the basis for a new generation of encryption applications.

“Someone can create a hidden service just for you that only you would know about, and the presence of that particular hidden service would be non-discoverable,” says Mathewson, who helped to code some of the first versions of Tor in 2003, “as a building block, that would provide a much stronger basis for relatively secure and private systems than we’ve had before.”

Most darknet sites today make no secret of their existence, widely publicizing their “.onion” web addresses on the regular web and social media for potential visitors. Any whistleblower can visit WikiLeaks’ anonymous upload system, for instance, by pasting wlupld3ptjvsgwqw.onion into their Tor browser, and many thousands of drug customers and dealers knew that the notorious dark web drug market Silk Road could be found at silkroadvb5piz3r.onion before the FBI took it offline.

But even without knowing a Tor hidden service’s address, another trick has allowed snoops, security firms, hackers, and law enforcement to discover them. Tor’s network comprises volunteers’ computers that serve as “nodes,” bouncing traffic around the globe. Anyone can position their computer as a particular sort of node - one of thousands of “hidden service directories” that route visitors to a particular hidden service.

For that routing system to work, all hidden services have to declare their existence to those directories. A study released at the hacker conference Defcon last year showed that more than a hundred of the 3,000 or so hidden service directories were secretly crawling every site whose address they learned – in a similar way to the way Google indexes the surface web -  in order to scan the dark web for previously undiscovered sites.

“The only people who should know about your hidden service are the people you tell about it,” says John Brooks, the creator of the Tor based chat program Ricochet, “that’s a pretty simple concept, and it’s currently not true.”

Now, the next generation of hidden services will use a clever method to protect the secrecy of those addresses. Instead of declaring their .onion address to hidden service directories, they’ll instead derive a unique cryptographic key from that address, and give that key to Tor’s hidden service directories. Any Tor user looking for a certain hidden service can perform that same derivation to check the key and route themselves to the correct darknet site. But the hidden service directory can’t derive the .onion address from the key, preventing snoops from discovering any secret darknet address.

“The Tor network isn’t going to give you any way to learn about an onion address you don’t already know,” says Mathewson.

The result, Mathewson says, will be darknet sites with new, stealthier applications. A small group of collaborators could, for instance, host files on a computer known to only to them. No one else could ever even find that machine, much less access it. You could host a hidden service on your own computer, creating a way to untraceably connect to it from anywhere in the world, while keeping its existence secret from snoops. Mathewson himself hosts a password-protected family wiki and calendar on a Tor hidden service, and now says he’ll be able to do away with the site’s password protection without fear of anyone learning his family’s weekend plans.

The next generation of hidden services will also switch from using 1024-bit RSA encryption keys to shorter but harder to crack ED-25519 elliptic curve keys. And the hidden service directory changes mean that hidden service URL’s will change, too, from 16 characters to 50. But Mathewson argues that change doesn’t affect the dark web addresses’ usability since they’re already too long to memorize.

Mathewson has bigger ambitions for the secrecy changes, too. He hopes they can create more tools that allow untraceable, private communication, like Ricochet and the Tor-based file sharing application Onionshare. Those apps automatically create Tor hidden services on their users’ machines for private communications, so preventing anyone from discovering those private Tor instances will make similar apps easier to build and more secure.

“It’s these things that are using hidden services as a building block that are going to get far stronger, with much more privacy than they had before,” says Mathewson.

Over the years the security, and privacy, of Tor hidden services has come under scrutiny since a massive FBI purge took dozens of dark web sites offline - including a reincarnation of the Silk Road, in late 2014. The attack that allowed that takedown of supposedly untraceable sites, now believed to have been developed by Carnegie Mellon University security researchers and obtained by the FBI with a subpoena, also took advantage of the network’s hidden service directories.

The researchers found a way to “mark” hidden services’ Tor traffic with a unique piece of data that could be recognized by both the node that hidden services first connected to - which knows the service’s IP address - and the address tracked by the hidden service directory - which knows its .onion address. By combining the data between those two computers, police had enough information to pin down the locations of servers running the illegal sites and seize them.

The Tor Project fixed the flaw that allowed those attacks within days of its discovery, says Mathewson. But even if a similar vulnerability were found in the future, the new hidden service directory system would in theory mean the most secret hidden services would remain safe, and that law enforcement wouldn’t be able to use the attack on any site whose address it didn’t know - although ones with widely publicized addresses might still be vulnerable.

That potential to foil law enforcement raises the inevitable question. Will undiscoverable hidden services become a magnet for the worst tenants of the Darknet, including markets for stolen data, hacking tools, or child pornography? Mathewson offers the answer that Tor and much of the rest of the encryption world has maintained for years - that strong privacy tools offer a societal trade off, and one that’s worth making.

“If the only way to ensure that socially deleterious uses of the internet were insecure is to make everyone insecure, I don’t think that leaves the world better off,” he says, “on the whole, humanity deserves privacy and does better with it than without it, even if some of the things people do with that privacy are things we’d prefer to control.”