You may have come across terms such as the Deep Web or the Dark Web. Often these terms are used interchangeably, but they actually mean something quite different. Let’s take a look at what the two terms mean and how they differ fundamentally.
How internet search engines work
Let’s start a bit broadly. If we take any common internet search engine such as Google, Bing or others, they offer us websites according to the query we ask. This way we can find products in e-shops, news, reviews, guides but also videos, pictures or anything else we can think of.
In order for search engines to be able to offer relevant content to our queries, they first need to know what is actually on the Internet. Search engines therefore use robots, called crawlers, which regularly crawl websites and try to evaluate as accurately as possible what is on the page and what queries a given page would be relevant to. If the Crawler’s algorithms deem it appropriate, the page in question is included in the search engine’s index, which means that it can appear in search results from that point on.
According to various criteria, the search engine then ranks the site and offers search results. So if we search for something, we get results virtually instantly. Search engines do not crawl the entire Internet in a few tenths of a second, because the crawler of that search engine has already done that sometime in the past.
So search engines regularly crawl sites and individual pages, looking for new content, or updating information about older content if it has changed. Crawlers also work in such a way that if they visit one page, they then proceed to visit individual links on that page. In this way, the crawler can go to more and more pages and discover content. Site owners can assist this process by, for example, submitting a so-called sitemap to search engines, which is just the content of the site and all the pages on it in a nutshell.
Deep web is content that search engines can’t see
We started broadly because this information is essential to understanding how the Deep Web works. The Deep Web is the part of the Internet that is not visible or accessible to search engines. Thus, crawlers cannot visit, crawl, and index these sites for some reason.
It would seem that this content, again, is not that much, but the opposite is true. In fact, the Deep Web is an absolutely essential part of the Internet. It is, for example, the following sites:
- Password-protected sites or services – Search engines can’t get to these sites because it’s only possible with knowledge of the password. For example, we could include in this category email accounts that only we have private access to
- Sites that search engines don’t know about – Some sites are intentionally hidden and have no links to them. Search engines are then unaware of their existence and cannot crawl them. In this category we could also include, among other things, very cluttered sites that do not have a clear structure and search engines simply do not deal with them because they are not of good quality.
- Sites that the search engine will not pass for technical reasons – In order for a crawler to pass a page, it needs to meet some technical requirements. Some sites may not meet these requirements, or are programmed intentionally so that the crawler will not understand them and therefore not crawl them.
There is a lot of content on the Deep Web, and basically we all use some of this Deep Web. It’s nothing special, and that search engines can’t access some of the content is perfectly fine and entirely desirable.
The Dark Web is only accessible with the help of special software
The Dark Web, or Dark Web, is the part of the Internet that is hosted on special networks called the Darknet. These networks make use of the classical Internet, but in order to connect to them, special software or configuration is needed. The Dark web is thus a part of the Deep web because it cannot be visited just like that, but some requirements need to be met. This can be smaller networks, but also large networks like Tor, Freenet or I2P.
Each of these networks is connected to in a different way, but typically need software that anonymises the user and encrypts the content using different methods. The dark web is thus a place where it is all about full user anonymity, but also completely zero censorship of content.
Is it legal to connect to the Dark Web?
Yes. Using the Dark Web is perfectly legal and there are a number of perfectly reasonable reasons why this complete anonymity is required. A large portion of the Dark Web population is just people like you who want to use the Internet completely anonymously, bypass censorship, and protect their privacy. It’s a safe way for whistleblowers, activists or journalists to share information without risking persecution from their government or anyone else. Various social networks, such as Facebook, also work on the Dark Web, because users from countries where they would face sanctions for using the network can connect and thus have access to uncensored information.
However, since this is a completely uncensored part of the Internet, it’s probably not surprising that the Dark Web is also a place where a lot of illegal activities are taking place, which you can probably figure out on your own.
Simply put, therefore, by accessing the Dark Web itself, it does not break any laws, but its use can be very dangerous if we do not protect ourselves.