Majestic-12 : DSearch : Breakdown by top level domain
The following table shows breakdown of URLs by their top level domain name. All newly found
URLs will be grouped into buckets relating to the same top level domain. This enables those
who run client software to make a choice of domains they prefer to crawl.
||Top level domain name
||Total number of URLs
||Estimated size (GB)
||Issued to nodes
||Already crawled URLS
Last updated: 16 Oct 2018 16:00:04:190 GMT
Notes: some of the top level domains designations (such as .ipaddr) were used for internal purposes and do not
actually exist in official list of Top Level Domains.
Estimated size is based on observed average of 20KB per URL, this takes into account average number of errors, this does not include size of the same urls being recrawled - merely an esimated size of all that url space.