Majestic-12 logo
Distributed Search Engine
Enter domain or URL:
Home | DSearch | Projects | Stats | Download | Forum | Blog | About

Majestic-12 : DSearch : User Info

Nickname:

|MatMan|

Country:

Germany Germany

ID:

1251

Team:

Not in a team yet. Join one now!

Joined:

29/08/2006 10:20:00

URLs crawled:

70,857,052,207

Data (MB)*:

1,987,402,602

Peers (nodes):

Version

Name

Last seen

Uptime

Downstream
(Kbits)**

Upstream
(Kbits)**

Current run successes
(failures****)

Today
URLs (MBs)***

Last 30
days

Total

v1.7.16 (.NET 4.5)   Linux

MatVM-XU01

19/04/2018 15:35:16

46 mins 2 secs

6,268

60

77.3%
(DNS: 1.8%, Conn: 0.8%
T-out: 0.9%, 404s: 6.5%
Robots: 2.4%, Other: 10.3%)

2,947,407
(82,532)

92,257,798
(2,830,311)

3,168,106,917
(88,153,435)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU05

19/04/2018 15:38:48

2 hours 31 mins 5 secs

6,965

58

71.1%
(DNS: 1.9%, Conn: 1.6%
T-out: 1.4%, 404s: 5.7%
Robots: 4.1%, Other: 14.1%)

2,577,045
(70,402)

93,689,793
(2,878,382)

2,074,379,703
(57,405,917)

v1.7.16 (.NET 4.5)   Linux

MatSAN-XU07

19/04/2018 15:40:01

16 mins 1 sec

4,122

39

65.8%
(DNS: 2.7%, Conn: 0.3%
T-out: 1.0%, 404s: 9.7%
Robots: 3.0%, Other: 17.5%)

2,544,563
(71,344)

80,166,609
(2,519,914)

1,879,046,077
(53,309,507)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU12

19/04/2018 15:45:15

46 mins 2 secs

7,533

71

67.8%
(DNS: 3.2%, Conn: 1.7%
T-out: 2.0%, 404s: 8.5%
Robots: 2.4%, Other: 14.4%)

2,769,474
(74,561)

92,412,899
(2,823,679)

397,142,364
(12,782,397)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU07

19/04/2018 15:45:33

1 hour 46 mins 3 secs

5,783

55

73.1%
(DNS: 2.4%, Conn: 1.1%
T-out: 1.4%, 404s: 5.9%
Robots: 3.7%, Other: 12.5%)

2,838,232
(80,852)

94,242,004
(2,906,944)

851,861,271
(25,246,652)

v1.7.16 (.NET 4.5)   Linux

MatSAN-XU04

19/04/2018 15:46:42

5 hours 31 mins 12 secs

5,346

54

74.0%
(DNS: 2.0%, Conn: 1.0%
T-out: 1.5%, 404s: 6.0%
Robots: 3.0%, Other: 12.4%)

2,654,351
(75,894)

80,105,275
(2,522,384)

3,139,160,567
(86,150,016)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU10

19/04/2018 15:49:33

16 mins 0 sec

6,151

79

69.6%
(DNS: 3.8%, Conn: 2.1%
T-out: 2.1%, 404s: 6.3%
Robots: 2.6%, Other: 13.5%)

2,722,690
(75,475)

93,619,682
(2,882,620)

654,354,616
(20,136,759)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU05

19/04/2018 15:49:57

4 hours 46 mins 8 secs

6,553

73

73.4%
(DNS: 2.2%, Conn: 0.8%
T-out: 1.3%, 404s: 5.9%
Robots: 3.6%, Other: 12.8%)

2,910,285
(83,886)

76,982,562
(2,391,448)

738,277,175
(24,545,069)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU10

19/04/2018 15:49:58

4 hours 46 mins 8 secs

6,529

68

73.5%
(DNS: 2.0%, Conn: 0.9%
T-out: 1.1%, 404s: 6.1%
Robots: 3.7%, Other: 12.8%)

2,869,724
(80,198)

74,698,114
(2,324,878)

579,160,090
(19,216,834)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU04

19/04/2018 15:49:58

4 hours 46 mins 9 secs

6,478

64

72.1%
(DNS: 2.3%, Conn: 0.7%
T-out: 1.2%, 404s: 6.7%
Robots: 4.0%, Other: 13.0%)

1,022,193
(31,545)

72,216,923
(2,261,491)

769,816,508
(25,582,469)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU02

19/04/2018 15:49:58

4 hours 46 mins 9 secs

6,069

73

73.1%
(DNS: 1.4%, Conn: 0.6%
T-out: 1.2%, 404s: 6.4%
Robots: 2.8%, Other: 14.5%)

2,668,703
(77,238)

77,596,097
(2,401,786)

781,728,655
(25,986,838)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU09

19/04/2018 15:49:58

4 hours 46 mins 9 secs

6,708

75

73.5%
(DNS: 2.4%, Conn: 0.9%
T-out: 1.0%, 404s: 6.3%
Robots: 3.2%, Other: 12.9%)

2,976,310
(86,742)

64,886,536
(2,047,141)

684,694,767
(22,760,223)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU08

19/04/2018 15:49:59

4 hours 46 mins 10 secs

6,317

71

71.3%
(DNS: 1.7%, Conn: 1.0%
T-out: 1.4%, 404s: 6.8%
Robots: 3.1%, Other: 14.6%)

2,792,294
(81,097)

72,584,299
(2,260,362)

739,732,901
(24,586,811)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU07

19/04/2018 15:49:59

4 hours 46 mins 9 secs

6,604

66

73.3%
(DNS: 1.8%, Conn: 0.9%
T-out: 1.4%, 404s: 6.1%
Robots: 3.3%, Other: 13.2%)

2,925,829
(85,076)

60,240,343
(1,814,871)

722,490,225
(23,989,148)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU01

19/04/2018 15:49:59

4 hours 46 mins 10 secs

6,298

61

72.5%
(DNS: 1.8%, Conn: 0.7%
T-out: 1.6%, 404s: 6.1%
Robots: 3.6%, Other: 13.8%)

2,843,257
(81,892)

76,886,348
(2,384,091)

893,111,131
(28,514,111)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU06

19/04/2018 15:49:59

4 hours 46 mins 10 secs

6,635

69

73.6%
(DNS: 1.8%, Conn: 0.7%
T-out: 1.2%, 404s: 5.8%
Robots: 3.2%, Other: 13.7%)

2,712,980
(80,120)

77,712,851
(2,417,082)

770,516,857
(25,610,965)

v1.7.16 (.NET 4.5)   Linux

MatServ-XU03

19/04/2018 15:50:02

4 hours 46 mins 7 secs

6,465

71

72.3%
(DNS: 2.0%, Conn: 0.7%
T-out: 1.0%, 404s: 6.6%
Robots: 3.9%, Other: 13.5%)

2,884,566
(83,997)

78,071,369
(2,418,581)

785,121,393
(26,089,783)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU03

19/04/2018 15:50:21

16 mins 1 sec

4,564

74

67.8%
(DNS: 1.6%, Conn: 1.2%
T-out: 2.3%, 404s: 9.2%
Robots: 3.6%, Other: 14.3%)

2,773,002
(76,555)

94,151,482
(2,882,827)

3,256,498,610
(90,721,527)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU02

19/04/2018 15:50:45

16 mins 0 sec

5,020

101

72.4%
(DNS: 4.1%, Conn: 1.0%
T-out: 2.7%, 404s: 7.9%
Robots: 1.8%, Other: 10.3%)

3,005,399
(82,471)

95,388,317
(2,904,924)

3,045,953,595
(85,577,120)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU09

19/04/2018 15:50:46

16 mins 0 sec

4,172

77

59.9%
(DNS: 11.1%, Conn: 0.4%
T-out: 2.2%, 404s: 7.4%
Robots: 1.2%, Other: 17.8%)

2,768,999
(75,597)

40,742,593
(1,235,026)

550,245,287
(16,890,015)

v1.7.16 (.NET 4.5)   Linux

MatSAN-XU06

19/04/2018 15:50:57

1 hour 16 mins 2 secs

5,904

68

68.7%
(DNS: 1.8%, Conn: 1.1%
T-out: 1.4%, 404s: 6.1%
Robots: 5.5%, Other: 15.4%)

2,490,456
(73,780)

79,794,207
(2,505,997)

2,906,084,371
(80,049,794)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU06

19/04/2018 15:51:06

16 mins 1 sec

4,464

59

65.9%
(DNS: 2.0%, Conn: 1.2%
T-out: 1.7%, 404s: 6.7%
Robots: 5.7%, Other: 16.7%)

2,717,091
(77,517)

93,478,990
(2,864,404)

1,491,582,262
(41,973,979)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU08

19/04/2018 15:51:17

16 mins 0 sec

6,006

63

69.7%
(DNS: 2.7%, Conn: 0.9%
T-out: 0.8%, 404s: 9.2%
Robots: 2.9%, Other: 13.9%)

2,877,190
(81,580)

93,438,669
(2,874,509)

801,563,498
(23,922,070)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU11

19/04/2018 15:51:18

16 mins 1 sec

6,155

102

67.0%
(DNS: 2.8%, Conn: 1.9%
T-out: 2.5%, 404s: 5.8%
Robots: 2.5%, Other: 17.5%)

2,803,897
(76,644)

93,229,689
(2,872,239)

472,288,598
(15,313,366)

v1.7.16 (.NET 4.5)   Linux

MatVM-XU04

19/04/2018 15:51:59

16 mins 0 sec

5,320

75

70.1%
(DNS: 3.5%, Conn: 0.8%
T-out: 1.3%, 404s: 7.3%
Robots: 2.3%, Other: 14.7%)

2,838,651
(76,216)

93,575,102
(2,867,779)

3,208,319,556
(88,797,322)

v1.7.16 (.NET 4.5)   Linux

MatSAN-XU03

19/04/2018 15:54:26

1 hour 16 mins 2 secs

5,252

64

72.2%
(DNS: 3.4%, Conn: 1.2%
T-out: 1.6%, 404s: 7.0%
Robots: 3.2%, Other: 11.5%)

2,669,459
(74,893)

79,557,278
(2,486,152)

2,923,198,261
(80,948,704)

v1.7.16 (.NET 4.5)   Linux

MatSAN-XU05

19/04/2018 15:55:21

1 hour 16 mins 3 secs

5,955

67

72.4%
(DNS: 2.3%, Conn: 0.8%
T-out: 2.8%, 404s: 6.5%
Robots: 2.7%, Other: 12.4%)

2,586,807
(74,042)

80,033,178
(2,521,577)

3,021,066,452
(83,473,632)

v1.7.16 (.NET 4.5)   Linux

MatSAN-XU08

19/04/2018 15:56:21

1 hour 16 mins 3 secs

5,630

70

71.9%
(DNS: 2.3%, Conn: 0.7%
T-out: 1.5%, 404s: 7.5%
Robots: 3.3%, Other: 12.8%)

2,617,600
(76,021)

79,124,309
(2,493,201)

1,605,155,884
(43,631,191)

v1.7.16 (.NET 4.5)   Linux

MatSAN-XU01

19/04/2018 15:57:37

1 hour 16 mins 2 secs

5,643

66

69.9%
(DNS: 4.7%, Conn: 1.2%
T-out: 1.4%, 404s: 6.2%
Robots: 3.6%, Other: 13.0%)

2,578,865
(72,092)

79,958,740
(2,519,694)

3,012,591,121
(83,213,411)

Total (29 nodes)

170,909

1,993

-

78,387,319
(2,220,259)

2,360,842,056
(73,114,294)

45,923,248,712
(1,324,579,065)

The following chart shows |MatMan|'s contribution (blue) to overall crawling effort by day.

Last updated: 19 Apr 2018 15:58:20:503 GMT

* data volumes were counted for nodes v1.0.0 or higher from 11 Apr 2005. Once indexing is complete data volumes will be updated retrospectively.
** sustained overall bandwidth usage over whole period of that session
*** URLs and Megabytes (MBs) of data crawled in that periods (Today, Last 30 days, Total)
**** DNS: DNS resolves errors, Conn: connection errors, T-out: timeouts, 404s: URL Not Found code, Robots: robots.txt blocks, Other: everything else

Copyright © Majestic-12