Dave's Brain

Crawlers

Crawlers in the last week

CrawlerVisits
BLEXBot/1.0667
MJ12bot/v1.4.8154
Cliqzbot/1.0135
SeznamBot/3.2133
DotBot/1.1118
bingbot/2.0109
MJ12bot/v1.4.790
Googlebot/2.186
YandexBot/3.072
Yahoo! Slurp67
istellabot/t.1.1364
Windows NT 10.0; WOW6454
BUbiNG (+http://law.di.unimi.it/BUbiNG.html)53
proximic50
ia_archiver38
coccocbot-web/1.038
DeuSu/5.0.237
AhrefsBot/5.237
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)35
Qwantify/2.4w27
GrapeshotCrawler/2.023
Baiduspider/2.022
WOW6422
SemrushBot/1.2~bl21
archive.org_bot20
Windows NT 5.1; rv:6.0.220
Linux x86_6417
Clickagy Intelligence Bot v215
SiteExplorer/1.1b12
U9
CCBot/2.0 (http://commoncrawl.org/faq/)9
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36/Nutch-1.139
Daum/4.18
Steeler/3.57
Exabot/3.07
istellabot/t.16
Plukkie/1.66
Windows NT 6.1; WOW646
Findxbot/1.05
GetIntent Crawler (http://getintent.com/bot.html)5
MSIE 10.05
linkdexbot/2.05
MaxPointCrawler/Nutch-1.10 (maxpoint.crawler at maxpointinteractive dot com)5
Dataprovider.com4
spbot/5.0.34
Windows NT 6.3; WOW644
Win643
Macintosh; Intel Mac OS X 10_12_33
Uptimebot/1.03
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36/Nutch-1.123
ltx71 - (http://ltx71.com/)3
Ubuntu2
IAS crawler (ias_crawler; http://integralads.com/site-indexing-policy/)2
Macintosh; Intel Mac OS X 10_11_62
MSIE 9.02
-2
Macintosh; Intel Mac OS X 10_9_22
SafeDNSBot (https://www.safedns.com/searchbot)2
GarlikCrawler/1.2 (http://garlik.com/, crawler@garlik.com)2
amd64 Windows 2003 5.22
Intel Mac OS X 10.92
oBot/2.3.12
Windows NT 5.1; rv:6.02
Macintosh; Intel Mac OS X 10_9_32
AwarioSmartBot/1.0 (+https://awario.com/bots.html; bots@awario.com)2
panscient.com2
SEOkicks-Robot2
SMTBot/1.02
linkdexbot/2.22
yoozBot-2.22
Macintosh; Intel Mac OS X 10_11_41
crawler4j (https://github.com/yasserg/crawler4j/)1
amd64 Linux 3.16.0-4-amd641
MegaIndex.ru/2.01
X11; Linux x86_641
YisouSpider1
compatible; ips-agent1
Nutch/Nutch-2.3.11
Windows NT 5.11
rogerbot/1.1 (http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, rogerbot-crawler+pr4-crawler-15@moz.com)1
netEstate NE Crawler (+http://www.website-datenbank.de/)1
Contacts-Crawler/0.1 (+http://www.scrapinghub.com)1
Wotbox/2.01 (+http://www.wotbox.com/bot/)1
Python-urllib/2.71
MSIE 7.01
psbot/0.1 (+http://www.picsearch.com/bot.html)1
Mozilla/5.0/Firefox/42.0 - nbertaupete95(at)gmail.com1
nutch-1.4/Nutch-1.41
Windows NT 6.11
Yeti/1.11
Prlog/1.01
Trident/7.01
SurdotlyBot/1.01
amd64 Linux 4.9.0-3-amd641
Aboundex/0.3 (http://www.aboundex.com/crawler/)1
http://temnos.com; rv:38.01

Copyright © 2008-2017, dave - Code samples on Dave's Brain is licensed under the Creative Commons Attribution 2.5 License. However other material, including English text has all rights reserved.