Dave's Brain

Crawlers

Crawlers in the week ending 2018-apr-15
CrawlerVisits
DotBot/1.1877
MJ12bot/v1.4.8190
YandexBot/3.0146
bingbot/2.0124
istellabot/t.1.13120
Cliqzbot/2.0102
MauiBot (crawler.feedback+wc@gmail.com)97
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)97
SeznamBot/3.293
Googlebot/2.162
BUbiNG (+http://law.di.unimi.it/BUbiNG.html#wc)55
BUbiNG (+http://law.di.unimi.it/BUbiNG.html)52
GrapeshotCrawler/2.048
AhrefsBot/5.247
proximic40
ia_archiver38
Windows NT 10.0; WOW6434
SemrushBot/1.2~bl32
coccocbot-web/1.030
Windows NT 5.1; rv:6.0.229
Qwantify/2.4w29
MJ12bot/v1.4.720
python-requests/2.18.420
compatible;contxbot/1.017
MaxPointCrawler/Nutch-1.10 (maxpoint.crawler at maxpointinteractive dot com)16
Windows NT 6.1; WOW6414
MBCrawler/1.0 (https://monitorbacklinks.com)12
Linux x86_6411
Barkrowler/0.7 (+http://www.exensa.com/crawl)11
Yeti/1.111
ExtLinksBot/1.510
ltx71 - (http://ltx71.com/)10
Exabot/3.08
Steeler/3.58
archive.org_bot7
CCBot/2.0 (http://commoncrawl.org/faq/)7
Macintosh; Intel Mac OS X 10_10_17
Clickagy Intelligence Bot v26
DeuSu/5.0.26
Macintosh; Intel Mac OS X 10_13_15
crawler (crawler.feedback@gmail.com)5
U5
Win644
Macintosh; Intel Mac OS X 10_13_34
linkdexbot/2.04
python-requests/2.11.03
spbot/5.0.33
YisouSpider3
SEOkicks-Robot3
Python-urllib/1.172
Dataprovider.com2
Windows NT 6.3; WOW642
yoozBot-2.22
WOW642
uCrawler/1.02
Hatena::Fetcher/0.01 (master) Furl/3.062
Windows NT 6.12
oBot/2.3.12
linkdexbot/2.22
SafeDNSBot (https://www.safedns.com/searchbot)2
SiteExplorer/1.1b2
CRMNLCrawlAgent/Nutch-1.15-SNAPSHOT1
null1
Intel Mac OS X 10.91
NetSeer crawler/2.01
DomainStatsBot/1.0 (http://domainstats.io/our-bot)1
AHC/1.01
NYU Internet Census (https://scan.lol; research@scan.lol)1
Macintosh; Intel Mac OS X 10_9_31
panscient.com1
Macintosh; Intel Mac OS X 10_9_41
Intel Mac OS X 10.101
Macintosh; Intel Mac OS X 10_12_11
Mappy/1.01
X11; Linux x86_641
MSIE 9.01
Macintosh; Intel Mac OS X 10_11_41
Windows NT 6.1; rv:28.01
-1
Macintosh; Intel Mac OS X 10_9_21
GarlikCrawler/1.2 (http://garlik.com/, crawler@garlik.com)1
SurdotlyBot/1.01

Copyright © 2008-2018, dave - Code samples on Dave's Brain is licensed under the Creative Commons Attribution 2.5 License. However other material, including English text has all rights reserved.