User-agent WebZip
User-agent larbin
User-agent b2w0.1
User-agent psbot
User-agent Python-urllib
User-agent URL_Spider_Pro
User-agent CherryPicker
User-agent EmailCollector
User-agent EmailSiphon
User-agent WebBandit
User-agent EmailWolf
User-agent ExtractorPro
User-agent CopyRightCheck
User-agent Crescent
User-agent SiteSnagger
User-agent ProWebWalker
User-agent CheeseBot
User-agent LNSpiderguy
User-agent ia_archiver
User-agent ia_archiver1.6
User-agent Alexibot
User-agent Teleport
User-agent TeleportPro
User-agent MIIxpc
User-agent Telesoft
User-agent Website Quester
User-agent moget2.1
User-agent WebZip4.0
User-agent WebStripper
User-agent WebSauger
User-agent WebCopier
User-agent NetAnts
User-agent Mister PiX
User-agent WebAuto
User-agent TheNomad
User-agent WWW-Collector-E
User-agent RMA
User-agent libWebclsHTTP
User-agent asterias
User-agent httplib
User-agent turingos
User-agent spanner
User-agent InfoNaviRobot
User-agent Harvest1.5
User-agent Bullseye1.0
User-agent Crescent Internet ToolPak HTTP OLE Control v.1.0
User-agent CherryPickerSE1.0
User-agent CherryPickerElite1.0
User-agent WebBandit3.50
User-agent NICErsPRO
User-agent Microsoft URL Control - 5.01.4511
User-agent DittoSpyder
User-agent Foobot
User-agent WebmasterWorldForumBot
User-agent SpankBot
User-agent BotALot
User-agent lwp-trivial1.34
User-agent lwp-trivial
User-agent BunnySlippers
User-agent Microsoft URL Control - 6.00.8169
User-agent URLy Warning
User-agent Wget1.6
User-agent Wget1.5.3
User-agent Wget
User-agent LinkWalker
User-agent cosmos
User-agent moget
User-agent hloader
User-agent humanlinks
User-agent LinkextractorPro
User-agent Offline Explorer
User-agent Mata Hari
User-agent LexiBot
User-agent Web Image Collector
User-agent The Intraformant
User-agent True_Robot1.0
User-agent True_Robot
User-agent BlowFish1.0
User-agent JennyBot
User-agent MIIxpc4.2
User-agent BuiltBotTough
User-agent ProPowerBot2.14
User-agent BackDoorBot1.0
User-agent toCrawlUrlDispatcher
User-agent WebEnhancer
User-agent suzuran
User-agent VCI WebViewer VCI WebViewer Win32
User-agent VCI
User-agent Szukacz1.4 
User-agent QueryN Metasearch
User-agent Openfind data gathere
User-agent Openfind 
User-agent Xenu's Link Sleuth 1.1c
User-agent Xenu's
User-agent Zeus
User-agent RepoMonkey Bait & Tacklev1.01
User-agent RepoMonkey
User-agent Microsoft URL Control
User-agent Openbot
User-agent URL Control
User-agent Zeus Link Scout
User-agent Zeus 32297 Webster Pro V2.9 Win32
User-agent Webster Pro
User-agent EroCrawler
User-agent LinkScan8.1a Unix
User-agent Keyword Density0.9
User-agent Kenjin Spider
User-agent Iron331.0.2
User-agent Bookmark search tool
User-agent GetRight4.2
User-agent FairAd Client
User-agent Gaisbot
User-agent Aqua_Products
User-agent Radiation Retriever 1.1
User-agent Flaming AttackBot
User-agent Oracle Ultra Search
User-agent MSIECrawler
User-agent PerMan
User-agent searchpreview
Disallow 

User-agent 
Disallow cgi-bin
Disallow images