User-agent: * Disallow: /cgi-bin/ Disallow: /admin/ Disallow: /poll/ Disallow: /upload/ User-Agent: DISCo Pump, Wget, WebZIP, Teleport Pro, WebSnake, Offline Explorer, Web-By-Mail Disallow: /