User-agent: * Disallow: *.txt Disallow: *.php Disallow: *.html