# Allow all search engines to crawl all content User-agent: * Disallow: User-agent:Googlebot Allow :/ User-agent:Googlebot-Mobile Allow :/ User-agent:Googlebot-Image Allow :/ User-agent:Mediapartners-Google Allow :/ User-agent:Bingbot Allow :/ User-agent:Slurp Allow :/ User-agent:msnbot Allow :/ User-agent:msnbot-media Allow :/ User-agent:yahoo-mmcrawler Allow :/ User-agent:Yandex Allow :/ # Disallow specific directories that are not meant to be indexed Disallow: /login/ Disallow: /admin/ Disallow: /admin/login/ Disallow: /cgi-bin/ Disallow: /temp/ Disallow: /public/ Disallow: /vendor/ Disallow: /node_modules/ Disallow: /config/ Disallow: /bootstrap/ Disallow: /resources/ # Disallow specific files that should not be indexed Disallow: /login Disallow: /register Disallow: /password/reset Sitemap: https://www.trimurty.com/sitemap.xml # Block specific crawlers (example: BadBot) User-agent: BadBot Disallow: / User-agent: EvilScraper Disallow: /