User-agent: * Disallow: /ai-assistant/ Disallow: /convert-pdf-black-white/ Disallow: /convert-pdf-to-kindle/ Disallow: /extract-invoice-data-pdf/ Disallow: /pdf-to-text/ Disallow: /remove-watermark/ Disallow: /searchable-pdf/ Disallow: /chat/ Disallow: /Gensw_51ccac142d894c5.html # Pre-launch: the 200M paper detail pages and journal pages are noindex — # keep crawlers off them until PAPERS/JOURNALS_INDEXING_ENABLED flip on # (drop the two Disallow lines at launch). /papers/top/* topic pages and # the /papers listing are LIVE indexable surface: longest-match Allow wins. Disallow: /papers/ Allow: /papers/top/ Allow: /papers/$ Disallow: /journal/ Sitemap: https://paperguide.ai/sitemap_index.xml