User-agent: * Disallow: /admin/ Disallow: /storage/ Disallow: /vendor/ Disallow: /config/ Disallow: /bootstrap/ Disallow: /database/ Disallow: /resources/ Disallow: /tests/ # Phase 14.6: removed `Disallow: /*?*` and `Disallow: /search*`. # - `/*?*` blocked every query-string URL including pagination (?page=N) on # recipes/all, categories and blog. Those pages are meant to be indexable # (they self-canonicalize), so the block orphaned deep, paginated content. # Pagination is now governed by the pages' self-referencing canonicals. # - `/search*` matched nothing (the real route is /recipes/search) AND blocking # search via robots would stop Google from ever reading the search page's # correct page-level ``. Search stays out of the # index via that page-level noindex, not robots.txt. # Allow crawling of assets (site assets live under /assets/). Allow: /assets/ Sitemap: https://www.ofrecipes.com/sitemap.xml Sitemap: https://www.ofrecipes.com/video-sitemap.xml Sitemap: https://www.ofrecipes.com/image-sitemap.xml