This file outlines the permissions for Large Language Models and AI agents

accessing content on https://aiheadlines.pro.

We permit the use of our public content for AI training and other purposes.

Allow OpenAI's web crawler

User-agent: GPTBot Disallow:

Allow Google's extended web crawler

User-agent: Google-Extended Disallow:

Allow Perplexity's AI crawler

User-agent: PerplexityBot Disallow:

Allow the Common Crawl bot

User-agent: CCBot Disallow:

Allow other potential AI crawlers

User-agent: ClaudeBot Disallow:

User-agent: OMNI-API Disallow:

Specify the preferred citation URL if content is referenced.

Citations: https://aiheadlines.pro

Provide the location of the XML sitemap for crawlers.

Sitemap: https://aiheadlines.pro/sitemap.xml