LLMS CRAWLER POLICY
Website: https://www.altmansolon.com/
Purpose: Define permissions for Large Language Model crawlers
Reference: OpenAI, Anthropic, Perplexity, Google Extended guidelines

GLOBAL POLICY (Default)
User-agent: *
Allow: /
Disallow: /hubspot-form-submit/
Disallow: /_hcms/*
Disallow: /hs-search-results
Disallow: /private/
Disallow: /api/*
Disallow: /preview/
Disallow: /login/
Disallow: /thank-you/
Crawl-delay: 5

OPENAI CRAWLERS
User-agent: GPTBot
Allow: /
Disallow: /private/
Disallow: /api/
Disallow: /login/

User-agent: ChatGPT-User
Allow: /
Disallow: /private/

ANTHROPIC (Claude)
User-agent: ClaudeBot
Allow: /
Disallow: /private/
Disallow: /api/

GOOGLE EXTENDED (AI Training)
User-agent: Google-Extended
Allow: /
Disallow: /private/
Disallow: /api/
Disallow: /login/

PERPLEXITY
User-agent: PerplexityBot
Allow: /
Disallow: /private/
Disallow: /login/

META / FACEBOOK
User-agent: FacebookBot
Allow: /
Disallow: /private/

COHERE
User-agent: CohereAI
Allow: /
Disallow: /private/

NOTES
This file is optional and does not affect SEO or Google ranking.
It exists only to manage permissions for AI crawlers.