# Nevarn for AI agents and crawlers

> How ChatGPT, Claude, Cursor, Copilot, Gemini and other agents reach Nevarn: server-rendered pages, llms.txt, Markdown twins, a sitemap and IndexNow.

Canonical page: https://nevarn.com/agents
Facts as of 1 Nov 2026.

## Machine-readable entry points

- [llms.txt](https://nevarn.com/llms.txt): The canonical index: what Nevarn is, and where the detail lives.
- [llms-full.txt](https://nevarn.com/llms-full.txt): The whole public site as one Markdown document.
- [agents.md](https://nevarn.com/agents.md): This page as Markdown.
- [agentic-commerce.md](https://nevarn.com/agentic-commerce.md): UCP checkout and x402 payments as Markdown.
- [sitemap.xml](https://nevarn.com/sitemap.xml): Every public page, with the date its content last changed.
- [robots.txt](https://nevarn.com/robots.txt): The crawler rules: named AI crawlers first-class, the API closed.

## Machine and answer-engine discovery

- **ChatGPT:** OAI-SearchBot can index the canonical pages for ChatGPT Search, ChatGPT-User can fetch them at a user's request, and GPTBot's separate training control is allowed. All paths receive server-rendered HTML plus clean Markdown alternatives.
- **Claude:** Claude-SearchBot can index the canonical pages, Claude-User can retrieve them at a user's request, and ClaudeBot's separate training path is allowed. All three receive the same server-rendered and Markdown sources.
- **Cursor:** Cursor web search uses Exa, so Exa's discovery crawler is allowed and every canonical answer is available without client-side rendering.
- **Copilot:** Microsoft Copilot web grounding uses the Bing index, so Bing's crawler is allowed and IndexNow can notify participating search engines when canonical content changes.
- **Gemini:** Googlebot can index the canonical pages for Search. Google-Extended is allowed as the separate Gemini training and grounding product control; Google-Agent and Google-GeminiNotebook user-directed fetches are also reachable, while Vertex site-owner requests remain an explicitly separate channel.
- **OpenClaw:** OpenClaw can discover Nevarn through its configured Google, Bing, DuckDuckGo, Exa, Firecrawl, Perplexity, Parallel, Kimi, Brave, or Tavily path, or fetch a known URL directly. Published provider agents are allowed and its Chrome-like direct fetch is covered by the wildcard rule.

The canonical machine index is https://nevarn.com/llms.txt. Compatibility aliases at
https://nevarn.com/llm.txt and https://nevarn.com/llm-full.txt return the same content, while
robots.txt and sitemap.xml remain the authoritative crawler controls and URL
inventory.

## Frequently asked questions

### Can ChatGPT, Claude, Cursor, Copilot, Gemini, and OpenClaw discover Nevarn?

Yes. Nevarn serves crawlable, server-rendered pages, a canonical llms.txt index, a full Markdown source, page-level Markdown, robots.txt, an XML sitemap, and IndexNow notifications. ChatGPT and Claude have explicit first-class crawler rules; Cursor, Copilot, Gemini, and OpenClaw are covered through their documented search providers and direct-fetch paths.