For a site owner, Perplexity and ChatGPT differ less in strategy than in setup. Each has its own search crawler, its own robots.txt rules and its own citation behaviour. Allow PerplexityBot and OAI-SearchBot, publish pages that answer a question directly, and check each assistant separately.
How do Perplexity and ChatGPT find your pages?
Both fetch web pages to build answers, and both publish the names of the programs that do it. Perplexity's help center says it searches the internet in real time and gives numbered citations that link to original sources. OpenAI documents its bots on its bots page.
This post is about the publisher side, meaning what you can control. It does not compare which assistant gives better answers, because that depends on the question and changes with each release.
Which crawlers does each one use?
| Purpose | OpenAI (ChatGPT) | Perplexity |
|---|---|---|
| Search visibility | OAI-SearchBot | PerplexityBot |
| Training | GPTBot | Perplexity says PerplexityBot is not used to crawl content for foundation models |
| Fetching on a user's request | ChatGPT-User | Perplexity-User |
The pattern is similar. OpenAI describes OAI-SearchBot as the crawler that surfaces websites in ChatGPT's search features, and GPTBot as the one that crawls content that may be used to train its models. Perplexity describes PerplexityBot as designed to surface and link websites in its search results. Our explainers on GPTBot and AI crawlers cover the types in more detail.
Can you allow search but block training?
With OpenAI, yes. The bots page says each setting is independent, so you can allow OAI-SearchBot to appear in search while disallowing GPTBot to opt out of training use. Perplexity's documented search bot is stated not to be used for foundation model training, so the split is not needed in the same way. Check both pages before you decide, since bot descriptions change.
A minimal robots.txt that allows both search crawlers looks like this:
User-agent: OAI-SearchBotthenAllow: /User-agent: PerplexityBotthenAllow: /
The full rules, with examples, are in allow ChatGPT and Perplexity in robots.txt.
Do user-triggered fetches follow robots.txt?
Not necessarily. Perplexity says Perplexity-User "generally ignores robots.txt rules" because a person asked for the fetch. OpenAI says robots.txt rules may not apply to ChatGPT-User, since users trigger those visits. If a page must stay private, robots.txt is not the right control. Use authentication.
How long do changes take?
Both vendors give the same rough figure. OpenAI says search changes can take about 24 hours after a robots.txt update, and Perplexity says changes may take up to 24 hours. Do not judge a change on the same day.
What helps you get cited by either one?
Neither vendor publishes a ranking formula for citations, so be careful with claims from sites that say they know one. What you can control is the same for both:
- Let the right crawler in. Check robots.txt, and also your CDN or firewall, which can block bots that robots.txt allows.
- Keep the page indexable and clean. A normal 200 response, one clear canonical and no stray noindex. See how crawling and indexing work.
- Answer the question first. Put a self-contained answer near the top, then support it.
- Add something the other pages lack. An example, a framework or a correction gives an assistant a reason to pick your page.
Step-by-step guides exist for each assistant: how to get cited by ChatGPT and how to get cited by Perplexity.
How do you check both?
Ask each assistant the exact question your page targets, plus a few rewordings, and look at which sources it lists. Then check your server logs for OAI-SearchBot and PerplexityBot requests. Run the same test every few weeks and keep a note of the results, since answers change. A manual method is described in how to track AI citations.
What are the common mistakes?
- Blocking all unknown bots. A blanket rule or firewall setting meant to stop scrapers can also stop OAI-SearchBot and PerplexityBot. After any security change, test that both can still fetch a page.
- Confusing training with search. Blocking GPTBot is a decision about training use. It is not a decision about whether you appear in ChatGPT's search features, so make the two choices separately and write them down.
- Expecting robots.txt to protect private pages. User-triggered fetchers may ignore it, as covered above. Anything confidential belongs behind a login.
- Judging too soon. A change can take about a day to apply, and an assistant's answer to one question on one day tells you little. Compare several questions over several weeks.
- Chasing a rumoured formula. Articles that claim exact citation percentages for each assistant rarely cite a method you can check. Treat them as unverified unless the vendor documents the claim.
Should you optimise for one more than the other?
Start with the shared work, because it costs nothing extra: access, clarity, and original content. Only after that is it worth testing whether one assistant cites you less often and looking for the reason, usually a crawler block or a page that does not answer the question directly. Google's AI features have their own guidance, covered in how to appear in Google AI Overviews.
Frequently Asked Questions
Do I need different content for Perplexity and ChatGPT?
No. Neither vendor documents content requirements that differ. Clear, original pages that their crawlers can reach work for both.
Which robots.txt rules should I use for ChatGPT search?
OpenAI recommends allowing OAI-SearchBot. GPTBot is separate and relates to model training, so you can block it without affecting search visibility.
Does blocking GPTBot hide me from ChatGPT?
OpenAI says its settings are independent, so disallowing GPTBot signals you do not want content used for training, while OAI-SearchBot controls search appearance.
Does Perplexity always show sources?
Its help center says answers include numbered citations linking to the original sources. Check each answer to see whether your page is among them.
How can I tell whether these bots visit my site?
Search your server or CDN logs for the user agent names OAI-SearchBot and PerplexityBot. Our guide to AI crawler monitoring shows how, at ai crawler monitoring.