AI search platforms
Perplexity built its product around one idea: every answer should show where it came from. It runs its own web index, its own crawler and, since October 2025, a free AI browser called Comet. For a business, that makes Perplexity one of the most transparent assistants to work with, and one of the most literal about what your pages actually say.
Short answer
Perplexity answers questions from its own web index, built by PerplexityBot, and fetches live pages with Perplexity-User when a person asks. It splits pages into smaller passages and ranks those against the question, then cites them. Allow PerplexityBot, and write pages where each section answers one question clearly and can stand on its own.
Get Found · Get Leads · Grow
Get Found
We allow PerplexityBot and write pages in self-contained passages that each answer one question, the format Perplexity likes to cite.
Get Leads
Perplexity shows its sources up front, so cited pages make the next step obvious: text, call or request a free visibility review.
Grow
We track which pages Perplexity cites and for what, and work to earn mentions on the trusted sites it pulls from.
How it works
Perplexity publishes documentation for its crawlers. When it launched its Search API in September 2025, it also described how its index works.
Perplexity says PerplexityBot is used to surface and link websites in its search results and is “not used to crawl content for AI foundation models.” It honors robots.txt, and Perplexity recommends allowing it if you want to appear.
When a person asks a question, Perplexity-User may visit a page to answer accurately and link to it. Because a person requested the fetch, Perplexity says this fetcher “generally ignores robots.txt rules.”
Perplexity says its index covers “hundreds of billions of webpages” and breaks documents into smaller units that are “individually surfaced and scored.” In practice, a single clear paragraph can be what earns the citation.
What makes it different
Google and ChatGPT can answer without showing many links. Perplexity puts numbered sources front and center. That makes it easy to see who it trusts for your industry and city, and easy to check whether it describes your business correctly.
Perplexity says it runs its own index, built by its own crawler. A page that ranks well on Google isn’t automatically in Perplexity’s index, and the reverse is also true. Crawl access to PerplexityBot matters on its own.
OpenAI and Anthropic each run a separate bot for model training. Perplexity says its declared crawlers aren’t used to train foundation models, so the decision is simpler: allow PerplexityBot if you want to be found.
Comet, Perplexity’s AI browser, became free worldwide in October 2025. It can summarize pages, compare options and handle tasks while people browse. That means Perplexity may read your site while a customer is actively deciding.
One caution worth knowing: in August 2025, Cloudflare publicly accused Perplexity of using undeclared crawlers to reach sites that had blocked its bots. Perplexity disputed the claims and said the bot Cloudflare named wasn’t one it operates. For most businesses that want to be found, the practical takeaway is simply to allow Perplexity’s declared bots.
Checklist
How MarketPros helps
We can’t promise Perplexity will cite you. We can make your pages the kind of clear, specific source answer engines prefer.
FAQ
PerplexityBot. It builds the index Perplexity uses to surface and link websites. Perplexity-User fetches pages on behalf of a person and, per Perplexity, generally ignores robots.txt.
Perplexity says it runs its own index of hundreds of billions of webpages, built by its own crawler. Ranking well on Google doesn’t guarantee you’re in it, so crawl access matters.
Perplexity says its declared crawlers, PerplexityBot and Perplexity-User, aren’t used to collect training data for AI foundation models.
Comet is Perplexity’s AI web browser. It became free to download worldwide in October 2025 and can summarize pages and compare options while people browse.
Next step
Want to see which sources Perplexity cites for your services and city? Text (404) 877-2184 to set up a Zoom discovery meeting, or request a free visibility review.
Sources
Last reviewed October 23, 2026 by Don Stormo, founder of MarketPros.AI.
