LLM search: how ChatGPT, Perplexity and Gemini choose their sources
Three architectures, one common principle, and the list of what your site must make possible.
To be cited by an AI assistant, you first need to understand where it looks. Contrary to a widespread idea, the business answers of ChatGPT, Perplexity or Gemini do not come from the model's memory but from a live web search, followed by reading a few pages. Each engine has its own circuit, and the differences matter.
This article describes these circuits as we observe them at the date of publication, then lists what a site must put in place to be found, read and cited.
Contents
Three search architectures
ChatGPT, when it performs a search, relies on a third-party index, mainly Bing's, complemented by its own robots. A page absent from Bing is therefore invisible to ChatGPT; this is why Bing Webmaster Tools and the IndexNow protocol have become GEO tools.
Perplexity built its own index with its robot, PerplexityBot, and cites its sources by default, with numbered links. It is the most transparent of the three and the one that most readily picks up well-structured niche pages.
Gemini and Google's AI Overviews rely on the Google index. A page that ranks well on Google therefore has a head start, but the AI Overview selects passages, not pages: the first result is not necessarily the first cited.
The robots to allow
The robots.txt file decides who can read the site. Several companies blocked AI robots in 2023 for fear of training, and are now missing out on search. The main agents to know:
GPTBot (OpenAI, collection for training), OAI-SearchBot (OpenAI, ChatGPT search index) and ChatGPT-User (access triggered by a user).
PerplexityBot (Perplexity, index and citations).
ClaudeBot (Anthropic).
Google-Extended (controls the use of content by Gemini for training; does not change Google indexing or AI Overviews).
Bingbot (Bing, and through it ChatGPT search and Copilot).
How a page is selected
The engine rephrases the user's question into several queries, sometimes about ten, exploring different angles: the category, the location, the comparison, the price, the evidence. It keeps a small number of pages, extracts the passages that answer most directly, then writes. Three criteria come up in our tests: freshness (a dated, updated page is preferred), precision (attributable facts rather than generalities) and corroboration (information repeated by several independent sources).
The consequence for writing is simple: every section must be extractable on its own and remain understandable, with the company name, the fact and its source in the same paragraph.
llms.txt: useful, free, no guarantee
The llms.txt file, proposed in 2024 as a convention, describes a site in plain language for models: what the company does, its main pages, its references. No major engine has publicly committed to reading it, but it costs nothing to produce, it forces you to state the essentials in a few lines, and it is already used by some tools. We publish it for our clients at the root of the site, next to robots.txt.
Structured data and entity identity
Schema.org markup (Organization with addresses, founders and areas of expertise, Service, Article with author, FAQPage) describes the page to the machine and ties its content to an identifiable entity. It does not guarantee a citation, but it reduces ambiguity: an AI that knows who it is talking about cites more readily. The identity must then be identical on LinkedIn, in directories and in the press.
IndexNow and Bing, the short route to ChatGPT
IndexNow is a protocol that notifies Bing, and the engines that share its index, as soon as a page is published or modified. For a company targeting ChatGPT, it is the fastest way to make a new page known. We automated it on our own site: every release submits the modified pages within seconds.
What we observe
On our question sets in science communication, Perplexity cites specialized agencies as soon as they have precise pages on the subject. ChatGPT favors companies picked up by third-party sources: agency directories, press, comparisons. Google's AI Overviews closely follow the organic ranking, with an advantage for pages that answer the question in one sentence. These differences justify measuring engine by engine before acting.
Going further
Frequently asked questions
Does ChatGPT use Google to search?
No. ChatGPT search relies mainly on Bing's index and OpenAI's robots. A page absent from Bing does not appear in its answers, hence the value of Bing Webmaster Tools and IndexNow.
Should you block GPTBot to protect your content?
Blocking GPTBot limits the use of content for training, but blocking OAI-SearchBot or Bingbot makes the site invisible in ChatGPT search. For a company that wants to be cited, open access is the rule; the choice is made robot by robot.
Is the llms.txt file mandatory?
No. It is a convention proposed in 2024, with no commitment from the major engines. It is free, quick to produce and risk-free, which justifies putting it in place without expecting a guaranteed effect.
Let's talk about your project
Leading an innovative research project? Looking for a partner who can bring your idea to life beyond the scientific community?
Which type of engagement suits you best?