GPTBot
GPTBot is OpenAI’s collection crawler. It feeds model training; its relatives OAI-SearchBot and ChatGPT-User serve real-time search and on-demand browsing requested by ChatGPT.
In practice
Three agents, three functions, three distinct robots.txt lines: blocking GPTBot does not prevent ChatGPT Search from citing a site via OAI-SearchBot. Our position for group sites: open everything — visibility inside assistants is an acquisition channel, not a data leak. Each publisher decides based on their own business model.