robots.txt 1994–2023: User-agent: * Disallow: robots.txt 2023–?: User-agent: CCBot User-agent: ChatGPT-User User-agent: FacebookBot User-agent: Google-Extended User-agent: GPTBot User-agent: Omgilibot Disallow: / #web #ai #robotstxt