TikTok’s parent launched a web scraper that's gobbling up the world’s online data 25-times faster than OpenAI
==========

ByteDance, TikTok's parent company, launched a web scraper named Bytespider in April 2023; Bytespider scrapes data 25 times faster than OpenAI's GPTbot and 3,000 times faster than Anthropic's ClaudeBot; aggressive scraping activity has increased over the last six weeks; ByteDance is under pressure to sell TikTok or shut it down due to U.S. national security concerns; Bytespider does not respect robots.txt, raising copyright infringement issues; ByteDance aims to improve TikTok's search function with a new LLM; TikTok recently updated its search function to allow real-time keyword searches for advertisers.

#Bytedance #Tiktok #WebScraping #Ai #Bytespider #Gptbot #Claudebot #NationalSecurity #Copyright #SearchFunction

https://fortune.com/2024/10/03/bytedance-tiktok-bytespider-scraper-bot/