OpenAI GPTBot Crawls Company Site to Death, CEO Calls It a DDoS Attack

OpenAI's GPTBot causes a website outage for Triplegangers, leading to high AWS costs. Discover the impact of aggressive AI crawling on small businesses and data scraping.

I never would have expected that the culprit behind the downtime of a company’s website would be OpenAI’s aggressive web crawler—GPTBot.

GPTBot is a tool released by OpenAI a few years ago to automatically crawl and collect data from the internet.

Just a few days ago, a small company with a 7-person team (Triplegangers) experienced a sudden website outage. The CEO and the staff scrambled to figure out where the problem was.

To their shock, they discovered the culprit was OpenAI's GPTBot.

From the CEO’s description, OpenAI's crawler "attack" seemed a bit wild:

We have over 65,000 products, each with its own page, and each page has at least three images. OpenAI is sending tens of thousands of server requests, trying to download everything, including hundreds of thousands of photos and their detailed descriptions. After analyzing the company's logs from the previous week, the team found that OpenAI had used more than 600 IP addresses to crawl the data.