Question

ChatGPT wrote a scraper, now the site blocked my IP

Solved · 7 views · asked last month by kai_makes · edited

Wanted product prices from a supplier. The script ran every minute. Now I'm blocked.

What I’ve tried

Waited a day. Still blocked. Searched here and in the docs, found nothing that matched my case.

Comment

1 answer

Marked as helpful by the asker

lena_opslast month · edited

Once a minute is an attack from their side. Before scraping anything: check for an API or a data export, read robots.txt, and if you must scrape, once a day with a real User-Agent and a contact email in it. For the block: email them, explain, apologize. That works more often than you'd think.

Comment

Keyboard shortcuts

ChatGPT wrote a scraper, now the site blocked my IP

What I’ve tried

1 answer

Your answer