Always fun to argue with AI bots about why they're ignoring robots.txt.
Senza categoria
1
Post
1
Autori
0
Visualizzazioni
-
Always fun to argue with AI bots about why they're ignoring robots.txt.
$ grep -A1 ChatGPT robots.txt
User-agent: ChatGPT-User
Disallow: /
$ grep -c ChatGPT-User access_log
54Me: "Yo, wtf?"
AI: "I did not crawl or access the site directly."
Me: "Uhm, yeah you did, I *just now* saw you fetch that URL."
AI: "Wasn't me - was my, uhm, 'web tool'. That's it." -
undefined Maronno Winchester :antifa: ha condiviso questa discussione