Conversation
Yo, ClaudeBot. You see that f'n robots.txt? Yes, it's for you.

Oh, you don't care? Well, then, choke on a bucket of 403s.
5
3
11

@monsieuricon
they don't care .... I ban them with fail2ban

0
0
0

@monsieuricon what really is needed is feeding poison to these bots rather than a 403 -- some kind of cheap Markov model creating english is stuff.... if they ignore robots.txt they then deserve what they get

0
0
1

@monsieuricon In another thread it was said ClaudeBot respected robots.txt... but only when explicitly told so with it's user agent!

It might **also** end up in an infinite loop refreshing your robots.txt, likely hoping you change your mind.

0
0
0