Conversation

K. Ryabitsev-Prime ๐Ÿ

So, the aggressive crawler that pretends to be b4 in the user-agent is back hitting lore full force, this time hitting EU instead of APAC.

Thankfully, knowing a thing or two about b4, I can easily tell fake b4 traffic from real b4 traffic. Enjoy a ton of 404's, jerk.
2
2
20
and if you want to feed lore to your shitty AI learning system, JUST CLONE THE DAMN GIT REPOS. IT'S NOT LIKE WE DON'T MAKE IT ALL EASY TO MIRROR.
3
1
15

@monsieuricon I have thought of changing the 403's we are sending the scrapers to 406's...

https://http.cat/406

1
0
0

@nirik @monsieuricon Iโ€™d get a giggle telling them payment is required.

https://http.cat/status/402

0
0
0

@monsieuricon this is the most infuriating part about the scraping, straight up just dumb

0
0
0

@monsieuricon @monsieuricon instead of 404โ€™ing them, send them a fork bomb :D

0
0
0

@monsieuricon hey, on the topic of "just clone the repos": does that happen very often? I ask because I've been paying close attention to the development of git server side advertising of static repo packfiles via bundle-uriยน and where Eric might hook into that in the public-inbox infrastructure, and I was wondering whether optimizing that path is even worthwhile given how often or rarely it happens in the wild.

ยน: https://git.kernel.org/pub/scm/git/git.git/tree/Documentation/technical/bundle-uri.adoc

0
0
0