LLM crawlers are aggressively destroying important community infrastructures but sadly there is not an easy fix. Still: Blocking those crawlers should be high on your list of todos
(Original title: LLM crawlers continue to DDoS SourceHut)
Hi everyone, for several hours we are currently fighting with potentially malicious crawling. We are seeing distributed access on Codeberg that could have been crawling, however it seems to only use the most expensive routes. And each time we block an access pattern, a new one pops up immediately.
We are doing our best to keep the performance up.
Today, I presented the b4 tool to the Linux council at our office (talk was geared towards developers only though).
My colleagues frequently encountered patch corruption issues with mail servers. They were impressed when I introduced the concept of web endpoint submission.
Thanks to @monsieuricon for such a wonderful tool.