I once again ask you to please stop using gmail for mailing lists.

Attached is a screenshot of *just one* of the 4 nodes delivering various kernel mailing list traffic with a deferred queue of 100,000+ messages, all for gmail.

We are currently sitting on about 500,000 deferred messages, all for gmail. I am very, very close to just banning gmail addresses from large lists.

I am currently considering forcing all gmail.com addresses to digest delivery for lists like LKML, netdev, and a few others.

Of 2447 total subscribers to LKML, 1187 are gmail.com addresses.

@monsieuricon
Do... it... 👻

@monsieuricon Are they rate limiting you because you have so much traffic to send to them?

@monsieuricon I am very happy to never voluntarily have used a gmail etc account; self-hosted the moment I got my own domain (for which I had to use a temp "free" email to be able to register it though ;)
One would hope the people who run gmail have some clout with their own service to fix this up though...

@monsieuricon and how many more are hosted on google workspace.
I wonder if you’d still have this problem if the mailing list server was on GCP

@monsieuricon What about employer-owned emails that are hosted by Google workspace etc?

@epilys This is a tiny fraction as far as I can tell.

@monsieuricon Just do it. Gmail is a bug in the internet at this point not a feature.

@monsieuricon Debian Mailinglists see similar numbers of mails-in-deferred-queues

@monsieuricon I force all google destinations over ipv4 because ipv6 defer / reject rates are far higher

@monsieuricon It's been a pain for the PHP mailing lists too. We've tweaked some delivery params with postfix and that has alleviated the problem fairly well. They're also really keen of having SPF and DKIM, but the latter is tricky with our MLs addkmfea footer.

No idea how we're going to make one click unsubscribe work, without it being a a shit show with virus scanners and Outlook that just follow every link...

@monsieuricon @epilys includes Linaro though...

@monsieuricon is it due to some quirk of gmail's servers?

@monsieuricon Gmail is arguably the largest source of spam my mail server receives from, despite all the SPF/DMARC/DKIM anti-competitive nonsense they forced us all to adopt.

@monsieuricon, why exactly are they being deferred though?

@monsieuricon do it; push the button.

@monsieuricon "Due to a high amount of unsolicited mail"?

@monsieuricon question, is that just @gmail.com addresses, or also custom domain addresses hosted by google (iirk its called workspaces)?

@monsieuricon what's the reason or the problem that the mails get deferred? That's an insane amount 😧

@never_released
Yeah we use Google as our corporate it service including email. Usually we set up local SMTP servers so the tooling works and they plug into into Gmail with per-app passwords. I take it the problem there is no one at Google to talk to about what large list servers need to do to keep Gmail happy? For my own mail server I just kept iterating with DKIM and the various other extensions until it was happy. However I guess the volume of email is a lot less.
@monsieuricon @epilys

@etchedpixels I have some sympathy given how prevalent spam is in the email world. I still see loads on my server compared to my Gmail and I guess that's down to their aggressive attempts to limit spam.
@monsieuricon

@monsieuricon Googlemail is bad for FOSS collaboration. I can’t even believe people use that.

@monsieuricon do it !

@monsieuricon and this is happening because gmail is not accepting these messages? is it some sort of greylisting? aside from the fact that "don't use gmail" is good advice, are they _also_ technically misbehaving?

@monsieuricon I wish we did not have to subscribe, in the first place. For ex-vger lists this is achievable and I unsubscribed myself some years ago, but I cannot do that for infradead.org, launchpad, xen and few others.

git grep "moderated for non-subscribers" | wc -l
238

@monsieuricon Can you explain for someone who's on dozens but has never run an ML in his life what this means?

@lproven @monsieuricon I don't run an ML either but based on the term "deferred" I suspect Google is limiting the number of messages per day that it will receive from a single sender. This is probably justified as a spam prevention measure but has the effect of putting otherwise reputable senders in the same category as spammers, and to some ML users make the ML admins look like they are not doing their job correctly.

Doing something about it would require either manual intervention (from Google? hah!) or a more sophisticated automatic reputation system.

Corrections to the above are welcome.

@monsieuricon is migadu the best option for people that want to get an email?

@pdp7 for kernel work, I usually refommend fastmail. Migadu has some outgoing mail throttling that can make it difficult to send large patch series.

@monsieuricon I have kernel.org for sending which is great. But for receiving, my korg forwards to pdp7.com which is a Google workspace. I've been really disappointed in Google mail handling over the past 3 months so I would like to switch where I have my mail hosted.

@monsieuricon BTW I'm really happy I got my korg account because sending through my Google workspace, pdp7.com, had been getting flagged as spam by gmail accounts over the past couple months. And it turns out several korg addresses forward to Gmail, at least for the subsystems I normally work on, so my patches would always bounce and I would have to hope those maintainers would at least have my mail via the list server.

@monsieuricon for reading lists, I have a stupid system where I've been using gmail, which predates my Google workspace, for all my list subscriptions for 15+ years. I have hundreds of label filters which I rely on to not miss things that matter to me. But I have to go grab the thread with b4 so I can respond with my korg account in mutt (or previously my pdp7.com Google workspace account).

I knew my setup is stupid and inefficient but your post made me realize it is also harmful to infra.

@monsieuricon it sounds like now is the time to ween myself off my Gmail labels and convert my hundreds of Gmail filtering rules to lei queries. All the lists I care about are on lore so I don't really have a good excuse anymore.

Given all of that, it sounds like fastmail might be a good option for me to move my pdp7.com to?

That is what my korg forwards to. I only got my korg account recently and it's not subscribed to anything - it's just in maintainers.

@monsieuricon final question :)

Is there any efficient way for me to unsubscribe my Gmail from all those kernel lists? I think I ended up subscribing to most of them on vger and infradead over the years.

@pdp7 @monsieuricon

I'm in progress of switching from mailing list delivery to just pulling the mails from public inbox git archives directly to maildir files. It bypasses so many useless steps.

I just wasn't happy with any of the alternatives for this, so I'm a bit delayed by having to write my own... 🙄