Sometimes when people whine about the kernel email stuff and how painful reviews are and hint at wanting something more github/lab/etc.-like (or forges) I wonder if they've ever actually done reviews with these platforms.

They're not exactly a parcel of unbelievable joy either.

Another factor is, everything is public. If we made it super easy for people to just click on a line of code to make a comment, how many random knob heads are going to drive-by useless noise?

There's more to this stuff but people generally find that throwing tantrums on social media and getting cheered on by people who have never and likely will never contribute is more conducive to the ego than thinking through these things...

This is not to say btw that git forges or whatever aren't the way forward, everybody knows that the email situation sucks, but there's more to it than 'omfg these auld lads are just so stuck in their ways with this terrible stuff and I am a victim woe is me!'

Pause. Breath. Think. Chill.

My 'favourite' aspect of this is seeing people who have put in the effort to think about this like @corbet and @monsieuricon reply in these threads and get ignored/dismissed.

Social media doesn't reward the right things, even here.

@ljs Not sure about the random knob head argument—the kernel is already on GitHub and if you use the same email for commits and your GitHub account, you will get useless noise sent to you via email.

(Never happened to me btw but I only have a single digit number of commits last of which was 10+ years ago.)

@liskin I'm not sure what you mean about the kernel being on github, it's mirrored and has no actually valid PRs there whatsoever. Some people still submit them despite it very clearly stating 'submissions not accepted', but it's just a mirror. And not sure what useless noise you mean?

My point is if people submit reviews via PR/MR, and all it takes is a github/gitlab/etc. account to engage in it, you WILL get people doing random drive-by stuff. Because it'll be easy and have the prestige of being on linux.

You only have to look at some of the main proponents of this stuff who... oh go on about bad behaviour from people on web platforms they use for other things 🤣

Yes it can be managed to some degree but in every single community on the internet when you lower the barrier to entry this stuff significantly increases, that's just how it works.

I'm not saying don't do it, I'm saying you have to take this stuff into account, those up on the soapbox screaming about how everybody's stupid for not immediately switching seem not to want to think about these things.

@ljs Okay, that clarifies it. I though you're worried about random people commenting on random commits.

Kinda like degenerate me sometimes does - "hey I've bisected it to this commit and it's wrong because so and so and I'm fucking tired so no I won't create a new issue or fight with mailman and greylists if I can just comment on this shitty piece of work you committed 3 years ago, kthxbye, lol"

@ljs For almost 10 years, I used to do all my development work through mailing lists (DRM, xserver, Wayland, ...). I was also involved with patchwork-fdo and CI for all patch series on IGT's and intel-gfx mailing lists... And there is no way I would do this again!

The email workflow is IMO fundamentally broken, and fixing it would require creating an AI to guess the intent of the author. Heck, even I couldn't always make sense of the authors' intent!

Mixing code and comments is a bad idea.

@mupuf yeah everybody agrees it sucks, the point is there's more to it than is let on in the typical diatribe on the subject.

The way out imo is better tooling around the existing email process (b4, lore), after which we could move towards swapping out the underlying process to something better.

It really does have to be iterative, realistically. And also with an acceptance that alternatives also come with their own price.

The way you hear people talk about it you'd think other approaches are nothing but peaches and cream and that is far from the truth.

@ljs @liskin So, your point is that emails are hard to get working so it provides a good barrier of entry?

The thing is, I can easily flood mailing lists by creating tons of email addresses. Good luck preventing that... Heck, even normal email spam makes it to development mailing lists (I used to admin a few lists on FD.o).

Forges both can and do a better job at spam management. Again, I would know as an admin for gitlab.fd.o.

Not sure what could be considered a benefit of emails, really.

@mupuf @liskin No that's not my point, and I'm starting to wonder if you're actually in good faith here :/

My point is that this issue is a real concern that the various 'rah rah rah emails!' tantrum throwers ignore.

Your comment about spam shows you missed my point, which I did try to outline in detail there.

Yes people can do that, and there is spam, but most people can't be bothered.

However if you have the ability for a random to do a couple clicks and drive-by a series featured on phoronix say you're going to see a lot more noise.

And keep in mind that's NOT spam, it's just low-quality engagement because you make it really easy to engage.

It's a concern and issue that has to be considered.

I'm not defending email so much as saying 'it isn't as simple as that', as all of the reasonable people who have tried to engage in these kinds of discussions have said.

Ultimately I suspect git forge IS the right way forward, and great if it really is good at dealing with additional low-quality engagement.

But there are a whole host of concerns to address.

Not least the fact the kernel has to iterate towards improvements.

The lack of respect shown for efforts made in that direction with b4 and lore etc. is another bug bear on this for me tbh. People are trying.

Some are, others whine.

@ljs @mupuf @liskin I actually think email and the way the Linux kernel development has been run so far has provided one of the lowest possible barriers of entry you could ever dream of. The lists are open to everybody, no subscription required, and you could even email Linus directly if you wanted to. That's how patches were incorporated originally, right?

It's just now the technology has changed and it's no longer the case.

I really think the low barrier of entry is a virtue we should keep.

@vegard @mupuf @liskin The barrier to entry currently is in finding the relevant thread.

You know as well as I do, if phoronix ran a story on [latest time Linus was angry] or some angry dude with 20k followers misrepresented a situation, and people could FIND the relevant thread and reply to somebody in it with a couple clicks, they WOULD do that.

If you don't believe me, go read the comments in these places.

For somebody capable of contributing to the kernel the barrier to entry is relatively low (if annoying). Any sensible alternative approach would not increase it for contributors, it'd reduce it. So nothing to worry about there.

The important freedom in linux is that anybody can contribute anything if they put enough effort in and have the talent to. This is emphatically NOT true of most open source projects.

I hope that stays, and I hope tooling improves.

But I do get tired of the 'hey everybody email dev is broken!' threads. Like yeah, dude we know.

It's the 'and this justifies me being incredibly toxic!' bit that comes immediately afterwards that I have a very big problem with.

@vegard @ljs @liskin Originally, it was indeed the lowest barrier of entry.

Nowadays, how likely is it that a developer's email provider would be compatible with it? B4 helps with that but that's a new tool to learn just to interact with kernel people, on top of learning about email headers to know how to reference emails.

I agree, the world changed...

@ljs @liskin Re-reading you again, I once again come to the conclusion that the friction provided by the email workflow would reduce spammy (in the sense of unwanted) messages and other low-cost abuses.

Sorry for being unclear in my original answer and not acknowledging that you are right (damn 500 character limit) before saying that forges have ways of mitigating this issue in ways mailing lists can't.

kernel.org should be its own forge, would address your concern!

@mupuf @liskin yeah sorry this is an emotive issue and it's easy to get wound up.

I'm glad forges do handle these things well, that's good. I do suspect they're the way forward tbh, just like I suspect rust SHOULD play a key role in the kernel.

But in both cases it's a question of iterating from here to there.

I don't have a 500 char limit on this instance :P and for that I am grateful and people reading my verbal diarrhea are not

@ljs @liskin He he.

You know what? I had an hour to spare, so I gave the email workflow another try by upstreaming some patches I had lying around for Linux and u-boot... And only managed to send one, for u-boot :o

Here were the pain points, from bad to worst:

* Needed to install the perl deps for git send-email
* Struggled to create an app password with runbox, my paid email provider... Had to store my main password in ~/.gitconfig
* Tried to set-up patch-manager... The code is broken...

@ljs @liskin

* Reading multiple pages of how to contribute
* Subscribing to the ML
* Setting up IMAP filters to redirect that to its own folder

Now I can't wait for code review and monitoring master to see if it got merged :D

Yep, I did not miss it :s

@mupuf @liskin

@mupuf @liskin yeah the setup is no fun.

I mean my process is a disaster because I'm too scared of fucking things up

so I do a git branch, do the work, do get_maintainer.pl and figure out who to send it to, add anyone missed, filter out the commit signer nonsense, git format-patch, checkpatch, move patch set to own directory, write in cover letter, discover problems and rinse and repeat before git send-email line which I constructed in a buffer in emacs.

That's before we get onto the whole -fix-patch debacle and people telling you arcane things you didn't realise only when you fuck them up.

And yeah the set up is a pain.

I do use lei for mail and that's a HUGE improvement, I think b4 would tackle a lot of the above.

@mupuf @liskin you can skip this IMAP stuff with lei, seriously.

In fact you can skip subbing to the mailing list too.

It's a major improvement:

https://josefbacik.github.io/kernel/2021/10/18/lei-and-b4.html

Don't do any of that crap

@ljs @liskin I did not look into lei/b4 because I thought it was Linux-only... Turns out it wasn't: https://lists.denx.de/pipermail/u-boot-custodians/2022-June/000658.html

So yeah, outdated information in https://u-boot.readthedocs.io/en/latest/develop/sending_patches.html

Will give it a try for the other commits.

@ljs @mupuf I was hoping to find that b4 helps with figuring out where to apply the patch, too, but it's not mentioned. Last time I wanted to apply a series of yours, I spent over half an hour just figuring out where the damn tree is (you'd expect to just google for "linux mm git" but you get all sorts of nonsense and never the actual git URI you need) and where in its history I need to check it out for the series to apply without conflicts. Any idea what am I missing?

@ljs @mupuf With GitHub, I just "gh pr checkout 123" and it's immediately obvious what commit the author based their work off...

@liskin @mupuf yeah that's an issue.

I always base my stuff off of mm-unstable unless it's merge window which then confusingly goes to linus's tree.

There's a way to specify it but yeah that sucks

@ljs @mupuf @liskin even b4 b4 and lei existed, you most certainly wouldn't have to subscribe in order to send a series, as most ml's accept mails without subscription, and you should get replies directly.

@vbabka @liskin @ljs Sure, but it's not that simple, right?

A. Most lists will send your email through moderation if you are not a member

B. Not everyone remembers to answer all... And then I may miss some answers.

With lei, I would be more comfortable not subscribing. I'll give it a try tomorrow.

@mupuf @vbabka @liskin @ljs For the kernel the majority of lists aren’t moderated for non-subscribers - the main kernel list hosts have always been very good at anti spam stuff (partly just through rejecting HTML mail), and realistically people forgetting to reply to all has always been very rare - it does happen but it’s so infrequent it’s not worth worrying about.

@liskin @mupuf @ljs b4 will try to guess where the series belongs. And if the series was sent with b4 itself, it would also have a base-commit, which will tell you exactly where in the tree that belongs, too.

Also, if you try b4, you won't really have to configure your email server, as we also support submitting a patch series via a relay service.

There's still lots of work left to do, but we *are* listening.

@mupuf @ljs @liskin my usual concern with running a forge is that suddenly there's a single target that someone can attack and stop all kernel development. Imagine you have a 0-day that lets you own billions of devices worldwide. Just knocking out forge.kernel.org prevent it from being fixed. Yes, we can fall back to coordinating everything via mail when this happens, but that means we haven't really fixed the underlying problem.

@monsieuricon @mupuf @liskin @ljs I doubt that is really true though, it's rarely anything comes solely through kernel.org and most vendors would not rely on it in the 0 day billions of devices marketplace. I really think this is one excuse you should dismiss until we have a forge.

@monsieuricon @liskin @ljs aren't lore or mailman single points of failures though?

But yeah, being cognisant of vulnerabilities in our development workflow is important. I however don't feel like the current model is particularly robust, though.

@mupuf @monsieuricon @liskin cc'ing people still works.

@ljs @mupuf @liskin @monsieuricon not if the people sending the patches and people receiving the patches are reliant on workflows using lore or mailman or patchworks or whatever.

@airlied @mupuf @liskin @monsieuricon you don't need to be 'reliant' on it, I've literally encountered this issue IRL and you just carry on using email as the underlying medium.

Yes it sucks to not be able to use lore for [period of outage] but it doesn't end workflow.