Ever wondered why @torvalds coined the #Linux #kernel's "no regressions" rule? He just explained it again here: https://lore.kernel.org/all/CAHk-=wgtb7y-bEh7tPDvDWru7ZKQ8-KMjZ53Tsk37zsPPdwXbA@mail.gmail.com/

'"[…] I introduced that "no regressions" rule something like two decades ago, because people need to be able to update their kernel without fear of something they relied on suddenly stopping to work. […]"'

Follow the link for context and other statements that did not fit into a toot.

#LinuxKernel #regressions

@kernellogger @torvalds I'd appreciate having this rule being applied to the "stable" kernel too.

@kernellogger @torvalds I think a key bit of this that the less... charitable people need to pay attention to is that sometimes mistakes are made, so it's best effort, but as soon as a regression is noticed kernel developers will work to rectify the situation asap.

An imagined 'no regressions and totally perfect so never make a mistake' rule does not exist.

@oleksandr @kernellogger @torvalds

Whats the issue? this is already the case to my understanding 😊

This does of course not mean that the stable kernel do not contain regressions

@oleksandr @torvalds

it is applied to stable series as well; but accident happen (maybe too many, yes) and the way we need sometimes is not ideal[1].

That's a very long and complex story very short.

[1] it for example might take weeks for fixes to arrive in stable trees when the problem happens in mainline as well, as the fix then has to be mainlined first to avoid later regressions. Improving that process somehow (by temporary dropping patches or merging fixes quicker) is on my agenda.

@gromit @kernellogger @torvalds Too many backporting is happening without proper review. For instance, for v6.9.3 there are ~400 commits pending ATM, the majority of those were autoselected.

@kernellogger @torvalds The effect of this backporting race is quite opposite to what "stable" maintainers expected. I'm forced to stop using it and do backports on my own. I'm curious to see what's in your plans.

@oleksandr @gromit @torvalds

FWIW, "autoselected" makes it sound like it was "AUTOSEL" that chose them; that is not the case for those patches afaics.

And what you mention here is a slightly different topic and thus could be seen as a straw man ( Linus talked about dealing with regressions, not about preventing them).

@kernellogger @torvalds Can #HomeAssistant please adapt this rule?

I am having more issues with Home Assistant issues than Linux kernel issues.

@ljs @kernellogger @torvalds Yeah, it is more important how you react when by a plain mistake a regression is introduced, than not introducing them at all, i.e. transparency and brutal honesty is the key IMHO :-)

And also, for instance with this fresh bus encryption feature for TPM chips, despite getting only a single performance regression report from a private user so far, I scaled "default y" down to "default X86_64" because that is the only platform where I've been able to test it successfully even with an old Intel Celeron NUC.

I would also emphasize this as a maintainer: even if the new feature is exciting, scale it down *eagerly* before it hits a release. It is always easier scale up later on, than scale down. I tend to label "uncertainty" as a regression despite not having better knowledge rather than "wait and see". I'd label any other behavior as pure ignorance: you knew that shit might hit the fan but did absolutely nothing.

@oleksandr @gromit @kernellogger @torvalds I'm trying to review those, but often Sasha simply ignores feedback. :-(

@chris @kernellogger @torvalds The first rule of Home Assistant is never update. The second rule of Home Assistant is never update.

@ZS @kernellogger @torvalds Oh, an Update! (click)

@oleksandr @torvalds

different small solution would be needed afaics, among them:

* some of the fixes that wait till the merge window should not waited for it to distribute things better
* those that should wait ideally should be marked for "backport only after -rc4" or so) (see https://git.kernel.org/torvalds/c/2263c40e65255202f6f6d9dfa31d23906995ff7c)
* maybe also some distinction for "this is urgent, backport quickly", so others can wait longer
* ideally: make more subsystems use stable tags (and ignore anything not tagged by them)

@kernellogger @torvalds I realized that the recent misunderstanding was probably because when occasionally the message shifts from "no regressions" to "WE DO NOT BREAK USERPACE", the intended audience are kernel developers that should do better, and not the outside world being bragged to.

@kernellogger @torvalds linux-stable-v2? linux-steady? linux-really-stable?

2/ Also note that Linus' message[1] indirectly explains why you might not be able to claim "no regressions" when you only find a problem after updating from one longterm aka LTS #Linux #kernel series to a later one:

By then others might have started relying on the new behaviour, hence fixing the regression might be impossible without causing a regression for those other people – and then you might lose out.

[1] https://lore.kernel.org/all/CAHk-=wgtb7y-bEh7tPDvDWru7ZKQ8-KMjZ53Tsk37zsPPdwXbA@mail.gmail.com/

#LinuxKernel #regressions

@oleksandr @torvalds

That's the wrong way IMHO, I prefer fixing the real problem then to work around them; and who would do all the work?

The thing that IMHO would help the most from my point would be a git tree with two branches for each relevant series, where one branch contains regression fixes that are queued for mainlining already while the other contains reverts and fixes under review (as mentioned already a week or two ago).

@kernellogger @oleksandr @gromit @torvalds you mean not autosel but fixes tag without cc stable? There are too many of those and are not filtered for severity, violating the documented stable kernel rules. If they come from the merge window and are backported and releases in stable shortly after rc1, then the stable kernel is similarly stable as rc1 is.

@vbabka

I think that's quite a common case with the kernel development discussion.

Things which happen within kernel development and make a lot of sense within the context of that development, get misinterpreted by a crowd of spectators and then used to harass other projects.

And then those other projects get into a fight with the kernel, while the core issue is that you should not use any (good and bad) development rules, concept or ideas to harass other people.

@kernellogger

@vbabka @gromit @kernellogger @torvalds less stable because entropy grows, especially with random cherry-picking

@kernellogger @oleksandr @torvalds I'd really like to see description of a bug it fixes before patch being merged. Less than 10% have that. We have rules saying "bugs only" and "no theoretical bugs", but warning fixes are merged all the time and so are workarounds for various checkers.