A big issue in the kernel are asymmetries between review resource and patch submission.

If you have 20 inexperienced kernel engineers all doing big difficult work and submitting large series that need TONS of review (and sometimes essentially development-via-review) and only a few reviewers, you have a problem.

What compounds this is that not all review is equal.

It seems most people will only really look at the surface, and rather happily give tags.

I think that technical skill is not enough for review [as I have said very often - soft skills are the actually core ones] - you have to be willing to say no, you have to be willing to consider the wider picture.

Finding people who have the right skills, are willing to handle the stress + thanklessness, & who have the time is not easy.

So the asymmetry I think is insoluble.

It'd be good if we could have stricter controls to throttle incoming series though.

Here’s a progress update on my diet! It’s been five weeks now.
- Five workout sessions per week (focusing on losing fat)
- Eating only 2 sandwiches per day (most of the time!)

Target weight is about 70-75kg... so still a long long way to go.

Prototype for type-based partitioning of Linux kernel slab caches: https://discourse.llvm.org/t/rfc-a-framework-for-allocator-partitioning-hints/87434/24?u=melver

Compiler seems to be doing a good-enough job of inferring allocated types per /proc/slabinfo. With the diagnostic -Rpass=alloc-token I can see about 965 allocation sites where it failed to infer the allocated type, but most of them are "bag of bytes", and only few with complex sizeof calculations like struct_size that are too opaque right now (but can be fixed).

Working on saving 8 or 16 bytes per slab object in certain slab caches that fall into special cases, most notably 0.7%-0.8% or 1.5%-1.6% (depending on the configuration) memory savings for the inode cache (ext4 and xfs).

When memory cgroup and memory allocation profiling are enabled (the former being very common in production and the latter less so), the kernel allocates two pointers per object: one for the memory cgroup to which it belongs, and another for the code location that requested the allocation.

In two special cases, this overhead can be eliminated by allocating slabobj_ext metadata from unused space within a slab page:
- Case 1. The "leftover" space after the last slab object is larger than the size of an array of slabobj_ext.
- Case 2. The per-object alignment padding is larger than sizeof(struct slabobj_ext).

Thanks to @vbabka who suggested an excellent general approach to cover Case 1 and 2 with a minimal performance impact on the memory cgroup charging code (more details in the cover letter)

For these two cases, one or two pointers can be saved per slab object. Examples include the ext4 inode cache (case 1) and the xfs inode cache (case 2). That results in approximately 0.7-0.8% (memcg) or 1.5-1.6% (memcg + mem profiling) of the total inode cache size.

https://lore.kernel.org/linux-mm/20250827113726.707801-1-harry.yoo@oracle.com/

I should have been a photographer ?!

I didn't really work on my legs much before, but ever since I started PT, my legs hurt with every move I make

On diet day 3

Uh I don't want to misread code

First day of PT, quite tired.
…now my trainer says only two sandwiches are allowed every day. Oh no!

For a while now, the kernel's configuration and build systems have been an area of concern for me. Almost nobody truly understands those complex subsystems, which were handled by a single maintainer.

That maintainer, Masahiro Yamada, has just stepped down after eight years on the job:

https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8d6841d5cb20

Happily, Nathan Chancellor and Nicolas Schier have agreed to pick up the build system. The configuration system, instead, is now unmaintained. That ... seems less than optimal.

Thanks to Masahiro for doing this work all these years, and to Nathan and Nicolas for stepping up!

Still digesting it, and may be slightly incorrect, but a summary of what I've learned: properly using statistics to compare the performance of computer systems [1] [2].

0. When you get a sample and calculate sample mean, it is likely different from the population mean. Simply comparing two sample means from different computer systems may therefore lead to misleading conclusions, since the observed difference could be due to random sampling variation rather than a real performance difference (especially when the variance is high).

1. Central Limit Theorem indicates that, regardless of the underlying distribution, the sampling distribution of the mean tends to follow normal distribution when the sample size is sufficiently large (typically n >= 30). But to apply CLT, the observations should be independent and collected from the same distribution.

2. Based on the CLT, we can estimate how close the sample mean (to be precise, any population parameter) is likely to be to the population mean. A confidence interval [x, y] with a confidence level of p% means that if we repeated the sampling process many times under the same conditions, about p% of those intervals would contain the population mean.

3. To compare two distributions, confidence intervals can help determine whether the difference between mean values is statistically significant. This can be done by:

- Checking whether the confidence intervals of the two samples do not overlap, or
- Examining the confidence interval for the mean of the differences to check if it does not include 0.

4. Caveat: The CLT assumes that the data is collected independently, meaning one observation does not affect another. In computer systems, this assumption usually does not hold. Caches, memory layout, scheduling decisions etc. can introduce some degree of dependencies between observations.

This can be mitigated by 1) reducing dependence between experiments as much as possible or 2) by applying the bootstrapping method [3].

[1] Jan Kara, Measuring performance regressions, https://youtu.be/HAHhW13ofrg?si=drgegMwXUDegHsQf
[2] Dev Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling, https://www.amazon.com/Art-Computer-Systems-Performance-Analysis/dp/0471503363
[3] https://en.wikipedia.org/wiki/Bootstrapping_(statistics)

Got a new small bookshelf but the user experience isn’t great

Oh god let me fall asleep

@sj recently started using drgn to add more DAMON selftests, which is a super interesting use case that I didn't envision: https://lore.kernel.org/all/20250628160428.53115-1-sj@kernel.org/. It has already found a real bug! https://lore.kernel.org/all/20250719181932.72944-1-sj@kernel.org/

linux-stable is a vibe-backported frankenkernel

All microconferences (MCs) at LPC 2025 have been accepted! It is time to submit topics to your favorite MCs.

Please check out our latest blog post for the list of MCs, and how to create a ideal MC topic.

https://lpc.events/blog/current/index.php/2025/07/25/all-microconferences-have-been-accepted/

I've been playing with the LLM code assistants, trying to stress them out with the Linux kernel code. So far, I've had success with them writing reasonable unit tests:
https://lore.kernel.org/lkml/20250717085156.work.363-kees@kernel.org/
https://lore.kernel.org/lkml/20250724030233.work.486-kees@kernel.org/
These both saved me some time since it emitted quite a lot of good boundary testing code, but I had to massage them both a bit until I was happy with the coverage. But it was a net win on time spent.

And then I walked it through fixing a buffer overflow. This one didn't save me any time because I had to tell it how to look at the problem. Since it was a shorter/simpler session, I included my exact prompts just for anyone interested in what I did:
https://lore.kernel.org/lkml/20250724080756.work.741-kees@kernel.org/

This graph is the one I'm most excited about: the lifetime of security flaws in Linux is finally starting to get shorter (and the number of fixed flaws continues to rise).

https://hachyderm.io/@LinuxSecSummit@social.kernel.org/114750428620118674

Recently, I was debugging some intermittent boot failures and discovered that, on x86, forgetting to synchronize kernel page tables when installing new PGD entries can lead to kernel crashes.

Most of the time, PGD entries are populated during the boot process, and new tasks inherit page tables derived from the swapper process, so everything works fine.

However, if you're adding memory at runtime via memory hotplugging and populating new PGD entries for the vmemmap (i.e., the struct page array) and the direct mapping area, you need to iterate over all page tables in the system and update them accordingly to make it visible to all tasks.

Failing to handle this properly can lead to a situation where you think the page tables are set up correctly, but in reality, only init_mm.pgd (page table of the swapper process) was updated, not the page table of current task. This leads to kernel crashes since the PGD entry is not set up properly.

x86 code has mechanisms to handle this kind of synchronization, but it’s easy to be overlooked and introduce kernel crashes as the code evolves. Here’s my patch series that addresses the issue and aims to make it more robust: https://lore.kernel.org/linux-mm/20250709131657.5660-1-harry.yoo@oracle.com/

GNU Butt