Today is my last day at Meta. This has been the best team I’ve ever been on, and I’ve been on some great teams. Next week I start a new chapter, I will be joining Anthropic to help them scale out their infrastructure and put my decades of kernel and systems experience to use. I will be stepping back from kernel development as my primary job for the first time in my career. I’m sad to leave my colleagues, but I’m excited to try something new and see where it takes me.

A patch series[1] for making DAMON supports ARM (32bit) with LPAE has just landed on mm-new tree. It was made by a great and joyful collaboration between I and Huawei (Quanmin Yan and Zuo Ze). Hopefully this will land on Linux v6.18.

[1] https://lkml.kernel.org/r/20250822093420.2103803-2-yanquanmin1@huawei.com

#linux #kernel #damon

Looks like the @opensuse Tumbleweed kernel will soon (when 6.17 is final and passes QA) produce a warning when mounting a bcachefs instance... https://bugzilla.opensuse.org/show_bug.cgi?id=1248109

DAMON results are arguably hard to understand. And no single visualization fits all. Hence DAMON user-space tool provides a feature[1] that lets users program their own visualization in Python code.

A visualization recently suggested was showing size of memory for different hotness, and sort the size values by the hotness. It can intuitively show cold memory tail. We implemented it as another custom visualization program[2].

The script outputs the visualization in text format, that can easily be plotted as a chart, like attaching.

[1] https://github.com/damonitor/damo/blob/next/USAGE.md#damo-report-access-programming-visualization
[2] https://github.com/damonitor/damo/blob/next/report_access_exec_scripts/idle_time_mem_sz.py

#linux #kernel #damon #damo

Exploring mails on deep threads with hkml is not that easy. It is easy to miss on what mail currently the cursor is on, and which mail is replying to which mail.

To help this, below three new key bindings are implemented and pushed to master branch of hkml[1].

'w' and 'b' key move the cursor to next and previous words, respectively, similar to those on 'vim'.

'g' key highlights and unhighlights (toggles) the row and the column of the current cursor.

The word-based cursor movement is useful for moving it to beginning of the mail.

The highlighted lines can be useful for seeing on what mail the cursor is on, and what mails are replying to what mails.

[1] https://github.com/sjp38/hackermail

#hkml

I continued this to allow monitoring accesses that made by only specific group of CPUs. You can do that using damo, like below.

```
$ sudo ./damo start --exp_ops_use_reports y --exp_ops_cpus 0-3,7-8,10
```

Again, note that this depend on gross, buggy, un-upstreamable-as-is hacks.

The initial proposals of these works go back to 2022. So I'm happy to share my first prototype that can at least be tried. So wrote a blog post after a while about this: https://damonitor.github.io/posts/write_only_cpus_only_monitoring/

#linux #kernel #damon #damo

RE: https://social.kernel.org/objects/6090d05e-4257-4b43-99fe-d104cba95054

With kernel on the latest damon/next tree[1] and DAMON user-space tool on the latest next tree[2], users can try write-only monitoring via following command.

```
$ sudo ./damo start --exp_ops_use_reports y --exp_ops_write_only y
```

Please note that it is totally experimental, not in an upstreamable shape, and I did nearly no test. It may have many bugs, will be changed a lot, and might not be upstreamed at all. But, you can at least run the experiment.

#linux #kernel #damon #damo #write-only

[1] https://git.kernel.org/pub/scm/linux/kernel/git/sj/linux.git/log/?h=damon/next
[2] https://github.com/damonitor/damo/tree/next

MM pull request[1] for Linux 6.17-rc1 is merged. There are so many interesting changes. As usual I highly recommend to read Andrew's great summary of the changes. If I pick only two changes that make me happy to see among only DAMON changes,

"mm/damon/vaddr: Allow interleaving in migrate_{hot,cold} actions" from Bijan and Ravi from Micron[*] made DAMON useful for yet another domain that I never expected.

"selftests/damon: add python and drgn based DAMON sysfs functionality tests" increased DAMON test coverage so much thanks to the power of 'drgn', and actually found a real bug[2].


[*] Andrew introduces it as came from me, but the real workers are Bijan and Ravi!

[1] https://lore.kernel.org/all/20250730152806.16f5c618e3af0d3b8dea3159@linux-foundation.org/
[2] https://social.kernel.org/notice/Awb8b5Dbt5hBDsyIdc

#linux #kernel #damon #drgn

@sj recently started using drgn to add more DAMON selftests, which is a super interesting use case that I didn't envision: https://lore.kernel.org/all/20250628160428.53115-1-sj@kernel.org/. It has already found a real bug! https://lore.kernel.org/all/20250719181932.72944-1-sj@kernel.org/

BTW, this year we have quite an early "save the date" landing page for LSF/MM/BPF 2026 https://events.linuxfoundation.org/lsfmmbpf/

And AFAIK the first time it's going to be happening outside of North America!

With great stress comes great stress reliever.

All microconferences (MCs) at LPC 2025 have been accepted! It is time to submit topics to your favorite MCs.

Please check out our latest blog post for the list of MCs, and how to create a ideal MC topic.

https://lpc.events/blog/current/index.php/2025/07/25/all-microconferences-have-been-accepted/

Intel Announces It's Shutting Down Clear Linux

The most depressing news of the week: Intel is ending their performance-optimized Clear Linux distribution. Over the past decade the Clear Linux operating system has shown what's possible with out-of-the-box performance on x86_64 hardware... Not just for Intel platforms but even showing extremely great performance results on AMD x86_64 too. But with the cost-cutting going on at Intel, Clear Linu…
https://www.phoronix.com/news/Intel-Ends-Clear-Linux

Bypass Kernel Barriers: Fuzzing Linux Kernel in Userspace With LKL

Xuan Xing & Eugene Rodionov gave a talk about fuzzing the Linux kernel interfaces fully in user space using LKL (Linux Kernel Library).

Video: https://www.youtube.com/watch?v=Wxmi-2ROYNk
Slides: https://static.sched.com/hosted_files/lssna2025/01/Bypass%20Kernel%20Barriers_%20Fuzzing%20Linux%20Kernel%20in%20Userspace%20with%20LKL.pdf

Bijan's DAMON patch series[1] for dynamic NUMA memory weighted interleaving that shows ~25% performance improvements on a test is now merged into mm-new, and looks good to me. I'm more than exciting to see this is on the track and grateful to have a chance to participate!

[1] https://lore.kernel.org/20250709005952.17776-1-bijan311@gmail.com

#linux #kernel #damon #interleaving

And OSSummit NA 2025 DAMON talk video is now available: https://youtu.be/Ou4BQQ0Ved8?si=ZrhjCGI4gRyJmODn

#linux #kernel #damon #ossummit #ossummitna #ossummitna25

RE: https://social.kernel.org/objects/f03afaa9-5786-49eb-8c1d-2b6b6261685e

Access pattern heatmap is powerful visualization, but exploring the time/space is not that easy[1]. To help that, we added an interactive zoom and scroll mode of the heatmap. You can try that with `damo report heatmap --interacrive_edit`. Attaching a demo gif.

[1] https://damonitor.github.io/posts/why_the_heatmap_is_not_showing_the_expected_access_patterns/

#linux #kernel #dmon #damo

OSSummit NA was great. I met old and new great people. DAMON talk was also done very well, thanks to great audiences. The video would be available soon. Meanwhile, you could get the slides for the talk at the conference web site[1] and GitHub[2].

[1] https://sched.co/1zfmE
[2] https://github.com/damonitor/talks/tree/master/2025/ossna

#linux #kernel #damon #ossna

Bcachefs may be headed out of the kernel

https://lwn.net/Articles/1027289/ #LWN

This graph is the one I'm most excited about: the lifetime of security flaws in Linux is finally starting to get shorter (and the number of fixed flaws continues to rise).

https://hachyderm.io/@LinuxSecSummit@social.kernel.org/114750428620118674