DAMON Self-Tuned Memory Tiering Shows Nice Improvement For Linux Servers

Linux developer SeongJae Park has posted a set of patches for the Linux kernel's wonderful DAMON code to provide for self-tuned memory tiering that "just works" and is racking up some nice performance wins...
https://www.phoronix.com/news/DAMON-Self-Tuned-Memory-Tiering

I just posted an RFC patch series for making DAMON-based memory tiering self-tuned and just works. It contains evaluation of DAMON-based memory tiering that shows 4.42 % performance improvement over baseline and ~10% improvement over NUMA Balancing in memory tiering mode under a setup.

[1] https://lore.kernel.org/20250320053937.57734-1-sj@kernel.org

#linux #kernel #damon

'damo report heatmap' frequently confused users due to its internal map drawing target area selection. I implemented a feature for making it easier to control, and posted a blog for the problem and the new feature: https://damonitor.github.io/posts/why_the_heatmap_is_not_showing_the_expected_access_patterns/

#damo #faq #damon #linux #kernel

PSA: please don't use git.kernel.org directly for your CI needs. It is not viable for us to provide enough capacity to withstand dozens of CI nodes all trying to clone linux.git at the same time for multiple CI farms. You *will* get ip-blocked. Please set up a local mirror and hit that instead.

I just posted[1] DAMON monitoring intervals auto-tuning patch series without RFC tag. It is now queued on mm-unstable for 6.15-rc1. This version also includes its test results on a real-world server workload. To quote the conclusion part of the evaluation:

"The auto-tuning feature was able to adaptively react to the
dynamic access patterns of the workload and reliably provide consistent
monitoring results without manual human interventions. Also, the
auto-tuning made DAMON consumes only necessary amount of resource for
the required quality."

[1] https://lore.kernel.org/20250303221726.484227-1-sj@kernel.org

#linux #kernel #damon

It is my honor to be designated as Godfather of the 2025 edition of Kernel Recipes. The conference organizers made it quite clear that they would like some focus on increasing the speaking skills of the presenters. This blog series is a first step in that direction: https://people.kernel.org/paulmck/kernel-recipes-2025

A change in maintenance for the kernel's DMA-mapping layer

https://lwn.net/Articles/1011819/ #LWN

An academic paper showing DAMON-based memory tiering can be further improved using h/w-assisted promotion has been published: https://uu.diva-portal.org/smash/get/diva2:1927657/FULLTEXT01.pdf

#linux #kernel #damon #paper

Very happy to announce my book The Linux Memory Manager is now available to pre-order at

https://nostarch.com/linux-memory-manager

It's a comprehensive 1,300 page exploration of how memory functions in Linux that goes into great depth on the subject, and is the first book of its kind for 20 years :)

Well well, look what the cat dragged in
https://nostarch.com/linux-memory-manager

I just published a hands-on tutorial walking through a real kernel bug investigation using drgn, both as a video: https://youtu.be/s5TvkvMiV_M and a writeup: https://drgn.readthedocs.io/en/latest/tutorials/blk_rq_qos_crash.html. It includes a core dump that you can download to follow along. It covers stack traces, local variables, and a few drgn helpers for identifying unknown addresses and structures. Check it out!

I just posted an RFC patch series[1] for automatically tuning DAMON monitoring intervals, like auto-exposing cameras. Hopefully this will mitigate one of big blockers for making DAMON just works and runs on all Linux machines by default.

[1] https://lore.kernel.org/20250213014438.145611-1-sj@kernel.org

#linux #kernel #damon

The FOSDEM25 hkml talk video is also now released: https://mirror.as35701.net/video.fosdem.org/2025/h2215/fosdem-2025-4572-hkml-a-tool-for-working-on-mailing-lists-driven-projects-without-subscribing.av1.webm

#hkml #fosdem #fosdem2025 #linux #kernel

RE: https://social.kernel.org/objects/5071f7e3-2a27-44d7-8352-262c1dfe5bde

The FOSDEM25 DAMON talk video is now released: https://video.fosdem.org/2025/ud2208/fosdem-2025-4396-damon-kernel-subsystem-for-data-access-monitoring-and-access-aware-system-operations.av1.webm

#linux #kernel #damon #fosdem #fosdem2025

RE: https://social.kernel.org/objects/5071f7e3-2a27-44d7-8352-262c1dfe5bde

I presented DAMON and hkml last Sunday FOSDEM. I finished reviewing of the record, and the videos are under transcoding now. Seems it woudl take some time. The slides are already available, though.

DAMON: https://fosdem.org/2025/schedule/event/fosdem-2025-4396-damon-kernel-subsystem-for-data-access-monitoring-and-access-aware-system-operations/

hkml: https://fosdem.org/2025/schedule/event/fosdem-2025-4572-hkml-a-tool-for-working-on-mailing-lists-driven-projects-without-subscribing/

#linux #kernel #damon #hkml #fosdem #fosdem2025

Received the LSFMMBPF[1] invitation this morning, and just registered. Looking forward to meet people and discuss about DAMON and Linux kernel!

[1] https://cvent.me/4ldr5l?sms=7&cn=EKkfkKZzQzqP2fCBdr2aRg

RE: https://social.kernel.org/objects/19008587-0a35-45d0-80e3-01d5f4d76875

For 2025, here is a updated and hopefully-useful notice about Linux kernel security issues, as it seems like this knowledge isn't distributed very widely based on the number of emails I still get on a weekly basis:

- The Linux kernel security team does not have any "early notice" announcement list for security fixes for anyone, as that would only make things more insecure for everyone. The number of organizations that fail to understand this is way too high.

- The kernel community DOES assign CVEs, as we are a CNA, please see https://www.kernel.org/doc/html/latest/process/cve.html for how they are handled and assigned. Side note, we were #2 in quantity for CVE assignments in 2024 despite only doing so for 10 1/2 months, averaging about 10 CVEs per day. Any process you might have where you feel you need to research each CVE on an individual basis manually is going to be a major time suck, automate it! All CVE entries are provided with proper git commit ids for the vulnerable release ranges for you to check yourself, AND we have tools and other formats that you can use to check this yourself. See https://git.kernel.org/pub/scm/linux/security/vulns.git/ for the tools and raw data for you to pull from directly if you don't want to deal with the cve.org json feed.

- Kernel CVE entries are constantly updated over time, you can not just look a them only when created, and then ignore all updates. Too many groups are missing revoked CVE entries and tightening of vulnerable kernel ranges that we are updating on a weekly basis. By ignoring the updates, you are causing yourself more work, not less. cve.org provides an "updated" feed in their git tree, use it!

- Along the lines of the huge number of recorded CVEs, you HAVE to take all of the stable/LTS releases in order to have a
secure and stable system. If you attempt to cherry-pick random patches you will NOT fix all of the known, and unknown, problems,
but rather you will end up with a potentially more insecure system, and one that contains known bugs. Reliance on an "enterprise"
distribution to provide this for your systems is up to you, discuss it with them as to how they achieve this result as this is what you are paying for. If you aren't paying for it, just use Debian, they know what they are doing and track the stable kernels and have a larger installed base than any other Linux distro. For embedded, use Yocto, they track the stable releases, or keep your own buildroot-based system up to date with the new releases.

- Test all stable/LTS releases on your workload and hardware before putting the kernel into "production" as everyone runs a different %
of the kernel source code from everyone else (servers run about 1.5mil lines of code, embedded runs about 3.5mil lines of code, your mileage will vary). If you can't test releases before moving them into production, you might want to solve that problem first.

- A fix for a known bug is better than the potential of a fix causing a future problem as future problems, when found, will be fixed then.

Join the sched-ext crew at @fosdem for the Scheduler Contest. Submit your own Linux scheduler (written with eBPF and sched-ext) or scheduling policy idea, and with a bit of luck, win a sched-ext T-Shirt and a cup: https://mostlynerdless.de/blog/2025/01/28/sched-ext-scheduler-contest-fosdem25/

MM pull request[1] for Linux 6.14-rc1 is out. This includes many interesting changes including below six DAMON patchsets.

- "mm/damon: add sample modules"
- "mm/damon: replace most damon_callback usages in sysfs with new core functions"
- "mm/damon: enable page level properties based monitoring"
- "mm/damon: remove DAMON debugfs interface"
- "mm/damon: extend DAMOS filters for inclusion"
- "Docs/mm/damon: add tuning guide and misc updates"

I highly recommend to read the summaries on the pull request[1] to find exciting MM changes for Linux 6.14!

[1] https://lore.kernel.org/20250124162248.60104eec848619a187242392@linux-foundation.org/

#linux #kernel #mm #damon

My hkml lightning talk on FOSDEM'25 is now scheduled. 2025-02-02 13:10 at H.2215 (Ferrer). Looking forward to see you there!

https://fosdem.org/2025/schedule/event/fosdem-2025-4572-hkml-a-tool-for-working-on-mailing-lists-driven-projects-without-subscribing/

#hkml #fosdem #fosdem25

RE: https://social.kernel.org/objects/2bfd6f7e-ea3a-4f8e-ad8b-668977c52e57