I posted[1] an RFC patchset for DAMON debugfs interface removal a few days ago. Unless someone objects, the change will be merged soon, maybe into v6.14-rc1.

[1] https://lore.kernel.org/20241015175412.60563-1-sj@kernel.org

#linux #kernel #damon

@kernellogger It failed by default, as below:

$ bash kcbench -b -i 1 -j $(nproc --all)
[NOTE] Downloading source of Linux 5.15; this might take a while...
Processor:           QEMU Virtual CPU version 2.5+ [40 threads]
Cpufreq; Memory:     Unknown; 63163 MiB
Linux running:       6.12.0-rc2-mm-unstable-damon+ [x86_64]
Compiler:            gcc (Debian 12.2.0-14) 12.2.0
Linux compiled:      5.15.0 [.../.cache/kcbench/linux-5.15]
Config; Environment: defconfig; CCACHE_DISABLE="1"
Build command:       make vmlinux
Run 1 (-j 40):

Compilation failed. Aborting kcbench. Command used for compilation:
CCACHE_DISABLE="1" make --silent -C '.../.cache/kcbench/linux-5.15' O='/tmp/kcbench.yjn7s1H3h/worker-0/' -j '40' vmlinux
Tail from the output:
CCACHE_DISABLE="1" make --silent -C '.../.cache/kcbench/linux-5.15' O='/tmp/kcbench.yjn7s1H3h/worker-0/' -j '40' vmlinux
.../.cache/kcbench/linux-5.15/scripts/extract-cert.c: In function ‘display_openssl_errors’:
.../.cache/kcbench/linux-5.15/scripts/extract-cert.c:46:9: warning: ‘ERR_get_error_line’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
   46 |         while ((e = ERR_get_error_line(&file, &line))) {
      |         ^~~~~
[...]
cc1: all warnings being treated as errors
[...]
make: *** [Makefile:219: __sub-make] Error 2

After adding -s 5.17 following the workaround from @klausman [1], it succeed.

$ bash kcbench -b -i 1 -j $(nproc --all) -s 5.17
[NOTE] Downloading source of Linux 5.17; this might take a while...
[...]
Linux compiled:      5.17.0 [.../.cache/kcbench/linux-5.17]
Config; Environment: defconfig; CCACHE_DISABLE="1"
Build command:       make vmlinux
Run 1 (-j 40):       65.73 seconds / 54.77 kernels/hour [P:2858%, 45 maj. pagefaults]

[1] https://social.kernel.org/notice/AmuC8cjqppYS6XduT2

After about ten days from the last posting[1] of damo [2] monthly downloads record, the count has doubled (4,000 -> 9,000) again. The time for doubling the count has been reduced from 18 months to 7 months, then to 10 days this time.

Because daily stat graphs on PyPI are not very convenient for monthly download stats, I plotted the 'rolling 30-day downloads' count. It shows this might be a temporal bubble that will pop soon. But, doubling is doubling. I'll seize tonight :D

[1] https://social.kernel.org/notice/AmYHJtlc5aI4ObXnKS
[2] https://github.com/damonitor/damo

#linux #kernel #damon #damo #pypistats

An official video is also uploaded on YouTube: https://youtu.be/mRU1ZeNB9WY?feature=shared

#linux #kernel #damon #linuxplumbers

RE: https://social.kernel.org/objects/f6b89ea2-239f-40c8-b910-3eb3ef48c66b

At OSSummit Europe last month, I and Honggyu Kim presented[1] a few DAMON recipes including those for memory auto-scaling and CXL memory tiering. The video is now available at YouTube[2].

[1] https://sched.co/1ej2S
[2] https://youtu.be/xKJO4kLTHOI?feature=shared

#linux #kernel #damon #ossummit #asv2 #hmsdk

@qyousef @gregkh @oleksandr @axboe @kernellogger @ljs @vbabka It's clear we have many rooms to improve, and I'm grateful for these constructive discussions. I'd like to say, though, that the stable trees stably work pretty well for some use cases including my previous experiences at Amazon Linux (https://github.com/amazonlinux/linux), which is assumed to be one of the major kernels for AWS.

Again, we shouldn't be fully convinced about the current status since we have much room to improve. I just wanted to say the current status is not really completely broken, and appreciate people including the maintainers and people proposing improvements.

The next instalment of Linus vs. Kent – and Linus seems to be on the edge:

"[…] I'm contemplating just removing #bcachefs entirely from the [#Linux #kernel] mainline tree. Because you show again and again that you have no interest in trying to make mainline work. […]"

For the full discussion, start here: https://lore.kernel.org/all/cphtxla2se4gavql3re5xju7mqxld4rp6q4wbqephb6by5ibfa@5myddcaxerpb/t/#m631c24cd07f5820a4cbff8f25dff1d1a0c3cf2e7 (the quote is from one of the later mails from Linus). #LinuxKernel

"[...] Always remember: upstream Linux kernel development is highly social. 😉"

Wonder how many more "strikes" are needed? I think we've seen far too many already. Heck, even the Phoronix commenters seem to think enough is enough.

RE: https://masto.ai/@phoronix/113257686368572007

Just posted[1] 2024-Q3 DAMON news letter. It contains a number of news regarding new features, more users, repos reorganizations, and conference talks.

[1] https://lore.kernel.org/20241001191425.588219-1-sj@kernel.org

#linux #kernel #damon

Monthly downloads of DAMON user-space tool (DAMO)[1] from PyPI took about 18 months to surpass 2,000. About only 7 months after that, it exceeded 4,000 today!

[1] https://github.com/damonitor/damo

#linux #kernel #damon #damo

#sched_ext, which allows scheduling policies to be implemented as #BPF programs, has been merged for #Linux 6.12:

https://git.kernel.org/torvalds/c/88264981f2082248e892a706b2c5004650faac54

See https://lwn.net/Articles/922405/ for a description of what it does and https://lwn.net/Articles/972710/ for the controversy it caused that is the reason why it took so long to land in mainline.

Sched_ext Merged For Linux 6.12 - Scheduling Policies As BPF Programs

Linux 6.12 is shaping up to be a heck of a kernel update! Following real-time "PREEMPT_RT" going mainline after twenty years and many other kernel features merged this week, Linus Torvalds just pulled in the much anticipated sched_ext code!..
https://www.phoronix.com/news/Linux-6.12-Lands-sched-ext

My Kernel Memory Management LPC Micro Conference talk titled "DAMON: Long-term Plans for Kernel That {Just Works,Extensible}" was successful.
The video[1] and slides[2] are available. I was obviously nervous since I forgot contents of the slides due to the great talks from two previous speakers (I highly recommend you to watch those). I like it though, since it reminds me how humble I am ;)

[1] https://www.youtube.com/live/CTWQ-d7pj5s?feature=shared&t=20182
[2] https://lpc.events/event/18/contributions/1768/

#linux #kernel #damon #linuxplumbers

MM pull request for Linux v6.12-rc1 is posted[1] by Andrew Morton. Important fixes / cleanups for DAMON are included:

- mm/damon/lru_sort: adjust local variable to dynamic allocation
- misc fixups for DAMON {self,kunit} tests
- replace per-quota region priorities histogram buffer with per-context one
- Docs/damon: update GitHub repo URLs and maintainer-profile

I believe it is a good list of changes for next LTS kernel candidate.

[1] https://lore.kernel.org/20240920023317.f20c9ecbb2a2f4bf382d831c@linux-foundation.org

#linux #kernel #mm #damon

Lots of people thanked me for my work on b4 this week. It feels very nice, but I want to remind everyone that b4 wouldn't be possible without public-inbox. Please email Eric Wong (privately) and thank him for his work -- I am pretty sure he can use some good vibes.

Found an interesting academic paper preprint that optimizing THP using DAMON and BPF: "eBPF-mm: Userspace-guided memory management in
Linux with eBPF" (https://arxiv.org/pdf/2409.11220)

#linux #kernel #damon #bpf #thp

CONFIG_DAMON is enabled on Debian kernel! https://salsa.debian.org/kernel-team/linux/-/merge_requests/1172

#linux #kernel #damon

@gregkh
> but it's not obvious how and no one has come up with a way to do so. Maybe now they will have some more incentive :)

Not sure if this is what you want, but there is __attribute__((tainted_args)) since gcc 12.

This "untrusted data" patch series from Benno Lossin is the result of conversations at last weekend's Rust Linux kernel conference in Copenhagen:

https://lore.kernel.org/all/20240913112643.542914-1-benno.lossin@proton.me/

It's not a "silver bullet" for why we should be using rust in the Linux kernel, but it is a "big giant sledgehammer" to help squash and prevent from happening MANY common types of kernel vulnerabilities and bugs (remember, "all input is evil!" and this change forces you to always be aware of that, which is something that C in the kernel does not.)

I had always felt that Rust was the future for what we need to do in Linux, but now I'm sure, because if we can do stuff like this, with no overhead involved (it's all checked at build time), then we would be foolish not to give it a real try.

And yes, I've asked for this for years from the C developers, and maybe we can also do it there, but it's not obvious how and no one has come up with a way to do so. Maybe now they will have some more incentive :)