Testers wanted:

I did some changes to kcbench, my #Linux #kernel compile benchmark. If you have a few CPU cycles to spare, please give this a try and let me know in case anything goes wrong:

$ curl -O https://gitlab.com/knurd42/kcbench/-/raw/master/kcbench
$ bash kcbench -b -i 1 -j $(nproc --all)

If you have lots of CPU cycles to space, compile a "allmodconfig" kernel instead, which depending on your CPU might take quite a while

$ curl -O https://gitlab.com/knurd42/kcbench/-/raw/master/kcbench
$ bash kcbench -b -i 1 -j $(nproc --all) -m

@kernellogger I'd love to, but the main machine I would use has the openssl-hates-sha1 problem. Any easy way to work around that for just this run?

@klausman fedora or some other distro?

@kernellogger Debian Bookworm

@klausman hmmm, can't replace that in a container that I just set up with distrobox, so for now I can't tell; will take a closer look later.

Found another problem while trying: compilation failed with "cc1: all warnings being treated as errors"; wonder why, CONFIG_WERROR is disabled; 'CFLAGS=-Wno-error' did not help either. Wonder what sorcery I'm missing. 😕 #sigh

@kernellogger
~/linux ❯❯❯ bash kcbench -b -i 1 -j (nproc --all) -m
[NOTE] Downloading source of Linux 6.8; this might take a while...
Processor: AMD Ryzen 9 7950X 16-Core Processor [32 CPUs]
Cpufreq; Memory: powersave [amd-pstate-epp]; 63405 MiB
Linux running: 6.11.2-arch1-1 [x86_64]
Compiler: gcc (GCC) 14.2.1 20240910

@kernellogger
Linux compiled: 6.8.0 [/home/idoleat/.cache/kcbench/linux-6.8]
Config; Environment: allmodconfig; CCACHE_DISABLE="1"
Build command: make vmlinux modules
Run 1 (-j 32): 744.56 seconds / 4.84 kernels/hour [P:2981%, 1233 maj. pagefaults]

@idoleat many thx!

@klausman ohh, and just to be sure: the error message is something like "Aborting: build would fail, as your distribution's OpenSSL policy apparently rejects SHA1 checksums by default. Compile […]", yes?

@kernellogger yes, exactly

@klausman some googling says that "CipherString = ALL@SECLEVEL=0" in /etc/ssl/openssl.cnf might help; is there a line similar to that already? I do not have it here in the container.

https://serverfault.com/questions/1143995/tls-1-0-broken-with-newer-debian-openssl

@kernellogger any special considerations for compiling inside a container with rootless podman? Any preference for base container image? Is the compilation usually memory-/compute-bound? What is the minimum required memory per core?

Sorry for too many annoying questions!

@meisam kcbench kcbench is not a "universal" performance benchmark, the approach is more along the lines of "kcbench tries to compile a kernel and helps avoiding some pitfalls, but if kernel compilation fails for whatever reasons you get to keep the pieces".

That's why I have no answer to most of your questions, which also depends on the kernel version compiled, which depends on your compiler.

But Rootless podman should work. nd it's compute bound.

@deafbeef

thx; and ohh, intersting to compare the result to those of an AMD Ryzen 9 7950X that @idoleat posted here:

https://g0v.social/@idoleat/113289687526487269

@klausman you likely have heard the answer to that a few times in your life already:

it should (but I haven't tried). 😟 🥴 😬

@kernellogger How much would just building a newer kernel throw off the tests?

@klausman I don't care about the results, I only want to know if everything works

2/ FWIW, I don't care about the results, I only want to know if kcbench works.

@klausman I think I found the problem; it's the -Werror stuff and not the SHA1 stuff

@kernellogger I may have misunderstood the goal then. It's more like let's see if it works than let's see how it performs. I'll try it in a few different environments to see what happens.

@kernellogger
Processor: AMD EPYC 9754 128-Core Processor [512 CPUs]
Cpufreq; Memory: Unknown; 773416 MiB
Linux running: 6.12.0-rc2+ [x86_64]
Compiler: gcc (Debian 14.2.0-3) 14.2.0
Linux compiled: 6.8.0 [/home/axboe/.cache/kcbench/linux-6.8]
Config; Environment: defconfig; CCACHE_DISABLE="1"
Build command: make vmlinux
Run 1 (-j 512): 21.50 seconds / 167.44 kernels/hour [P:10016%, 230 maj. pagefaults]

@axboe That's ... not slow.
I know I'm old, but I remember coming kernels over night. 🤯
@kernellogger

@klausman yup; that -Werror comes from tools/objtool/Makefile and I have found no easy way to override this from Kcbench without modifying the sources (which I don't want to do).

So comping a newer version it is (5.17 seems to work)

@axboe thx!

And yeah, @larsmb, that's, ehh, "not slow" 😂

With modules (-m) it likely will take somewhat longer, but I guess even that will be quite fast. Not sure if I want to know the result.

But FWIW, compiled a recent allmodconfig kernel using clang19 on my laptop (AMD Ryzen 5 PRO 4650U) recently, that took a bit more than an hour…

@kernellogger I have tried to find a way to specify the kernel versionn, but --help showed nothing obvious. It also is aodd in that it claims:

On this machine kcbench by default will use a Linux kernel 6.8 configured
by 'make allmodconfig'.

but when starting it, it says:

Linux compiled: 5.15.0 [/home/klausman/.cache/kcbench/linux-5.15/]

So I am not sure there isn't something else that's off.

@klausman

"help showed nothing obvious" -> see screenshot.

@klausman that "On this machine kcbench by default will use a Linux kernel 6.8 configured" is a bug, thx!

@klausman

latest kcbench now catches this problem and aborts: https://gitlab.com/knurd42/kcbench/-/commit/fe5217827778c04df9fcd9033a0598d27ca2e359

thx again!

@kernellogger I used -s 5.17 and it seems to be running now (sure burns _all_ the CPU :) ). I'll report back when it's done

heh, just started the same...

@klausman @kernellogger

@wonka @kernellogger

Run 1 (-j 96): 365.64 seconds / 9.85 kernels/hour [P:8354%, 5013 maj. pagefaults]

😛 I had to show off the new machines at work

@klausman I wouldn't access the work machines at this time of day. Also, the biggest machines I could currently access would have only half the cores of that - but then, they're VMs anyway.

@kernellogger

```
Processor: AMD Ryzen 7 7840HS w/ Radeon 780M Graphics [16 CPUs]
Cpufreq; Memory: powersave [amd-pstate-epp]; 27849 MiB
Linux running: 6.10.13 [x86_64]
Compiler: gcc (Debian 12.2.0-14) 12.2.0
Linux compiled: 5.17.0 [/root/.cache/kcbench/linux-5.17]
Config; Environment: allmodconfig; CCACHE_DISABLE="1"
Build command: make vmlinux modules
```

and still running with `-j 16`.

@klausman @kernellogger

Aaaand then it failed.

```
Compilation failed. Aborting kcbench. Command used for compilation:
CCACHE_DISABLE="1" make --silent -C '/root/.cache/kcbench/linux-5.17' O='/tmp/kcbench.1YjKs0X4y/worker-0/' -j '16' vmlinux modules
```

A lot of "note:" and "warning:" in the "Tail from the output", but no errors.

@klausman @kernellogger

@wonka @klausman

that's unfortunate, but will happen sometimes; guess compiling an even newer kernel might do the trick (maybe try 6.8).

Might try to reproduce this here to add a check to catch this.

@deafbeef okay, I admit: I never would have come up with trying that 😄

@kernellogger I tried these containers with podman 4.9.4 on alma 9.4 host:

Almalinux 9.4 (latest) ships with gcc 11.4.1 and binutils 2.35.2 but the script complains that
“Aborting, seems you are trying to compile Linux 5.10 or earlier with binutils 2.37 or later, which is known to fail.”

The chosen kernel is version 5.7 but seems like it’s not detecting the binutils version correctly!

@axboe @kernellogger Does that P:10016% mean on average it was only using 100 of your CPUs?

@penguin42 @kernellogger seems plausible, more than half the time compiling such a small config on this box is spent in serialized steps like linking etc.

@axboe @kernellogger That sounds like an opportunity to optimise :-) Not that you've got any time to think of ideas in those ~25seconds; it's not even enough to fill your mug up.

@penguin42 @kernellogger oh for sure, it could be at least twice as fast if it wasn't for stuff like that. allmodconfig on the same box runs in under 3 min, and that's orders of magnitude more code.

@kernellogger It failed by default, as below:

$ bash kcbench -b -i 1 -j $(nproc --all)
[NOTE] Downloading source of Linux 5.15; this might take a while...
Processor:           QEMU Virtual CPU version 2.5+ [40 threads]
Cpufreq; Memory:     Unknown; 63163 MiB
Linux running:       6.12.0-rc2-mm-unstable-damon+ [x86_64]
Compiler:            gcc (Debian 12.2.0-14) 12.2.0
Linux compiled:      5.15.0 [.../.cache/kcbench/linux-5.15]
Config; Environment: defconfig; CCACHE_DISABLE="1"
Build command:       make vmlinux
Run 1 (-j 40):

Compilation failed. Aborting kcbench. Command used for compilation:
CCACHE_DISABLE="1" make --silent -C '.../.cache/kcbench/linux-5.15' O='/tmp/kcbench.yjn7s1H3h/worker-0/' -j '40' vmlinux
Tail from the output:
CCACHE_DISABLE="1" make --silent -C '.../.cache/kcbench/linux-5.15' O='/tmp/kcbench.yjn7s1H3h/worker-0/' -j '40' vmlinux
.../.cache/kcbench/linux-5.15/scripts/extract-cert.c: In function ‘display_openssl_errors’:
.../.cache/kcbench/linux-5.15/scripts/extract-cert.c:46:9: warning: ‘ERR_get_error_line’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
   46 |         while ((e = ERR_get_error_line(&file, &line))) {
      |         ^~~~~
[...]
cc1: all warnings being treated as errors
[...]
make: *** [Makefile:219: __sub-make] Error 2

After adding -s 5.17 following the workaround from @klausman [1], it succeed.

$ bash kcbench -b -i 1 -j $(nproc --all) -s 5.17
[NOTE] Downloading source of Linux 5.17; this might take a while...
[...]
Linux compiled:      5.17.0 [.../.cache/kcbench/linux-5.17]
Config; Environment: defconfig; CCACHE_DISABLE="1"
Build command:       make vmlinux
Run 1 (-j 40):       65.73 seconds / 54.77 kernels/hour [P:2858%, 45 maj. pagefaults]

[1] https://social.kernel.org/notice/AmuC8cjqppYS6XduT2

@kernellogger
Work fine here:
...
Processor: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz [12 threads]
Cpufreq; Memory: powersave [intel_pstate]; 64177 MiB
Linux running: 6.8.0-45-generic [x86_64]
...
Linux compiled: 6.1.0
Build command: make vmlinux
Run 1 (-j 12): 248.37 seconds / 14.49 kernels/hour [P:1080%, 1749 maj. pagefaults]

Computer was busy, the fans had increased speed - as expected.

@klausman

> On this machine kcbench by default will use a Linux kernel 6.8 […] but when starting it, it says […]

Thx again, hopefully now fixed, too: https://gitlab.com/knurd42/kcbench/-/commit/337ad1de7ece06b1b952380dae720e53ac512c65

@kernellogger in Alpine:3.20 container (gcc 13.2.1 compiling kernel 6.1) the compilation fails:

/root/.cache/kcbench/linux-6.1/scripts/basic/fixdep.c:92:10: fatal error: sys/types.h: No such file or directory
92 | #include <sys/types.h>
| ^~~~~~~~~~~~~
compilation terminated.

probably something essential is missing but the script is not checking it! [glibc issue?!]

@meisam likely needs musl-dev; but I had another earlier problem while investigating: time from busybox is not sufficient, kcbench needs gnu time; where did you get that? From the edge testing repo? https://pkgs.alpinelinux.org/package/edge/testing/x86_64/time

@meisam many thx, fixed:

https://gitlab.com/knurd42/kcbench/-/commit/97c878d8c561d8ba5aeef51d8d68a54418bdc5f9

@kernellogger in arch container (gcc 14.2.1 compiling kernel 6.8) it fails with some messages like:

./sync-check.sh: line 40: diff: command not found
Warning: Kernel ABI header at 'tools/include/linux/objtool_types.h' differs from latest version at 'include/linux/objtool_types.h'
…
/root/.cache/kcbench/linux-6.8/scripts/link-vmlinux.sh: line 297: cmp: command not found
Inconsistent kallsyms data

looks like diff and cmp commands are needed but not checked by the script!

@kernellogger " bash kcbench -b -i 1 -j $(nproc --all)" works after installing diff and cmp on Arch!
But "bash kcbench -b -i 1 -j $(nproc --all) -m" fails with:

/root/.cache/kcbench/linux-6.8/kernel/gen_kheaders.sh: line 17: type: cpio: not found
make[4]: *** [/root/.cache/kcbench/linux-6.8/kernel/Makefile:159: kernel/kheaders_data.tar.xz] Error 1

Looks like cpio is also needed and not checked by the script! After installing it, the allmodconfig compilation works.

@meisam thx, fixed now, too

https://gitlab.com/knurd42/kcbench/-/commit/313fe1edcaccaf179c0db5e44dd6bd40a979bab2

(and this time I did not forget to attribute this to you; sorry, forgot in the previous commit)

@kernellogger so, I don't know if you are interested in powersave cpufreq mode, but here you go. It's also cross compiling to RISC-V (no explicit flag to kcbench, but that's just the environment variables I have set on my bash session).

@mssola I don't care about the result, only about "does kcbench work". But it really really great to see that this setup works! And the result of course is interesting to know, too. So many thx!

@sam_ravnborg thx for testing!

@kernellogger To quote every bug triage ever: Works on my machine ;)
allmodconfig completes here, Fedora 40 on a Ryzen 9 5900x

@Cobinja thx for testing!

@kernellogger it works. Run 1 (-j 6): 3248.81 seconds / 1.11 kernels/hour [P:568%, 338 maj. pagefaults]

@letoams thx for testing!

@sj thx for testing; latest kcbench should abort and tell the users that 5.15 would fail https://gitlab.com/knurd42/kcbench/-/commit/fe5217827778c04df9fcd9033a0598d27ca2e359

@kernellogger 6.8 fails too:

```
CCACHE_DISABLE="1" make --silent -C '/root/.cache/kcbench/linux-6.8' O='/tmp/kcbench.kBGChNXa5/worker-0/' -j '16' vmlinux modules
make[4]: *** [/root/.cache/kcbench/linux-6.8/kernel/Makefile:159: kernel/kheaders_data.tar.xz] Error 127
```

(About the "/root/": I'm running that in a podman container.)

@klausman

@wonka @klausman thx for testing; pretty sure that should be fixed now thx to https://gitlab.com/knurd42/kcbench/-/commit/da615f9a6b6f6197d42dde5beed183dfd65be646

@kernellogger I did not install the gnu time. by default, /usr/bin/time is linked to busybox time in the container!

and you're right musl-dev is needed on alpine (which is not checked by the script). after installing it, compilation progresses further and fails because "stat", "grep", and "time" from busybox 1.36.1 do not recognize the used flags

stat: unrecognized option: file-system
grep: unrecognized option: silent
/usr/bin/time: unrecognized option: output

@kernellogger btw, libelf header files package on alpine is called "elfutils-dev" which is not mentioned by the script help when it's missing. I also got the OpenSSL header files from "libressl-dev" which is not mentioned either.

@meisam I guess in that case it doesn't make much sense mention what's needed on alpine (see the other toot you just sent) and just bail bail out if busybox is detected -- or do you disagree?

@kernellogger I'd suggest opening an issue and documenting the situation (what we know is needed and what is still missing for compilation). hopefully someone will find a way for making it work on alpine.
Maybe you can mention help-needed in bail out messages and direct people to an open issue.

@kernellogger Hmm, with now-current master it works with 5.17. I didn't need to install tar or xz though?

```
Run 1 (-j 16): 1626.82 seconds / 2.21 kernels/hour [P:1450%, 3616 maj. pagefaults]
```

@klausman

@wonka @klausman earlier you compiled 6.8; guess those tools at some point started to be a requirement for an allmodconfig builg; I did not bother to make the check config or version dependent (would that be that worth it?), as most systems likely have those tools installed anyway

@kernellogger Yep, but, as I edited in, tar and xz have been there then.

@klausman

@wonka @klausman ahh, okay, now I understand; guess I need to take a closer look then, as it likely is something else that fails; thx!

@meisam fixed, thx https://gitlab.com/knurd42/kcbench/-/commit/a0613b70568c08ab814436f1aec2c190ae470e26

( hope that "if messyness" really does what I want, but I checked in an alma container, there is works)

@kernellogger Currently, this works:

podman container run -it debian:latest bash -c 'apt update && apt install -y curl time build-essential procps bc bison cpio flex pkg-config libelf-dev libssl-dev && curl -O https://gitlab.com/knurd42/kcbench/-/raw/master/kcbench && time bash kcbench -b -i 1 -j $(nproc --all) -m -s 6.8'

With 5.17 it works too. Without supplying a version, it complains about some bug in some Makefile that would make it fail, as it did two days ago.

@klausman

@wonka hmmm, strange; so why did it fail earlier? Out of disk space maybe? Whatever, I think I leave the commit it, as tar and xz clearly will be needed for generating a kheaders_data.tar.xz (I have no idea why that is build, but I guess there is a reason)

@klausman

@kernellogger Out of disk space certainly not with ~90 GiB free at that time. Possibly somehow because of missing cpio, which you added as a requirement today?

@klausman

@wonka yeah, maybe it was that. 😄

@kernellogger not exactly a powerhouse, but all went well:

```
Processor: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz [8 threads]
Cpufreq; Memory: powersave [intel_pstate]; 23802 MiB
Linux running: 6.10.11-amd64 [x86_64]
Compiler: gcc (Debian 14.2.0-6) 14.2.0
Linux compiled: 6.8.0 […]
Config; Environment: defconfig; CCACHE_DISABLE="1"
Build command: make vmlinux
Run 1 (-j 8): 553.11 seconds / 6.51 kernels/hour [P:732%, 33 maj. pagefaults]
```

@olepbr thx for testing!