Jarkko Sakkinen
jarkko
After writing few hand optimized inference engines my conclusions are:
1. Inference engine for family of model should be written like a video codec by hand optimizing hot paths and really putting opcodes to their right places.
2. Inference runtimes are vibe coded.
Google's "special AI hardware" is sort of like duct tape for this.
1. Inference engine for family of model should be written like a video codec by hand optimizing hot paths and really putting opcodes to their right places.
2. Inference runtimes are vibe coded.
Google's "special AI hardware" is sort of like duct tape for this.
0
0
0
@jan and another blocker is lack of inline assembly (coming in future) and I get some FFI overhead in inference from external assembly files :-)
Still I got Qwn3-VL-2B 15-20% faster than I got in Rust :-) With Zig I can further take away unnecessary overhead.
Still I got Qwn3-VL-2B 15-20% faster than I got in Rust :-) With Zig I can further take away unnecessary overhead.
1
0
0
Jarkko Sakkinen
jarkko
@jan I might actually end up implementing ReadSeek's next version Zig.
It has one advantage especially for NPM releases i.e. same compiler binary generates all platforms. It such a huge advantage over any language feature in cross platform plugins.
Deeply acknowledged this in the middle of Odin rewrite. It's more complex with any other language than with Zig.
It has one advantage especially for NPM releases i.e. same compiler binary generates all platforms. It such a huge advantage over any language feature in cross platform plugins.
Deeply acknowledged this in the middle of Odin rewrite. It's more complex with any other language than with Zig.
1
0
0
Jarkko Sakkinen
jarkko
@peterkorsgaard OK fair enough thank you :-) I do dogfeed myself with them, shouldn't be a problem.
0
0
0
@peterkorsgaard, Would bootc, composefs and toolbx by any means make sense for the Buildroot's upstream?
https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd-test.git/tree/package?h=main
https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd-test.git/tree/package?h=main
1
0
0
Jarkko Sakkinen
jarkko
Edited 3 days ago
Backported more features from my bootc OS to my kernel testing harness: https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd-test.git/. It now runs on a bootc container and is booted by a UKI kernel.
It also has scripts/build.sh, which is developed and tested to work with Apple's own container manager called, well, 'container' (brew install container && brew services start container).
Getting the build to work on top of Apple's own container manager makes a three year old M2 Pro blazingly fast build server that outputs both x86 and ARM images in no time (much faster than any PC I have).
On a Mac:
CONTAINER=container CONTAINER_MEMORY=16G CONTAINER_CPUS=4 scripts/build.sh
It's a big over-engineered image for the purpose but I've had these features as a side-effect of my OS project so why not collect the fruits and create a more realistic environment.
Right and image has Podman and toolbx, if in the middle of testing something needs to be built :-)
#buildroot #bootc #linux #kernel
It also has scripts/build.sh, which is developed and tested to work with Apple's own container manager called, well, 'container' (brew install container && brew services start container).
Getting the build to work on top of Apple's own container manager makes a three year old M2 Pro blazingly fast build server that outputs both x86 and ARM images in no time (much faster than any PC I have).
On a Mac:
CONTAINER=container CONTAINER_MEMORY=16G CONTAINER_CPUS=4 scripts/build.sh
It's a big over-engineered image for the purpose but I've had these features as a side-effect of my OS project so why not collect the fruits and create a more realistic environment.
Right and image has Podman and toolbx, if in the middle of testing something needs to be built :-)
#buildroot #bootc #linux #kernel
1
0
2
@jan yeah, it's the only book i found about it :-)
Odin is widely popular in game industry and outside of it is quite alien language so far.
I neither love Rust nor Zig because I don't understand them and what I mean by "understanding" is to understand the language from end to end all the time. They get progressively worse and it's a goa for the corresponding projects. I liked Rust in its early years but then it was quite simple and nice systems languuge; no weird language features such as async :-)
Odin is widely popular in game industry and outside of it is quite alien language so far.
I neither love Rust nor Zig because I don't understand them and what I mean by "understanding" is to understand the language from end to end all the time. They get progressively worse and it's a goa for the corresponding projects. I liked Rust in its early years but then it was quite simple and nice systems languuge; no weird language features such as async :-)
1
0
1
Jarkko Sakkinen
jarkko
Edited 3 days ago
I learned Odin's full grammar in 2-3 hours (there's a pretty nice Kindle book about it). Now I know it well enough already that I can even write it even to paper with a carbon pen.
This is what I appreciate also in C, and is a non-existent quality in any other modern systems language.
This is what I appreciate also in C, and is a non-existent quality in any other modern systems language.
1
0
1
Jarkko Sakkinen
jarkko
I've got hooked into Odin. It makes more sense for a C programmer than any other new systems language.
I'm also in process of rewriting ReadSeek CLI. If I had been aware of it I would have used from the get go.
It's easier to maintain in long-term because Odin, like C, does really change.
I'm also in process of rewriting ReadSeek CLI. If I had been aware of it I would have used from the get go.
It's easier to maintain in long-term because Odin, like C, does really change.
1
0
0
Jarkko Sakkinen
jarkko
I think this a nice Buildroot tweak:
❯ make sdk BOARD=puu_amd64 && make sdk BOARD=puu_arm64
Using cached SDK: /home/jarkko/.cache/puu-os/sdk/2026.05.1/linux-x86_64/puu_amd64/c4bc9208266dda1f4180340abcb7eead8df05982c6728d8e6d665d0e14ef64b9/puu_amd64-c4bc9208266dda1f4180340abcb7eead8df05982c6728d8e6d665d0e14ef64b9_sdk-buildroot.tar.gz
Using cached SDK: /home/jarkko/.cache/puu-os/sdk/2026.05.1/linux-x86_64/puu_arm64/ff5125e9c7b2ea625d15d7fb0eccfede5100f82872964e49d8b0e6de93189dd9/puu_arm64-ff5125e9c7b2ea625d15d7fb0eccfede5100f82872964e49d8b0e6de93189dd9_sdk-buildroot.tar.gz
SDK cache does a lot to compilation times.
:-)
❯ make sdk BOARD=puu_amd64 && make sdk BOARD=puu_arm64
Using cached SDK: /home/jarkko/.cache/puu-os/sdk/2026.05.1/linux-x86_64/puu_amd64/c4bc9208266dda1f4180340abcb7eead8df05982c6728d8e6d665d0e14ef64b9/puu_amd64-c4bc9208266dda1f4180340abcb7eead8df05982c6728d8e6d665d0e14ef64b9_sdk-buildroot.tar.gz
Using cached SDK: /home/jarkko/.cache/puu-os/sdk/2026.05.1/linux-x86_64/puu_arm64/ff5125e9c7b2ea625d15d7fb0eccfede5100f82872964e49d8b0e6de93189dd9/puu_arm64-ff5125e9c7b2ea625d15d7fb0eccfede5100f82872964e49d8b0e6de93189dd9_sdk-buildroot.tar.gz
SDK cache does a lot to compilation times.
:-)
0
0
0
Jarkko Sakkinen
jarkko
i reinitiated my audio project from early year with odin as it is easy to import and migrate from C and somehow find entertaining. it's like programming basic :-)
and also optimal language similarly as C for just programming without special tools or LLMs given that you can keep it in your head.
so with that angle slowly breathing life to that project...
and also optimal language similarly as C for just programming without special tools or LLMs given that you can keep it in your head.
so with that angle slowly breathing life to that project...
0
0
0
Jarkko Sakkinen
jarkko
Edited 5 days ago
I'm most proud of my fat ARM test builds in linux-tpmdd-test :-) Actually felt achieving something. Same disk image covers generic EFI, RPI400 and RPI3B.
RPI3B probably does not have a modern supplement. It has that broken TrustZone, which makes it great
RPI3B probably does not have a modern supplement. It has that broken TrustZone, which makes it great
0
0
0
@gimulnautti business is cruel in the way that what it sees that influence can make people kill each other, it can probably also used to make money 🤷
1
0
0