Conversation
Conor Dooley
conor
# v6.3 linux-riscv recap thing
For v6.2's release, I had to go back and dig through the PRs to figure out what had changed in the
release, so I figured for v6.3 I would try and keep on top of the PRs as they went out.
So here's one I prepared earlier I guess!
Perhaps the biggest change for v6.3 is the addition (finally!) of the devicetrees for the various
Allwinner D1 based platforms. There's been a bunch of people involved in that support, including
Heiko who worked on the T-HEAD cpu support, but Samuel Holland did the lions share of the work on
the Allwinner parts of that. I think Arnd pointed out that that amounted to 10 boards?
(Reading this back now, feels like they've been in far longer!)
Bit of business-as-usual for the other vendors, a new dev board for Microchip and some minor changes
on the Renesas stuff now - a negative diffstat actually, as some /delete-node/ macros were removed
from the devicetree as support arrived in drivers for the RZ/Five.
Across the kernel, the bits of support for the JH7110 have started to land, including the mmc,
pinctrl and pmu (that being power management, not perf) drivers.
As for architectural changes in v6.3, things mostly centred around text patching and the handling of
extensions, with work from Heiko Stuebner & Jisheng Zhang *dominating* the first RISC-V PR[0] for
the merge window.
Firstly, Jisheng overhauled how we handle enabling extensions. Since a kernel must be able to run
on all hardware, regardless of what (additional) extensions it supports, we need to enable that
code at runtime. To do so in an efficient manner, we patch the kernel at runtime, during early
boot[1]. This mechanism is called "alternatives" Previously, we stored the absolute addresses of the
"standard" code & the extension's replacement version of that operation, and Jisheng converted that
to use relative addresses instead. All that work, in a complex corner of the kernel (IMO!) just so
that we can use ISA extensions properly in the VDSO!
They also squeezed in some cleanup of our linker script into their schedule somehow..
Heiko then had another big patchset, refactoring extension handling with the aim of allowing
function calls in these "alternatives", so that we can use the Zbb bit manipulation extension in
string comparison routines. Zbb is present on the StarFive JH7110, so, once the devicetrees & clock
drivers etc land for that platform you can go test it out if you have a VisionFive 2.
Unfortunately between those series, a lot changed and the inevitable bugs surfaced, the most serious
of which (preventing boot, mounting a rootfs etc) were fixed before -rc1 by Samuel and Björn Töpel!
Other than those, the first PR also featured:
- Ftrace optimisations from Guo Ren, the first couple patches from a larger series he has been
working on for some months.
- A fix from Andy Chiu, that's been in the works since Linux Plumbers, that caused panics in ftrace
relating to disabling preemption.
- Another one that's been in the works since Plumbers, RISC-V previously had a mix of SOC_ and ARCH_
symbols for vendors, depending on whether a vendor had prior support in other architectures. There
are now ARCH_ versions of the SOC_ symbols as part of an effort to gradually convert everything to
ARCH_ to match what is done in other parts of the kernel.
- Splat decoding support for RISC-V in the decodecode script, from Björn Töpel.
- Sergey Matyukevich upstreamed part of a fix for a tricky looking cache update bug, that seems to
occur on uniprocessor systems running CONFIG_SMP=y kernels.
In the netdev PR, which is where bpf patches for RISC-V (and other archs) get into mainline, Pu
Lehui landed support for both the bpf trampoline and 64-bit kfunc support. Björn seemed pretty happy
about that, so it must be important!
The second PR, as might be expected, was substantially smaller. It did bring with it a KASAN rework
from Alex Ghiti, aiming to substantially simplify what he described as RISC-V's "intricate"
implementation and a bump in COMMAND_LINE_SIZE that should allow syzbot to start fuzzing RISC-V.
In terms of fixes, we got yet another round of fixes for handling the (highly frustrating) decision
to move the csr and fence.i instructions out of the I extension, specifically for mixed toolchains
where some components pre-date that move.
As of -rc3, the most interesting fix this cycle was from Sergey Matyukevich, reverting some
ill-fated changes of his for flushing stale TLB entries and replacing them with another fix.
As far as I can tell from reading the patches, the issue was mainly hit on uniprocessor systems
running SMP kernels - at least the drawn out discussion certainly featured discussion of that
configuration!
Nathan Chancellor took over a bit of work I was doing, the nth patchset of late that has to deal
with the fallout of RVI's decision to move some instructions out of the base I extension.
Great craic really.
A fairly "major" fix for how late in the game it came was Alex Ghiti's conversion of RISC-V back
to using the fixmap for dtb mapping. It was moved out of the fixmap in 2020, with the logic that it
was not actually needed & wasted a lot of space.
However, I ran into some issues where either:
a) reserved memory node names would use "early" virtual
addresses (and thus cause a panic when accessed in a driver)
b) reservation was done too late to prevent early memblock allocations being done in a section of
reserved memory. This would cause the reservation to fail, a bit of an issue if that memory is
not actually usable by the kernel...
The most straightforward thing was to just move back to using the fixmap & sidestep the issue.
Thanks to Alex for doing something promptly & saved me significant time flailing around if I had had
to do it myself!
Of course there were a bunch of other various fixes, with several focusing on different bits of TLB,
and some for the text patching code that had recently been changed by Jisheng et al.
One of those fixes being because somehow we all forgot that the aforementioned alternatives
framework is not always enabled, and if it wasn't the FPU could not be used!
0 - https://lore.kernel.org/all/mhng-9df18be9-c02d-41f8-93ee-7e1e124a31ec@palmer-ri-x1c9/
1 - We also do it when modules are loaded
For v6.2's release, I had to go back and dig through the PRs to figure out what had changed in the
release, so I figured for v6.3 I would try and keep on top of the PRs as they went out.
So here's one I prepared earlier I guess!
Perhaps the biggest change for v6.3 is the addition (finally!) of the devicetrees for the various
Allwinner D1 based platforms. There's been a bunch of people involved in that support, including
Heiko who worked on the T-HEAD cpu support, but Samuel Holland did the lions share of the work on
the Allwinner parts of that. I think Arnd pointed out that that amounted to 10 boards?
(Reading this back now, feels like they've been in far longer!)
Bit of business-as-usual for the other vendors, a new dev board for Microchip and some minor changes
on the Renesas stuff now - a negative diffstat actually, as some /delete-node/ macros were removed
from the devicetree as support arrived in drivers for the RZ/Five.
Across the kernel, the bits of support for the JH7110 have started to land, including the mmc,
pinctrl and pmu (that being power management, not perf) drivers.
As for architectural changes in v6.3, things mostly centred around text patching and the handling of
extensions, with work from Heiko Stuebner & Jisheng Zhang *dominating* the first RISC-V PR[0] for
the merge window.
Firstly, Jisheng overhauled how we handle enabling extensions. Since a kernel must be able to run
on all hardware, regardless of what (additional) extensions it supports, we need to enable that
code at runtime. To do so in an efficient manner, we patch the kernel at runtime, during early
boot[1]. This mechanism is called "alternatives" Previously, we stored the absolute addresses of the
"standard" code & the extension's replacement version of that operation, and Jisheng converted that
to use relative addresses instead. All that work, in a complex corner of the kernel (IMO!) just so
that we can use ISA extensions properly in the VDSO!
They also squeezed in some cleanup of our linker script into their schedule somehow..
Heiko then had another big patchset, refactoring extension handling with the aim of allowing
function calls in these "alternatives", so that we can use the Zbb bit manipulation extension in
string comparison routines. Zbb is present on the StarFive JH7110, so, once the devicetrees & clock
drivers etc land for that platform you can go test it out if you have a VisionFive 2.
Unfortunately between those series, a lot changed and the inevitable bugs surfaced, the most serious
of which (preventing boot, mounting a rootfs etc) were fixed before -rc1 by Samuel and Björn Töpel!
Other than those, the first PR also featured:
- Ftrace optimisations from Guo Ren, the first couple patches from a larger series he has been
working on for some months.
- A fix from Andy Chiu, that's been in the works since Linux Plumbers, that caused panics in ftrace
relating to disabling preemption.
- Another one that's been in the works since Plumbers, RISC-V previously had a mix of SOC_ and ARCH_
symbols for vendors, depending on whether a vendor had prior support in other architectures. There
are now ARCH_ versions of the SOC_ symbols as part of an effort to gradually convert everything to
ARCH_ to match what is done in other parts of the kernel.
- Splat decoding support for RISC-V in the decodecode script, from Björn Töpel.
- Sergey Matyukevich upstreamed part of a fix for a tricky looking cache update bug, that seems to
occur on uniprocessor systems running CONFIG_SMP=y kernels.
In the netdev PR, which is where bpf patches for RISC-V (and other archs) get into mainline, Pu
Lehui landed support for both the bpf trampoline and 64-bit kfunc support. Björn seemed pretty happy
about that, so it must be important!
The second PR, as might be expected, was substantially smaller. It did bring with it a KASAN rework
from Alex Ghiti, aiming to substantially simplify what he described as RISC-V's "intricate"
implementation and a bump in COMMAND_LINE_SIZE that should allow syzbot to start fuzzing RISC-V.
In terms of fixes, we got yet another round of fixes for handling the (highly frustrating) decision
to move the csr and fence.i instructions out of the I extension, specifically for mixed toolchains
where some components pre-date that move.
As of -rc3, the most interesting fix this cycle was from Sergey Matyukevich, reverting some
ill-fated changes of his for flushing stale TLB entries and replacing them with another fix.
As far as I can tell from reading the patches, the issue was mainly hit on uniprocessor systems
running SMP kernels - at least the drawn out discussion certainly featured discussion of that
configuration!
Nathan Chancellor took over a bit of work I was doing, the nth patchset of late that has to deal
with the fallout of RVI's decision to move some instructions out of the base I extension.
Great craic really.
A fairly "major" fix for how late in the game it came was Alex Ghiti's conversion of RISC-V back
to using the fixmap for dtb mapping. It was moved out of the fixmap in 2020, with the logic that it
was not actually needed & wasted a lot of space.
However, I ran into some issues where either:
a) reserved memory node names would use "early" virtual
addresses (and thus cause a panic when accessed in a driver)
b) reservation was done too late to prevent early memblock allocations being done in a section of
reserved memory. This would cause the reservation to fail, a bit of an issue if that memory is
not actually usable by the kernel...
The most straightforward thing was to just move back to using the fixmap & sidestep the issue.
Thanks to Alex for doing something promptly & saved me significant time flailing around if I had had
to do it myself!
Of course there were a bunch of other various fixes, with several focusing on different bits of TLB,
and some for the text patching code that had recently been changed by Jisheng et al.
One of those fixes being because somehow we all forgot that the aforementioned alternatives
framework is not always enabled, and if it wasn't the FPU could not be used!
0 - https://lore.kernel.org/all/mhng-9df18be9-c02d-41f8-93ee-7e1e124a31ec@palmer-ri-x1c9/
1 - We also do it when modules are loaded
2
15
14