There is a new presentation about the improvements to the XiangShan Kunminghu V3 RISC-V Vector implementation.

Previously the RVV implementation was suvierly bottlenecked to the point of unusability.
Now it looks like we'll have a good open-source out-of-order RVV implementation soon.

Apparently the implementation was completed in September, but they are now successively releasing the changes over the next few months.

https://github.com/OpenXiangShan/Talks-and-Publications/blob/master/slides/20261020-RVSC-面向通用高性能计算的向量扩展设计.pdf

(the slides I post here were translated by google translate)

#riscv #rvv #simd

First thing, they fixed the decoder to maintain full 8 uop/cycle throughout reguardless of LMUL.

vtype gets propagated in IBuffer.

I personally prefer the idea of handling RVV after scalar/fp rename, so you can propagate vtype in rename and handle RVV renaming/cracking in a slightly decoupled and narrower stage, but apparently they were happy with this solution.

They improved the memory uop generation improvements for strided/indexed load/store:

fault-only-first load instructions are now usable as well:

They are now not only treating v0 as a mask register, but make it dependent on the instruction semantics. Previously they already had seperate mask and vector files.

and also now fixes the previous LMUL>1 mask write serialisation

gains in libc str* look nice (this is probably mostly the fault-onlt-first load improvements)

Permutations have also been improved. I am a bit worried if permutation unit blocks the other execution units, that wasn't apparent from the slides.

still, these LMUL>1 latency/throughput numbers look great.
16 cycle latency, 12 cycle throughout LMUL=8 vrgather.vv is really good.
It's also nice to see that the rules of thumb: prefer vslideup over down and .vi over .vx hold here as well

Finally some other improvements:
* RVV move elimination support
* strides/indexed load/store can generate 3 ld or 2 st per cycle, so the complex memory operations should basically always match or exceed the equivalent scalar code
* segmented load/store in `2*nf` cycles, makes sense. Presumably nf memory accesses followed by nf register writebacks

@camelcdr IMO where you place the vector config handling in the pipeline really depends on what performance characteristics you're looking for. Handling it after scalar rename adds latency in the common case, but gets rid of a bunch of complexity that can result in really bad issues in the uncommon cases.

The way the ISA is designed certainly favors the higher-latency designs, but IMO the right decision depends on what sort of workload characteristics you're expecting to see. Unfortunately it's a hard tradeoff to make: the latency-sensitive workloads also tend to be the irregular ones that are harder to cover pre-silicon, so the chance of missing an edge case that destroys performance is high.

So designers are sort of forced to decide between rolling the dice on the lower-latency implementation or sticking with the higher-latency implementation and accepting the inherent performance penalties on some workloads.

I suspect we're going to see two very different styles of RVV implementation show up over the next few years as vendors optimize for one style or the other, and then various ISA changes go through that work around the issues in each.

@palmer From my perspective the "higher-latency" design, is higher latency for RVV but lower for scalar, which generally aligns with what you want in software.

Consider scalar1, m8rvv, scalar2. In a 4-wide design, the "latency" focuses RVV implementation would decode scalar2 after 3 cycles, but the "throughput" one on the first.

I envision a design that puts a small vector buffer after scalar rename, from which it cracks/renames at a smaller bandwidth corresponding to the vector backend. The buffer would increase the reorder window substantially, because it store LMUL bundles. I think this would add 1-2 cycle latency for RVV.

But that additional latency is not for RVV -> RVV dependencies, it's only for RVV -> scalar or branch recovery. And then again, branch recovery for scalar instructions would be faster.

With this design Ovlt is also much more feasable. You "just" use vl when cracking from the vector buffer, if vl is already know (scalar runs ahead), otherwise fall back to LMUL.

@palmer I suppose if you mostly run LMUL=1 code, then it's worse.
But I'd imagine the reduced complexity would still make it attractive.

@camelcdr

> Consider scalar1, m8rvv, scalar2. In a 4-wide design, the "latency" focuses RVV implementation would decode scalar2 after 3 cycles, but the "throughput" one on the first.

That's assuming the latency-focused design doesn't predict the vector config, which is where the rabbit hole of edge cases comes from...

@palmer There is nothing it can do if it's actually vlmax LMUL=8, the renamer is only so wide (here 4).

Edit: Well I said nothing. I suppose somebody could figure out LMUL wide renaming, but that seems very hard and not possible without defragmentation.

@camelcdr yes, right, I wasn't looking closely enough at the actual op.

I guess I was thinking of that as a different issue: that's sort of the long-decoder issue, where you have to decide how well the implementation is going to handle instructions that generate more uops than it can deal with in a group (which may or may not be equal to your rename bandwidth). That's what I've seen be the issue with long-LMUL performance, and that doesn't change when you move around the V rename point. Renaming V later just lets you know what the actual uop requirement is without predicting.

and FWIW: it's definitely possible to decouple the rename granularity from the architectural register size. I agree it's probably not a good idea ;)

@palmer With scalar RISC-V you don't have to crack anything before rename, so putting LMUL support into that pipeline directly seems very messy. (well that one 2W atomic, but that doesn't have to be high frequency)
If you separate it, you can scale it down to fit your backend size reducing the complexity further.
And because implementors can choose their VLEN, the backens size is bounded.
I doubt anybody would close to build a 6x128 (like X925) design over a 3x256 design, if software support for both is the same.