Jarkko Sakkinen
Posts
5175Following
336Followers
512OpenPGP: 3AB05486C7752FE1
Jarkko Sakkinen
jarkko
Edited yesterday
Not posting this any time soon but now I think swcam has a decent uAPI where vidioc configuration is decoupled from producer of the stream. The producer provides a dataset of <pix_format, frame_rate> pairs that constraint the vidioc API upon creation and via SWCAM_IOC_WAIT gets the specs for the currently playing stream, always in the expected space of configurations.
The streaming pipeline itself has remained the same from the get go.
See:
https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd.git/tree/include/uapi/linux/swcam.h?h=vcam
Just wanted to put this to rebasable and complete state just in case (and I will continue to rebase the branch).
The streaming pipeline itself has remained the same from the get go.
See:
https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd.git/tree/include/uapi/linux/swcam.h?h=vcam
Just wanted to put this to rebasable and complete state just in case (and I will continue to rebase the branch).
1
0
0
Jarkko Sakkinen
jarkko
drafted my first ever himmelblau idm patch for dynamic user credential resolution :-) will take a while before any prs result.
0
0
0
Jarkko Sakkinen
jarkko
-MODULE_DESCRIPTION("V4L2 virtual camera driver");+MODULE_DESCRIPTION("V4L2 software camera driver");
a cosmetic change but describes better the scope and purpose. I.e. vcam -> swcam
a cosmetic change but describes better the scope and purpose. I.e. vcam -> swcam
0
0
0
Jarkko Sakkinen
jarkko
Edited 2 days ago
the next item in my backlog (i have a project to do something to every item in my personal backlog) are container objects. expected outcome is similar as for vcam initially i.e. "fuck no".
but since these are totally sidechannel things and nothing i take too personally i can still work on these topics if nothing else for my own entertainment :-) two years in a review cycle - i have zero problems with that.
i do give up on patch sets even if i've made myself a fool promoting them every single time when objective facts fight against them that i can understand with my limited brain capacity. up until that i go against the wind :-)
but since these are totally sidechannel things and nothing i take too personally i can still work on these topics if nothing else for my own entertainment :-) two years in a review cycle - i have zero problems with that.
i do give up on patch sets even if i've made myself a fool promoting them every single time when objective facts fight against them that i can understand with my limited brain capacity. up until that i go against the wind :-)
0
0
1
Jarkko Sakkinen
jarkko
It is good to remark libcamerify from libcamera-tools is LD_PRELOAD based solution for "virtual camera", not ubiquitos solution really. It is also across the board much more stressing and higher latency solution for system than a well-designed loopback device.
0
0
0
Jarkko Sakkinen
jarkko
1
0
2
Jarkko Sakkinen
jarkko
For the v2 I think I have sane way to constrain the state i.e., in VCAM_IOC_CREATE:
__u32 nr_states;
__u32 states;
Each state (32 bytes per state) specifies configuration with pixel format, geometry, stride etc.
Obviously this means some redundancy perhaps because one has to address e.g. same width and height in different configurations but I think in the end this database approach is the most robust pick.
__u32 nr_states;
__u32 states;
Each state (32 bytes per state) specifies configuration with pixel format, geometry, stride etc.
Obviously this means some redundancy perhaps because one has to address e.g. same width and height in different configurations but I think in the end this database approach is the most robust pick.
1
0
0
Jarkko Sakkinen
jarkko
Edited 2 days ago
Test code I have cleaned up relocated:
https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/vcam-test.git
From this i spin off at least some kind of initial kselftest.
https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/vcam-test.git
From this i spin off at least some kind of initial kselftest.
1
0
0
Jarkko Sakkinen
jarkko
Edited 3 days ago
Despite the feedback does not look very accepting, continuing with vcam makes common sense to me enough that I continue improving and rebasing it.
In order to lift it up from RFC phase at least property negotiation needs some rework, there's some compliance suites for V4L2 to try out and of course some test code needs to be supplemented.
Up side with resistance is at least that it is antidote for a loose send-finger :-) I'll focus on making each iteration considerably better than N - 1.
In order to lift it up from RFC phase at least property negotiation needs some rework, there's some compliance suites for V4L2 to try out and of course some test code needs to be supplemented.
Up side with resistance is at least that it is antidote for a loose send-finger :-) I'll focus on making each iteration considerably better than N - 1.
1
0
0
Jarkko Sakkinen
jarkko
v4l2loopback has highly scattered but at the same time wide base. It even includes stuff like scientific research. Here's one recent'ish example: https://dl.acm.org/doi/epdf/10.1145/3664647.3681007. This also implies that it is a tainted driver likely deployed to multiple critical organizations and environments.
This type of feature is also quite robust tool for many uses where emulation is required but we still need sufficient figures for latency and throughput. It could be applied e.g., to drone platform testing where vision can be mocked with virtual devices providing simulated environment or pre-recorded environment (for training AI).
It is also good to keep in mind not every single piece of legacy software can be magically turned into PipeWire clients, and neither it is a great fit all possible deployments of Linux.
This type of feature is also quite robust tool for many uses where emulation is required but we still need sufficient figures for latency and throughput. It could be applied e.g., to drone platform testing where vision can be mocked with virtual devices providing simulated environment or pre-recorded environment (for training AI).
It is also good to keep in mind not every single piece of legacy software can be magically turned into PipeWire clients, and neither it is a great fit all possible deployments of Linux.
0
0
1
Jarkko Sakkinen
jarkko
OK, there's 6.19-rc8. Has not happened for a while, or at least don't recall
0
0
0
Jarkko Sakkinen
jarkko
IMHO vcam is a bit like ntsync. In other words, it is somewhat use case optimized but the use case is just relevant enough to make it sense to have the feature in the mainline kernel.
0
0
0
Jarkko Sakkinen
jarkko
I didn't really know all the time what I was doing but I at least the choice of tying the pole providing the service tightly to dma-buf was a right design choice :-) This has been somewhat experimental project.
0
0
0
Jarkko Sakkinen
jarkko
Edited 4 days ago
early RFC: https://lore.kernel.org/linux-media/20260201133342.335680-1-jarkko@kernel.org/T/#u
I'm happy with approach to mm where caller allocates all the resources, which is great for accounting from the correct victim. For any driver that is actually often a non-trivial problem. Other stuff, I'll hold for feedback...
I'm happy with approach to mm where caller allocates all the resources, which is great for accounting from the correct victim. For any driver that is actually often a non-trivial problem. Other stuff, I'll hold for feedback...
1
4
6
Jarkko Sakkinen
jarkko
Edited 4 days ago
any Rust crates for streaming AirPlay?
would be nice at least in some level test the main use case for vcam before sending any patches :-) murphy's law is strong in this...
would be nice at least in some level test the main use case for vcam before sending any patches :-) murphy's law is strong in this...
0
0
0
Jarkko Sakkinen
jarkko
Now vcam is internally running in dma-buf based architecture only:
https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd.git/tree/drivers/media/vcam.c?h=vcam
The key function is vcam_dequeue_frames, which pairs queued output and capture frames. When they match the same dma-buf it won't copy anything. So it works like zipper (or that's how I imagine it in my head) :-)
I think it is quite sane and elegant pipeline.
https://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd.git/tree/drivers/media/vcam.c?h=vcam
The key function is vcam_dequeue_frames, which pairs queued output and capture frames. When they match the same dma-buf it won't copy anything. So it works like zipper (or that's how I imagine it in my head) :-)
I think it is quite sane and elegant pipeline.
0
0
2
Jarkko Sakkinen
jarkko
One last thing inherited from v4l2-loopback is kcalloc(), which I dislike given that it is unaccounted memory for a process.
The solution is to make "server side" using vcam ioctl API exclusive dma-buf based because they provide all the robustness. Depending on allocator they can be memcg accounted and they can be also sent between processes.
I.e. I had some harmful robustness that is really unnecessary given that user space camera driver is specialized software to begin with.
It does not affect /dev/videoX, which runs on different VB2 queue supporting userptr, mmap and dma-buf.
The solution is to make "server side" using vcam ioctl API exclusive dma-buf based because they provide all the robustness. Depending on allocator they can be memcg accounted and they can be also sent between processes.
I.e. I had some harmful robustness that is really unnecessary given that user space camera driver is specialized software to begin with.
It does not affect /dev/videoX, which runs on different VB2 queue supporting userptr, mmap and dma-buf.
0
0
1
Jarkko Sakkinen
jarkko
Edited 5 days ago
I think vcam driver is almost ready for RFC submission. It is a dual-VB2 queue architecture and intermediator that moves frames between the queues. Flow can be either based on copying or shared DMA-BUF's and memory allocation can be either vmalloc or dma-sg based.
Right now VCAM_IOC_CREATE locks in "physical characteristics" of the device. It might be nice to be able to change them without re-creating the device but this is *purposely* left out as nothing prevents extending uAPI that way later on.
SLOC is now varying between 2-2.5 KSLOC, which "feels" about right for me considering complexity of the flow etc.
I'll hold for a week so that I discover low-hanging stupid stuff and send it late next week.
Right now VCAM_IOC_CREATE locks in "physical characteristics" of the device. It might be nice to be able to change them without re-creating the device but this is *purposely* left out as nothing prevents extending uAPI that way later on.
SLOC is now varying between 2-2.5 KSLOC, which "feels" about right for me considering complexity of the flow etc.
I'll hold for a week so that I discover low-hanging stupid stuff and send it late next week.
2
1
0
Jarkko Sakkinen
jarkko
Starting documenting vcam driver for RFC patch set.
I probably don't convert my test yet to kselftest. For the time being, seeing is believing so I'll just link the test program Git URL to the cover letter. Probably based on feedback I'll also have better information what would be the most appropriate kselftest.
I probably don't convert my test yet to kselftest. For the time being, seeing is believing so I'll just link the test program Git URL to the cover letter. Probably based on feedback I'll also have better information what would be the most appropriate kselftest.
1
0
0
Jarkko Sakkinen
jarkko
Edited 6 days ago
For a parentless video device, is it too unorthodox embed dma_mask, dma_parms to your private context struct?
I just put vdev->dev.dma_* to point to those fields.
It seems to work at run-time for vb2_dma_sg_memops.
I don't know really what I'm doing TBH but after trying out different things this seems to work.
I'm just a dude who wants stream Zoom calls with a phone camera - definitely not a V4L2 expert ;-)
I just put vdev->dev.dma_* to point to those fields.
It seems to work at run-time for vb2_dma_sg_memops.
I don't know really what I'm doing TBH but after trying out different things this seems to work.
I'm just a dude who wants stream Zoom calls with a phone camera - definitely not a V4L2 expert ;-)
0
1
1