Jarkko Sakkinen
jarkko
One last thing inherited from v4l2-loopback is kcalloc(), which I dislike given that it is unaccounted memory for a process.
The solution is to make "server side" using vcam ioctl API exclusive dma-buf based because they provide all the robustness. Depending on allocator they can be memcg accounted and they can be also sent between processes.
I.e. I had some harmful robustness that is really unnecessary given that user space camera driver is specialized software to begin with.
It does not affect /dev/videoX, which runs on different VB2 queue supporting userptr, mmap and dma-buf.
The solution is to make "server side" using vcam ioctl API exclusive dma-buf based because they provide all the robustness. Depending on allocator they can be memcg accounted and they can be also sent between processes.
I.e. I had some harmful robustness that is really unnecessary given that user space camera driver is specialized software to begin with.
It does not affect /dev/videoX, which runs on different VB2 queue supporting userptr, mmap and dma-buf.
0
0
1
Jarkko Sakkinen
jarkko
Edited 3 months ago
I think vcam driver is almost ready for RFC submission. It is a dual-VB2 queue architecture and intermediator that moves frames between the queues. Flow can be either based on copying or shared DMA-BUF's and memory allocation can be either vmalloc or dma-sg based.
Right now VCAM_IOC_CREATE locks in "physical characteristics" of the device. It might be nice to be able to change them without re-creating the device but this is *purposely* left out as nothing prevents extending uAPI that way later on.
SLOC is now varying between 2-2.5 KSLOC, which "feels" about right for me considering complexity of the flow etc.
I'll hold for a week so that I discover low-hanging stupid stuff and send it late next week.
Right now VCAM_IOC_CREATE locks in "physical characteristics" of the device. It might be nice to be able to change them without re-creating the device but this is *purposely* left out as nothing prevents extending uAPI that way later on.
SLOC is now varying between 2-2.5 KSLOC, which "feels" about right for me considering complexity of the flow etc.
I'll hold for a week so that I discover low-hanging stupid stuff and send it late next week.
2
1
0
Jarkko Sakkinen
jarkko
Starting documenting vcam driver for RFC patch set.
I probably don't convert my test yet to kselftest. For the time being, seeing is believing so I'll just link the test program Git URL to the cover letter. Probably based on feedback I'll also have better information what would be the most appropriate kselftest.
I probably don't convert my test yet to kselftest. For the time being, seeing is believing so I'll just link the test program Git URL to the cover letter. Probably based on feedback I'll also have better information what would be the most appropriate kselftest.
1
0
0
Jarkko Sakkinen
jarkko
Edited 3 months ago
For a parentless video device, is it too unorthodox embed dma_mask, dma_parms to your private context struct?
I just put vdev->dev.dma_* to point to those fields.
It seems to work at run-time for vb2_dma_sg_memops.
I don't know really what I'm doing TBH but after trying out different things this seems to work.
I'm just a dude who wants stream Zoom calls with a phone camera - definitely not a V4L2 expert ;-)
I just put vdev->dev.dma_* to point to those fields.
It seems to work at run-time for vb2_dma_sg_memops.
I don't know really what I'm doing TBH but after trying out different things this seems to work.
I'm just a dude who wants stream Zoom calls with a phone camera - definitely not a V4L2 expert ;-)
0
0
1
Jarkko Sakkinen
jarkko
Right, I obviously need separate "IDLE" and "STREAMING" bits in status word despite always being mutually exclusive. Otherwise it is impossible to define meaningful "wait for bits turned on" op :-)
0
0
1
Jarkko Sakkinen
jarkko
Lot's of stuff I'm doing with vcam driver are really a versions of ideas re-used that I did for SGX driver :-) Everything is a memory manager of some sort in the end of the day... That driver was really a grand tutorial of Linux device model for me and still helps me to navigate through it today.
0
0
2
Jarkko Sakkinen
jarkko
Yet another iteration in ioctl API. Since VIDIOC bindings were dropped from outside completely some complexity of them existing is just unrolling.
Now it is five calls total:
1. VCAM_IOC_CREATE
2. VCAM_IOC_QUEUE
3. VCAM_IOC_DEQUEUE
4. VCAM_IOC_STATUS
5. VCAM_IOC_WAIT
The last two will replace and remove control_fd previously returned by VCAM_IOC_CREATE. It existed because output was using a subset of VIDIOCs previously.
Now it is five calls total:
1. VCAM_IOC_CREATE
2. VCAM_IOC_QUEUE
3. VCAM_IOC_DEQUEUE
4. VCAM_IOC_STATUS
5. VCAM_IOC_WAIT
The last two will replace and remove control_fd previously returned by VCAM_IOC_CREATE. It existed because output was using a subset of VIDIOCs previously.
0
0
0
Jarkko Sakkinen
jarkko
Now that the uapi in vcam is improving by factors daily that will rather make me postpone sending RFC by a week or two than send it too soon.
When this trend stagnates a bit I'll send the RFC :-)
When this trend stagnates a bit I'll send the RFC :-)
1
0
0
Jarkko Sakkinen
jarkko
Edited 3 months ago
OK with some tweaking vcam ioctls reduced down to three ioctl play:
1. VCAM_IOC_CREATE
2. VCAM_IOC_QUEUE
3. VCAM_IOC_DEQUEUE
Most of stuff went to create ioctl, there's now sysfs attributes for limits and available formats, and frames are anyhow freed when fd is closed.
1. VCAM_IOC_CREATE
2. VCAM_IOC_QUEUE
3. VCAM_IOC_DEQUEUE
Most of stuff went to create ioctl, there's now sysfs attributes for limits and available formats, and frames are anyhow freed when fd is closed.
0
0
1
Jarkko Sakkinen
jarkko
Edited 3 months ago
Flattened ioctls:
VCAM_IOC_CREATE
VCAM_IOC_QUERY
VCAM_IOC_SET_FORMAT
VCAM_IOC_ALLOC_FRAMES
VCAM_IOC_IMPORT_DMABUF
VCAM_IOC_QUEUE
VCAM_IOC_DEQUEUE
VCAM_IOC_FREE_FRAMES
The interface is available from fd opened by /dev/vcam, thus making it the life-line of /dev/videoX. The result struct of VCAM_IOC_CREATE provides "control fd", which ATM provides status flag for stream on/off state.
VCAM_IOC_CREATE is one-shot:
1. Before the ioctl is called, other ioctls result -ENOTTY.
2. After successful invocation, the other ioctls become available, and VCAM_IOC_CREATE will result -ENOTTY up until the end.
In other words, no broken use of VIDIOC API and /dev/videoX for outputing frames. Instead, vcam provides a dedicated and consolidated API to perform its task within the constraints what makes sense for virtual webcams and such.
Probably getting ioctls acceptable for reviewers will take a few rounds but I still think that this is the right design choice to get something that we can eventually ship in the mainline kernel.
VCAM_IOC_CREATE
VCAM_IOC_QUERY
VCAM_IOC_SET_FORMAT
VCAM_IOC_ALLOC_FRAMES
VCAM_IOC_IMPORT_DMABUF
VCAM_IOC_QUEUE
VCAM_IOC_DEQUEUE
VCAM_IOC_FREE_FRAMES
The interface is available from fd opened by /dev/vcam, thus making it the life-line of /dev/videoX. The result struct of VCAM_IOC_CREATE provides "control fd", which ATM provides status flag for stream on/off state.
VCAM_IOC_CREATE is one-shot:
1. Before the ioctl is called, other ioctls result -ENOTTY.
2. After successful invocation, the other ioctls become available, and VCAM_IOC_CREATE will result -ENOTTY up until the end.
In other words, no broken use of VIDIOC API and /dev/videoX for outputing frames. Instead, vcam provides a dedicated and consolidated API to perform its task within the constraints what makes sense for virtual webcams and such.
Probably getting ioctls acceptable for reviewers will take a few rounds but I still think that this is the right design choice to get something that we can eventually ship in the mainline kernel.
1
0
1
Jarkko Sakkinen
jarkko
Edited 3 months ago
Here vcam-test outputs to /dev/vcam and in sequence streams /dev/video0 and renders the output to KMS/DRM initialized frame buffer.
I need to flatten the ioctl interface of /dev/vcam a bit but otherwise this is looking pretty reasonable...
I need to flatten the ioctl interface of /dev/vcam a bit but otherwise this is looking pretty reasonable...
0
0
1
Jarkko Sakkinen
jarkko
As per kselftest for vcam, it will by default do frame comparison without displaying anything but will contain also debug mode, which uses KMS/DRM directly to show the results. It's run mostly run in a VM without window manager anyhow.
0
0
0
Jarkko Sakkinen
jarkko
There's uxplay but what are options of libraries capable for streaming AirPlay?
0
0
0
Jarkko Sakkinen
jarkko
Edited 3 months ago
I replaced the misuse of VIDIOCs on driver side with a dedicated ioctl interface:
VCAM_OUTPUT_IOC_QUERY
VCAM_OUTPUT_IOC_SET_FORMAT
VCAM_OUTPUT_IOC_ALLOC_FRAMES
VCAM_OUTPUT_IOC_IMPORT_DMABUF
VCAM_OUTPUT_IOC_QUEUE
VCAM_OUTPUT_IOC_DEQUEUE
VCAM_OUTPUT_IOC_FREE_FRAMES
This is accessed through output_fd. In addition vma_ioc_new returns control_fd, which is read/poll interface for the streaming status.
With these two interfaces it should be possible to implement drivers for streaming sources such as smartphones.
I feel that this project is going to really awesome direction :-)
#linux #kernel #video
VCAM_OUTPUT_IOC_QUERY
VCAM_OUTPUT_IOC_SET_FORMAT
VCAM_OUTPUT_IOC_ALLOC_FRAMES
VCAM_OUTPUT_IOC_IMPORT_DMABUF
VCAM_OUTPUT_IOC_QUEUE
VCAM_OUTPUT_IOC_DEQUEUE
VCAM_OUTPUT_IOC_FREE_FRAMES
This is accessed through output_fd. In addition vma_ioc_new returns control_fd, which is read/poll interface for the streaming status.
With these two interfaces it should be possible to implement drivers for streaming sources such as smartphones.
I feel that this project is going to really awesome direction :-)
#linux #kernel #video
0
0
1