Pavel Machek
Posts
1990Following
95Followers
179
Pavel Machek
pavel
@martijnbraam @bart gstreamer is optional for preview. But you really need it for encoding, and so using it for preview is easy.
Main message from that document was that frames should be copied not from v4l, but from gpu (after downscale and probably YUV conversion), because that's a way to do movie recording.
Main message from that document was that frames should be copied not from v4l, but from gpu (after downscale and probably YUV conversion), because that's a way to do movie recording.
0
0
0
Pavel Machek
pavel
@martijnbraam @bart
If you are doing big redesign... here's how to do video recording.
# Design
ARM has some limitations with DMA, which results in data in DMABUF
being uncached. That is problem on in both V4L->CPU and GPU->CPU
directions, still we can get good video recording on existing
hardware. Here is how to do it.
Put sensor in 4MPix or so mode, and capture to DMABUF.
Reading whole DMABUF from CPU pretty slow, but you can sample data to
do AF/AE. You can also set buffer aside for future reading, and thus
take high-resolution raw photos during video recording.
GPU can work with uncached memory without penalty, and debayer with
downscale is pretty simple operation. It will also provide better
results than downscale in sensor followed by debayer.
GPU can multiply with color conversion matrix easily, and conversion
to YUV format is simple to do. Using subsampled color mode such as
YUY2 enables faster movie encoding and reduces ammount of data
transfered GPU->CPU. As that's uncached and slow, YUV is a big win.
Getting data from GPU->CPU is slow and tricky. You probably want GPU
to compute on frame n while CPU thread is copying from frame n-1 to
CPU memory.
Rest is simple. You can use gstreamer to both encode movie and provide
preview, you can do another GPU pass for preview data. Above can get
0.8MPix, 30fps recording on Librem 5.
If you are doing big redesign... here's how to do video recording.
# Design
ARM has some limitations with DMA, which results in data in DMABUF
being uncached. That is problem on in both V4L->CPU and GPU->CPU
directions, still we can get good video recording on existing
hardware. Here is how to do it.
Put sensor in 4MPix or so mode, and capture to DMABUF.
Reading whole DMABUF from CPU pretty slow, but you can sample data to
do AF/AE. You can also set buffer aside for future reading, and thus
take high-resolution raw photos during video recording.
GPU can work with uncached memory without penalty, and debayer with
downscale is pretty simple operation. It will also provide better
results than downscale in sensor followed by debayer.
GPU can multiply with color conversion matrix easily, and conversion
to YUV format is simple to do. Using subsampled color mode such as
YUY2 enables faster movie encoding and reduces ammount of data
transfered GPU->CPU. As that's uncached and slow, YUV is a big win.
Getting data from GPU->CPU is slow and tricky. You probably want GPU
to compute on frame n while CPU thread is copying from frame n-1 to
CPU memory.
Rest is simple. You can use gstreamer to both encode movie and provide
preview, you can do another GPU pass for preview data. Above can get
0.8MPix, 30fps recording on Librem 5.
1
0
0
Pavel Machek
repeated
repeated
Martijn Braam
martijnbraam@fosstodon.org
2
3
1
Pavel Machek
pavel
@martijnbraam @bart Yes, I believe doing release now with known limitations is best solution.
Can I somehow convince you to mv gopixels rustpixels? :-)
Can I somehow convince you to mv gopixels rustpixels? :-)
1
0
0
Pavel Machek
pavel
@martijnbraam @bart I assume gtk-3 is quite similar to gtk-4? But you are right, something like SDL would also make sense.
But that also will not be small change, so perhaps should not be done between alpha and release? Maybe release version 2 with "either use old gtk-4 with this or avoid it on Librem 5 etc", and then start another rewrite in megapixels-3?
https://blog.brixit.nl/megapixels-2-0-progress/
But that also will not be small change, so perhaps should not be done between alpha and release? Maybe release version 2 with "either use old gtk-4 with this or avoid it on Librem 5 etc", and then start another rewrite in megapixels-3?
https://blog.brixit.nl/megapixels-2-0-progress/
1
0
1
Pavel Machek
pavel
@martijnbraam @bart Postmarketos still ships gtk-3, could we simply use that?
1
0
0
Pavel Machek
pavel
@bart @martijnbraam Could we get megapixels 2, pretty please? :-). Changes from v1 are pretty significant, and I have some work that would be best done post-release...
1
0
1
Pavel Machek
repeated
repeated
@fifisch.cz jsem boostoval předtím... je to blbost, samozřejmě, jen narážím na míru obskurnosti
0
1
0
Pavel Machek
repeated
repeated
Pavel Machek
pavel
@martijnbraam I was lucky Librem 5 did not have this quirk. I guess reason is I420 support which is good format for video encoding.
Anyway, megapixels-getframe can deal with that, good! :-).
Anyway, megapixels-getframe can deal with that, good! :-).
0
0
0
Pavel Machek
pavel
Trying to get #openplus6 camera to work with #megapixels. Here's start: https://gitlab.com/tui/tui/-/blob/master/cam/oneplus,enchilada.conf?ref_type=heads
You may be able to get single frame from megapixels-getframe... and then kernel crashes.
You may be able to get single frame from megapixels-getframe... and then kernel crashes.
0
0
2
Pavel Machek
pavel
@NekoCWD @postmarketOS Congratulation, you won this round :-). I tried getting Clicks Machine on op6 (not much luck so far), now I'm playing with Megapixels (getting kernel crashes so far).
0
0
1
Pavel Machek
repeated
repeated
Vasiliy Doylov
NekoCWD@nekocwd.link🎉 Now focus works fine with #OnePlus6 main camera.
Thanks to cool cats from sdm845 mainlining for camera, c-phy and other stuff 😻
I will clean up my changes and push it in a few days ⬆️
@postmarketOS@social.treehouse.systems
#mainlining #shotonmainline #mobilelinux #linuxonmobile
2
11
1
Pavel Machek
pavel
To build Megapixels on #postmarketos, do sudo apk add mg g++ libconfig-dev git meson libconfig-dev ninja-build
libraw-dev libexif-dev ninja
sudo apk add opencv-dev
# sudo apk add scdoc -- not really needed
sudo apk add gtk4.0-dev
sudo apk install feedbackd-dev
sudo apk add zbar-dev
sudo apk add pulseaudio-dev
..then proceed as usual.
libraw-dev libexif-dev ninja
sudo apk add opencv-dev
# sudo apk add scdoc -- not really needed
sudo apk add gtk4.0-dev
sudo apk install feedbackd-dev
sudo apk add zbar-dev
sudo apk add pulseaudio-dev
..then proceed as usual.
1
1
2
Pavel Machek
pavel
@martijnbraam Yep, "fun". When I realized it is using mplanes with single plane, I felt kind of betrayed.
(I also believe that "secondary" back sensor is 2x zoom; as "primary" one does not work, that was another betrayal).
I tried to get clicks machine working there, but passing buffer between v4l and gpu does not work and I'm kind of stuck.
(I also believe that "secondary" back sensor is 2x zoom; as "primary" one does not work, that was another betrayal).
I tried to get clicks machine working there, but passing buffer between v4l and gpu does not work and I'm kind of stuck.
1
0
0
Pavel Machek
pavel
How to crash #oneplus6? Just do cat /dev/video0 , or attempt to get frame with yavta. Nothing works with that camera, and I now know why: it uses MPLANE mechanism. I got a lot of crashes while playing, but now I have code to get a frame. Unfortunately, I have it in git, and the repository is corrupted :-(.
1
2
5