@libcamera has recently received an implementation proposal of an auto-focus algorithm for Rockchip RK3399. A great occasion to continue the work done at @fosdem to support the @PINE64 PinephonePro cameras.

Here it is a quick test of the auto-focus algorithm running in "continuous auto-focus mode" on the PinephonePro IMX258 rear camera.

Let's try to merge it soon upstream!

PS:
Sorry for my shaky hands during the first 10 seconds, let's say I was trying to make things harder for the algorithm

@PINE64 @fosdem @libcamera credits to Daniel Semkowicz from Theobroma Systems as original patches author https://patchwork.libcamera.org/project/libcamera/list/?submitter=126

(note: edited to correct a previous error)

@jmondi @libcamera @fosdem @PINE64

What program are you using for this gif shot? I have a Pinephone Pro with Manjaro Plasma (and also with Mobian). On them I use Megapixels for the camera (which works on Manjaro Plasma, though not on Mobian). But, so far as I know, that program doesn't take videos/gifs, and the photos it takes are not as clear as what I see in your posted gif.

@jmondi @libcamera @fosdem @PINE64

So great to see this - especially for the use-case previewed, detecting QR codes, in apps like Authenticator (https://flathub.org/apps/details/com.belmoussaoui.Authenticator)

@markgrieveson @libcamera @fosdem @PINE64 oh this is just 'cam' (one of the libcamera test applications) with SDL output. It's just for testing, shouldn't be considered a real camera application.

However, at fosdem we have (well, Kieran has, actually) just presented how is it possible to run any camera application that goes through pipewire/gstreamer with libcamera on the pro and mentions a few other libcamera-based camera application such as pinhole.

https://fosdem.org/2023/schedule/event/linux_camera_apps/

@rmader @libcamera @fosdem @PINE64 uh that's interesting! does this goes through pipewire/gst as well ?

@jmondi @libcamera @fosdem @PINE64 Yep, it's even using the xdg-camera-portal to optain the pipewire socket using ASHPD (see https://flathub.org/apps/details/com.belmoussaoui.ashpd.demo for the demo). So yeah, there are real apps already out there that use the stack \o/
Rotation support is also missing though, see https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/284

@markgrieveson @jmondi @libcamera @fosdem @PINE64 he records the screen with wf-recorder

For everyone but @jmondi I should probably add that Wireplumber does not yet prefer libcamera over v4l2 by default, so things don't yet work out-of-the-box on devices requiring that (like most mobile phones, including the PPP). But we hope to fix that soon - feel free to follow https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/379

@markgrieveson Apparently this was one of the libcamera test apps - "cam" with SDL backend. The important point here is, however, that libcamera can be used with many different frontends - once a few more things land upstream, this is what you'll get in Gstreamer based apps like Cheese, in webrtc apps like Firefox and Chromium - you name it.

@markgrieveson
Unfortunately there's no real good mobile friendly camera app making use of the stack yet - there are several ones in development though. If you want to play around, you can give this a go: https://gitlab.gnome.org/rmader/camera/-/issues/1

It uses libcamera through pipewire and gstreamer - hopefully the future standard stack for GTK apps. It currently requires a non-standard pipewire config, disabling the v4l2 camera backend (so only libcamera is used).

@jmondi @libcamera @fosdem @PINE64
it does not look much like focusing at all, but may be the quality of the camera is just so 😭

@insl @libcamera @fosdem @PINE64 shaky hands don't help and for sure it's perfectible, but it indeed focuses (more visible after the first 10 seconds of video).

Of course it perfectible, but it's a starting point at least :)

I was honestly surprised myself, as I was told contrast based continuous auto-focus was a no-go, while it can achieve decent results.

@jmondi @libcamera @fosdem @PINE64 looks very impressive!

@PINE64 @fosdem @libcamera for the image-tuning nerds, be aware that the #PinephonePro back camera is the PDAF capable version of the Sony IMX258 sensor.

PDAF provides an estimate of the image contrast and sharpness (which an algorithm needs to maximize by getting the focus distance right) through some "special" pixels and it's way more efficient than purely software-based algorithms where the scene contrast is estimated using the ISP provided statistics. It's way more efficient and performant, but needs dedicated hw support (which the ppp has).

The @Raspberry_Pi backend in @libcamera is, as usual, one step forward and provides a PDAF based algorithm for their newest v3 camera module which can be used as a base for extending the support to other platforms and sensors. It's not a negligible effort but the code is there and just need someone to work on that.

@markgrieveson you mean that you get pictures with megapixel on mobian on a pinephone pro ??? On mine megapixel just does not start... Do I have to do something special ? @jmondi @libcamera @fosdem @PINE64

@RLetot @markgrieveson @libcamera @fosdem @PINE64 this is separate development from megapixels, which doesn't use libcamera.

@RLetot @markgrieveson @jmondi @libcamera @fosdem @PINE64 For @mobian on the PPP you can now (since last night) `apt install pinhole` and get the camera working (using @libcamera, of course!)

@awai @RLetot @markgrieveson @jmondi @libcamera @fosdem @PINE64 footnote: still using the Mobian staging repository, rather than the stable one, most probably.

@mobian @RLetot @markgrieveson @jmondi @libcamera @fosdem @PINE64 nope, it's even in our bookworm repo (there likely weren't any user of this app so far on our side, so I migrated it quickly 😉 )

@awai @mobian @RLetot @markgrieveson @jmondi @libcamera @fosdem @PINE64
It works!!!