synchonous image capture with >1 Lepton 3.5

[llarrieu]

We're trying to take images using multiple Lepton 3.5 cameras at the same time and are running into issues. We're using a Raspberry Pi 4 and flirpy. We're able to get pseudo-synchronous capture using a for-loop, but the process takes approximately 5-6 seconds to complete, which is too long for our purposes. We've tried using Pool from the Multiprocessing Python library and found that only 1 is able to finish its entire process, whereas the others will start but will only produce empty data arrays(flirpy produces the "cannot open image capture device" error).
Has anyone else tried to do something similar and gotten better/faster results, or know of other ways to accelerate the process of getting images from multiple cameras?

[jveitchmichaelis]

Have you looked the threaded camera class here? It's not super well documented, but you can find a Boson implementation here as well. If you adapt it for the Lepton, you should be able to spawn a bunch of them.

https://github.com/LJMUAstroecology/flirpy/blob/main/src/flirpy/camera/threadedcamera.py

At a minimum i would try running six instances of Python to see if you can capture from each one continuously in parallel. Traditionally in machine vision the way you'd do this is slave the cameras to each other via hardware trigger, otherwise you can do it in software with a timer that triggers each thread more or less simultaneously.

You may be running up against limitations of Video4Linux and some of these questions (including why you only get even device numbers - that has nothing to do with flirpy) may best be asked there. For example if you're using the PT boards, Flirpy uses OpenCV for capture underneath, nothing special.

[llarrieu]

we haven't looked into using the threaded camera class yet, but we'll check that out; we were thinking about using a hardware trigger via the vsync pin but didn't really pursue it because we weren't sure how to approach accessing and using it via the PT boards.
I'll also check and see if there's any OpenCV-centered documentation regarding this and the device ID issue too, and ask the community there if the documentation is sparse.