Hi,
A couple of years ago I developed a multispectral camera based on the compute module of the Raspberry Pi. Hardware wise, the camera is pretty much done, and I'm planning to launch a campaign in Kickstarter to produce the final product.
Now, there is some work to be done in terms of data processing. So far, the camera captures images with the raw bayered data attached in a numpy array, which is demosaiced and then separated in 4 different bands and saved using opencv as a tiff image.
As expected, the raw, bayered, data is quite dark and has twice as many green pixels due to the raw format being rggb. I would like to know if there is someone who would like to collaborate improving the processing pipeline.
Below a test photograph I took sometime ago. I will follow up with some better pictures once I receive new lenses and NIR filters.
Regards,
Mayke
It would be amazing if you could do this. I tried to capture RAW data but could never figure out how to de-mosaic, color balance and scale from 10 to 8 bits. Makes you appreciate everything the picamera does to make images/video .
One option to consider for post processing is image sequencer. I don’t think it accepts TIFF but its open source format makes it adaptable to new tasks. http://sequencer.publiclab.org/examples/#steps=
See below for an image sequence that tries to scale plants to green/blue and objects to yellow/red.
Reply to this comment...
Log in to comment
Hi MaggPi,
Thanks for your comment. Picamera has a very simple de-mosaic algorithm. The way I've been analysing the images is by load them directly from the RPi into Matlab, where I further de-mosaic them, adjust the white balance and so on. Here you can see an example of a raw image processed in Matlab. I'm busy now installing opencv directly into the CM, which seems to be very straight forward but it's not. The 4GB size of the CM makes you very limited in terms of what you can install. IMG_0000_undistorted.tiff
Reply to this comment...
Log in to comment
Hi! This is awesome. You can find some more on related projects at #pi-camera and #raspberry-pi-infragram
We're also working on a cool project to make RPi images easier to pregenerate for download - you should check it out! https://publiclab.org/n/17262
Would your workflow be possible using Image Sequencer? #image-sequencer
Thanks for sharing!!!
On Fri, Oct 12, 2018 at 9:41 AM \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Hi @warren, I like #image-sequencer. It would be very interesting to develop a variation of it customised for aerial imaging. It would have to include things like radiometric calibration, vignette correction, gradient (which it's already included), and maybe an image registration or mosaic builder. People could then buy the hardware, capture images and process them using #image-sequencer. Let me know what you think. Regards, Mayke
Reply to this comment...
Log in to comment
Hi, I think that would be great. I'm trying to do an install of
infragram
in one recipe here, and if that works, I may do the same forimage-sequencer
, and set it up to run automatically on all images taken. If you're interested in trying to open a PR for this, I'm happy to help get it moving! It'd be installingimage-sequencer
instead ofinfragram
:https://github.com/publiclab/pi-builder/pull/37
Reply to this comment...
Log in to comment
CM Module carrier board designed for the multispectral camera.
Reply to this comment...
Log in to comment
Instant trigger in both cameras:
Reply to this comment...
Log in to comment
WOW!!! Was that generated onboard the Pi, or in post-processing? @xose @imvec take a look!
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Ah, but perhaps you're using the wrong filter setting in Image Sequencer -- it looks like potentially the NDVI values are inverse?
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Is this the correct order of the filters?
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Registration is done in Matlab as per this tutorial: https://uk.mathworks.com/help/vision/examples/find-image-rotation-and-scale-using-automated-feature-matching.html
Reply to this comment...
Log in to comment
Here is the picture for better appreciation:
Reply to this comment...
Log in to comment
Wow, very cool. Could you post a pic of the dual cameras too?
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Sure.
Reply to this comment...
Log in to comment
Some new pictures taken with the Multispectral camera:
Reply to this comment...
Log in to comment
Reply to this comment...
Log in to comment