Question: Cheap multispectral camera

maykef is asking a question about general: Follow this topic

maykef asked on October 12, 2018 09:54
598 views | #17281


Hi,

A couple of years ago I developed a multispectral camera based on the compute module of the Raspberry Pi. Hardware wise, the camera is pretty much done, and I'm planning to launch a campaign in Kickstarter to produce the final product.

Now, there is some work to be done in terms of data processing. So far, the camera captures images with the raw bayered data attached in a numpy array, which is demosaiced and then separated in 4 different bands and saved using opencv as a tiff image.

As expected, the raw, bayered, data is quite dark and has twice as many green pixels due to the raw format being rggb. I would like to know if there is someone who would like to collaborate improving the processing pipeline.

Below a test photograph I took sometime ago. I will follow up with some better pictures once I receive new lenses and NIR filters.

Regards,

Mayke

image description



16 Comments

It would be amazing if you could do this. I tried to capture RAW data but could never figure out how to de-mosaic, color balance and scale from 10 to 8 bits. Makes you appreciate everything the picamera does to make images/video .

One option to consider for post processing is image sequencer. I don’t think it accepts TIFF but its open source format makes it adaptable to new tasks. http://sequencer.publiclab.org/examples/#steps=

See below for an image sequence that tries to scale plants to green/blue and objects to yellow/red.
colormapraw.png

Reply to this comment...


Hi MaggPi,

Thanks for your comment. Picamera has a very simple de-mosaic algorithm. The way I've been analysing the images is by load them directly from the RPi into Matlab, where I further de-mosaic them, adjust the white balance and so on. Here you can see an example of a raw image processed in Matlab. I'm busy now installing opencv directly into the CM, which seems to be very straight forward but it's not. The 4GB size of the CM makes you very limited in terms of what you can install. IMG_0000_undistorted.tiff

Reply to this comment...


Hi! This is awesome. You can find some more on related projects at #pi-camera and #raspberry-pi-infragram

We're also working on a cool project to make RPi images easier to pregenerate for download - you should check it out! https://publiclab.org/n/17262

Would your workflow be possible using Image Sequencer? #image-sequencer

Thanks for sharing!!!

On Fri, Oct 12, 2018 at 9:41 AM \<notifications@publiclab.org> wrote:

Public Lab contributor maykef just asked a question entitled ' Cheap multispectral camera':

Help them by posting an answer here: https://publiclab.org/questions/maykef/10-12-2018/cheap-multispectral-camera


Hi,

A couple of years ago I developed a multispectral camera based on the compute module of the Raspberry Pi. Hardware wise, the camera is pretty much done, and I'm planning to launch a campaign in Kickstarter to produce the final product.

Now, there is some work to be done in terms of data processing. So far, the camera captures images with the raw bayered data attached in a numpy array, which is demosaiced and then separated in 4 different bands and saved using opencv as a tiff image.

As expected, the raw, bayered, data is quite dark and has twice as many green pixels due to the raw format being rggb. I would like to know if there is someone who would like to collaborate improving the processing pipeline.

Below a test photograph I took sometime ago. I will follow up with some better pictures once I receive new lenses and NIR filters.

Regards,

Mayke

image description


You received this email because you are subscribed to some or all of the following tags: near-infrared-camera,remote-sensing,ndvi,wetlands,agriculture,software,near-infrared,soil,infragram,tool,19,question:general,method,agricultural,fertilizer,remote-.

To change your preferences, please visit https://publiclab.org/subscriptions.

Report spam and abuse to: moderators@publiclab.org

Check out the blog at https://publiclab.org/blog | Love our work? Become a Public Lab Sustaining Member today at https://publiclab.org/donate

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hi @warren, I like #image-sequencer. It would be very interesting to develop a variation of it customised for aerial imaging. It would have to include things like radiometric calibration, vignette correction, gradient (which it's already included), and maybe an image registration or mosaic builder. People could then buy the hardware, capture images and process them using #image-sequencer. Let me know what you think. Regards, Mayke

Reply to this comment...


Hi, I think that would be great. I'm trying to do an install of infragram in one recipe here, and if that works, I may do the same for image-sequencer, and set it up to run automatically on all images taken. If you're interested in trying to open a PR for this, I'm happy to help get it moving! It'd be installing image-sequencer instead of infragram:

https://github.com/publiclab/pi-builder/pull/37

Reply to this comment...


CM Module carrier board designed for the multispectral camera.

Screen_Shot_2018-10-23_at_19.27.51.png

Screen_Shot_2018-10-23_at_19.27.41.png

Reply to this comment...


Instant trigger in both cameras:

multispectral_cam_sync.jpg

Reply to this comment...


WOW!!! Was that generated onboard the Pi, or in post-processing? @xose @imvec take a look!

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Ah, but perhaps you're using the wrong filter setting in Image Sequencer -- it looks like potentially the NDVI values are inverse?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Is this the correct order of the filters?

index-2.gif

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Registration is done in Matlab as per this tutorial: https://uk.mathworks.com/help/vision/examples/find-image-rotation-and-scale-using-automated-feature-matching.html

Reply to this comment...


Here is the picture for better appreciation:

clip_Raspicam.png

Reply to this comment...


Wow, very cool. Could you post a pic of the dual cameras too?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Sure.

IMG_4087.JPG

IMG_4085.JPG

IMG_4086.JPG

Reply to this comment...


Some new pictures taken with the Multispectral camera:

front_house1.jpg

ndvi1.jpg

ndvi.jpg

Reply to this comment...


side_house1.jpg

NDVI3.png

NDVI2.png

Reply to this comment...


Log in to comment