Wow, amazing! I can also send you some developed film, i have a whole roll.

Question! This sentence caught my eye!

The dangling bit coming off the circuit board is a light sensor that helps the webcam find the proper exposure levels.

WHAT!? really? I assumed it was a microphone! I've been clipping it out of all the spectrometers! If that's true, we have another way to control exposure on the spectrometers... would be very exciting. Can you confirm this?

Thanks, great writeup, don, and AMAZING to see it starting to work!!

HAH! We can absolutely not confirm the statement in that sentence. That assessment (guess) was made late at night, after much whiskey. If you've been clipping it off and things have been working fine, then our assessment is likely wrong :) In fact, I shall revise the post to include a more tentative statement about the nature of that dangling bit.

Thanks so much re: the developed film! I'll rummage about here to see if I can find any (was also thinking I might ask CVS in case they discard it regularly and wouldn't mind giving some to me) -- but if I somehow can't find any, and it's easy to mail it along, then that'd be cool!

OK, still interested if that's possible.

Re: film - you can prob. get a cheap roll of color film, pull it entirely out of its roll, roll it up, and hand it to the 1 hour photo place. That's good for some shocked looks too :-P

I was able to get film for $3.50 and get it made into negatives for $3.

Oh, cool! Yeah, let me just do that! I will make sure to mutter "citizen science -- no time to explain" as I unroll the film in-store.

Hi Guys,

I am one of the SimpleCV devs and I noticed your post. I think we work together to fix your issues and create a better library all around. Can you either toss a post up on our help forum (http://help.simplecv.org) or file an issue github and we'll take care of it just as soon as we can.

Don and Craig, This is incredible progress. It's really reassuring that Craig is already doing affine registration. It boggles the mind that the necessary libraries might fit on an SD card and run on a Pi.
I assume you have each camera plugged into its own USB port? What is the trick to synchronizing the shutters? Can you override or pregame the camera's autoexposure routine so it does not contribute to unequal shutter lag? Would it be possible to connect the two cameras with a Y cable so one signal from a USB port went to both cameras?

Before your next hack session maybe you should tidy up that dresser a bit.

@warren on March 5, 2013 - 14:53.

Wow, amazing! I can also send you some developed film, i have a whole roll.

Hey, if you're giving out free film, maybe you can send me some too? You can put it in with my mobile spectrometer kit shipment -- if you have any control over that!

@cfastie on March 6, 2013 - 23:46.

It boggles the mind that the necessary libraries might fit on an SD card and run on a Pi

Heck, I got 6 gigs left on my 8 GB SD after installing Debian 6 on my RPi. Software - at least in the Linux world - is small these days compared to storage media and data files.

What is the trick to synchronizing the shutters?

Having a stationary target ;) In all honesty, we haven't thought too much about this; although, we did run two fswebcam processes asynchronously so they wouldn't block each other and that made up for most of the lag.

Can you override or pregame the camera's autoexposure routine so it does not contribute to unequal shutter lag?

Possibly, we are looking at some Python library (SimpleCV/opencv) that will give us more control than fswebcam. Unfortunately, I have been struggling with some issues in the underlying linux drivers (v4l2) that prevent me from getting two webcams running simultaneously at high res - the nasty "Cannot allocate memory" bug that plagues many users. But, there is hope... Don's computer seems to be immune to this issue and he is running a new version of the software... so I guess its about time for me to do a distribution upgrade. I'll report back whether or not this issue still exists in Ubuntu 12.10 with v4l2 v0.8.8, since I heard Jeff was having a similar problem.

Would it be possible to connect the two cameras with a Y cable so one signal from a USB port went to both cameras?

I don't think so, because these UVC devices rely on a two-way link, unlike - perhaps -- CHDK cameras.

Oh that's right, the photos have to come back from the cameras into the USB ports so it would be hard to share one port.

Shutter sync won't be that important for ground based plant cam uses, but aerial work usually requires pretty good sync. This note has a link to a nice timer with a big 1/100 second display which is great for documenting the sync you are getting. Unfortunately the timer is Windows only.

That's nuts that the Pi operating system inhabits only 2 GB.

Sadly, the v4l2 upgrade did not fix the USB bandwidth problem for dual webcams; this issue probably isn't a bug but really a limitation of the cheap webcam hardware with the expected usage case of the drivers (full video streaming). The webcams have no on-board mjpeg encoder, which is one method of lowering the bandwidth requirements. The reason that Don's laptop works must be that his USB ports are on separate conrollers so are not fighting for the bandwidth - I don't think this is the case with the RPi's two USB ports. Short of finding (or writing!) a new driver that allows low FPS high res. capture, we are left with a semi-satisfactory work-around: force the stream data structure to unload after taking the capture, but before loading the other camera's stream; unfortunately, this introduces a ~0.5 second lag on my fast computer, which may be worse on the RPi. (Since this is one program, the timing can be done in situ by requesting timestamps from the OS [easier than it sounds].) The lag would only be about 0.005 seconds or less if we didn't have to reload the stream every time. I am going to muck around a bit more with the low-level stuff and see if we can't do better.

For those of you who really want to know the details of the USB bandwidth problem, the UVC driver site has this FAQ item.

Wow, it sounds like you are operating right on the edge of the system's capabilities. That problem is related to streaming video, and for the IRcam you need only transfer a single frame from each camera. But I guess the problem is that with little or no RAM for buffering, the cameras must transfer the photo as soon as it is captured. And for synchronous photos, that means two full frames trying to get through one USB controller at the same time and maxing out the limit. Is one option an additional USB controller, or would that be more trouble and expense than just using two Pis? An upgrade to the cameras might help if the new ones had more RAM or MJPEG compression, but that would probably cost more than a second Pi.

Maybe the land-based Plant Cam can have asychronous capture and a more expensive option is needed for aerial work.

Chris, your comments are right on the "money" (pun intended). It looks like the best we can do with these budget cams, short of writing a new driver (if even possible given the hardware constraints), is have two modes: high res (1280x1024)/high lag (~0.5 sec); low res (320x240)/low lag(~0.005sec).

Is one option an additional USB controller, or would that be more trouble and expense than just using two Pis?

I'm not sure if that is possible and cheaper, but maybe we could do one USB camera and one ethernet camera, since on the Model B they are separate hardware controllers. Two RPi's is also a realistic option though synchronization would be tricky, but the money might be better spent on fancier cameras.

Maybe the land-based Plant Cam can have asychronous capture and a more expensive option is needed for aerial work.

I'm mostly in agreement, but I'm going to toss out a hypothetical to the experts: For the aerial mapping application, maybe the lag isn't as bad as it seems -- remember that the scene isn't moving, only the perspective is. If we take enough shots to get coverage, then maybe the images can be affine-aligned, then stitched into two sheets, IR and VIS, then overlaid. Another problem might be motion blur, but as you know, this could be algorithmically corrected too. We could call it an "IR/VIS point and shoot (and stitch) camera".

You're right that for aerial work a 0.5 second lag is not terminal. The whole rig will move some, but the photos from the two cameras will still overlap a lot. They can be registered for making NRG or NDVI, but the overlap will sometimes be only 60-80%, so the cropped composite will generally be much smaller than a full frame. So more photo pairs will be needed to get complete coverage of the scene, more photo pairs will have to be registered, normalized, and combined into multispectral composites, and there will be more composite images to stitch together into multispectral maps, which will have more stitching seams and artifacts. But it will still work.

In fact, it provides some interesting information. If the lag is constant, photo pairs that do not overlap a lot will have been taken while the rig was moving a lot. Those photos will have more motion blur and probably be less vertical than photos from pairs that overlap a lot. So a solution is to take a lot of pairs and throw out the pairs that overlap less than x. The result will be better quality than your average pair of perfectly synchronous kite photos.

Stitching all the VIS into one map and all the NIR into another and then rectifying the maps is a non starter. The rectification will be lousy. And the stitching process is manual, so you only want to do it once. Or three times if you want VIS, NRG, and NDVI. Or once if somebody figures out how to use the information in the first manually stitched map to create the other two. You would have to record all of the manual placement and affine transforms for each image. No problemo, Baby.

Reposting a comment that somehow slipped through the cracks:

Comment by kscottz: Hi Guys, I am one of the SimpleCV devs and I noticed your post. I think we work together to fix your issues and create a better library all around. Can you either toss a post up on our help forum (http://help.simplecv.org [1]) or file an issue github and we'll take care of it just as soon as we can.

Hi kscottz, We are trying to do image registration (affine alignment) with shots taken from slightly different viewpoints. The issue I believe was already raised a while ago in this thread on your help forum. In the reply by "Rishi Mukherjee" he gives a link to a patch that he wrote which makes a change to the homography matrix of the method ImageClass.Image.findKeypointMatch in this github pull request. I cannot vouch for the sanity of this patch, but I can confirm that the homography matrix that it exports is compatible with the function cv2.warpPerspective; whereas, the method in the latest pip install SimpleCV (v1.3) exports a homography matrix that gives an incorrect/incompatible transformation that is rotated and somewhat distorted. I don't know that much about image manipulation, but maybe the problem has something to do with the different conventions used?

follow the technical Python code discussion here: https://github.com/ingenuitas/SimpleCV/issues/320

In pursuing the image registration problem, some more bugs were found in SimpleCV that cause some colorspace image formats to get accidently inverted during geometrical transformations. (Really the fault occurs because of row-major vs. column-major array formatting contentions - a tired old example of computational conventions that don't "really matter" but no one can agree on!)

Don't get me wrong, I am glad that the SimpleCV project exsists and I will continue using it as long as the bug fixes keep coming. Its because of these silly format incompatibilities and the awful legacy of C/C++ syntax that Python OpenCV library is not already simple!

For technical code discussion these issues can be tracked at: https://github.com/ingenuitas/SimpleCV/issues/339