I was just reading on the pi website about the rolling shutter and I was wrong about the global shutter (it said only cameras with physical shutters can use a global shutter mode). I think we are stuck with rolling shutter and just have to think of ways to overcome the resulting noise.

Hi! I think the question of rolling shutter might deserve it's own question posted, to really dig into - it's a great question! One thought I had was that the orientation of the camera probably has a big effect on the banding. Banding I've seen can often look like venetian blinds, and doesn't seem to have an effect that's inconsistent across each cross-section of the image. That is, the banding has typically happened at a resolution that makes bands in parallel to the cross-sections, limiting the damage they can do. But if the camera is rotated 90 degrees, that'd change a lot. Probably we need to start citing specific spectra/images to really dig into this topic. Thanks for bringing it up!!!

Ok, the project concepts are taking shape which always prompts more thoughts.

1) By 2D mapping the light spectrum and relative intensity, you will have some arrays of data characterizing the light source. That suggests two approaches: A) average the data (and possibly rotate the plants or B) keep the plant layout and apply the 2D mapping to the plant data. Either way, the plans would have be uniquely identified but with averaging, data would be tossed which generally obscures later analysis.

2) True, some light sources do not have 'steady-state' power sources (eg. HPS lamps in gymnasiums which pulse at 60 Hz and wreck havoc with normal digital photography). Others might 'flicker' at higher rates. To avoid the issue either the sampling rate must be much faster (and then process the data to get an average) or use sensor integration which is much longer than the pulse rate to let the integration time perform the averaging. Fortunately, plant growth is a very slow process relative to all of this.

2a) An 8MPix sensor has about 2800 lines and at the typical 30 fps max scan rate, lines are read at about 85kHz max. Since fluorescent lights with a ballast run at about 20kHz, the higher scan rate could probably produce 'banding'.

2b) So, what to do? I'd suggest dropping the sensor scan rate to increase integration time and then collect multiple spectrum data sets for time-dependent post-measurement analysis. Multiple spectra data sets are a good idea anyway to A) assure the data is valid and B) look for errant low-frequency noise and stability issues. 3D 'waterfall' plots can be very useful for analyzing spectral plots over time.

2c) Averaging. Caution is always warranted when averaging. It is best, when possible, to have more raw data to analyze because noise can take many forms. One large "glitch", which represent only single uncorrelated anomalies, can significantly affect an average value -- but that error will be hidden. Also, averaging always throws data away. Think more in terms of extracting useful information than just tossing information away.

Averaging true Gaussian noise is extracting the true signal from totally uncorrelated noise. However, the noise in PLab spectrometer plots is not all Gaussian so simple averaging does toss useful information which can be extracted by looking for raw data correlations over multiple spectra data sets. Again, fortunately, plants are slow critters.

3) You might look into the variance of sunlight vs a Solux source. While the Solux source does requires a few minutes warm-up, it is very stable. Yes, both sun and Solux are stable, they do have slightly different spectra. From my own observations, I suspect that a Solux lamp (reserved only for doing 'calibrations' so as to drop lamp aging to effectively zero) would provide a more repeatable reference.

Here's the question: if, with periodic checks against the sun, there was some variation. Do you decide that variation was the Solux lamp or just atmospheric water vapor, time of year, smog, etc. -- or some of each? My thought: Use a commercial device (with known, published measurement uncertainties) to measure the Solux. Do that a number of times with repeat measurement setups (to account for those variations) and check the stability. This is not to avoid using a solar reference, but as a potential means of detecting which source is the variable in terms of calibration.

Sorry for the long post; just tossing around some ideas to consider.

Dave

Observation of some basic system parameters will help define the limits.

1) A single stable source (Solux and CFL) plus the PLab-class spectrometer will allow determining the measurement limits.

2) 2D mapping of an example plant light source (testing a range of mapping resolutions) will find the measurement limits of 2D mapping.

3) Repeat and long-time measurements (spectrometer +Solux, +CFL, +PlantLight, +attenuatedSolar, +etc) will uncover measurement error limits

4) Monitoring a stable source while inducing mechanical stress to the spectrometer adds another error limit.

Identifying the various measurement errors, then allows making a set of measurements relative to a transfer standard (eg. a Solux) which means the data will have a known reference so future data sets can be accurately compared.

The smaller Solux run on 24VDC so power modules and achieve 0.5% stability [ https://publiclab.org/notes/stoft/04-14-2016/spectrometer-stability ]. PLab-class spectrometer mechanical stability is often worse.

Just a hint - Mechanical rigidity of a PLab-class spectrometer cannot be over-stressed and 'fixing it' later by averaging data generally throws away a lot of useful information.

Deciding how much data to collect is likely dependent on the above observations. If the plant light sources are variable, additional data might be warranted to allow finding additional correlations in the data. Look for repeatability, drift, synchronous and asynchronous noise and stability-related errors.

Oops responded to the wrong comment, moving here:

Hi all! Great input here. Just noting that we've had a major revision to the PL spectrometer design that should address a lot of the rigidity issues, though perhaps not as much as @stoft's wood design. In any case a major improvement over the paper box based design. You can read more about it at https://publiclab.org/lego

This may be OBE but Hufkens has posted files of the PI response curves:

https://khufkens.github.io/pi-camera-response-curves/

which were also referenced here:

https://publiclab.org/notes/MaggPi/06-17-2018/how-can-you-use-computer-vision-to-reduce-spectral-overlap

I am not familiar with your project to understand all the issues but one way to avoid the 60 Hz flicker issue (even though I am not certain it is an issue) is to use an LED source.

With non steady-state sources, measurement error results from asynchronous data sampling. The 'fix' is either long integration time (assuming the source RMS level actually is stable) or a measurement rate several times the nyquist rate so the sample data represents the actual signal.

Wavelength calibration via CFL (using PLab-class devices) is typically limited by system noise which causes spectral peak 'smearing' and 'jitter' -- though a bit of processing can help.

Finally, assuming stable DC light sources, with PLab-class devices, the biggest measurement error source tends to be broadd-band, non-Gaussian noise generated by mechanical instability.