Public Lab Research note


Planning for Spectral Workbench 2.0

by warren | February 10, 2015 20:30 10 Feb 20:30 | #11572 | #11572

Hello Spectral Workbench community!

Do you wish you could do more with Spectral Workbench?

Do you have trouble with a particular feature or recurring bug?

In coming weeks we'll be beginning a major revamp for Spectral Workbench, similar to the scale of the recent work on MapKnitter 2. Towards that end, and because SW is open source software, we're asking everyone to consider making suggestions for new features, bugfixes, refinements, etc. -- and to help prioritize these so that we can decide what to do first.

In the comments below, please offer suggestions and ask questions! Please provide:

  • a succinct description of the thing you'd like
  • a description of your problem if applicable
  • a few words of context for why you think this is important
  • (optional) suggestions for how to implement such a fix, or how it might be designed

The Public Lab developer community

P.S. Interested in contributing code to Public Lab open source software projects? Join the developers group at http://publiclab.org/wiki/developers


24 Comments

Issue 1: Set curve downloads There is a selection of file types for a single curve plot data download but that option is not available for a displayed 'set' of curves. If it is, but is just hard to find, then it should be made available in the same way and same visibility on the page.

Reply to this comment...


Great suggestion, Dave. There was an issue open for that by @cfastie, which I've moved over to https://github.com/publiclab/spectral-workbench/issues/37

Reply to this comment...


In the live view, I should be able to mouse over a point on the spectrum to read the wavelength.

In the live view, I should be able to see a time-running average of the spectrum.

Reply to this comment...


I'd really like a stand alone version so I can store, organise and delete captured data and take measurements without an internet connection. If at all possible some method of built in camera control. I know there were support problems with this, but I use camera control all the time and switching ownership of the camera backwards and forwards between the browser and the camera control program is a bit cumbersome.

Reply to this comment...


i ll use it on my website ;)

Reply to this comment...


Great, thanks Forrest (that's you, right?) -- I created two issues from yours:

Do you have a Github account?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


MrBumper - saw your note in Liz and my post about modes of research. I was thinking, what about a mode in the software that says "you're in training mode; your spectra will be labeled as 'testing' until you're ready to start posting experimental data" or something? So user accounts would have a "mode" which would start out as 'testing' and move on to 'experimenting' -- although those terms are a bit ambiguous, so if you have a better idea please chip in.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Issue 3: PLab 3.0 on Android: I attempted to use SWB on a Galaxy S4 with the PLab 3.0 connected via the USB port -- which would allow using the PLab 3 remotely; nice idea. However, the interface refused to detect the PLab camera. Yes, smartphone code is much too hardware/firmware/release dependent but maybe ... it's a minor glitch? At least cam selection code is already there.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hello I am new to the Spectral Workbench Community.

I am having trouble saving files so I can get students to do things like graph spectra in Excel.

I read that the workbench software did have to ability to do this, but I cant figure it out, so if it does please help me figure out how to do it. If not please make a feature:

Spectra should be downloadable into something like a csv file so they can be graphed in something like Microsfot excel.

Thanks

Reply to this comment...


Hi, @spiderweb311 - yes, you can download individual images by right clicking them in the Analyze page, but you can also download various formats in in the small button bar on the left side, as screen-shotted here:

Screen_Shot_2015-02-12_at_2.55.58_PM.png

@stoft - that would be so great and I've tried it with a cable like this one; unfortunately it's up to Google and the handset and browser manufacturers to actually make such peripherals show up as USB webcams to the browser; I haven't seen any browser that can do it yet. Maybe we can file an issue on Firefox's bug tracker?

micro-usb.jpeg

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Issue 4: Calibration points: The calibration procedure is presently based only on the CFL Blue and Green peaks but because of distortions in the typical detected peak shape, it could be better to include the Red. Also, the procedure instructs the user to find the "highest" green peak. However, the Green peak is actually a double-peak and the highest is not always the "same" peak for all devices -- again, due to peak distortions and general device instability. The instructions should have the user pick the "right-most" peak -- ie. the one on the "red" side of the spectrum with the longer wavelength.

Reply to this comment...


Issue 5: Calibrated wavelength Data: I did two sets of measurments with PLab3.0 and, therefore, two separate calibrations. While the # of data points (and # of wavelength values) is the same, the wavelength values themselves are different ! OOPS! Yes, I'll admit they are close (just a few nm shifted) but for comparing supposidly calibrated spectra (wihtin SWB's present constructs) comparing two, calibrated (different cals) will have a residual offset error between the spectra. This was easily found from the downloaded 'raw' CVS files. There is no reason not to select a pre-defined set of calibration wavelengths for the data -- unless the horizontal resolution changes. IE. the displayed and downloaded data should not have a wavelength offset to the data sets -- but yes, two separate calibrations might have the CFL peaks at slightly different nm values..... but comparing the two data sets should not contain an 'X' axis shift to the data points.

@warren : I now suspect there could be a hardware issue as well (or instead of) ... smartphone USB ports can talk to thumb drives but not small pocket hard drives due to power limitations. If/when the smartphone-external-camera issue is investigated, it would be good to evaluate if a smartphone can actually power the camera. I couldn't tell with the SWB interface.

Reply to this comment...


Issue 6: Toggle RGB: Just a small bit -- the initial 'combined' spectral plot is 'white' to differentiate from the RGB plot colors .... but when toggled back, that same plot is 'red' which is likely confusing for most and a bit deceiving (unless you happen to remember the shape of the original combined data. Probably a 5-min fix.

Reply to this comment...


Dave, would you be able to add your notes to existing issues in our tracker, and/or create new ones for your excellent suggestions?

One I though of today and added was:

display if users have calibrated their devices

by scanning for their spectra tagged "calibration"

Also, perhaps we can show how good the (most recent) calibration is by running @Sreyanth's autocalibration matching code against it.

A display of what % of users have calibrated would also be nice, perhaps over time

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Issue 7: Curved spectrum lines - I have observed that the coloured wavelength lines of elements show up as being slightly curved. This is produced across the length of the slit and across the height of the normally viewed spectrum. I have also noticed that the curvature is more noticeable at the infrared end of the spectrum I am interested in this phenomenon because I want to be able to compare the spectra from two or more spectra obtained from multiple cuvettes placed along the slit length. This necessitates comparing readings from the same wavelength in two or more spectra. I started with various suggestions as to the cause of this phenomena, but none of them seemed to hold as true. I did multiple tests and ended up deciding that the phenomenon was caused by the camera being close to the diffraction grating. I have now tested four cameras; from cheap webcams, to a sophisticated digital camera called an Olympus SP-510UZ, details of which may be found on Google. All the cameras showed the same curvature when the camera lens was close to the diffraction grating. However, all the cameras showed an improvement towards straight lines as the camera was moved away from the diffraction grating. But the spectrum image became smaller, as expected. This phenomenon has no importance to Spectral Workbench users who are only interested in a single spectrum image. I have found that the Logitech C310 camera does produce only a small amount of curvature in the spectral lines. I have also observed that if I put my own eye close to the diffraction grating I can see the curvature; and when I move my eye away from the diffraction grating the lines become straighter. I have tried using an F8 aperture instead of the default F3.2 aperture on the Olympus camera, and also using the Super Macro mode; but those changes do not seem to lessen the curvature. Unfortunately I was not able to record experiments on my computer when using my mobile phone webcam. I have images on my computer of all four tested cameras if anyone wants to see them, but I have only attached images taken with the Olympus digital camera. The spectra images are very similar for all the tested cameras when viewed close to the diffraction grating and further away from the diffraction grating. I have yet to carry out tests with multiple cuvettes (up to 4) across a slit length. Attachments: Olympus_Super_Macro_mode_-_against_grating.JPG

Olympus_Super_Macro_mode_-_about_5cm_from_grating.JPG

Reply to this comment...


Issue 7: Gain-Correction: After some experimentation ( http://publiclab.org/notes/stoft/02-25-2015/plab-spectrometer-gain-correction ) it looks possible that a default gain-correction curve could be included as part of SWB at least as an option. While there are significant limits to the PLab devices from the intensity vs. wavelength calibration, as a true calibration, a first-order approximation seems like a major improvement in the direction of providing users with more meaningful spectral results.

@warren, I can add items to a tracking list though I've not noticed where such a list is located.

Reply to this comment...


I want to be able to: 1) capture spectra 2) have them calibrated to the capture calibration 3) find the other spectra I've named to make a comparison 4) make sets 5) compare spectra within those sets 6) save my comparisons scripts to share 7) embed my sets. most of these features are a part of the current SW spec, but none have ever worked every time without a hitch.

I want a direct link to my spectra. Currently they are cryptically under "dashboard" halfway down the page, under a second tab "my spectra" I want to be able to find them immediately.

I want to be able to classify my spectra in a useful way beyond tagging (or perhaps tags can help me organize better). I want to be able to make collections of spectra, i.e. all my oil tests. Within those collections I want to be able to make.

extra features would be including Dave's solar calibration features, increasing the resolution of camera capture, and having web controls for Gain, exposure, and white balance.

I'm not particularly fond of the current "waterfall" capture system. I would prefer if our capture interface was 2D, capturing the full spectrum, or preferably 3D, I.E. capturing every frame in a time series. That may be too data intensive, but could be used locally in the browser, and some limited set of data would selecting a capture line on that tiny little screen is rather difficult too. It would be nice to have a bigger interface.

Reply to this comment...


I got an idea about calibration...

I recently recorded spectrums of daylight, and actually got to see some Frauenhofer lines. Would those lines perhaps allow for more precise calibration?

Another thing, that's even more important. It has come to my attention that some camera sensors actually suppress some of the green light input. This can easily be seen by just searching for "sun spectrum" in here and compare with this: http://qdl.scs-inc.us/2ndParty/Images/Charles/Sun/SolarSpectrum2_wbg.png

Take a look at my attempts to get the solar spectrum right. Never does. My "white led spectrums" also suffer from this artifact. Could a script not be made, that could estimate how much the green light intensity of a given camera should be amplified in order to get a "solar spectrum" that to a higher degree resembles the actual solar spectrum? Please do that....please ^^

Cheers =)

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Issue: I use water as my control. Although the result has displayed all light can be through water, there's still a message saying that it is overexposed. In fact, it is not overexposed as all are in 100% that is good!

Reply to this comment...


It would be nice if the plots could be re-scaled to accentuate the peaks without having to distort the entire image (as I've done here): https://www.flickr.com/photos/28617364@N04/23151693665/in/dateposted/

Reply to this comment...


That's a great spectrum! I didn't know ruby fluoresced. Was that with a 405nm laser?

Do you mean amplifying the peaks, so they're "taller"? I believe the new SWB does that automatically, as here: https://spectralworkbench.org/spectrums/show2/55073

Can you link to your ruby spectrum?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


@chongyukwai - can you link to the water spectrum you're talking about?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


@DarkGreen - there's now a new calibration tool -- read about it here: https://publiclab.org/notes/warren/09-30-2015/new-wavelength-calibration-procedure-preview-for-spectral-workbench-2-0

The Fraunhofer lines would be a good way to calibrate, but the big ones are in the infrared range. I think we can get a lot of mileage out of the mercury spectrum!

Reply to this comment...


Hi Warren,

Thanks, Yes ruby's fluorescence is what led to them first being used as lasing material. I see the SWB indeed does scale the plot image. Here is the SW spectrum taken with my Project Star spectroscope. https://spectralworkbench.org/spectrums/62829

Another one that turned out interesting, autunite (mineral containing uranium ion): https://spectralworkbench.org/spectrums/62894

Reply to this comment...


Login to comment.