🎈 Public Lab: Graded oils using UV fluorescence

Graded oils using UV fluorescence

mathew was awarded the Basic Barnstar by phil3D for their work in this research note.

What we want to do

Me and @Warren are trying to replicate the heavy to light sorting of oils, as seen in "Laser Remote Sensing at Sea, A Quantitative Approach" byT. Hengstermann and R. Reuter and a bunch of other people's work.

Specifically, we're looking in a shift in the color of spectra emitted by fluorescing oils from red (heavy oil) to blue (lighter oil). look at Figures 1, 4a & b from Hengstermann & Reuter:

Our attempt and results

I tried to replicate this in spectral workbench, first on one spectrometer and then again with a bunch of spectrometers with different cameras in them. @warren has been working with the data and re-working spectral workbench to do cool things with the data. We've been talking on the phone too.

@warren tried smoothing the graphs, equalizing area, and equalizing height. The most legible and compelling, is to split the area of the graph and mark the exact middle. When that is performed, the progression of oil grades from light to dark (blue to red) is nice and clear.

To quote Jeff:

OK, I added a couple more features -- one which bisects a graph with a vertical line where the area of the graph is equally divided -- what's that actually called? Anyhow I've just labelled it "Find graph 'centers'":

I also made a "Crop view" feature so you can zoom in a bit. It kind of sucks so maybe I'll add a real zoom feature and let people scroll back and forth. But it's too late tonight. But this makes it much easier to see the sequence of heavy/light oils:

https://spectralworkbench.org/sets/show/1482, of the SYBA cam:

https://spectralworkbench.org/sets/show/1484, of the filterless Infragram Webcam:

And https://spectralworkbench.org/sets/show/1481, of the filterless Infragram Webcam:

Questions and next steps

The order of oil grades is similar between the two different spectrometers. But each camera still produces different centers.

To make these spectra comparable between cameras, it looks like some sort of calibration samples will be needed. There are a lot of moving parts in these measurements such as concentration, or the path of the laser. We're still using a variable light path-- moving the laser around-- to give us spectra that aren't blown out.

Blowouts are a big problem we both noticed, and we're thinking some blowout warning in Spectral workbench might be useful. It might aso be possible to just watch one color channel:

Why do we want to do this?

we're looking to make an inexpensive means of identifying oil in the field that works between different cameras and devices.

22 Comments

Looks promising. Maybe these are obvious but... - Pick 3-5 oils, or oil mixtures, as a reference test set -- low to high density -- then use them for every test. - Prepare the samples in exactly the same way -- to eliminate another variable - Pick one oil as a reference (mineral?) and adjust the laser setup for a specific signal peak (non clipped -- and that will also likely prevent clipping with other oil signatures) -- then change only the sample ... to eliminate another variable. - It looks like the curve shapes you measured are similar, in general, to those in the paper -- and, they are all relatively "smooth" curves. You might use this to your advantage: - Try acquiring a set of values for each curve -- say the signal average for a 20nm width for every 25nm from 400-600nm -- 9 values. Then normalize the data sets (from 3-5 oils) to a common peak value as you'd only be interested in matching curve shapes not intensities. - If each data set has repeatable ratios (between values within a set for separate measurements of the same oil) then you'd have enough stability and an easy way to match test data with "known" samples by a simple: SUM(ABS(Refs-Meas)) ~= 0 for a match. - Admittedly, this is assuming there is a strong correlation between curve shape and oil density. - If different oil types, but with the same density, have subtle curve differences, then that might be approached with an additional observation of some limited part of the spectrum -- though that could be more difficult due to the apparent signal noise. Cheers, Dave

Is this a question? Click here to post it to the Questions page.

Reply to this comment...

Good work here. I like the blowout warning idea. Experimenting with the oil density is def. another good step, but I agree that the distinct variation in spectra is important to define first on SWB. The density issue might be solvable with really good sampling methods if we are finding that is a problem. Something perhaps like you run 3 samples of the same substance at different concentrations and average the line?

Can you explain the vertical lines a little more?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...

I don't know the word for this kind of thing, but maybe @btbonval does :-)

The line divides the graph into two equal areas:

Reply to this comment...

@stoft Dave, we're getting similar brightnesses between our readings of different oils by varying the concentration and/or light path, which seem to change the brighness without effecting the emissions curve. With the crude oil its actually hard to get a dim enough spectra-- with olive oil we have the opposite problem.

It seems you'd prefer we keep the light path consistent and only vary the sample-- but dialing in the concentration to get the right emissions is going to burn through sample preparations/cuvettes. Do you think we could use neutral density filters to knock the brightness down instead? I started to use a polarized filter for this purpose and it looked effective.

With just a filter change, then we could keep the lightpath the same, keep sample prep and concentration of oil the same, and just switch out a filter.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...

@stevie running three samples and averaging them is a great idea. You can see in the labels that I started running 3-5 tests per sample to go for bright, medium, and dim readings. When those curves were normalized they matched eachother in shape pretty well (as long as there were no blowouts). I think we could include that in the method-- I don't think much would have to be added to spectral workbench-- just looking to see if the dividing lines were close would work.

As for the vertical lines--I'm not sure what it is supposed to be called, but essentially we're integrating the area underneath the graph and then dividing it so exactly 50% on one side and 50% on the other side of the line.

Reply to this comment...

@warren The line that you describe would be the weighted mean of the dataset (integral divided by two, aka area under the curve divided by two as @mathew noted). This would be super easy to find assuming the plot was a histogram. Unfortunately the data plotted from the frequncy vs amplitude is not a histogram as such, because we don't have those little balls used to create the histogram curve. We only have the one curve itself. So we have to integrate the whole thing, find what half the integral is, and then find /where/ half the integral occurs.

You might also be talking about linear discriminant analysis if the left side balls are quantitatively different from the right side balls. LDA is a clustering algorithm to separate classes of data. Again, since we don't have access to the balls, this is probably irrelevant.

Reply to this comment...

integral = approximate_integral_method(y, x) accumulator=0 for i in range 1..N-1: accumulator += y[i]/integral * (x[i+1]-x[i]) # accumulate the left reimann sum if accumulator >= 0.5, return x[i]

integral = approximate_integral_method(y, x) accumulator=0 target = 0.5 * integral for i in range 1..N-1: accumulator += y[i] * (x[i+1] - x[i]) if accumulator >= target, return x[i]

# establish that x and y are 1d vectors xdata = data[0,:] ydata = data[1,:] # perform cumulative scalar integration over left riemann sum accumintegral = numpy.diff(xdata).prod(ydata[:-1]).cumsum() # the last value in accumintegral is the full area under the curve. target = accumintegral[-1]/2.0 # find the first x-value for which the accumulated integration exceeds half the total integral return xdata[accumintegral >= target][0]

Public Lab Research note

Graded oils using UV fluorescence

What we want to do

Our attempt and results

Questions and next steps

Why do we want to do this?

22 Comments

spectrometer

spectral-workbench