Hi @Maggpi, I guess we should talk about GSOC, when/how do you want to connect? @Warren, is there another way of communicating I should be using? Amir

Hi, I am not really sure how this works bit I think GSOC discussions moves over to github, https://github.com/publiclab/plots2/issues/2653, which is replicated to Gitter. https://gitter.im/publiclab/soc. In any case, really impressed with your opencv work, https://publiclab.org/notes/amirberAgain/01-12-2018/python-and-opencv-to-analyze-microscope-slide-images-of-airborne-particles and wondering if you could recommend any opencv teaching guides or ‘how to’ manuals?

Thank you for the kind words! I was very impressed with some of your posts on microscopy! The two resources I personally like best are the openCV tutorials and PyImageSearch. https://docs.opencv.org/3.4.0/d6/d00/tutorial_py_root.html

https://www.pyimagesearch.com/ [Try looking at the older content to begin with, as he now works a lot on ML/AI].

@amirberAgain we have a chat room on gitter and also on slack. You people can communicate to each other or us on these platforms. and if you people are interested i will send you the invitation link for the same. Thanks

Hi @MaggPi! Actually the easiest way to reach the PublicLab chatroom is thru https://chat.publiclab.org/ . While we support bridges to other chat networks, our "home" chatroom is hosted by matrix.org and reachable with Riot https://about.riot.im/ - a libre messaging system. I will be helping with co-mentoring - also I am PublicLab.Org's sysadmin. My strengths are systems programming, web, and Python. I hope you will have a great summer project! Please ping us if you feel stuck at any time. Any community channel is good!

Hi, @maggpi - could you paste in your GitHub project link here? Have you thought of a name for your project, even provisionally?

Looking through your materials now! Welcome!

One change seems to be the desire to collect images by microscope. I have the equipment for this but need advice on what should be test images? I assume a calibration slide is appropriate https://publiclab.org/wiki/basic-microscope but it’s also not a very fun reference and has no color. It may be a good time to try to measure something readily available (like a dollar bill) so that anyone could have a reference without paying $10 for a calibration slide.

Great question -- there are some ideas here; https://publiclab.org/notes/partsandcrafts/02-26-2018/6-looking-at-dust-and-other-things -- i like looking at things that move, but they're hard to keep alive :-)

One concern is what is the best spectrometer for phase 3. After reviewing the different Public Lab kits, the lego spectrometer seems the best choice. Lego was selected since it was compatible with the Raspberry Pi camera and I could possibly share results with cedarlodge(from Public Lab), https://publiclab.org/notes/cedarlodge/03-07-2018/raspberry-pi-spectrometer-gui#comments, So is this OK?

That's great, yes. And if @programmer1200 is willing, maybe we could send you some of the attachments he's working on? ( #16042) @programmer1200 if you're still interested in swapping some with us, and could include a second set, we'd love to forward it along. But either way - we can just forward ours along too.

What is the policy for getting access to public lab kits and parts? Do I need to purchase or can Public lab provide? I would like to use 2 lego kits (one for visible and one for vis-IR ). Note I plan to use my own v2 cameras and Raspberry Pi. (Please tell me if there is there some reason why the lego kit spectrometer comes with v1 instead of higher resolution v2?)

Sorry, we actually already sent you things! I hope there's enough there - we can send you a second lego set. The camera thing is just a cost issue - we know people who want higher quality cameras can get them on their own... they're expensive for us to carry in quantity. But we'll probably start carrying both versions!

Hope this is helpful! Reading your revisions above now.

OK, revisions look good! As @amirberagain mentioned, maybe expanding your timeline/checklist a bit could help. I'd like to see an outline of how you plan to organize your code; maybe pointing at a well-organized library as an example?

In JavaScript, we're trying for relatively short files, which are require()'ed together, and named consistently -- this tends to be good for other people to be able to quickly read through, find relevant parts, etc. Here's an example: https://github.com/publiclab/image-sequencer/tree/master/src/

I'm not as familiar with Python but I imagine similar coding style could be achieved! If there's an example project you'd like to aspire to, structurally, please share!

Once your GitHub repository is set up, I think a good step will be to make a planning issue there with the tasks you have in mind. See these examples: https://github.com/publiclab/plots2/labels/planning

I hope this helps. Thank you very much @MaggPi!

I think you are just right @warren. possibly one of the outputs of the initial planning phase should be a list of functions (and objects?) which could then be mapped out and divided in a logical way, maybe even some state machine flows? Calling functions and objects from other Python libraries is not a problem and should be desired, basically, you just write a .py file and import it to another one.

Hi, this is first test of the chat in chatroom. I was wondering if there is any preference for Open CV python virtual environment vs a non virtual environment. I understand the virtual environment helps multiple programs use the same libraries but it also seems like another rthing to go wrong. Also, AmirberAgain is not listed in the chat name

HI @MaggPi With regard to Python virtual environments you shouldn't worry about the they are a convenience for the developer but make no difference on the actual code. They exist so that you can mix and match library versions without messing with your operating system's Python. For our purposes it's fine to use the system's Python interpreter and libraries. You can install libraries as your current user with pip install --user library.

If your program uses external libraries be sure to create a requirements.txt file.

Also on IRC - some of us leave bots but are not at the keyboard. It's a way to get a backlog when we come back. A similar functionality is provided by Matrix.Org/Riot.im - it's good to leave your question and stick around until somebody answers! Github issues are also good for open discussion.

Hey @maggPi and @icarito, let's try to sync up and try to have a call in the coming days. I was thinking Thursday 15:00 EST (the same time as the open call starts), how does that sound to you?

icarito amirberAgain

My IB French exam is Thus and Fri. Is is possible to set something up Friday afternoon? I know you are looking for more details so please read below.

This post reviews GSOC first month activities and goals/programs to be developed for the second month.

First month activities:

-Reviewed public lab/spectral workbench web site. – Have tried to review all sections related to microscopes /spectrometry and multispectral imaging.
-started building lego spectrometry, multispectral imaging – having trouble with consistent measurements. Tried different designs to see what works best. Working on adjustable camera mount for spectrometer.
-prepared visible/IR table top camera
-prepared several notes/questions, see: https://publiclab.org/profile/MaggPi -began learning spectral workbench calibration software -purchased 2 raspberry pi 3B+/installed opencv. -setbacks- New raspberry pi gave blinking light power supply error on start-up and had to be returned. Damaged a pi camera while trying to mount in spectrometer.
-established contact with mentors -received public lab equipment

Goals:

The second month will develop a series of programs/functions that will adjust cameral settings and display images with opencv. The programs will be tested with imaging (microscope), multispectral and spectroscopic applications.

-Programs will be based on the Raspberry Pi visible, NoIR cameras. This will permit controlling camera functions by picamera. The general goal is to establish a two way link between the measurement, camera settings and computer vision analysis so the programs can be used for the next phase.

-Programs will be not be written for video analysis. This will provide time for Raspberry Pi to process opencv tasks at high resolution. Video will be used for camera set-up and spectrometer alignment.

Program Checklist:

Phase 1 tasks concentrate on controlling and recording different camera settings. -annotation of jpeg picture (time and file name) from pi camera -collection of RAW image data from pi camera (this should provide 12 bit range, so far all I have been able to collect is 8 bit jpeg) -annotation of picture with camera settings (displays auto exposure settings) -collect and process RGB or other split channel images (for example: https://publiclab.org/notes/MaggPi/05-14-2018/is-there-a-way-to-use-the-rgb-spectral-overlap-to-make-better-spectral-measurements) -prepare routines for side-by side display of multiple images (permits comparison) -cycle routines that captures and display side by side annotated images at different camera settings

General Comments:

Biggest challenge so far is working with opencv. I worked with simplecv before and opencv is ---- not so simple. One example is simple tasks such as displaying side by side images often get errors. I know the opencv sidebyside command is hconcat but it only works if the arrays are formatted the same size. So this means you need to think how to scale all the images beforehand.

Realize now that getting this stuff to behave isn’t easy.

Questions: What type of format is required for the June 11 evaluation? Is there some type of grading criteria like how many lines of code were written?

hey @Maggpi, I was sure the exam you mentioned previously was last Thursday - sorry for that. Best of luck with your exams, Let's sync start of next week? As for OpenCV - this is not a hard thing to do, use the resize as explained here: https://docs.opencv.org/3.4/da/d6e/tutorial_py_geometric_transformations.html

I guess @warren will be able to share experience on grading. If it's based on the number of lines of code we will need to change the criteria...

Hi @maggpi, when would you like to talk?

Hi, @amiberAgain How about next Monday or Wednesday 9AM US time. Hopefully, I will have camera comparison results by then. What's the the best way to set up a phone call? Thx

The coming Monday doesn't work for me, how about Tuesday, June 5th at 13:00 EST? Maybe a bit later and just before the openhour? The esiest option for me is to use the Hangouts, but we could also set a zoom meeting. The main subject I want to discuss is the plan and goals for the rest of the summer and see if there are any questions you might have that I could help with. @icarito does that work for you as well?

1300 EST would work for me too actually, and I'd love to join!

@amirberAgain, @warren, @icarito

This note is a follow up to the June 6 meeting, a past work summary and outline of future goals.
a. Code organization b. Milestones and future goals c. Raspberry Pi

Code organization: There was a good discussion of different ways to organize Raspberry Pi software for different Public Lab users and applications at the June 6 meeting. The approach selected considers three categories (Picamera, USB and Image Processing) that are based on different ways the image is acquired. In addition, multiple software routines have been posted to accommodate different levels of experience. For example, a beginner will probably just want to capture an image, a slightly more experienced user will want to capture and annotate the image, and an experienced programmer will want to capture, annotate and process the image. (with opencv). All code is available at: https://github.com/MargaretAN9/Peggy. Demonstration of code output can be seen at https://publiclab.org/profile/MaggPi

Postings on github as of June 11, 2018 Picamera: Designed for the Raspberry PI with PI v2 camera/picamera software.

--PiCameraEffectsShow.py Display several processing modes available with picamera --PiCameraEffectsVid.py Same as PiCameraEffectsShow.py but also records video in h264 format --picameravidwithimagecapture.py
Displays video and captures image
--picamerawithimagecaptureopencv.py Displays video captures image (with open cv)

USB: Designed for Raspberry PI with a USB webcamera.

--USBTakesPic+Annotates.py -uses fswebcam for picture --USBVidWithImageCapture.py -uses pigame for video and image capture

Image Processing: This category is for users who want to process a stored image.

--AddsAxisToImage.py
-Adds resolution scale to image --RGBquad.py
-Displays RGB pictures in quad format --YUVQuad.py -Displays YUV pictures in quad format

Milestones and future goals: The third month will continue past work and concentrate on interacting with the camera/object versus just using the auto settings. The programs will be tested with imaging (microscope), multispectral and spectroscopic applications.

Phase 2 Code deliverables: Exposure control programs: (June 16 to July13 Evaluation) Week 5 - Optimization for longer exposure times, requires multiple settings to be adjusted and looping thru different settings. Week 6 -Manipulating RGB/HSV image, masked measurements and selection program. Region of Interest selection is useful for spectral data since you can record a slice of the 3240 x 2464 image. Week 7 -Intensity and histogram maps
Week 8 -Prepare documentation/final evaluation

Phase 3 Code deliverables: Spectroscopic Applications (July 14 to Aug 14 Evaluation) The last phase will concentrate on applying some of the general processing options developed earlier to specific applications. Applications to be considered are NIR/G/B microscopy and spectral calibration. The general approach for spectral calibration is to see if multiple views at different exposure settings and analyzing data from the higher spectral orders could be used for spectral analysis. I thought I would work on this since most everyone in computer vision works stuff like face recognition but few are working spectral recognition. A major challenge is overcoming spectral overlap issues in higher orders.
Week 9 -Controlling dynamic range with feedback from CV processed spectrographs Week 10 -Spectroscopic calibration for (near) real time spectroscopy Week 11 -Transmission spectroscopy experiment – objective is to extend the free spectral range Week 12 -Prepare documentation/final evaluation

Raspberry PI:

I know Public Lab already provides a pre-load image for streaming applications ( https://publiclab.org/notes/partsandcrafts/12-03-2017/setting-up-the-raspberry-pi-camera-for-wireless-streaming ) but it may be worthwhile to consider an image for non-streaming applications as well. This would permit work without WiFi and allow camera preprocessing capability. The image does not necessarily need to include opencv since python/picamera/ matplotlib also provide many image processing/display features.

@amirberAgain, @warren, @icarito This note is a GSOC phase 2 summary and updates phase 3 goals.

Phase 2: Phase 2 research notes are posted at https://publiclab.org/profile/MaggPi. Comments on software programs (posted at https://github.com/MargaretAN9) are listed below :

-Color detection – generates mask/ calculates HSV colorspace range. Primary contribution was the use of trackbars for the user to easily select color range(s). (HSVcolortrackbar.py) -Exposure control matrix - Developed to get a systematic way to adjust Raspberry PI manual camera settings. My real first experience with the multiple tunings necessary for long exposures (up to 10 sec) and low frame rate. (exposuremosaic.py) -Microscope grid overlay with calibration. Simple CV geometry functions permit scale reference. Amazed that this works as well as it did. (videooverlayfinal.py) -Insect infrared microscopy – Simple contrast adjustments permit enhanced detail. Effort demonstrates need for better focusing options and real time histogram optimization.

In addition to the research notes, additional ‘test’; code was tested on Fourier masks , real time measurement, histogram equalization and region of interest selection .
On setback is that I have been unable to use 10bit RAW Raspberry Pi camera data. I have tried different processing options but they all need color and lens corrections. I have started work on HDR techniques as an alternative. My initial review is that HDR may operate over a larger dynamic range than a 10 bit RAW image.

Phase 3 Code deliverables: Spectroscopy and Microscope Applications (July 14 to Aug 14) Goals were outlined in prior posts but are summarized below for convenience: ‘The last phase will concentrate on applying some of the general processing options developed earlier to specific applications. Applications to be considered are NIR/G/B microscopy and spectral calibration. The general approach for spectral calibration is to see if multiple exposure settings and analyzing data from the higher spectral orders could be used for spectral analysis’.

Week 9 –Real time spectrographs Week 10 -Spectroscopic calibration for (near) real time spectroscopy Week 11 -Transmission spectroscopy experiment – objective is to extend the free spectral range Week 12 -Prepare documentation/final evaluation

Thanks, this is very helpful for the eval writing. Much appreciated!

This note is a GSOC phase 3 summary and provides a complete set of links related to the project.
Google Summer of Code proposal available at: https://publiclab.org/notes/MaggPi/03-20-2018/gsoc-proposal-computer-vision-enhancements-for-raspberry-pi-based-public-lab-science-projects

Code descriptions can be seen at https://publiclab.org/profile/MaggPi. Video demos at: https://www.youtube.com/channel/UCbyyYOlNo87CXJ39h3wqXZA Github readme file contains a list of all programs with a short description: https://github.com/MargaretAN9/Peggy/blob/master/README.md

Phase 3 activities involved several areas: -https://publiclab.org/notes/MaggPi/07-17-2018/high-dynamic-range-hdr-imaging-revisited -https://publiclab.org/notes/MaggPi/08-09-2018/raspberry-pi-manual-camera-control -https://publiclab.org/notes/MaggPi/08-14-2018/computer-vision-observations-of-elodea-plant-cells -https://publiclab.org/notes/MaggPi/07-18-2018/ndvi-micrsocopy - https://publiclab.org/notes/MaggPi/08-03-2018/ngb-ndvi-video-optimization-red-blue-manual-gain-control -https://publiclab.org/questions/MaggPi/07-24-2018/ndvi-led-simulator

HDR imaging, manual camera controls and Elodea microscope observations look at different ways to extend dynamic range and display images.
NDVI was explored in NDVI microscopy and NGB-NDVI video optimization. In addition, a NDVI LED Simulator was proposed as a potential method to calibrate NDVI data.
Software also was developed to display real time spectral data output. Work to separate orders and conduct calibration by colormaps were frustrated by signal crossover between RGB channels.

In general, the programs demonstrate a transition from ‘snapshot’ imaging to processing data at video rates with manual control of camera settings. The ability to adjust image properties in real time provides a quick way to control image quality and optimize for different computer vision tasks. While resolution is limited in real time processing, the same settings can be used for higher resolution images which can then be processed by image sequencer (https://publiclab.github.io/image-sequencer/examples/#steps=).

My only regret is that I didn’t have the time to look at problems discovered along the way. Issues like real time flat field correction and keeping focus from visible to near infrared are two annoying areas that deserve a second look. It may be worthwhile to look at how computer vision could compensate for optical problems as well as provide different lens options.

My last comment is I would like to thank all my mentors @amirberAgain, @warren, @icarito as well as those who provided help, comments, and encouragement @cfastie @liz. I learned so much but I also know I have much more to learn. While the programs provide a way to get started with real time computer vision techniques for Public Lab kits, possible future applications are unlimited.

Hi, @maggpi - i'm just going through this now, thanks! Perhaps the .docx file could be uploaded as a .txt or .markdown file -- it's displaying oddly when I click the link.

Looking great so far, thank you!

In fact, as you've provided this link for your final work product, I think reformatting your .docx file as README.md (markdown) and including it in the root folder of your GitHub repository would be great, as that will appear as the default on https://github.com/MargaretAN9/Peggy/ -- would you be able to do that in the next day? Thanks!

@warren , updated readme file at https://github.com/MargaretAN9/Peggy/blob/master/README.md Thanks for your patience.

@MaggPi, Sorry for disappearing like that, had too many projects over the summer. Looks like you did an awesome job!