Public Lab Research note


GSOC 2018 Work on Image Sequencer

by tech4gt | August 09, 2018 11:17 09 Aug 11:17 | #16892 | #16892

Project Info

Project Repository

Invert Module Repository

Grunt Plugin Repository

Pull requests

Public Lab Proposal Note

Summer of code proposal

NPM

Demo

Introduction

Image Sequencer is a sequential Image Processing and analysis system for browser, Node.js, and the command line. It is non-destructive in the sense that a new Image is created after each processing step. It is targeted towards people trying to run analysis on scientific imagery for their research.

The Modular Module System

A module is a single image processing step that is applied to all the Images loaded into sequencer. Some of these modules are baked into sequencer by committing their code into the sequencer source itself. To make this system more scalable I have implemented a new module system which allows for modules to be distributed via npm separately and consumed into sequencer dynamically.

An example of this is the new Invert module

The following commands can be used to install modules which are distributed via npm:

CLI

sequencer --install-module "invert image-sequencer -invert"

This command installs the npm package image-sequencer-invert then adds it as a dependency of Image Sequencer locally and loads this new module into sequencer for usage.

Node.js

sequencer.saveNewModule("path to files")
sequencer.loadNewModule("module name")

In the Browser context the modules can be included as script files.

Coding a Module

A Module must follow a basic format which makes it compatible with Image Sequencer. This summer I worked on minimizing the amount of code required inside a module, a lot of redundancy was eliminated and the default UI functions are now called automatically rather than the module having to call them.

Old Module Structure: New Module Structure:

The New String Syntax

Image Sequencer encodes the applied sequence of steps and their settings into a compact string which is then appended to the URL. Following are the methods associated with stringifying and JSONifying a sequence.

sequencer.toString() // String for the sequence currently loaded 
sequencer.importString() // Imports the stringified sequence
sequencer.toJSON() // returns the current sequence in JSON
sequencer.importJSON() // imports a JSON sequence

A simple exaple of a stringified sequence is channel{channel:green},blur{blur:2}

Sequences can also be saved. In the browser the UI button can be used to save in the browser local storage.

Inside the cli the new --save-sequence option can be used.

Meta Modules

With the work done this summer it is now possible to reuse module code by consuming various modules into one meta-module. These meta-modules have their own meta-data and their inputs are the inputs of their constituent steps. I have made a Colorbar meta module which adds a colorbar to the image.

See it in action here colorbar{}

A meta module expands itself into its constituent steps giving them proper inputs at runtime itself which enables them to be loaded from the URL itself.

Other Improvements

A lot of other goals were achieved which include:

Optimizing sequencer.run method so all the steps are not re-run every time steps are added or removed.

Named exports in the CLI which enable output Images to be exported to a desired location with a desired name and a desired extension

sequencer -i "./something.png" -o "./example.jpg" -s "invert"

An addition to the UI which enables the user to see the pixel value of an ndvi image by hovering on a particular location

A grunt plugin which enables Image sequence4r to be run with grunt - grunt-image-sequencer

Contributing

During the later part of my project I created various FTO issues and we were lucky to welcome some newcomers on the project.

List of all first timer issues

For contribution guidelines please checkout contributing.md

For further reading please do checkout the Readme

Conclusion

I was able to accomplish a lot of core work this summer which itself adds some great features and sets the platform for future work. I would like to thank google and publiclab for this awesome opportunity which made my summer a blast. Also thanks to all the mentors who have been supporting and reviewing my work.

For now I would continue to work on Image Sequencer but now that summer is over I will shift my focus more towards boosting our community and welcoming even more people to the project. Once again thanks a lot to everyone involved :)


15 Comments

Looks fantastic, Varun!

Reply to this comment...


@warren Thanks a lot ❤️

Reply to this comment...


Should I submit the link to this on the GSOC evaluation page?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Yes, go ahead!

On Thu, Aug 9, 2018, 11:17 AM \<notifications@publiclab.org> wrote:

Hi! There's been a response to a discussion you're involved in. Do NOT reply to this email; click this link to respond:

https://publiclab.org/notes/tech4gt/08-09-2018/gsoc-2018-work-on-image-sequencer#c20353

tech4gt wrote:


Should I submit the link to this on the GSOC evaluation page?


Look like spam? Mark this as Spam

Reply at: https://publiclab.org/notes/tech4gt/08-09-2018/gsoc-2018-work-on-image-sequencer#comments

Report abuse to: moderators@publiclab.org

Check out the blog at https://publiclab.org/blog | Love our work? Become a Public Lab Sustaining Member today at https://publiclab.org/donate

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


I was trying to use the new image sequencer colorbar option and I noticed that the Import Image/add step does not load an image. I used it before to blend images for NDVI and now I can only load one image. Has there been any changes to the import Image code that prevents loading an image? @warren @tech4gt

Thank You, I am using image sequencer from windows

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


@MaggPi yes you are right, I'll look into this right now!!

Reply to this comment...


@MaggPi The issue has been solved here https://github.com/publiclab/image-sequencer/pull/330 we will push this out to production ASAP. Thanks a lot for pointing this out 😃

Reply to this comment...


Great catch both of your and thanks! Publishing a new version now.

Reply to this comment...


v2.2.3 should be live momentarily; npm already published.

Reply to this comment...


can you confirm? I have to leave right now but if you have follow-up fixes I can merge them in about 4 hours.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


@warren the issue seems to be all fixed up. Thanks!! ❤️

Reply to this comment...


Thx @warren and @tech4gt !!! I was able to use the import image step and have moved on to the colorbar step. The chart below compares the different image sequencer colormap/bars. The (@fastie) Fastie colorbar from ( https://publiclab.org/notes/cfastie/08-26-2014/new-ndvi-colormap#12/37.8950/-122.1230 ) was also posted for reference. Please let me know if I am doing anything wrong. Slide2.JPG

Reply to this comment...


Yes this looks right, and I believe the default one actually has an error! I opened an issue here -- if you'd be at all interested in submitting a fix, we'd love some help!

https://github.com/publiclab/image-sequencer/issues/332

Reply to this comment...


Proposed colormaps for ImageSequencer are pictured below. Please recommend any changes /new colormaps and we will try to add them with the next software update.

Slide1.JPG

ImageSequencer is a general purpose image processing program capable of creating NDVI composites. The proposed colormaps have been selected primarily for NDVI analysis but could also be used for other applications. The pictures below provide additional information about each colormap (ImageSequencer software code, origin data, past comments from public lab and application reference). More info at https://publiclab.org/wiki/image-sequencer

Slide2.JPG

Slide3.JPG

Slide4.JPG

Slide5.JPG

Slide6.JPG

Slide7.JPG

Slide8.JPG

Slide9.JPG

Slide10.JPG

Reply to this comment...


These look fantastic. Let's merge in a commit with these and consider additions or changes in a follow-up issue! Great work, @maggpi!

Reply to this comment...


Login to comment.