Public Lab Research note


(GSOC 2014 Proposal) Porting ImageDistortion class in the Mapknitter interface to Leaflet.js

by anishshah101 | March 08, 2014 09:31 08 Mar 09:31 | #10141 | #10141

Name: Anish Shah

Affiliation: BITS Pilani (B.E.(Hons) Civil Engineering)

Location: India

Email: anishshah101@gmail.com

Phone: +91 7877799788

Project(s) you're working on or want to: MapKnitter

Project title: Implementing Rubbersheeting in Leaflet.js as a first step to porting the Mapknitter interface to Leaflet.

What I want to do

Abstract: MapKnitter tool is used for 'combining a series of these aerial images into a flat, projected map ' which can be browsed in the same way as Google Maps. Currently Mapknitter is built on old and legacy code and uses Open Layers for it's editing interface. Open Layers is a complex library following strict design principles so it requires time to master and develop features on it, also it is a heavy library which takes time to load on smartphones and hand held devices. I want to port the Mapknitter editing interface from Open Layers to Leaflet and make the start by porting a few important features initially. Leaflet.js is a light weight library (33 KB) in which adding and editing features is easy because of it's object model.

The broader goal my project is working towards: Creating a simpler user friendly tool for making maps.

The specific need my project fulfills: My project aims to make the development and use of Mapknitter tool easy, thus making the development processes (rubbersheeting maps) quick.

How will my project meet this need: The porting of the interface to Leaflet will help in quicker feature addition and accelerated development and debugging. It will also significantly reduce the time it currently takes to load maps. This will help the users to use the tool through hand held devices and smart-phones. Also, the look and feel of Leaflet is more modern which will make the interface more intuitive and easy to use. Broadly steps which will be followed to do this are:

1) Check the implementation of the Control_point.js, warper.js and image.js to examine the code which supports draggable points, polygonal bounds and image distortion as used in the present Mapknitter tool.

2) Identify algorithms for Distortion as present in: https://github.com/jywarren/mapknitter/tree/master/public/cartagen/src/warper and Matrix classes in the present tool from the code present in: https://github.com/jywarren/mapknitter/blob/master/public/cartagen/lib/matrix.js

3) Emulate the above findings to extend the ImageOverlay class in Leaflet.js to support polygon bounds for images and build a distortion interface in it. (An alternate approach to this is the exact flipside where a similar solution might be possible by extending the vector polygon classes in Leaflet.js to support ImageOverlay instead of just color overlay. As of now I have discarded this because the distortions of vectors are scalable to any unit, a feature which is not possible with rasters. This can be tweaked and customized to limit the scale suitable for images so possibilities here can be explored.)

4) Port the existing ImageDistortion class and Matrix class into the above coded Leaflet plugin as a first step to porting the entire Mapknitter editor to Leaflet.

Timeline/milestones:

Throughout the development process I will be recording the ongoing development on a blog (once a week), making it easier for the mentor to track my progress.

Pre-Summer Period (up to May 19): Get familiar with MapKnitter and the code base. Investigate and learn the required tools required for the project. Bond with community while identifying required resources for learning or help. Check the implementation of the ImageDistortion and Matrix class in the present tool.

May 19 - May 26: Creation of a new JS file where I will be building the code for imageDistortion as a Leaflet Plugin. This file will comprise all the code snippets which are to follow. This will start with the Creation of DOM elements for the HTML canvas element.

May 26 - June 2: Generation of Markers for Image vertices using Marker Icon in Leaflet.

June 2 - June 16: (First challenging part) Placing these markers and the image on the canvas, also these markers should be at appropriate vertices of the image. The scale of the image and it's position will depend on image dimensions, map dimensions and their aspect ratios.

June 16 - June 22: Setting up Listeners.

June 22 - July 20: (The most important and challenging part) Rendering the distorted image when the user is dragging the vertices inside the canvas.

July 20 - August 11: Porting the existing ImageDistortion and Matrix Class to the newly coded Leaflet plugin.

Upto August 21: Test the overall code, fix bugs. Properly document the code developed.

In case the above formed timeline works quicker than planned then I would want to work on: a) Speed the process of distortion maybe by caching the distorted image once the user stops to distort it. b) Building multi-touch events for touch screen devices. This should make the current system of distortion more or less complete.

Another feature which really interests me is True warping, which is more advanced but I would want to work on it once simple distortion in complete.

My attempt and results

Ongoing involvement: I have built the ImageDistortion functionality in Leaflet for affine transformations of images, the code for which can be viewed here: https://github.com/anishshah101/ImageDistortLeaflet . The live demo of this can be viewed at: http://128.199.208.55/ .The process of building this has helped me understand the direction to follow and the tools to use to prepare new features in Leaflet. By the end of SOC we should have the imageDistortion functionality for non-affine transformations and Matrix class of Cartagen ported to Leaflet.js

Experience:

1) I have been a freelance web developer and designer for 3 years now. In the process I have helped various companies build websites, cell phone apps and digital and print branding material. My portfolio website can be found here: http://pingly.in

I have started (along with a colleague) an open source project:

2) Breakdown Database Management Software for Maintenance Division of Manufacturing Plants (Built for Windows 7 Platform on wxPython and Python). The code can be viewed here: https://github.com/anishshah101/DBS

Also, I have modified some open source projects for private use in hackathons, which include:

3) Balloons.IO Web Chat Application (Built on Node.js and Express.js)

4) Real time twitter feed displaying application (Built on Node.js and uses Twitter API)

Besides these I have worked as the Tech Lead in a web based start-up

5) www.Buzzbox.pw - An anonymous posting forum on the web (Built on Python Django framework, HTML, CSS and Javascript)

I have also been an active tutor on Instaedu.com for 6 months where I help students with programming Languages such as Python and Javascript.

Teamwork: I have worked in a team for the projects 2) and 5) mentioned above. One of the greatest learnings from working in a team has been to write properly designed and documented code so that it is reusable. Effective communication makes it easy to work or projects involving large code-bases. The presence of a team helps in in-house feedback and objective opinions which I think is necessary for any project.

Expertise: I have good experience with Javascript, Node.js and Python. I am also acquainted with the use of GIS and have quickly understood the code base of Leaflet.js so I think I can take this project up very well. Having helped many companies build their websites I now consider myself a professional in Web Development and UX/UI design. I think these experiences make me worthy of taking up a challenge like this project.

Questions and next steps

What resources I will need:

1) Algorithms which implement Image distortion in the current tool.

2) Active community support from people working on similar projects and critical feedback on my progress.

Why I'm interested

I have always been a fan of revolutionary ideas and the Mapknitter project by Public Lab appears to me a big revolution in map building and navigation. The concept of building maps by rubbersheeting clicked photos via balloons and kites is extremely fascinating and I believe the scale of the project once the tool gets easier to use is huge, something I am eagerly looking forward to.

Audience: My audience for the project include developers building features for Mapknitter tool and communtiy people who click aerial photos and rubbersheet them into maps.

Context: I deeply believe that the Mapknitter project holds lot of promise for the future in areas such as map building and navigation. The initial setting up the tool is filled with challenges I would love to face in a language I love to code in (Javascript). Even after the SOC gets over, I would love to remain involved with the project and keep on adding features to make it the best Rubbersheeting tool on the web.

Commitment: I do not have any other major commitments for this summer. I can devote 40 hours a week for my GSOC project.__


14 Comments

This is a great draft proposal, thanks Anish! One note: the OpenLayers.Layer.Image class is not used in the current implentation; actually the "Cartagen" framework is used to place a region on top of an OpenLayers map, and actually intercepts all the mouse events. The Cartagen layer displays the polygons and warped images ("warpables", though in the revised version we should not use that confusing name) and passes on drag interactions to OpenLayers.

However, this system is a cobbled-together Frankenstein and your Leaflet implementation should allow us to make a much simpler version. Ideally we stop using Cartagen as well as OpenLayers. The warping/distortion code you'll be porting will mostly be from Cartagen, though.

Did you not want to include some notes on advanced tasks you might attempt if you make a lot of progress and have time left? Also, perhaps you should link to the relevant code on Github that we've been posting in the discussion list and the ideas page, so people reading over this description can see what Matrix and other code you're referring to.

Great work so far!

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Oh, one more thing to think about -- will you create the ImageOverlay subclass separately from the interactive code where users can distort the image with their mouse or touchscreen? It makes sense that the ImageOverlay class would be a passive, self-contained class, and you would create the interactive event handlers and manipulations as part of MapKnitter's JS code, not as a native part of the class, right?

Also, perhaps you should mention the alternative way to approach the problem -- imagePolygon -- and why you're not doing it that way. If you change your mind later, or if someone else comes along and asks, you can point them at this explanation.

Finally - what will you call your new class? ProjectedImageOverlay? RectifiedImageOverlay? StretchedImageOverlay?

Super, thanks!

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Anish - i'm going to paste in some questions and thoughts here related to how your project overlaps with Vidun's proposal. Some of these are for Vidun, some for you, but both projects will be related and it'll be important to figure out with Vidun where your work fits together, and how you'll each ensure that your work will function even if the other's does not yet work. And how they'll work together if both succeed.

  • Upload interface is a partial, which can be included in various places
  • Recent images also
  • Separate JS code files
  • Leaflet work contained in Leaflet plugin
  • Could upload function first be integrated with current system, but clearly mark where it would begin to work with Leaflet system once that's done? Via agreed-upon callback and/or constructor method?
  • Upload of each image to server completes with a callback like "imageUploaded()" which the Leaflet system can overwrite when it's ready?
  • Leaflet system, if completed, will need a) transparency, b) outlining, c) locking
  • new Bootstrap interface overhaul for MapKnitter could be part of Upload project.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Initially I want to create the ImageOverlay subclass and the distortion code separately from the interactive code along with the interactive event handlers. This will help other contributors to modify the code more easily in the Beta-phase. When we take it out of the testing phase then I will modify the code to respond to events from the event handlers as a part of Mapknitter's JS code.

I had a look at Vidun's proposal and I think that our work is extremely complementing. The only doubt I have is with the timelines as Vidun plans to complete most of the overlapping features much before I plan to build the relevant plugin. I have made a simpler version of the plugin (https://github.com/anishshah101/ImageDistortLeaflet) which allows affine transformations using 3 corners of an image and it can be used as a reference for building the features Vidun wants. I will discuss this with him once. If this works out then we will be able to port the interface to Leaflet very quickly.

The features of transparency and outlining are already present in Leaflet. Locking will be easy to build.

Reply to this comment...


Hi, Anish - can you link to an online demo of the affine code you've written so I can try it out? Sounds interesting!

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


ya sure,

you can have a look at the demo at: http://128.199.208.55/ The overlay image might take a few extra seconds to load.

Reply to this comment...


Wow, excellent demo! I'm so glad you posted it. I think the interface will be more complex, but this shows that you're definitely capable of implementing it. Great proof of concept!

Reply to this comment...


It also occurred to me that you could start by making a page which displays existing maps without allowing users to save any changes to them, or perhaps even to manipulate them (at first). A static map display page which nonetheless uses Leaflet.

Once you accomplish this, you could add a button such as "Edit" which enables users to begin to manipulate and/or upload new images. Anyways I'm getting really excited about this proposal! Great work so far!

Reply to this comment...


Glad you like the demo :) Really appreciate the encouragement. By 'existing maps' are you referring to the maps which are already in the mapknitter database ? I can start with the static map display page, sounds like a good idea.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Yep, that's right. Super.

Reply to this comment...


I also really liked the demo. Jeff, how big is the MapKnitter database? Can we duplicate it easily? It'd be great for Anish to be able to have access to the past map layouts locally for testing purposes.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Exciting, it is a great demo!

Reply to this comment...


I currently have 2 big MapKnitter feature requests that are not part of this proposal that I would like to just state for the record (not to distract from this proposal!)

1)Masking or PNG/TIFF support for image overlays

2)User controlled layer hierarchy

Both of these feature requests are related in terms of giving MapKnitter users more power in making better maps. (1) allows users to remove unwanted parts of an image from a map (2) allows users to have their preferred arraignment of overlays for optimal map production

Reply to this comment...


So the db is only 8.7mb when bz2 compressed. I'll mail a link.

Reply to this comment...


Login to comment.