Public Lab Research note

Setting up MapKnitter

by justinmanley | September 01, 2014 20:49 01 Sep 20:49 | #11095 | #11095

justinmanley was awarded the Basic Barnstar by warren for their work in this research note.

How to set up MapKnitter from scratch

It can be tough to get MapKnitter up and running. It should be as easy as:

  1. Clone repository from GitHub
  2. Install ruby dependencies from Gemfile using Bundler.
  3. Download static assets using bower.
  4. Run MapKnitter using passenger: passenger start -p 3000.

In practice, it's more complicated than this, in part because MapKnitter relies on old versions of ruby, rails, and other dependencies, and this causes conflicts and strange errors.

Here's what I had to do to get the latest version of MapKnitter (refactored to use Bundler, bower, Sprockets, etc) up and running on our server at

  1. git clone <repo> mapknitter
  2. cd mapknitter
  3. bundle install # install ruby gems using Bundler
  4. If you don't have nodejs and bower installed, you'll need both to run the latest version of MapKnitter. I had trouble installing node, but finally got it to work using: sudo su curl -sL | bash - (as per installing-node.js-via-package-manager) apt-get install nodejs exit # leave sudo shell - don't want to be in there too long.
  5. sudo npm install -g bower
  6. bower install
  7. At this point, I tried to run MapKnitter, but got a bunch of errors. After reading this answer on StackOverflow, I finally figured out that these errors (complaints about source_index begin undefined for Gem:Module) had to do with incompatibilities between the version of Ruby and Rails that MapKnitter is built with and the latest version of rubygems. The solution was: rvmsudo rvm rubygems 1.8.25 --force Note that the --force flag tells rvm to install version 1.8.25 of rubygems even though that is lower than the version currently installed.
  8. Set up config files. For me, this was as simple as: cp ./config/database.yml.example ./config/database.yml cp ./config/config.yml.example ./config/config.yml.example
  9. Set up database: rake db:migrate
  10. Run MapKnitter: bundle exec passenger start -p 3000 This is if passenger is installed locally using bundler. If passenger is installed globally, you can simply run passenger start -p 3000.
  11. I had trouble uploading images once MapKnitter was running; passenger logged errors with the image_science gem, which complained that FreeImage.h was missing. I fixed this by installing the freeimage library: sudo apt-get install libfreeimage3 libfreeimage-dev

All done!


Thanks for documenting this, Justin. Someday we should port it to Rails 3 or 4 :-/

Bundler makes things so much easier; thanks for implementing that! Shall we put this in the README file, perhaps?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...

Ported to Rails 3!

I'm getting your annotation branch and attempting a merge in and thorough code walkthrough; we'll have to plan out the migration and see how backwards-compatible it is with old maps.

Reply to this comment...

Jeff - It's awesome that you ported MapKnitter to Rails 3! Like, so awesome.

FYI the annotation branch isn't ready for production or even beta use yet. Right now, it only allows map annotation - it doesn't allow for images to be warped. I substantially (say, 80%) integrated Vidun's work on file uploads, but I didn't have time to integrate Anish's image distortion project. That integration is key before my annotation branch is ready for the merge. Unfortunately, the old image distortion code can't be used because it was designed for OpenLayers, which has a completely different API. (As a side note, check out this recent Leaflet plugin: ScanEx/Leaflet.imageTransform - it may provide something close to what we're looking for. I haven't looked through the code, so I don't know what the quality is, but it may be worth looking in to).

I am still actively working on MapKnitter. Haven't done any work directly on the mapknitter codebase in a while, but I am actively working on a toolbar extension to enable the kind of map annotation editing (i.e. editing polylines by clicking on them and having the ability to modify annotation colors and styles on the fly) that we spoke about the last time I talked with you and @mathew. See: manleyjster/Leaflet.Toolbar.

The only modifications that I made to the database were to add a few fields to the schema - I didn't remove or rename any tables or fields, so I expect that, once the desired features are added in to the codebase, the new version should play nicely with existing data.

Reply to this comment...

I'm currently reworking Anish's work, so that's good that you've done more on Vidun's.

I'd like to integrate your work as soon as possible, so it doesn't fall too far behind the main trunk.

The main thing I worry about with @anishshah101's code is that by using CSS transforms (although they are super high res and super efficient! They run on my phone!) we may not be able eventually to use the code to run exports in JavaScript on the client side. But the advantages are clear for near-term, and once the plugin is structured and integrated a bit more and build into MapKnitter, it's modular enough that we can swap in a canvas-based (maybe webgl!) distortion solution later.

At the moment, I'm debugging, getting it to work even after the window's been resized, reorganizing where code lives, and merging the drag, distort, rotate, and scale functions so they can be run on the same image.

Reply to this comment...

Justin, is the annotation tool at a unique /map/annotate/foo route, so that we could roll out your code and keep the branches closer aligned, even while you continue work? We could just keep that URL off the main menus so nobody except those who we recruit as beta testers know about it?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


There are 2 things which will be a problem while trying to convert the CSS transform into Javascript:

1) The CSS 3D transform used here: uses CSS matrix3D transformations for which I couldn't create a javascript alternative. The reason being, I didn't fully understand the math behind it. If we remove the CSS transform then we might have to create a similar function in javascript.

2) A major mistake from my end which I didn't realize until recently, the CSS functions rotate and scale use the center of image as the reference point for these transforms while the ImageDistort function is created keeping an edge point as reference for distortion transform. When we merge them through a common CSS transform so that they run on the same image, the point of reference will remain same for all the transformations so the rotation + scaling will happen about an edge instead of the center. I am unsure as to how this can be sorted.

Reply to this comment...

@warren - nope.

The database is pretty much the same, but I significantly changed the route mapping in order to take advantage of the resourceful routing built in to Rails. The transition should be smooth once we transition 100% (b/c the database is pretty much the same), but it's going to be pretty tough to run my modifications alongside the existing MapKnitter.

I would recommend periodically making the new MapKnitter available on for beta testing when we implement new features, and then pushing it out wholesale once it's gone through enough beta testing. I know it's a pain, but I felt that the potential in having a straightforward API that other Public Lab sites might use to draw on the content held by MapKnitter was compelling.

If you want me to merge the modifications that you've been making into my annotations-interface branch in order to keep the branches aligned, I can do that.

Reply to this comment...

Justin - when you suggest running it on test.publiclab, you mean with its own database, right?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...

Yup, I suppose that's necessary. Shouldn't be too expensive to run a separate Amazon RDS database just for the duration of testing. I agree that it's a pain.

Reply to this comment...

Login to comment.