Public Lab Research note


GSoC Proposal: Mapknitter Rails 6 upgrade

by kaustubh_nair | March 26, 2019 20:28 26 Mar 20:28 | #18873 | #18873

About me

Name: Kaustubh Nair

Github: kaustubh-nair

Email: kaustubh.nair108@gmail.com

Affiliation: International Institute of Information Technology, Bangalore

Location: Bangalore, India

Project description

Abstract/summary :

In this project, my primary intention is to upgrade Mapknitter to Rails 6. Other small features are increasing test coverage and improving CI.

Deliverables

The following features should have been implemented at the end of GSoC

  • Codeclimate set up
  • Dangerbot set up
  • Increased test coverage
  • Redundant code removed
  • Upgrade to latest stable version of Rails ( most likely 6) for MapKnitter
  • Upgrade to latest Ruby version supported by Rails 6 for Mapknitter
  • System tests configured for Travis and Dangerbot

We are expecting a few features to be implemented before the GSoC period starts. So in case I end up ahead of schedule, I shall perform the same upgrades on spectral-workbench

Implementation/Details

  1. Test Coverage - Tests are important since they will show specific errors and guide us to what is exactly wrong and where the code is failing. If we don't have tests covering a breaking change, it will take more time to track and understand the change, thus wasting time. (Relevant issue: #304) Currently, we have some basic unit and functional tests. The test coverage for mapknitter is currently at around 60%. I would aim to reach around 80% coverage before starting the upgrade.
  2. CI improvements - I shall set up Codeclimate which is really useful for code review and integrate it with travis. I will follow the Codeclimate documentation for this. I will also fix Dangerbot for code and PR review.(Relevent PR: https://github.com/publiclab/mapknitter/pull/306). I shall follow the documentation for this as well.
  3. Code Cleanup - After we have considerable test coverage, I shall proceed to clean up and remove redundant sections in the code. A list of features will be found on this Github issue
  4. Rails upgrade workflow - The idea is to roll out upgrades in as small pull requests as possible. We should upgrade rails to the closest target version each time for minimum conflicts and errors.

The path that I will follow is,

  1. Rails 3.2 => Rails 4.2
  2. Rails 4.2 => Rails 5.0
  3. Rails 5.0 => Rails 5.1
  4. Rails 5.1 => Rails 5.2
  5. Ruby 2.4.4 => Ruby 2.5.0
  6. Rails 5.2 => Rails 6.0

We should be going to Rails 4.0 and 4.1 before jumping directly to 4.2. But while trying to upgrade mapknitter to Rails 4, I realized that Ruby 2.4 and Rails 4 are not compatible. (Ref: https://github.com/rails/rails/issues/27670). The closest available version is Rails 4.2.8

Workflow

image description

  1. Upgrade Rails: First of all we should upgrade the rails gem to the closest target version.

For this we need to change the rails version in the Gemfile. For example,

   gem 'rails', '~>4.2.11' 
  1. Upgrade gems: Next, we should upgrade the gem dependencies to the latest version. Before doing this, we need to first relax versions of all the gems in the Gemfile so that their versions can be updated. Now gems can be upgraded by running,
   bundle update
  1. Update rails config: Once all the dependencies are installed and compatible with each other, we need to start updating the code. I will start by first fixing the config files. For this, we can use RailsDiff which shows the configuration changes between rails versions.

We can also use the rake update task that is provided by rails for dealing with changes in old files and creation of new files. This is done by running

   rake rails:update
  1. Fix errors in test: Due to change in rails version, there is a possibility that the tests will encounter syntax and other kinds of errors. This needs to be fixed before running the tests.

  2. Run tests: Tests will be my primary way of finding out breaking points in the code. I will be fixing the code until all the tests pass. Useful reference guides during this process will be the Upgrade Guidesand Release Notes., which are amazingly documented.

  3. Fix deprecation warnings: I will try to fix all deprecation warnings before proceeding to the next upgrade since it will make things easier in the future. We can also configure the test environment to raise an exception during deprecation warnings by add the following line to test.rb

   config.active_support.deprecation = :raise

This makes it easier to spot and fix deprecation warnings

  1. Finding and fixing bugs not covered by tests: Since our tests suite might not cover everything, It is possible to encounter unexpected bugs. For this we can implement the upgrade to the unstable branch for testing before merging it to main. We can take help from the community for testing the upgrade. I will write tests for any such bugs found and then proceed to fix them.

  2. Configuring system tests - It will be really useful for mapknitter to support system tests. I will be integrating system tests with Travis and Dangerbot once mapknitter is up to Rails 5. The implementation will be similar to what has been done in plots2

Timeline

Community Bonding Period (May 6 - May 27): During this time, and prior to this,I will be performing the prerequisites for the Rails upgrade mentioned above.

This includes:

  • Expanding test coverage
  • Setting up codeclimate and dangerbot
  • Removing redundant code

Phase 1 (May 28 - June 24): Upgrade to Rails 4.2

Phase 2 (June 25 - July 22): Upgrade to Rails 5.2

Phase 3 (July 23 - August 25): Upgrade to Ruby 2.5, Rails 6, Set up system tests

Weekly timeline

Week 1: Upgrade to Rails 4.2 - Update Gems, Fix tests

Week 2: Upgrade to Rails 4.2 - Fix tests, Fix deprecation warnings and other bugs.

Week 3: Upgrade to Rails 4.2 - Fix tests, Fix deprecation warnings and other bugs.

Week 4: Buffer period ( For fixing bugs if necessary. Else move on to next milestone)

Week 5: Upgrade to Rails 5 - Update Gems, Fix tests

Week 6: Upgrade to Rails 5 - Fix tests, Fix deprecation warnings and other bugs.

Week 7: Buffer period ( For fixing bugs if necessary. Else move on to next milestone)

Week 8: Upgrade to Rails 5.1

Week 9: Upgrade to Rails 5.2

Week 10: Upgrade to Ruby 2.5

Week 11: Upgrade to Rails 6

Week 12: Set up system tests and clean up code

Week 13: Set up System tests

First-time contribution

Comments, to show overall community involvement (like helping others):

https://github.com/search?utf8=✓&q=commenter%3Akaustubh-nair+org%3Apubliclab&type=Issues

Issues

Pull Requests

Experience

I have been working on Ruby on Rails for around a year and a half (Check out my github repos!).

Also I have been working as a backend developer with a startup for around 9 months. Currently, most of the medical records in India are handwritten and thus decentralized. We aim to provide a platform that used technology to make healthcare in India easier. i have gathered considerable skill in the Rails, Docker, and Devops while working here.

Projects:

Contribution Tracker- I made this project for Zense as a kind of scoreboard for tracking contributions. It uses the github api to track commits, prs and issues to the organization.

Twitter Clone- Twitter implemented using Django as part of my semester project with 2 other classmates.

Teamwork

I'm an active part of our college developers' club Zense. Zense tries to promote learning through team projects and sessions. We use software to try and fix problems in our college and society. Here's a link to our github page.

Passion

I'm really interested in Public Lab. The thing that amazes me the most about this organization is how welcoming and empowering this community is! I'd love to work with Public Lab and help the Open Source Community grow.

Audience

My project is targeted towards the entire publiclab community. Upgrading rails will ensure rolling out new features will be less time consuming and easier. Upgrading rails is a great benefit for everyone!

Commitment

I understand that it is a serious commitment and I shall be spending around 5-6 hours of work daily for this project.


6 Comments

Hey @warren here's my first draft of the proposal. It would be great if you could review it! Also, I'm a bit skeptical about the timeline. I think 3 months might be too much for the upgrade? Should I add a few small subprojects as well? Thanks!

(Also I don't know why the editor is adding so many extra links at the end. I can't remove them.)

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Thanks, @kaustubh_nair - this is really great. I also really appreciate all the work and collaboration you've been putting into MapKnitter already 👍 🎉

Actually the biggest advice I could give is to look through @souravirus's posts and work over the last summer's SoC -- taking the plots2 project from Rails 3.x to Rails 5.x!

https://publiclab.org/notes/souravirus/08-09-2018/gsoc-2018-rails-5-upgrade

It was a gigantic project, and you can follow the PRs as well:

https://github.com/publiclab/plots2/pull/2927

https://github.com/publiclab/plots2/pull/2798

https://github.com/publiclab/plots2/pull/2726

https://github.com/publiclab/plots2/pull/2533

These may include other upgrades such as further work on Yarn, changing the asset pipeline setup, all kinds of things. Thanks for a great proposal!!!

Reply to this comment...


Also! I am wondering if substantial progress is made on MapKnitter before the summer begins. If that happened, would you have any interest in doing some upgrade work on Spectral Workbench? Just an idea in case we get ahead of the schedule on MapKnitter (maybe wishful thinking 😃 )

Is this a question? Click here to post it to the Questions page.

@warren I am positive of doing a considerable amount of progress on Mapknitter before the summer begins. Ideally I should finish the Mapknitter upgrade atleast 2-3 weeks ahead of schedule. So I can upgrade Spectral Workbench after that. Even if that doesn't happen, I'd love to work on the Spectral Workbench upgrade after the GSoC period is over! 😃


Reply to this comment...


Hi, please upload your proposal at the Google Summer of Code website at the earliest. Please ignore this comment if already done.

Reply to this comment...


@kaustubh_nair congratulations on getting into GSOC 2019. Can you please direct me to the planning issue/project which you and Alax have created? I will feel superb in case I will be able to help you all in the way. Thanks

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Login to comment.