Name Vidit Chitkara
Affiliation Guru Gobind Singh Indraprastha University, New Delhi, India-110078
Location Delhi, India-110034
Project Title Email Notification Overhaul
Email notification overhaul.
Abstract/summary (<20 words):
Reply to comments by email, send weekly digests to Publiclab users and a separate dashboard to handle digest settings.
The aim of this project is to help users to have a better User experience and have better involvement with PublicLab. The project aims to achieve the following milestones:-
- Reply to comments by email
- Daily digests for users (This would help getting all the preferred content at one place).
- Users will be able to configure digest related settings.
1. REPLY TO COMMENTS BY EMAIL: In the current implementation of plots 2, one needs to visit the website (to a particular node) in order to reply to comments or simply comment. One of the better implementations could be, to use the mailman gem which would help us reply via email. We need to follow the following steps:-
- Add mailman gem to gemfile and do bundle install to install the dependencies.
- Now we need a script to configure the mailman settings. This would go in the script folder (script/mailman_server).
- There could be 3 methods through which we could receive an email, we could use pop3 as the method. Pop3 would require receiver's information such as username, password, etc. The script would now extract the incoming mails in the form of a message object and then redirect it to a "receive_mail" method in Comment model.
- The "receive_mail (message)" method would contain logic regarding the comment. It could be implemented as:-
i) Identify node id from message subject using appropriate regex (something like- /^#(\d+)$/).
ii) Identify user from message object. (message object provides a message.from method to identify sender's email).
iii) Identify the comment body from the message object. The message.body.decoded method gives us the comment body.
2. Creating comment from received and decoded mail by mailman and design plan for them Now that we have all the necessary information required for making a comment, we need to follow the following steps to create a comment:-
- Firstly we need a email_reply (boolean) column in the comments model to identify between normal comments and reply by email comments. So a new migration file would be made which would add the email_reply column to the comments table.
- The next step would be to execute a create query on Comment model for the identified node. The query would be like node.comments.create(user: user, body: body, email_reply: true)
- The last step would be to render the comment on the associated node with an icon to differentiate between normal and email replies. A working screenshot of this step is shown in the following screenshot.
3. Daily digest Email design:
A basic template of daily digest mail is shown in the following screenshots:-
This part could be broken down into the following parts:-
- A separate method, e.g. "send_digest(user,top_picks)", where user and top_picks are method arguments corresponding to Publiclab user and their subscriptions/digest notes respectively. This method would go in the subscription.rb file.
- A possible design for the digest is shown in the above screenshot.
- A method is already there in user.rb (content_followed_in_period(start_time,end_time)) which gives us the digest notes. These notes would be passed (as top_picks) in the 'send_digest' method in subscription mailer.
4. Using activejob for sending digest mails:
Sending daily digests to thousands of users per day (by creating a basic synchronous request to server) would not be feasible at all. We require a service that can make asynchronous requests, so that there is not much load on the servers. Rails provide a built in service for handling asynchronous tasks known as "active-job". "Active job" is a framework for declaring jobs and making them run on a variety of queuing backends. These jobs can be everything from regularly scheduled clean-ups, to billing charges, to mailings. Anything that can be chopped up into smaller units of work and run in parallelly.
Currently we are using rails 4.1.16 in the Publiclab app, which does not support activejob. So we first need to upgrade our app from rails 4.1.16 to version 4.2.6 (a stable release of rails 4.2.6).
For queuing and executing jobs in production we need to set a queuing backend. We can use "Resque" as the adapter for handling background tasks. Resque is a redis backed library for creating background jobs, placing those jobs on multiple queues and processing them later.
Since we require to send digest emails daily to multiple users, there should be a scheduler which schedules activejobs at a particular time everyday. For this purpose we can us the whenever gem. Whenever is a Ruby gem that provides a clear syntax for writing and deploying cron jobs. The following architecture and steps explain the basic flow of sending daily digests using activejob:
a) Whenever gem creates a cron job which calls the job every 24 hours. A schedule.rb file needs to be created in the config folder. Here, all the jobs are schedule. e.g:-
every :day, at: ‘12:20’ do runner “DailyDigestJob.perform_later” end
b). Since there are thousands of mails which are going to be rolled out at once, they would be called asynchronously by resque (queuing backend). For making an activejob, there is a jobs folder in the app folder. For each worker a separate file is created (daily_digest_jo.rb in our case). All the logic which needs to be executed asynchronously resides here in a "perform" method.
class DailyDigestJob < ApplicationJob queue_as :default def perform(*args) # Do something later users = User.where(allow_send_email: true) users.each do |u| DigestMailer.send_digest(u,Node.all).deliver Rails.logger.info "Scheduled a job to send digest" end end end
c) Finally the enqueued jobs would be dequeued (handled by resque) and mails would be sent asynchronously.
5. Testing Digest email feature: For testing out the digest email feature, we could send mails to a limited number of people. Only users containing certain tags (e.g. email-tester) would be sent the mail. The tests need to be hardcoded in the source code.
6. Digest email settings: Finally, we should give users an option to configure email related settings. This could be done by adding a form with checkboxes on the subscriptions page. The steps for implementation would be as follows:-
- Make a migration file for adding boolean attributes (receive_digest, post_comments, etc.). This is done to store user preferences for receiving mails.
- Make a controller method (e.g. configure_user_preferences) which would receive a post request with params consisting of user preferences. This method would will make the relevant database entry in user model.
- Make relevant route entries (map route to controller,action).
- Finally write some front-end code for the notification design. This would be written in the home/subscriptions.rb file. The following screenshot shows a prototype of the design.
Community Bonding Period [April 23– May 14]-> Discuss with mentor, the relevant issues of the project and requirements for them. Break the feature into several first timer issues, to encourage greater participation. Then start with the prototype of the features.
[May 14 – May 20]-> Implement reply by email part. Read more about mailman gem and how to write best scripts for receiving the mails. Then write mailman script for handling incoming mails.
[May 21 – June 3]-> Make relevant changes in the user model and make the design for the comments. Do intensive testing of the feature.
[June 4 – June 10]-> Write tests for reply by email.
[June 11 – June 15]-> First Evaluation. Write documentation for the reply by email part.
[June 15 – June 21]-> Design basic template for digest email. Make send_digest mailer at the backend along with all the required logic.
[June 22 – June 28]-> Integrate redis/ resque (queuing backend) and testing it.
[June 29 – July 8]-> Make a separate job for sending digest emails. Refine the digest email template.
[July 9 – July 13]-> Second evaluation Write tests for digest email part.
[July 13 – July 19]-> Testing the digest email feature by sending mails asynchronously to a smaller set of people (having certain tags). Write documentation for digest email part.
[July 20 – July 26]-> Implement the UI notification part and write tests for them.
[July 27 – August 6]-> Brushing the code in the above parts, wherever possible. Do testing of all the parts as a complete system. Complete the documentation.
Yes I have forked the plots2 repository on my local machine which is up-to-date with publiclab/plots2.
I have been developing web applications for about an year. Ruby makes life much easier by providing gems for almost everything one would need to build a full fledged web application. This encouraged me to learn web development in rails framework.
I worked as a Teaching Assistant for Web development (Ruby on rails) course in an educational start-up, namely, Coding Ninjas (www.codingninjas.in).
Currently I am working as a web developer intern in a healthcare startup known as kvlabs (www.kvlabs.in). kvlabs.in is a rails based web application.
I have been contributing to open source softwares for about 6 months. Some of the organizations I have contributed to are openSUSE and Publiclab. In openSUSE I have contributed to OSEM (Open Source Event Manager). OSEM is an open source software tailored for managing free and open source conferences. In public lab I have contributed to plots 2.
Contributions to publiclab
I have been contributing to publiclab (plots2) for the past 5 months. I have 28 merged pull requests which could be viewed here. I have reported 14 issues out of which 4 are first timer issues. The reported issues could be viewed here.
Being an active github user, I personally feel that it is really easy and reliable to reply via comments on a particular pr or issue. There are a lot of answers, questions and comments being posted on publiclab, so it would provide great ease for users replying to questions, answers, comments,etc. via email. Also, regular digests would help publiclab users to remain updated about the content related to environmental issues. Being a pubiclab user, I would love to get to get all my preferred content at a single place. Hence, the above features would help users to get a better user-experience of Publiclab.
I understand that this project requires time commitment equivalent to a full-time job. Since, I am well versed with the project code, I am really pumped up to give my best for this project.