(This template is for students applying to summer code programs with Public Lab. Use this link to start writing a post: https://publiclab.org/post?n=899&tags=soc,soc-2018,soc-2018-proposals You can delete this line once you've started filling it out.)
About me
Name: Naman Gupta
Affiliation: Guru Gobind Singh Indraprastha University, New Delhi, India-110078
Location: Delhi, India-110034
Github: namangupta01
Email: 01namangupta@gmail.com
LinkedIn: https://www.linkedin.com/in/namangupta01/
Project: Email Integration Project
Web Link: http://www.namangupta.me/
Project description
Email integration project
Abstract/summary:
Implementing delayed background jobs, cron jobs for sending daily/weakly/monthly/ trending email digest to the subscribers.
Reply-by-email to comments feature.
Introduction
The current implementation of Email System in PLOTS2 doesn't implement Active Jobs i.e queue feature because it is currently using rails 4.1. And rails 4.1 doesn't implement Active Jobs on its own. We have to ways to resolve this either include activejob gem or upgrade to rails 4.2 (we will go with this). So currently, mails are sended synchronously. And if the sending of mail is failed due to some reason or we don't want to send the mail at the time of request/response or if we want that a mail should be sent after (1 day or any specific period of time) we can't implement it now.
Sometimes, we have to run jobs at particular intervals to, say, backup logs, send emails, etc. In rails application we can schedule such tasks using the whenever gem.
For integrating cron jobs with the plots2 we can implement whenever gem which will be very useful in sending daily "batch digest" email to the subscribers. We can specify the time when the email should be sent. More details on implementation is in Implementation part.
Another feature is the reply-by-email feature using mailman gem which currently have three types of receivers: Standard Input, Maildir, and POP3. POP3 is used most often.
Another feature that we can implement is of reply by tweet which is very convenient and easy for the user to do.
Project Goals
During this GSoC season I would deliver:
- Including Active Jobs in project.
- Implementing whenever gem to work with Active Jobs to perform delayed jobs.
- Implementing mailman gem which will be used in reply-to-comment via email feature.
- Implementing reply-to-comment via email feature.
- a user interface for managing digest settings and email notification settings.
- Reply by tweet feature.
Implementation
3.1 Including Active Jobs
Currently, plots2 is using rails 4.1 and rails 4.1 don't implements Active Jobs so it should be updated to rails 4.2 so that we can use delayed jobs with the help of Active Jobs. On which i am currently working.
3.2 Implementing whenever gem
For implementing cron jobs with our application we can use whenever gem. The steps for implementing this gem are specified below :
1. Include whenever gem in Gemfile
2. Type command wheneverize . in the root folder of the application which will create an initial config/schedule.rb
3. After typing above command also type command whenever which will create a file schedule.rb
4. Now edit the schedule.rb file to add cron jobs. The best things about whenever gem is that it makes it easy to write cron jobs in simple ruby language.
- For Example:
every 1.day, at: '4:30 am' do
runner "MyMailer.send_daily_digest.deliver_later"
end
every 1.day, at: '4:30 am' do
runner "MyModel.task_to_run_at_four_thirty_in_the_morning"
end
every 3.hours do # 1.minute 1.day 1.week 1.month 1.year is also supported
runner "MyModel.some_process"
rake "my:rake:task"
command "/usr/bin/my_great_command"
end
5. Now run command whenever --update-crontab update the schedule.rb file and to enjoy cron jobs.
3.3 Implementing mailman gem
User will receive an email whenever he is being mentioned in a comment or whenever someone is commenting in the node that the user is following. And if he wants to comment or want to reply the comment, this feature would make it very easy for him because he don't have to open the website and then comment. He can directly reply via email using this feature.
This feature can be implemented with the help of mailman gem. Mailman is an incoming mail processing micro framework that support POP3 and Maildir support, that works with rails "out of the box".
Implementation steps:
1. Add mailman gem in gemfile and then do bundle install.
2. Now we need to add a script in script/mailman_server. This script will configure the mailman setting.
3. Currently, there are three types of receivers in mailman: Standard Input, Maildir and POP3. POP3 is very commonly used so we will use POP3 receiver.
4. POP3 receiver is enabled when the Mailman.config.pop3 hash is set in script/mailman_server. This will poll every minute by default. Polling time can also be changed. After new messages are processed, they will be deleted from the server by default but if we want to save them we can also save them.
5. We have rails environment present in script/mailman_server which means we can access rails models there.
6. script/mailman_server also contains a module which will run when a mail is received. As specified above we can use our models there because rails environment is there. Here's the screenshot of how the whole script/mailman_server looks like.
7. Now receive_mail function in comment model will contain the code for finding node and Sender email, so that the comment can be created. So the question comes, how we will decode the node Id and how we will get the sender's email. We will get sender's email from the From of the mail which contains the sender's email address and we will get the node id from the subject of the email. For example, subject would be like publiclab node #1; here 1 is the node id, so we will use regex to find out the node id and then add comment to the node. This is exactly same that happens in Github where issue or pull request id is used in subject.
8. Add a icon when displaying comment to differentiate between normal comment and reply by email comment.
3.3 Implementing feature reply by tweet
Reply by tweet is an amazing feature that would led users to reply to any node via tweet. Currently, we are posting every node at http://twitter.com/publiclab (using IFTTT.com). We can easily do it by using Twitter Webhooks.
**
**
Implementation Steps:
1. We will use Twitter **Web-hooks **Api ( Webhook api ) to get all the things happening on the public lab twitter account whether it is tweets, reply, mentions and lots more.
2. Using web hooks, we will get all the data in real time, no polling is to be done using any gem, in polling permanent connection is made between twitter and server using sockets by handshaking of our server and twitter.
3. As we will get all the activities happening on twitter including reply, the next thing is to identify the node.
4. Now, all we need to find out to which node does the tweet belongs. This can be done in two ways. First way is creating a new model via migration to save the node vs key i.e in ift.tt/2G6IK69, 2G6IK69 is the key which we will store in reference to the tweet and we will store node_id in reference to the node. Or, secondly we can tweet an tweet_id_ along with the tweet and save tweet_id and node_id in our database.
5. As we will receive reply using web hooks, we just have to identify the tweet to which the reply belongs which can easily be find by the service provide by twitter.
6. After identifying, we can add comment in our database.
7. We can also add a icon to differentiate between reply-by-tweet comment and normal and reply-by-tweet comment.
Reference links:
About webhooks: https://stackoverflow.com/questions/20644019/what-is-a-webhook-and-why-should-i-care
Timeline/Milestones
Community Bonding Period [23 April to 14 May] -> Get to know team and the mentors and also start discussing, analyzing the areas and discussing the work with mentors.
[14 May to 20 May] -> I have my exams from 14 may to 22 may so my working frequency will be less in this period (from 21 may to 22 may).
[21 May to 27 May] -> I will start working on including active jobs so that further work of performing delayed jobs can be done.
[28 May to 3 June] -> I will start working on the feature to include whenever gem so that we can perform weekly/email/monthly/ trending email digest.
[4 June to 10 June] -> I will start working on implementing the feature to implement mailman gem so that we have an reply by email feature.
Phase one evaluation [11 June to 17 June] -> Working on creating Mailers for digest.
[18 June to 24 June] -> Start writing tests of what is done till now.
[25 June to 1 July] -> Integrate twitter webhooks to get realtime activity from twitter of what happening on the publiclab twitter account.
[2 July to 15 July] -> I will start making model and writing documentations and working on logic to add comment on a particular node.
[16 July to 22 July] -> Finally working on improving code structure and working on view of how the comment looks.
[23 July to 29 July] -> Writing tests of the code written up till now.
[30 July to 6 August] -> Finally take a week to scrub code, improve documentation, etc.
Experience
I have been working on the the Web Technology from more than 1.5 year. And have developed many applications on rails as well as on other technology.
I have also worked as Tech Head in Google Developers Group - BVCOE which is a technical community and also took various workshops and sessions as mentor on different technology.
I have also worked as Teaching Assistant for Web Development Course (which is a Ruby On Rails course) in Coding Ninjas (www.codingninjas.in) where my job was to mentor the student and help them in their Web Projects.
I am currently working as Web Developer Intern in Edusquare and working on building their online test platform for preparing for prestigious Engineering Entrance Exams in India. These platforms are build using various technology -- Ruby On Rails, Python, Flask, AngularJs and Backbone Js.
TeamWork
I have been actively volunteering in various community where i have showed my skills of teamwork and hard work. I am currently volunteering in Google Developers Community and i have also organised hackathon in my college and worked as one of the organiser.
Open Source Contribution
I have been working on the open source projects from the past eight months. I have contributed to organisations like Public Labs and OpenSUSE.
Passion
The things that interest me the most about public lab is the science and environments because i am very passionate and have very deep interest in science particularly in physics --understanding how the law works in nature.
Audience
This project will make the publiclab user experience good whether it is reply by email feature or getting daily/weekely/ monthly/treading email digest to the subscribers.
Commitment
Yes, i do understand this is a serious commitment and i will give my full dedication and commitment to this project .
42 Comments
Wheeee! A third proposal for email integration! I'm going to re-iterate what I wrote to @gauravano and @vidit -- this is a highest-priority project for us, so we'd love to see multiple people tackling this from different sides. It's not a problem if multiple people are accepted; we will have to break the broader project down into parts so that it's possible for multiple people to work on at the same time.
For all three of you, do look at proposals by others doing Rails 5 upgrade work: @souravirus, and @meghana-07 have both proposed this. #15826 #15814 There may be some cooperation and work sharing on the Rails 4.2 upgrade which ActiveJob requires -- even before the summer begins!
Perhaps with three people interested, my comments about how to break it up could be taken even further -- discuss! I also proposed a possible "reply by tweet" concept on @gauravano's post, so maybe that's another area where a third person could get into things.
Awesome, please read one anothers' posts and provide each other with some feedback! And don't worry about overlapping -- as i said before, it's a powerful motivator to break things up into smaller parts so that multiple people can work in parallel on different parts of the problem. Thanks!!!
Reply to this comment...
Log in to comment
https://publiclab.org/notes/gauravano/02-18-2018/gsoc-proposal-email-integration-project
https://publiclab.org/notes/vidit/02-16-2018/gsoc-18-email-notification-overhaul
Reply to this comment...
Log in to comment
@warren i would love to help in upgrading to rails 4.2 and even i am currently working on this. And as you said about the reply-by-tweet concept on @gauravano's i am happy to work on this. I will update the proposal regarding this concept soon.
Reply to this comment...
Log in to comment
And @warren what do you think of having an email creating editor for the admin to send emails to the users ?
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
@warren i just have one doubt in reply-by-tweet concept. If this is going to be used for replying to comments then how will the user will do tweet in reference to particular comment? We have to give some reference in the tweet so that the comment is identified and rest of the things can be handled by cron jobs using whenever gem.
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
The idea with "reply by tweet" is that we automatically tweet out each note -- this we do already, at http://twitter.com/publiclab (using IFTTT.com) -- so any replies to that tweet would get copied back to the original note.
You're right -- the reference back may be tough. We could:
tweet:___ID___on the nodei'm sure there are other ways too!
Reply to this comment...
Log in to comment
This is interesting -- what about something similar, where we have a way to notify groups of people, like how we do on GitHub like
@publiclab/reviewers-- a "group" handle. Onplots2, we could do:@g/partsandcrafts-- to notify people who have the profile tagpartsandcrafts. We may also want to ensure that only moderators can do this, or it'll create a lot of email flying around!Let's make an issue for this last idea and add it to the list of email features. It's not high-priority but could be pretty easy to implement, given profile tagging and the existing email templates.
Reply to this comment...
Log in to comment
@namangupta as the timeline is near can you please update the remaining sections so that the potential mentors can review it ??
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
@warren i have updated about twitter reply feature...have a look
Reply to this comment...
Log in to comment
Hi @namangupta !
This is really creative thinking especially reply by twitter feature .
I have some experience with webhooks and a couple of things to add here :
1.) If you plan to use Webhooks than why not then create a webhook to post tweets real-time , instead of using IFTTT ? Does this makes sense ?
2.) PublicLab has all the URL's and API's open , i.e anyone can access them . The URL used for Webhook provided by PublicLab will be open , right ?
For a big organization like PublicLab , do you think it would be a safe option ? Especially what if someone gains access to this URL and sends TB's of data here , which will result in spam data and ultimately server crashes . Our website will be flooded with bad data in this case .
This may be a extreme case but just trying to be cautious here ! You may be aware of the recent DDoS attack on Github , where TB's of data was transmitted (it is the biggest attack till date) .
3.) I remember that it took me like atleast 2-3 weeks to get my account active for Webhooks . Is the approval time decreased now ?
4.) If in production , there is any error in our application, then the tweets that will be posted in that timeframe will be lost till the error is resolved (bcoz the twitter webhook will not pay attention) , makes sense ?
5.) How to you plan to debug ?
6.) The key (generated by IFTTT) comes after the tweet gets published . right ? So how will you add Key vs Node entry in database ?
7.) And you may have answer to this but just to add here - how do you plan on getting replies of a tweet as Twitter API does not directly give this directly ?
The Email part looks perfect to me and I learnt a lot :) Thanks !
I hope this helps !
Thanks so much :)
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
I just realized i asked too many questions here ! Sorry about that !
As @warren said here , you do not have to answer them before the deadline .
So Sorry again ! Thanks !
Reply to this comment...
Log in to comment
@sagarpreet no problem. I am glad to see that you are so much interested in my proposal. Thanks :)
Reply to this comment...
Log in to comment
Great !
Reply to this comment...
Log in to comment
another reply foe testing
On Wed, Jul 4, 2018 at 10:44 AM, \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Hmm, my email didn't seem to arrive... i wonder why!
Reply to this comment...
Log in to comment
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
another thet
Reply to this comment...
Log in to comment
testcomment
Reply to this comment...
Log in to comment
second check to check if mailman is timeout
On Wed, Jul 4, 2018 at 10:02 PM, Naman Gupta \<naman18996@gmail.com> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
why it is not working man!
On Fri, Jul 6, 2018 at 2:21 PM, Naman Gupta \<naman18996@gmail.com> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Mailman was down, I started it! I wonder if it crashed, will try to find it in the logs.
Reply to this comment...
Log in to comment
Mailman was down, I started it! I wonder if it crashed. In any case the local SMTP was not relaying email for the docker IP address - testing it now!
Reply to this comment...
Log in to comment
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Testing again
On Fri, Jul 6, 2018 at 10:59 PM, \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Testing mailman again, sorry for the noise!
Reply to this comment...
Log in to comment
this is reply to @icarito
On Sat, Jul 7, 2018 at 12:03 AM, \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
checking again
On Sat, Jul 7, 2018 at 12:09 AM, \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
checking again
On Tue, Jul 10, 2018 at 2:59 AM, Naman Gupta \<naman18996@gmail.com> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
checking again
On Tue, Jul 10, 2018 at 3:07 AM, Naman Gupta \<naman18996@gmail.com> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
looking good but it didn't trim out the reply text... that code is live - do you want to see my full text email?
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
This is same as question..if this is working then question's reply by comment should work
On Wed, Jul 11, 2018 at 10:13 PM, Naman Gupta \<naman18996@gmail.com> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
testing again!
On Wed, Jul 11, 2018 at 12:43 PM \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
me testing toooo!
On Sat, Jul 28, 2018 at 2:05 AM, \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
testing....
On Sun, Jul 29, 2018 at 6:28 PM, \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
@liz you can test here if you want?
On Mon, Aug 6, 2018 at 1:28 PM \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
great! here goes my reply by email!
--
+1 336-269-1539 / [@lizbarry](/profile/lizbarry) / lizbarry.net
On Wed, Aug 8, 2018 at 9:49 AM, \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Looks like it got your signature. @namangupta can we add a standard filter for signatures looking for the newline + "--" ?
On Wed, Aug 8, 2018 at 9:46 AM \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
That's a great idea to block it from including signatures below the "--"
That would make what i'm about to say here a non-issue, that the code misinterpreted the link to my twitter account "twitter.com/lizbarry" (text is @lizbarry) as a call out to a Public Lab account which it expanded in markdown in the comment to [[@lizbarry](/profile/lizbarry)](/profile/lizbarry).
--
+1 336-269-1539 / [@lizbarry](/profile/lizbarry) / lizbarry.net
On Wed, Aug 8, 2018 at 9:53 AM, \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Testing
On Aug 6, 2018 8:04 PM, "Naman Gupta" \<naman18996@gmail.com> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Testing it guys.
On Fri, 24 Aug 2018, 09:57 Naman Gupta, \<naman18996@gmail.com> wrote:
Is this a question? Click here to post it to the Questions page.
@naman18996 username tagging is working too?
On Sun, 9 Aug 2020, 12:15 , \<notifications@publiclab.org> wrote:
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Login to comment.