Public Lab Research note


SoC proposal: GSoC: Websocket Implementation for Real-time Usage and Sensor data and Display Library

by namangupta | | 180 views | 17 comments |

Read more: publiclab.org/n/18452


About me

Name : Naman Gupta

Github : https://github.com/namangupta01

LinkedIn : https://www.linkedin.com/in/namangupta01/

Email : 01namangupta@gmail.com

Gitter : namangupta01

Location: New Delhi

Affiliation: Bharati Vidyapeeth's College Of Engineering, New Delhi

I am in my final year persuing Computer Science Engineering. I am Full Stack Developer with more than 2.5 years of Experience in this Domain.

Project description

Websocket Implementation for Real-time Interaction and Sensor data and Display Library.

Abstract/summary (<20 words):

Project includes two parts, First is implementing the ActionCable for implementing sockets and second is making a library for Sensor data and Display Library.

Project Goals

  1. During this GSoC season I would deliver:
  2. Integrate Sockets for Notification System.
  3. Integrate Sockets for Real-Time addition of Comment.
  4. Integrate Sockets for Real-Time addition of Answers.
  5. Integrate Sockets for Real-Time liking and user is typing.
  6. Basic setup of Sensor data and display library.
  7. Add csv parser in the library.
  8. Integrate chart library.
  9. Implement various chart.
  10. Finally implement the library in the plots2 project.

Problem

As already stated above that project is divides into two parts. First is Implementation of sockets for real-time use which will help in better user interface for the users and second is Sensor data and Display Library which will help in better analysing the sensor data via visualizing charts.

Implementation

Real-Time Notification System by implementing WebSockets using ActionCable in rails.

Real-Time communication is performed by Using WebSockets which is a web protocol for bidirectional communication. WebSocket is not a request-response protocol where only the client can request. It is a socket(very similar to TCP socket). Mean once the connection is open, either side can send data until the underlying connection is closed. While in HTTP, only client request server responds.

Rails also support the implementation of WebSockets using ActionCable. Using actioncable we can implement sockets in client and server side as well. The client is subscribed to a channel also known as room and everything that broadcast to the particular room will be received by all the sockets or user present in the room or channel or we can say that to all the users who have subscribed to the room or channel.

For implementing real-time notifications we will perform the follow the following process:

  1. First of all setup and configure the Redis which is a key-value based DB which will be used by the actioncable to store the sockets information in a key-value form. For this import the Redis adapter for using Redis and configure the Redis credentials in cable.yml.

  2. Whenever a user opens the website and if he is logged in then create a channel and make a subscription of the users socket to that channel.

  3. When ever new comment or new answer is added to the page find all the users that follow that page or tags used in that page and then broadcast notification those users using ActionCable.

  4. Handling the broadcasted notification data on the client side in a function named received to show the notification. Because whenever the data is broadcasted on the server then received function of that channel is called.

  5. Get the Notification Permission

  6. Implement browser notification API for displaying a notification.

    Here is the notification:

  7. Perform all the notification process in ActiveJob to perform this job in the background.

But there is one problem in the above approach which is multiple tab issue. Multiple tab issue is the issue in which whenever the user has opened multiple tabs then those tabs sockets are subscribed to the same room and everything that is broadcasted to that room will be received by all the tabs which will lead to the multiple same notifications.

There is no way in ActionCable to transmit data to a single socket. So in order to solve this issue, we can use a separate room for every tab i.e one room or channel per tab.

This will be done by using the long token as a postfix of users:id namespace as shown:

And at the time of broadcasting, we can find all the sockets having name space of users:id as a prefix as shown.

By doing this we can prevent multiple tab issue.

Real-Time addition of Answers

For real-time addition of answers, we will use WebSockets using ActionCable here also. Whenever a new answer or comment is created we will find all the peoples who are following that node and then perform Broadcasting operation in the Background using active jobs.

Steps to implement this:

  1. Create an Answer Channel.
  2. Whenever a user loads a particular node page than subscribe to that node’s channel to receive the real-time answer.
  3. Add an after_create call back which will find the users following the node and then call the Active Job to perform the Broadcasting Function.
  4. Handle the node’s answer channel or room event on the client side.

Real-Time addition of Comments

Real-Time addition of Comment is similar to that of an addition of Answer. We have to create a separate channel and add an after_create callback to call a background function which will call an active job perform function which will Broadcast to the specified users. And on the client side subscribe to those channels for getting the data.

Steps to implement this: 1. Create a Comment Channel. 2. Whenever a user loads a particular node page than subscribe to that node’s comment channel to receive the real-time comment. 3. Add an after_create call back which will find the users following the node and then call the Active Job to perform the Broadcasting Function in the background. 4. Handle the nodes comment channel or room event on the client side.

Real-Time: User is typing and likes

Similarly in this case we are going to make rooms for particular notes and whenever there is a new like or dislike on the notes it will get updated in real-time. It have the similar implementation as of the above cases.


Sensor data upload and display library

With the sensor data upload and display library on public lab website we can use interactive graphs to show the data. And since it is going to be the library so we can use this in any of the platform.

For making this library lets divide this into steps:

  1. Uploading on csv from local or provide online link to a csv for parsing and extracting the data to analyze

    There will be two ways to uploading the csv file. One of them will be upload from local and other will be via a link of csv present online.

    After selecting the file from the browser menu, we will create a CSV File object which will be used for parsing in next steps.

  2. The next step is parsing the CSV file that we have browsed in order to extract out the data.

    After we got the File object from the file browsing in First step the next step is to parse the CSV file in order to get the data for the analysis.

    We are going to use PAPA Parser to parse the CSV file. PAPA Parser is a famous open source CSV parser. For more information about the parser go to https://github.com/mholt/PapaParse.

    After passing the File Object and callback into the parser we will receive the parsed result in out function callback’s argument which will contains the array of arrays containing the csv data. And we will have a option to look at the parsed data via open data in a model. As shown in the figure.

    Before Uploading CSV file :

    After uploading CSV fIle

    After Using View CSV data a model will be opened showing csv files data.

    Parsed data in the form of an Array of array

  3. Next step is to include a Standard Open Source Graph Library

    Chart.js is a standard widely used open source graph library with wide variety of feature. We are going to use this for showing the Graphs or charts.

  4. Next step is to select the graph type:

    After uploading the csv file, there will be a dropdown for selecting the graph type as shown :

  5. After selecting chart type now we have to select the columns for plotting in chart.

    No of columns selected will depends upon the type of graph selected. For example: if we select the line graph type then there will be two columns selected one for X-axis and another for Y-axis. Similarly, if we select radar graph we can select multiple columns as well.

  6. After selecting columns we can now have generated chart Now after selecting the columns from the drop down we will generate chart . The chart will be plotted between the columns selected between in the above x-axis and y-axis drop down. As shown in the figure.

  7. There will be multiple chart supported from line, bar, area to radar and Pie. And all this will be easy to integrate by just using id and then call a function with this id as

Timeline/milestones

Time Duration Work to be Done
27 May to 2 June Basic Action Cable Setup in Plots 2 project
3 June to 9 June Implement Sockets for Notification
10 June to 17 June Implement Sockets for realtime addition of comments and Answers
18 June to 23 June Tests for Socket for Notification
24 June to 30 June Basic setup for Sensor Data and Dispay Library
1 July to 7 July Implement csv input part and Implement csv parser
8 July to 14 July Integrate Chart.js library with the project
15 July to 21 July Integrate Line chart with the project
22 July to 28 July Tests for Line chart in the project
29 July to 4 August Integrate Bar and Pie chart in the Project
5 August to 11 August Tests for Bar and Pie Chart in the Project
12 August to 18 August Add export options for csv data in the library
19 August to 25 August Integrate this library into the Plots 2 and show the lists of csv data uploaded

Contributions

I have been contributing to PublicLab from more than a year now and have also worked with PublicLab in GSoC 2018. Link to my contributions are: https://github.com/publiclab/plots2/pulls/namangupta01


Experience

I am a Full Stack Developer with more than 2.5 years of Experience. I have been contributing to PublicLab from more than a year now and have also worked with PublicLab in GSoC 2018 as a Student Developer. I have previously worked as Project Development Intern in Tata Consultancy Service where i worked on making an Online Code Evaluation Platform using RoR, Js, Docker, Shell Scripting. I have also worked in many startups where i worked on many technology like Angular, Flask, Dokcer, Node, Ruby on Rails etc. Currently, i am working in Indias most loved edu-tech startup Coding Ninjas where i am working as Software Engineer Intern where i am working on the technology like: Node, WebSockets, Angular, Ruby on Rails etc. where i have worked on making features like Video Calling, Chatting etc.


Teamwork

As stated above, I have worked with Public Lab and many Companies where i used to work in team which concludes that i am strong team player.


Passion

I am passionate about solving general problems using technology.


17 Comments

Your proposal is pretty awesome. Will you be able to manage this much stuff during GSoC? I think you may require more time. I really love the mockup, sequence diagrams and the implementation part.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Will you mind if I will mentor this project Naman, lol we both did a lot of coding last year at time of GSoC?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Actually I was lean towards implementation of this project and I never got time for doing this project. Will there be similar notifications like facebook too?

image.png

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


image.png

I really love this. We can have Author X commented on your timeline with their profile pictures and comment body. That will be awesome. What do you think?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hey, @bansal_sidharth2996 I too am writing a proposal on Sensor data and Display Library and would love to work on it as well.

Reply to this comment...


Will you mind if I will mentor this project Naman

Haha I won't mind

Reply to this comment...


No it won't be exacty like facebook. It will be the browser notification. But i loved the idea of implementing notification like facebook. @warren what do you think?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Suppose I am watching a movie and having fun time. I will get these browser notifications. Then I will be willing to turn off them. So, I think it will be great if we can have it like facebook too. We can also create notification page which can list all the notifications. What do you think about this page @warren @namangupta?

@IshaGupta18 oh your and his idea is similar. It will be great to review your proposal too.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


And @bansal_sidharth2996 what if that browser notification is important to you, you will be not willing to turn them off. Lol 😆 😂

Reply to this comment...


But I liked your idea of Notification list at a place. So that one can be aware of all the recent things happened in order.

Reply to this comment...


Thanks a lot @bansal_sidharth2996 and @namangupta. i will soon be sharing it with you!

Reply to this comment...


I think it could be done but we may want to think about how separate we can maintain the two systems so that the initial notifications API can be developed first, and the "notifications you missed" later?

On Sat, Mar 2, 2019, 12:10 PM \<notifications@publiclab.org> wrote:

Hi! You were mentioned by bansal_sidharth2996 in a comment on the research note SoC proposal: GSoC: Websocket Implementation for Real-time Usage and Sensor data and Display Library. You can reply to this email or visit this link:

https://publiclab.org/notes/namangupta/03-02-2019/soc-proposal#c22011

bansal_sidharth2996 wrote:


Suppose I am watching a movie and having fun time. I will get these browser notifications. Then I will be willing to turn off them. So, I think it will be great if we can have it like facebook too. We can also create notification page which can list all the notifications. What do you think about this page @warren @namangupta? @IshaGupta18 oh your and his idea is similar. It will be great to review your proposal too.


Reply at: https://publiclab.org/notes/namangupta/03-02-2019/soc-proposal#comments

Report abuse to: moderators@publiclab.org

Check out the blog at https://publiclab.org/blog | Love our work? Become a Public Lab Sustaining Member today at https://publiclab.org/donate If this email title has an ID in the format #0000, you can reply with the email you use at PublicLab.org and your response will be posted as a comment on the website.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


From what I understand from the above context @bansal_sidharth2996 wants to say is not "notifications you missed" but "all the notification" just like Facebook. Am I right @bansal_sidharth2996 ?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Yeah. We will be having all the notifications at the notification page. The notifications which are unread can be highlighted. 

On Sat, Mar 2, 2019 at 11:43 PM \<notifications@publiclab.org> wrote:

Hi! You were mentioned by namangupta in a comment on the research note SoC proposal: GSoC: Websocket Implementation for Real-time Usage and Sensor data and Display Library. You can reply to this email or visit this link:

https://publiclab.org/notes/namangupta/03-02-2019/soc-proposal#c22016

namangupta wrote:


From what I understand from the above context @bansal_sidharth2996 wants to say is not "notifications you missed" but "all the notification" just like Facebook. Am I right @bansal_sidharth2996 ?


Reply at: https://publiclab.org/notes/namangupta/03-02-2019/soc-proposal#comments

Report abuse to: moderators@publiclab.org

Check out the blog at https://publiclab.org/blog | Love our work? Become a Public Lab Sustaining Member today at https://publiclab.org/donate If this email title has an ID in the format #0000, you can reply with the email you use at PublicLab.org and your response will be posted as a comment on the website.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


@warren Please have a look let me know where it can be improved. Thanks

Reply to this comment...


I like this proposal - thanks! Noting 2 things -

  1. there's some good discussion of JS vs. Ruby for charting in Isha's proposal, which would be good to comment on!
  2. noting that, as we did last year, it's not a problem for proposals to overlap. We work very collaboratively and we have always found ways to share work and break things up into related parts. In fact, it often results in better code if 2 brains team up on a project 🤝

Reply to this comment...


I really like your proposal Few suggestions We are looking forwards for supportive folks. Make sure to do 1. at least one PR review each week and

make some FTO issues in each week to involve new contributors inside your projects. People will love to be part of big projects. It is a great feeling to help others.

Please take out a day from timeline from each month at end of each phase of SoC fellowship to write about what you learnt and did in the period. Earlier it was not compulsory. But let's make it compulsory. By this mentors will be able to access your progress. Don't forget to mention the ftos which you created during this period. Also mention the PR reviews which you did.

We are planning for video calls I'm each month during SoC program. So, it will be great to see you all.

It will be great if you can review each others proposals and give some suggestions.

These are small activities which can lead to better and supportive community at PL. All participants all programs are encourage to write these in their proposals.

Thanks and best of luck.

Reply to this comment...


Login to comment.

Public Lab is open for anyone and will always be free. By signing up you'll join a diverse group of community researchers and tap into a lot of grassroots expertise.

Sign up