Public Lab Research note

GSoC proposal: v2 API development

by rishabh07 |

Read more:

About me

Tell us about yourself!

Name : Rishabh Kumar Singh




Gitter: rishabhptr

Affiliation: Guru Gobind Singh Indraprastha University, Delhi, India

Location: Delhi, India

Project description

V2 API development and 3rd party app integrations.


(<20 words):

Further development of the RESTful API to migrate old legacy code to the new Swagger system that we are using and make new end points along with documentation and 3rd party app integrations.


We already have a RESTful API which includes a Grape/Swagger interface with documentation but now we need to migrate the older API code onto this new Swagger interface.Also we need to provide more end points for the statistical data that exits at Public lab website.

Additionally we need to provide 3rd party app integrations.

  • We are using grape-swagger gem for our RESTful API services and already providing end-points using it for various search interfaces. But we also have various end points that exists in the legacy API which exist at different locations in the code,these would be need to be migrated to this new system and possibly consolidated at a single location in the code.
  • Apart from this we need to provide more endpoints for the overall statistical data based on time ranges. This too needs to needs to implemented in the new swagger system along with proper documentation and requires tests to be written.
  • We already have a token-based API system where-in user(or bots) can post comments using API by using a user access-token that is accessible on user profile, we need to further develop and this feature so that we can provide more 3rd party integrations to our application based on abilities of a user i.e we need to authenticate user status (user, moderator, admin) and provide further more actions to perform.



  • Swagger specification( OpenAPI specification) is a definition standard for RESTful API's. It is widely used as it provides a good interface for both developing and consuming API's.
  • Not only it makes the API's easier for a human to understand but also it makes it machine readable, which means that the same documentation can be used by tools to display the API, generate servers and client. Some of such tools are Swagger-UI and Swagger Code-gen.


Grape is a API micro-framework in ruby which is widely used on frameworks such as rails and Sinatra for the various features it provides such as.

  • Raising custom exceptions.
  • Versioning
  • Authentication
  • Support multiple formats.
  • Helps organise code by organising end points in a module - resources hierarchy ,much like one would do in routes.rb


When we are already using Grape for developing our API, we can use grape-swagger gem which helps in automatically generating swagger-complaint documentation which can be seen in Swagger-UI.

The way this works is :

  1. We mount all the different modules in a single Grape::APIsuperclass and then we can use add_swagger_documentationto provide user-information of our whole API sub-system, select where mount the swagger documentation( normally /swagger_doc) etc.
  2. We can describe our modules and their individual end points indicating their use also describing the parameters and response options such as showCount and pageNum to implement pagination of response.
  3. Mounting the documentation route in config/routes.rb mount GrapeSwaggerRails::Engine, at: "/location"

Now, presently all above steps are already done and swagger documentation is already live at api/docs.

Presently we have two modules made using Grape::API which are srch and typeahead where the latter provides pointers for better searches and the former provides detailed information using a search string accross all models in the project Tags,Users,Nodesetc.

These are located in api folder and their utility classes search_service and typeahead_service which contain helper function for queries are located in services folder. Also

Therefore the swagger documentation of them is already made live as both are mounted on API superclass.

Migrate old API code:

We have legacy end points eg.: notes by tags, wikis by tags, rss feeds etc which were not made using Grape hence are isolated from this system so they need to be implemented in this system and have swagger documentation and they could exist as modules which would help clean up the code.

The required steps would be:

  1. To Create separate modules with appropriate :resources and their end-points.
  2. Define helper functions if needed in utilities class under services folder.
  3. Handle serialization of output, pagination etc.
  4. Mount these modules to API super class.
  5. Add Unit and functional test for these end-points.

One idea, that can be used while implementing this is the Versioning feature of grape where-in we can have two separate versions of api which can indicated in various ways including in header. /:version/resource/method.:format This way we could work with a different version while developing and have only one version afterwards which ensures we don't break anything.

Add Statistical end-points:

We have lots of stats data at we should implement in our API.

  1. Using the above described approach we can add the various end points for research notes published,wiki-pages , comments made etc. where the user can fetch query these based on a time-period, search period like tag,author etc.
  2. These end-points need to be nested under appropriate :resources eg:/questions/tag/..,\tag\etc
  3. Because of bulk of data it would be advisable to include optional start date and end date parameters in the request.
  4. Would need to add the required tests for these too testing their response.


We are serving response in various formats such as .xml, rss feeds ,json etc and there are some formats we could still add such as .csv.

  • For csv , rails have a pre-existing library which can be included using require csv and by adding a respond_to for csv format we should we able to generate specific columns passed through request params and also provide download for the same.
  • We can add this response format in grape by format: csv but we might need to use a CSV formatter for the same.

Documentation and Integration of Token-based API

We need to add documentation for our pre-existing token system which already includes a feature to post comments by sending the token with the POST request.

One more important aspect left uncovered is authentication in an API. Grape supports a variety of authentication methods , token-based authentication being one of those. But we dont actually need to add any external functionality to implement this feature and create unique tokens as we already have user token available on each user profile.

So, we could use these token in requests and for the endpoints that we would want to secure we can use !authenticate helper to verify the token and identify the user, which would help us expose some end-points to user with pre-defined roles.

Also various new options for a bot/CLI could be added such as marking a post as spam,approving it,adding tags etc


Will update soon.


Break your project up into small projects -- one per week!

See this page for guidance on breaking your plan up into small, self-contained parts:

I will update this soon.


All the documentation regarding swagger-grape is available online and also some documentation for the pre-existing API code in code base already exists. I would of course require the guidance of my mentor.

First-time contribution

Have you looked over our welcome page and are you familiar with how to make your first contribution? Have you already made it? We're eager to see that you've had a good experience making a small initial contribution to our site. Please check out our welcoming page: _**

I have made some contributions in public labs in the time span of last few months.**


Pull requests:


Tell us how you've learned about writing software; what languages you've been learning, if you've worked on other projects, links to GitHub or other code repositories or samples.

I had been learning web development using HTML,CSS, JavaScript from my second year of college and last year I started with Ruby on Rails using it as a back-end and I am still enjoying working with it because it enables you to get things done faster and has rich community along with actively maintained packages(gems) for providing different functionalities.I have made some small projects in Ruby on Rails during my learning process when following various online courses.

Prior to that I had some experience in Android development , networking and working with cloud services.

From past 6 months I have been actively contributing to Open-Source organizations such as OpenSUSE and Public lab which has really helped with get comfortable with the framework even more as they both have projects on Ruby on Rails.


Describe teams you've worked with before, whether open or closed source, and in what capacity you participated. Cite examples of how you were self-motivated and self-sufficient._

Last year I had been a part of team that worked on a Android Application for providing resources for Olympiads to students. My contribution generally comprised of generating the backend of the application and providing integration to the various services and libraries that were being used. This helped me to understand the importance of working in team and how to work with different parts of code and integrating it with what others have worked upon.


What about our projects, and Public Lab, interests you? What are you passionate about? Open science, environmental justice?

Environmental degradation is a alarming situation which is becoming a even more dangerous threat for the past few decades. Seeing that how at Public lab , people with different set of skills have come together to volunteer to make a change in the present scenario motivates me to make a contribution myself that can actually create some impact in this field even though at a small scale as I believe that is what true essence of Open Source is.


Whom do you want your work to help? We especially appreciate proposals which make technologies and techniques more welcoming and friendly to those who've often been excluded.

Public lab hosts a large scale of information and with this project this information can be made more accessible through the REST API and with the new endpoints and integrations, the services offered can made more reachable in various forms.

Also with swagger system and additional documentation along with separating the API code from the rest of the code it would be easier for developers for utilizing these interfaces and enable future developers to easily build upon this system.


Do you understand this is a serious commitment, equivalent to a full-time summer job? Tell us how you'll structure your schedule from day to day!

Yes, I understand that this is a full time commitment. If selected I would not have any other job apart from this and would be able to chip in 6-8 hours every day for this.

I would however need 7 days for my exams that would lie in between the stipulated time frame but I would make up for it by putting in extra work after the exams.

software gsoc soc gsoc-2018 soc-2018 rgsoc-2018 soc-2018-proposals soc-2018-api

response:13975 implementation:


Hi! This is looking great -- thanks! A few points and requests:

these would be need to be migrated to this new system and possibly consolidated at a single location in the code.

Would we be able to maintain legacy APIs, or would we redirect them and provide documentation to people to upgrade? What's a list of legacy API endpoints to be sure we migrate?

Also, a lot of the comments I left for @Raounak apply here too - take a look:

Thanks a lot! Don't forget to take a look at related proposals by @milaaraujo, @stefannibrasil, @Raounak, and @sukhbir -- and reach out to talk with them about their proposals too! #soc-2018-api

Thanks for all your great work in contributing already! Very much appreciated. :-)

Is this a question? Click here to post it to the Questions page.

Hi, when we would move the end points in the present Grape system the path will change like /api/module-name/.. so they can still exist but I think it would be better to redirect them along with documentation and futher try to get rid of them eventually.

I have made some changes regarding implementation based on the amount of work that has already been done in this project. I will try to add more lists of end-points, mock-ups etc soon

Nice work out there @rishabh07 . It seems to be well explanatory. You can have a look on @warren suggestion to make it more perfect.

You must be logged in to comment.