Help save net neutrality! A free, open internet is once again at stake—and we need your help. Learn more »

Public Lab Research note

by bsugar |

A wonderful thing has befallen us!

@icarito discovered a tool to export GoogleGroups data!!

I'll leave the instructions to the various pages for the time being. I followed directions to a T without issue but feel free to ask questions below if you get stuck. I'll do a more detailed write up later.

1) Download Python >= 3.5 I use Anaconda, and since other tutorials regarding this subject will be in Anaconda Python 2.7, you might find it easier to install that version.

2) After installing Anaconda 2.7, we'll need to install Anaconda Python >= 3.5 and we'll want to do it such that it will not overwrite the 2.7 installation. This can be done by following the directions here.

3) This will also require having the web inspector installed on your browser of choice. As a rule I do not use Chrome for an unhealthy obsession with thinking I can stick it to the (G)man. However, I suppose using Chrome in this instance would be the best way to stick it. I'll be using Firefox but tools will pretty much look the same so the instructions should go for all major browsers: Firefox, Chrome, Safari

4) Then enter the following command into the terminal:

pip install googlegroupexporter


5) Then enter BUT DO NOT HIT RETURN YET:

ggexport my-private-group --cookies " "


7) Open up the network inspector:

8) Click on a domain that is groups.google.com:

9) This will open up the headers panel. Scroll down until you see "cookie". Click on the text and copy it.

10) Go back to the terminal, paste the contents between the quotes, hit return, and sit back and enjoy!

ggexport my-private-group --cookies "VERY-LONG-COOKIE-STRING-COPIED-FROM-MY-BROWSER"


11) Report back here with your success or lack thereof.

Hi Benjamin! Excited to see progress. What are you building? Also, curious as to why you think both versions of Python are necessary? I find Python3 more enjoyable to use.

Is this a question? Click here to post it to the Questions page.

Also it would be very cool if we could share a Jupyter notebook of this ;-)

Absolutely!

Oh, didn't see your other comment. Both versions of Python are not necessary in the strictest sense. What I meant was that since I'll be using 2.7 in future tutorials, one should install both so as to make the tutorials as 'plug n' play" as possible. When I first got into data science, 2.7 was recommended but that was two years ago so that thinking may have changed. Switching to 3.5 would certainly make working with unicode easier! That said, this is the first time two years that I have ever encountered the need for 3.5. I was really pleased that Anaconda implemented switching so easily. It is probably good practice to use virtual environments anyway.

Oh. Didn't see your other other question. As far as what this is for we should really try to create a larger page with history. In the meantime, it's inspired by a presentation about OpenStreetMaps mailing list dynamics by Alyssa Wright et al. using the no longer maintained OpenThreads library. We'd like to do this, but for Public Lab. As an aside, the openThreads library is really great it's just that it was written for Mailman (I think?) and not GoogleGroups.

Is this a question? Click here to post it to the Questions page.

Here is a jupyter notebook of how to use the library programmatically (e.g. not via the command line). This notebook does not include the instructions above. I'll have to get to that later but for now, use the instructions above, and then see this notebook:

You must be logged in to comment.