Public Lab Research note


Exporting GoogleGroups

by bsugar | July 25, 2017 07:52 25 Jul 07:52 | #14679 | #14679


image description


---Dictated, not read---

A wonderful thing has befallen us!

@icarito discovered a tool to export GoogleGroups data!!

I'll leave the instructions to the various pages for the time being. I followed directions to a T without issue but feel free to ask questions below if you get stuck. I'll do a more detailed write up later.

1) Download Python >= 3.5 I use Anaconda, and since other tutorials regarding this subject will be in Anaconda Python 2.7, you might find it easier to install that version.

2) After installing Anaconda 2.7, we'll need to install Anaconda Python >= 3.5 and we'll want to do it such that it will not overwrite the 2.7 installation. This can be done by following the directions here.

3) This will also require having the web inspector installed on your browser of choice. As a rule I do not use Chrome for an unhealthy obsession with thinking I can stick it to the (G)man. However, I suppose using Chrome in this instance would be the best way to stick it. I'll be using Firefox but tools will pretty much look the same so the instructions should go for all major browsers: Firefox, Chrome, Safari

4) Then enter the following command into the terminal:

pip install googlegroupexporter

5) Then enter BUT DO NOT HIT RETURN YET:

ggexport my-private-group --cookies " "

6) Log into GoogleGroups

7) Open up the network inspector:

image description

8) Click on a domain that is groups.google.com:

image description

9) This will open up the headers panel. Scroll down until you see "cookie". Click on the text and copy it.

image description

10) Go back to the terminal, paste the contents between the quotes, hit return, and sit back and enjoy!

ggexport my-private-group --cookies "VERY-LONG-COOKIE-STRING-COPIED-FROM-MY-BROWSER"

11) Report back here with your success or lack thereof.


I did this Help out by offering feedback!


People who did this (0)

None yet. Be the first to post one!


6 Comments

Hi Benjamin! Excited to see progress. What are you building? Also, curious as to why you think both versions of Python are necessary? I find Python3 more enjoyable to use.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Also it would be very cool if we could share a Jupyter notebook of this ;-)

Reply to this comment...


Absolutely!

Reply to this comment...


Oh, didn't see your other comment. Both versions of Python are not necessary in the strictest sense. What I meant was that since I'll be using 2.7 in future tutorials, one should install both so as to make the tutorials as 'plug n' play" as possible. When I first got into data science, 2.7 was recommended but that was two years ago so that thinking may have changed. Switching to 3.5 would certainly make working with unicode easier! That said, this is the first time two years that I have ever encountered the need for 3.5. I was really pleased that Anaconda implemented switching so easily. It is probably good practice to use virtual environments anyway.

Reply to this comment...


Oh. Didn't see your other other question. As far as what this is for we should really try to create a larger page with history. In the meantime, it's inspired by a presentation about OpenStreetMaps mailing list dynamics by Alyssa Wright et al. using the no longer maintained OpenThreads library. We'd like to do this, but for Public Lab. As an aside, the openThreads library is really great it's just that it was written for Mailman (I think?) and not GoogleGroups.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Here is a jupyter notebook of how to use the library programmatically (e.g. not via the command line). This notebook does not include the instructions above. I'll have to get to that later but for now, use the instructions above, and then see this notebook:

Reply to this comment...


Login to comment.