Public Lab Wiki documentation



Evaluation



A page for all things related to evaluation at Public Lab.

Many different evaluation efforts are ongoing across Public Lab and we try to coordinate our efforts to minimize survey fatigue or redundancy.

@liz leads the evaluation team! Ask questions below to find out more.

Community Survey

We do an annual community-wide survey to better understand our community:

2017 Annual Community Survey


Stakeholder interviewing

A series of stakeholder interviews was done in 2017! You can read them here:

Title Author Updated Likes
Interview: Yvette Arellano @stevie 4 months ago
Interview: Jim Gurley @stevie 5 months ago
Interview: Nayamin Martinez and Gustavo Aguirre Jr. @stevie 6 months ago
Interview: Karen Savage @stevie 6 months ago
Interview: Ramsey Sprague @stevie 7 months ago
Interview series with Grassroots Community Organizers @stevie 7 months ago
What fuels a movement? @stevie about 1 year ago


User interface design

See the User Interface page for more on design work towards user interface and user interaction improvements.


Questions

Title Author Updated Likes
How can I make a tag graph visualization? @bsugar 29 days ago
Help with a standard mini-evaluation for assessing software outreach efforts? @warren 4 months ago

Ask a question  or help answer future questions


Related work

Title Author Updated Likes
How can I make a tag graph visualization? @bsugar 29 days ago
How do we do user interface design work in a community process? @warren about 1 month ago
Help with a standard mini-evaluation for assessing software outreach efforts? @warren 4 months ago
Exporting GoogleGroups @bsugar 7 months ago
Your input kindly requested on the 2017 Community Survey! @liz 8 months ago
growing Github contributors @liz 9 months ago
Evaluation: Listen4Good round one results @stevie about 1 year ago
How useful is a map? @clauds about 1 year ago
First draft of tag graph @liz over 1 year ago
Creating an evaluation framework for Public Lab @Shannon almost 2 years ago
Show 3 more
Intended Purposes for Different Tools and Techniques @gretchengehrke over 2 years ago
2013 Barnraising evaluation summary @Shannon over 4 years ago
Measuring our community growth using web traffic @warren about 5 years ago


Older page content

(From 2011 via @warren, interesting!)

On this page we are in the process of summarizing and formulating our approach towards self-evaluation; as a community with strong principles, where we engage in open participation and advocacy in our partner communities, this process is not that of a typical researcher/participant nature. Rather, we seek to formulate an evaluative approach that takes into account:

  • multiple audiences - feedback for local communities, for ourselves, for institutions looking to adopt our data, for funding agencies, etc
  • reflexivity - we may work with local partners to formulate an evaluative strategy, and this may often include questionnaires, surveys, interviews which we take part in both as subjects and as investigators
  • outreach - by publishing evaluations in a variety of formats, we may employ diverse tactics to better understand and refine our work; its publication in diverse venues (journals, newspapers, white papers, video, public presentation, etc) offers us an opportunity to reach out to various fields (ecology, law, social science, technology, aid)
  • location - our evaluations should be situated in geographic communities, examining the effects of our tools and data production in collaboration with a specific group of residents

Goals

Good evaluative approaches could enable us to:

  • quantify our data and present it to scientific, government agencies for use in research, legal, and
  • provide rich feedback for field mappers (in the case of balloon mapping and other public scientists to improve their techniques
  • assess the effects of our work on local communities and situations of environmental (and other types of) conflict
  • involve local partners in the quantification and interpretation of our joint work
  • ...

Approaches

We're going to use a few different approaches in performing (self-)evaluation -- each has pros and cons, but we will attempt to meet the above goals in structuring them.

Approach A: Logbook questionnaire

The logbook is an idea for a Lulu.com printed book to bring on field mapping missions for balloon mapping.

Although this strategy can be reductive, compared to interviews, videos, etc, its standard approach yields data which we can graph, analyze and publish for public use. The results will be published here periodically. Any member of our community may use them for fundraising, outreach, or for example to print & carry to the beach to improve mapping technique.

Read more at the Logbook page.

A mini version of this questionnaire was used by Jen Hudon as part of her Grassroots Newark project and can be found here:

Approach B: Community Blog

The community blog represents a way for members of our community to ... critical as well as positive...

To contribute to the community blog, visit the Community Blog page

Approach C: Interviews

We're beginning a series of journalistic/narrative interviews with residents of the communities we work with. Read more at the interviews page.


community-metrics evaluation