DIY Oil Testing: Community

This is Part II of DIY Oil Testing: Progress toward community oil pollution analysis

II. Working with communities

Part of the Public Lab approach is to build collaborations with people and community groups directly facing environmental problems. During the Homebrew Sensing project we did outreach to both online and offline groups. The section below will explain some of our methods, results and further observations on the outreach strategies we employed. Taking this project into the future, we've developed a workshop series that outlines what it looks like to work with a group through the oil testing process.

Workshops

Community sampling of neglected oilfield site in Tangipahoa Parish, north of New Orleans; photo by @eustatic

Community Outreach

Over the course of the project we identified several community groups who were interested in the Oil Testing Kit. However, a group’s level of involvement in the project was based on several factors, distinguished by urgency and ability.

A community group had a degree of urgency in the problem if there was an imminent threat to their health or environment from something the kit addresses. This means their interest in learning and using the kit was important to them and often times personal.

The second factor that influenced a group's involvement in the project was ability. Ability here is defined in terms of resources and outcomes. The group was able to work on the project if they had:

time to commit,
energy to learn, and
a desired outcome was achieved when they learn or use the kit

Although the degrees of urgency and ability varied in the groups we worked with, they generally broke down into these categories:

	Urgent	Not Urgent
Able	Will work on the project and can use it in their community or work setting.	Will work on the project for interests other than their specific environmental concerns.
Not Able	Will not work on the project based on project limitations but has the most need for what the kit could do in the future.	Will not work on the project. Could potentially use the kit, but does not have resources to dedicate to it or a direct need for the kit as it exists.

Groups who will work on the project:

Urgent/Able

We worked with two groups during the project who feel under the category “Urgent/Able.” One of these groups was a non-profit “committed to uniting and empowering people to protect and restore the natural resources of the Gulf Region.” They were interested in the kit for its ability to be used in its current state as a detection tool and in its potential use in accountability and reporting. Their involvement in the project was mostly conducted through a Public Lab organizer who works for the organization and had a vested interest in tool development and the tools use.

The other group we worked with who fell in this category was a community group that has organized against the threat of fracking in their parish. They have done work lobbying their local government, exposing the issue to the media, and educating local people about the threat of fracking. Their interest in the kit in detection and the kit’s potential use in the fight against fracking. They have had one training of 15 people on the kit and plan to train 15 more. They also plan to work with the kit as it develops and is more able to address their community interest.

Not Urgent/Able

We worked with two groups who fell into this category. One group was an education based organization where students spend a semester exploring environmental issues and pathways for science and environmental careers. Their interest in the project was mostly academic, and saw the oil testing kit as a good use of science in education. They were also interested in use cases for the kit around their community. While there was not an urgency for them to use and understand the kit, their interest in exploring it has allowed us to use them as a sounding board for tool development and in experimental design.

The second group we worked with in this category was a group who had an early interest in the kit based on their experiences during and after the BP oil spill. They had spent a lot of money sending oil samples to the lab for testing and saw the kit as a screening tool in that process. Their urgency on the kit is not currently great, but their interest in learning about it and their ability to dedicate time to training on it has led them to take the kit into the classroom.

Groups who would not work on the project

During the project we worked with two groups who initially learned about the kit, what it could do and what it could not do and decided against taking their work on the project to the next level.

Urgent/Not Able

We worked with one group in this category. They are the group who has most to benefit from a kit like this, but is stretched the thinnest for resources and would not get desired outcomes from working with the kit as it is. The group we worked with has been dealing with a facility that has been taking hazardous chemical waste and processing it in open pits for many years. The plague of this facility on the community has brought about concerns for human health, daily nuisance of chemical odor, dumping in the waterway, and flooding waste ponds into the ditches and roads during rain events. They have the urgency of health, well being well beyond any other group we worked with for this project. However their ability to work with the kit was limited several factors. They had a lack of time to dedicate to using the kit. They identified that they did not have the technical computer skills that were needed to use the kit in it’s current state. They identified that the kit in it’s current state would probably not help them collect information that would push their community towards the outcomes they desired.

Not Urgent/Not Able

The group we worked with who fell in this category identified that the current state of the kit limits their urgency and ability to take it on as a project. This group works with both fenceline communities and advocacy as well as water issues as their staff are also River Keepers. The limitations on their urgency with the kit is that it does not currently answer to a enough of their pressing issues. Their limitations on ability to work with the kit is the time it would take to learn and use it, and the lack of desired outcomes from using it. For them, desired outcomes would be the kit’s ability to positively identify oil and scan samples taken from water.

Conclusion

If the kit develops in a way, as desired, to answer more environmental questions, take less time to learn, and produce more results that are actionable, we will see an increase in the number of community groups who have an ability and an urgency to learn and use the kit.

Sharing Space

Over the course of this project we identified that while there were a number of groups who were interested in working on developing the Oil Testing Kit and groups who were interested in using the kit for their own purposes of environmental exploration, the two did not naturally paths on the project development trajectory. This is something Public Lab has been working on for a number of years. However, it should be noted that despite this sometimes inevitable divide between users and developers, something quite different happens when people from these two groups share the same physical space.

One example of this is at the 2015 Public Lab Barnraising where both developers and users of the kit shared the same space. This physical act of sharing a space facilitated a knowledge sharing among the groups that dosen’t seem to happen on the wiki or on the google groups. We could observe developers situating the Oil Testing Kit in the narrative of a community story, and community members recognizing the needs and challenges of tool development. This sharing personalized the project and process for many people and is an excellent tool that should continue to be used.

Workshops and the learning process

In environmental exploration the learner must understand the reasons, uses and limitations of a tool in order for the learning process to be effective. The tool must also be made accessible by providing educational experiences that satisfy the learner’s unique learning preferences. Finally the tool must be situated in a problem that the learner can understand and explore.

During the course of this project, the most successful learning strategy was not to place the kit in front of someone and have them explore tool development itself as a process. Rather it was more successful to walk through the problem definition as it relates to the learner, before exploring how people sought to answer the problem. This process has been duplicated in this toolkit as the Workshops. (see Oil Testing Workshops)

Online Outreach

Spectral Challenge

Over two years, we've had the opportunity to pilot a number of different strategies for building community interest and involvement in technology and methods development. In 2013, we developed a competition to promote innovative work to further the analysis of pollutants with do-it-yourself tools, called Spectral Challenge. Launched at SpectralChallenge.org and announced on the Public Lab website, this was an "X-prize style competition" in two stages, the first to promote and reward good collaborative practices and excellence in documentation, and the second to reward concrete advances in analytic methods, including improvements to DIY hardware. In an additional twist, the prize was crowd-funded, meaning that anyone who wished to support these goals financially could add to the prize pool.

Ultimately, the challenge failed to attract much participation. In retrospect, we feel that the terms of the challenge were too open; rather than prescribe a specific improvement such as a precision or resolution target, the prize was left open to the team that demonstrated "identification of a sample in a real-world scenario, such as from an actual soil or water sample" -- and was even more open-ended in Stage 1, which called for teams to, among other things, "refine research questions and describe & execute tests which we'll need to produce credible data - including identifying problems, but especially suggesting solutions." We feel that the prize would have benefitted from much narrower goals and more scaffolding; potential participants weren't sure where to begin. Although we value openness and diversity at Public Lab, we feel that providing strong models for participation can mitigate a tendency towards aimlessness in distributed online groups.

Still, much about Spectral Challenge represented the values we aspire to in the Public Lab community; entries were required to cost less than $200, and projects were required to post "simple, legible, open source documentation." Collaborations and locally-relevant work were especially preferred. The main legacy of Spectral Challenge is that many of these priorities have been adapted into the "Open Open Hardware" process (/wiki/contributing-to-public-lab-hardware) announced in fall of 2015, which aims to structure and standardize collaborative practices in order to set clear timelines and expectations for hardware projects, as well as address the "tyranny of structurelessness" by establishing roles and patterns for open source hardware development.

Oil Testing Kit: Kickstarter

Another initiative to promote participation in the Oil Testing Kit project was a Kickstarter campaign launched in the fall of 2014, which offered a prototype kit and represented a more participatory and focused campaign than previous campaigns Public Lab has run. In contrast to previous campaigns, we focused exclusively on a single type of environmental problem -- oil pollution -- and asked backers to become involved in testing towards that goal.

The campaign was not successful -- a surprise, given Public Lab’s four previous consecutively successful Kickstarter campaigns. We believe this was due to a number of different factors, and have summarized these thoughts in an update posted just after we ended the campaign. Ultimately, we concluded that while Kickstarter can be (and has been) a powerful venue for certain types of Public Lab projects, it was not a good match for the kind of campaign we sought to launch.

The Beta Program

What was the beta program?

The Oil Testing Kit Beta Program was designed in April/May of 2015 with the intent to move forward with design of the oil testing kit, provide proof of concept that the kit did infact allow users to distinguish oil and identify potential use case for the kit.

The initial outreach materials for the program were sent out in July 2014. These advertised that the program was “offering prototype laser-cut versions of the new Oil Testing Kit free of charge for 20 people interested in helping test and refine the kit. This is an exciting opportunity to help improve our prototype DIY methods for classifying unknown petroleum samples by weight. Our eventual goal is for this kit to be usable to test and compare oil spill residues, and we need your help! Anyone can join the Beta Program, no experience required, you just need time to assemble the kit, use it (please see obligations on the sign-up wiki) and troubleshoot.”

How was it set up?

The requirements of the Beta program initially required all participants to have completed the program by November, 2015. The program details are posted at https://publiclab.org/wiki/oil-testing-kit-beta, and the program involved:

Creating a profile on PublicLab.org (if you don't already have one)
Joining the plots-spectrometry list (in the left-hand sidebar of this page!)
Posting unboxing photos & tweet them with @PublicLab & #OilTestingKit!
Building and document the beta kit and sample preparation process
Running triplicate spectra of each known and unknown sample (the kit includes five labeled and five unlabeled samples)
Uploading all of the spectra, tagged with "oil-testing-kit", add them to a set, and post a research note of their tests (more details on this to be posted soon)
Running a sample that you find locally. This can be anything from suspected motor oil residue on storm drain to the tar like substance you sometimes see by train tracks!
Attending two meet-ups to chat with other Beta Program members (times to be announced)

A fellowship was created to run the Beta program and analyze the results of the sets that were put on SpectralWorkbench.org. There were two main forms of communication set up for beta participants. On was through a closed email group where participants were BCCed, the other was bi weekly chats in the Public Lab chat room.

How did it play out?

Below is a timeline of the program as it played out:

7/14 Announced Beta program
7/22 First chat on Beta program was held before the announcement of who would be in it was made. This chat had 25 attendees.
8/5 Announced participants (20 selected out of 31 applicants)
9/14 The kits were shipped.
9/12 The status of the application for the Beta program was changed from “accepting current applicants” to “sign up if you’re interested in future testing.” 12 people applied to be apart of the beta program after participants were selected.
9/30 The Second Beta Chat was held, this was set after participants should have received the kits (8 people). The chat was run to address people’s questions on kit construction.
10/7 Third Beta Chat. This chat was set to answer people’s questions on Running Samples with the kit. Three attendees joined the chat.
10/14 Fourth Beta Chat was available to help answer any questions people had working with the kit. Three people were in attendance.
10/19 In the emails out to the Beta participants, we stop BCCing and directly added in people. The Fellow running the program sent out an encouraging email to participants to post research notes and start running sets.
10/22 Full beta program sets started coming in.
10/28 Fifth Beta Chat was held to answer questions and address problems people were having finishing the beta program. Two people attended this chat.
11/2 The Fellow running the program sent emails to those we hadn’t heard from.
12/7 OpenHour on the OTK (20 people, no beta participants)
12/11 Last outreach for group we didn’t hear from and outreach to new participants.
12/18 In person meetups and staff participated in the beta program to add 6 other full data sets.

While 9 of the 20 beta participants posted research notes about working with the kit (mainly on constructing the kit), as of January 2016, only four of the original beta participants completed the entire beta program.

Special Notes: It should be noted that between June and August, unforeseen staffing changes affected the timeline of kit production and shipping. Also, changes in a few of the materials put into the kit occured right before the kits were shipped which delayed shipping as well.

Oil Testing Kit Beta, as shipped

The Oil Testing Kit beta; photo by Cindy Regalado (@Cindy_ExCites)

Reflections and suggestions

Outcomes:

There was a lot of momentum around the Beta Program when it was first announced. A lot of people participated in the first chat and contributed great ideas to how they could foresee using the kit in their scientific exploration.

There was a lot of really good feedback about constructing the kit as people began the program.

There were also several good posts about people completing the beta program:

Several participants in the program took their exploration further than was originally outlined. Their contributions to working on oil testing have been extremely valuable in furthering the kit design and use.

Unknown samples from the oil testing beta, photo by @Cindy_ExCites

Communication

While the original idea to loop in beta participants on a separate email from the spectrometry google group was probably a good idea, moving forward it is best that these groups are not done with BCCs, but rather allow them to see everyone’s emails. It seemed to build a sense of responsibility in completing the program.

The OpenHour was well attended by people who were interested in hearing how they Beta program went, but was not attended by any Beta participants. This could be because it was a daytime call, but also, since so few people finished the program participating in a public call on it might have felt out of their reach.

Timelines and Deliverables

While a number of people completed part of the responsibilities of the beta program, very few finished all the tasks outlined. This could have been for a number of reasons:

We advertised the program as being “for anyone”, and required only that participants assemble, use and troubleshoot the kit (with a note to see the full list of to-dos on the wiki). The deliverables for those participating in the program ended up being a good bit more involved and took a lot of time to complete. Future beta program should probably include time estimates for participants. This could go so far as estimating time to complete: construction, any experimental aspects, posting back to the community, troubleshooting and participating in chats (or events such as OpenHour) on the tool.

We ran behind on shipping the kits which could have attributed to the waning participation we saw by individuals. In the future it could be a strategy to halt kit changes and development, order for the program and then announce it once the full kits are in the Portland office ready to ship.

While a number of Public Lab staff and the Fellow all worked on the Beta Program, there was not one person who was overseeing the entire program from shipping to results. In the future it would be useful to have one person manage the beta program as an overseer for the entirety of the program.

Another option moving forward would be to set up a different type of system when people get the kits to ensure more commitment from them, for example a “security deposit” to be refunded once they complete the project.

In general, we look forward to building on these lessons to develop a stronger co-design process with clearer short- and long-term goals, and entry points for potential collaborators. Toward this end, we've outlined a new workflow that addresses some of these issues; read on.

[cuvette frame](https://i.publiclab.org/system/images/photos/000/013/822/original/IMG_20160125_081251.jpg)

A prototype cuvette frame and UV light, based on the Oil Testing Kit Beta Program process

Open open hardware

One of the outcomes of the various community outreach and engagement efforts of the project was a new format for technology and methods development, which we are provisionally calling “Open open hardware” -- a reference to the fact that many “open hardware” projects are developed in private and only published openly upon completion. By contrast, the process we’ve proposed and begun to adopt is one where the goals include:

clear instructions for contributing to a design
low barrier to entry for new contributors
predictable revision timeline
a transparent roadmap for reviewing and integrating changes
regular iteration and feedback on proposed changes to help them get prepared for the next release due date
a "maintainer" for each project who will help coordinate contributions, as well as support and promote dialogue and transparency with contributors
a single, consistent, versioned, "baseline" design for the project, emphasizing simplicity & low cost, but upon which advanced mods may be made

While this does focus more on technical contributions, it is part of an attempt to blur the distinctions between technology and problem-focused contributions, and the barriers between contributors with different backgrounds, by establishing clear and open communication and expectations. You can read more about the Open open hardware process in the original announcement at https://publiclab.org/notes/warren/11-16-2015/an-open-open-hardware-development-cycle

We have begun to adopt this process for the Oil Testing Kit and hope to expand its use throughout Public Lab’s tools over the next year. View Public Lab’s hardware contributions page at https://publiclab.org/wiki/contributing-to-public-lab-hardware

Continue to Tools and Methods, part III of this document.

Public Lab Wiki documentation