It might be good to agree on definitions of some basic terms. Here is one suggestion.
1. Reproducibility, Reproducing the results of an experiment:
Repeating an experiment and getting a result that leads to the same conclusion. More technically, using the same experimental procedure to test the same hypothesis and coming to the same conclusion about the hypothesis (confirming or rejecting it). Also performing a different, related experiment that produces a result consistent with the conclusion of the original experiment.
This can also be applied to doing something that does not appear to be an experiment. For example, if someone uses a mercury thermometer to measure the temperature of water in an ice bath and gets a result of 32.1 ± 0.8°F (n= 10 measurements), this result could be reproduced using a Riffle and DHT sensor. If the Riffle results are 32.4 ± 1.1°F (n= 10 measurements), then the result has been reproduced (i.e., there is no statistical difference between those two results). Although this appears to be just two measurements and not an experiment, it could be done so that all of the requirements of an experiment are fulfilled:
- A stated hypothesis (e.g., the measurement of ice water temperature is not different from 32°F)
- A procedure appropriate for the system (e.g., lots of ice and water)
- A number of replicates (multiple measurements) which is appropriate to describe the variability of the device (the thermometer) and the parameter (the water temperature).
- An appropriate statistical test.
This would be a very simple experiment, but it is nonetheless an experiment. Therefore, its result should be reproducible. In this sense, the results of simple observations or measurements can be reproduced as long as the series of observations or measurements meet the above requirements and are therefore bonifide experiments.
2. Replication, Replicates:
Multiple units of study (samples, trials, measurements, study plots, days, populations, etc.) which are required to account for the different types of variability in the subject of study and in the method of study.
These replicates (or replicate samples, replicate measurements, etc.) must be collected under the same conditions and in the same way. This type of replication is the basis for all statistical analysis because multiple data points allow the variability in some parameter to be quantified.
Replication must be done at multiple levels depending on the question being asked (i.e., on the hypothesis being tested). For example, if asking the question Do these two air samples differ in the amount of suspended silica? then a lab procedure could be done on five replicate subsamples from each air sample. However, if the question is Do silica mines pollute the air? then the experiment might require collecting 10 replicate air samples at each of 10 replicate sites near each of 10 replicate mines and also 10 replicate control locations on 10 different replicate days, and then running five replicate lab analyses on each sample. The number of replicates required at each level depends on how much the measured parameter varies at that level and is often not known until the samples are measured.
3. Repetition, Repeating a procedure:
Doing something multiple times.
When there is no way to test whether the outcome of repeating a procedure is the same every time it is done, then the concept of reproducibility does not apply.
Building and modifying devices
Following someone else’s instructions to build or modify a device is not reproducing a result unless a test can be done to determine if some predetermined specifications have been met. If such a test is available, then building or modifying a device can become part of an experiment. If the above requirements of an experiment are met, then the test can be used to determine if the build or modification has reproduced the results of the original. Multiple builds might be required because each would be a replicate in the experiment.
In most cases, the requirements of an experiment are not met when people follow instructions to build something, often because there are too many variables to control. However, anything can be part of an experiment if careful planning is done and replication is sufficient to account for the inherent variability.
If five people each use their Riffle to monitor the water temperature in a stream near their house, this is repetition. It might be difficult to argue that these activities are part of a single experiment or are replicates or are reproducing a result. It’s just five people more or less repeating the same activity. Additional measurements, restrictions, replications, or controls could allow this type of activity to be part of an experiment, but monitoring by itself is not an experiment and often the results cannot be reproduced (environmental variation can make this difficult).