Luke said it mostly happened in bright sunlight, but also indoors. I just did a test in (admittedly cloudy) sunlight vs. closing the blinds, and covering the device with a thick blanket, and didn't see a big difference. It doesn't seem likely to me that light is leaking in through this big dark folded-over blanket. How does this compare to your noise, @lukebrown? Is it possible you have a small light leak as opposed to it going through the paper itself? I'm just trying to reproduce the issue first.
This has had 10 pixel rows of data smoothed to try to reduce noise: