Above: The data logger shield (left) snaps onto an Arduino Nano (right).
[Update: This data logger is now available as a DIY kit which requires no soldering: http://kaptery.com/product/nano-logger-kit]
Digital logging of environmental data requires a device that can gather data from sensors and store it. Arduino microcontrollers are well suited for this task when a few components are added:
- Basic Arduinos cannot store much data, so a microSD card is usually added.
- Basic Arduinos do not have clocks which keep very good time and they do not remember the time when powered down. So a real time clock (RTC) should be added.
- Arduinos do not have environmental sensors, so these must be added.
- Arduinos need a source of power.
Above: The logging shield includes a slot for a microSD card.
Enterprising technologists have built custom printed circuit boards (Riffle, Mayfly logger) that combine an Arduino compatible chip with an RTC and a slot for a microSD card. These elegant solutions sell for many times the price of a basic Arduino (e.g., $60.00 compared to $6.00).
Above: The logging shield includes an RTC chip, a crystal to keep accurate time, and a backup battery so the clock runs when the Arduino is not powered. It's the same things that are inside a cheap digital watch. A clock is required so an accurate time is associated with each data logging event.
An inexpensive route to building a data logger is to buy individual components and solder them together. The Cave Pearl Logger is capable of logging in the field for more than year and is made from less than $10.00 worth of electronic components. It takes a few hours of careful soldering to make your own.
Above: The logging shield (top) has both male and female headers, so when the Arduino Nano (bottom) is plugged in all of the male pins are still exposed.
In search of an easier and less expensive route to using an Arduino to log data, I bought a few logging shields on eBay for $4.00 apiece. This gadget incorporates a microSD card slot and a RTC and allows an Arduino Nano to be attached without soldering. It also allows sensors and power to be connected without soldering.
Above: To connect sensors or other devices, Dupont wires can be slid onto the pins on the logging shield.
It was easy to make logging software run on this device because I had already figured out how to log data with an Arduino Nano and the individual components included with the logging shield. I had started with a simple version of Ed Mallon's sketch for his breadboarded Arduino Uno logger.
There was one very important difference between the breadboarded version and the Nano logging shield. The shield includes a different RTC than the one commonly sold as a breakout board or built in to data loggers (the DS3231 RTC). The logging shield uses a DS1307 RTC which is missing a key feature -- it does not have alarms. In order for field loggers to collect data for a long time, they must be put to sleep between logging events so the batteries don't run down. To wake up the logger, an alarm is set on the RTC. If the RTC does not do alarms, sleeping can't happen (at least the waking up part can't happen). At least this made the sketch simple because it did not need any of that confusing code about sleeping and waking up.
Above: The shield has a screw terminal for connecting power lines. Power lines can also be connected to the VIN and GND pins on the Nano, and the device can also be powered via the USB port on the Nano.
The lack of alarms would seem to be terminal flaw, but it probably makes very little difference because of another flaw. The Arduino Nano, like the Uno, is hard to put into a sleep that saves very much power (I think) because it has an UART chip (for the USB port) which is always on. So alarms would allow the SD card to be slept, but that saves far less power than is being burned by the UART chip all the time. I'm not too sure about the details, but this seems to be the general situation. The Nano is not the best Arduino to use if you want your logger to continue collecting data for a month or more on battery power.
Above: A BME280 sensor (right) for pressure, temperature, and humidity is connected with Dupont wires to the shield. The BME280 is an I2C sensor. To connect additional I2C sensors, some soldering is probably required.
I did a test deployment of the shield logging pressure, temperature, and humidity every five minutes. I was aware that the Nano was going to run the batteries down quickly, so I powered it with six AA batteries. I left it outside and checked in on it periodically to see if the red LEDs were still lit.
Above: This is the configuration for the test deployment with six AA batteries.
The device continued to log for longer than the breadboarded version, but only by a couple of hours longer. It logged very reliably for a little more than two days.
Above: The logger logged for 55.6 hours before the batteries got too low. It logged three variables (plus time stamps) every five minutes (655 logging events).
This logging shield, and any logger based on an Arduino Nano or Uno, is not going to log for very long on small batteries. But it is a very easy way to log data: load a sketch, plug in a few wires, and turn it on. So in certain situations, the shield can make data logging quite convenient because no soldering is required. This includes places where mains power is available (1. indoors, 2. close to a building, 3. close to any outlet) or where a solar charger or other alternative power source is available. It is also a good solution for remote logging when only a day or two of logging is required. In these situations, the convenience of swapping out sensors using easily plugged in wires can be an important benefit.
There is still one bug in this system that I can't catch. The RTC is not doing everything it is supposed to do. It gets the correct time from the computer when a sketch is loaded onto the Nano, and the sketch gets the current time from the RTC whenever the sketch asks for it. But when the logger is not powered, it does not keep time. This is one of the primary roles of the RTC and the reason it has a coin cell battery. When the logger is powered on again, the clock is reset to the original time the sketch was loaded, even if that was days ago. I have replicated this error many times using all three of the logging shields I bought and three different Nanos. I have also tested the coin cells and tried various other tests. It is possible that the logging shields are defective, but since all three of them do the same thing, it seems more likely that my sketch is causing the error.
There is one clue about this glitch that should be diagnostic. If I load the sketch onto a Nano at 10:00 AM and run the logger, the logger starts keeping time from 10:00 AM. If I restart the logger an hour later, the logger starts keeping time at 10:00 AM again. If I move that Nano to another logging shield, the logger starts keeping time at 10:00 AM, not the time when new logging shield had it's RTC set. So the Nano is determining the start time for the RTC even though it is the RTC that should be doing this. Somehow data stored on the Nano is being used to start the RTC's timekeeping. This really has me baffled. Here is the sketch I am using: Deek2bme.txt
Let me know if you have any ideas about how to make the RTC behave.
[UPDATE] I figured out what I was doing wrong with the RTC. See comment below.
[Update: This data logger is now available as a DIY kit which requires no soldering: http://kaptery.com/product/nano-logger-kit]
18 Comments
Combined with this i2C hub even if not so cheap for it's function = no extra soldering.
https://www.seeedstudio.com/Grove-I2C-Hub-p-851.html
Reply to this comment...
Log in to comment
That's a nice little I2C hub. Maybe it would be easy to make something like that for DuPont wires from strips of male header:
But somebody would end up doing some soldering.
Reply to this comment...
Log in to comment
I love that Grove sells premade wires of different lengths with those connectors on them. Are the pin "pitches" (isn't that what you call the spacing?) the same? I've definitely used some connectors that weren't originally designed for an Arduino just straight onto the pins -- two is easier than four, though, if the spacing isn't just right.
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
A nice thing about DuPont jumper wires is that each wire has a separate connector. So pin pitch does not matter (if it is at least 0.1 inch, the Arduino standard). And one end of your cable can connect to four adjacent pins while the other end connects to four non-adjacent pins (important for Arduino). The single connectors can be removed from the wire end, so you can make your own wires of any length with some (really cheap) crimp connectors.
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
I figured out what I was doing wrong with the real time clock. This line in a sketch:
RTC.adjust(DateTime((__DATE__), (__TIME__)));
will set the time on the RTC when the sketch is loaded from the Arduino IDE. This line does not have to be present in the sketch after the sketch is first loaded because the RTC will always keep time. If this line is present the next time the Arduino is powered on, and no sketch is loaded, the Arduino will somehow reset the RTC to the time of the previous sketch load. I don't know how the Arduino stores this old time, but it does. So this line must be removed from the sketch after the RTC time has been set.
Done.
Now I am ready for some indoor, or short-term, or extension-cord environmental data logging. For about $15, with no soldering, I can log pressure and temperature. For a few dollars more for each sensor, I can add humidity, light intensity, motion, orientation, compass direction, or infrared radiation. Other sensors cost a little more, and I have sent enough of my PayPal money to China for now.
I should be able to find something worthwhile to monitor.
Reply to this comment...
Log in to comment
Xose, Thanks for pointing to those Grove I2C hubs. I had been pondering how to fork the I2C line. Here is my attempt to make an I2C hub from male headers. I soldered stranded wire along one end of the pins and slid the plastic part down against the wire. Then I hot glued the four header sections together. Now I can connect five I2C sensors to the Nano logger.
But how can I make one of these without soldering?
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Amazing! I'll definitely give it a try ;)
Reply to this comment...
Log in to comment
That lack of alarm is pretty irritating considering that the 1337 has them built in, and they could have simply used that chip instead. Watchdog timers work ok for jumping through short sleeps, but you could also try setting the SQW frequency to 1Hz, and then use that output as an interrupt alarm on rising or falling. Every time you wake, increment a counter and compare it to the number of seconds you want to stay asleep sleep, and set a flag if you have reached that many wakeup/seconds.
My guess (?) is that you should be able to put the FT232 into some kind of USB suspend mode, or perhaps by pulling one of the pins low? Otherwise I would look at ch340 based nano clones to see if their uart goes into sleep modes more easily. There is a post with some other tips on the Arduino forum:
https://forum.arduino.cc/index.php?topic=418299.0
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Thanks Ed,
It's good to know there is hope for the Nano. That's an excellent thread you linked to. I might try removing the LED resistor and the regulator. Just removing the resistor reduces the sleep current by almost half. But trying to wake it up with the square wave could be beyond me.
I'm doubtful about powering it with a boost converter. I tried that with a Riffle and the sensor data was really noisy. I assume the noise came from the converter.
If only they had made that shield for a Pro Mini and used a better RTC. I'm surprised someone has not recognized the value of such a device.
Chris
Reply to this comment...
Log in to comment
I know that USB cables sometimes come with a barrel on them that's designed to reduce electrical interference, but I forget what it's called (I had one on my mp3 player that recharged through USB). Does that type of cable exist for the microUSB form factor?
If you wanted to power this from an electrical outlet, you could use an extension cord, a power to USB adaptor, and then a microUSB cable with one of those barrel attachments built in.
EDIT: The barrel is a ferrite core (https://superuser.com/questions/217772/what-is-that-cylinder-on-cables). Monoprice sells a microUSB cable with one for ~$4. Not sure if it would reduce interference enough in practice.
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
That would be terrific if a ferrite bead eliminated the noise I was seeing. My impression was that the bead was protection against radio frequency interference that was picked up by the cable (acting as an antenna). My other impression was that the noise I recorded with the Riffle was not picked up by the cable but was generated by the boost converter electronics and was micro variations in the current sent down the cable. Maybe the ferrite bead would filter that out too. But I don't really know anything about this topic so I should stop relating my impressions.
Reply to this comment...
Log in to comment
As cfastie noticed this line RTC.adjust(DateTime((DATE), (TIME))); reset the RTC to the same time every... time the setup is fired that's because the compiler substitute the compiling time into DATE and TIME just before compiling. You have to remove this line and possibly to add a system to set time because DS1307 may lose seconds every day or you have to switch to and from daylight saving time unless you use UTC.
Reply to this comment...
Log in to comment
Ah, I see. Those keywords, DATE and TIME, refer to the time the sketch was compiled and are saved on the Arduino until a new sketch is loaded.
That means that when that line of code sets the RTC time, it is already several seconds late because it takes several seconds to upload the sketch to the Arduino. So if you want more precise time on the RTC, you can do it manually from the serial monitor. Or maybe there is a less athletic way to do it.
Thanks zoomx,
Chris
Reply to this comment...
Log in to comment
good tutorial!!
Can I use HC-SR04 to data?
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Where I find the libraries : #include <SdFat.h> #include <SPI.h> #include <Wire.h> #include <RTClib.h>
#include <Adafruit_BME280.h> ???
Is this a question? Click here to post it to the Questions page.
The links are in the Arduino code - but I'll copy them here too!
https://github.com/greiman/SdFat/
https://github.com/adafruit/RTClib
https://github.com/adafruit/Adafruit_Sensor
https://learn.adafruit.com/adafruit-ccs811-air-quality-sensor/arduino-wiring-test
https://github.com/PaulStoffregen/SoftwareSerial
Reply to this comment...
Log in to comment
Hi, need help in using this module. i tried using this code Deek2bme.txt but my SD card says card failed. its a 32 GB samsung card formatted with fat32 and i placed a file named datalog.txt. but still the serial monitor says Card Failed. please tell me what am i doing wrong.
Hi. I had the same problem with sd-card. My PCB board was wrong.
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Login to comment.