Raspberry Pi, Solar Power, and Cellular IoT Communications - Lessons Learned on a Remote Island
"Never forget what you are. The rest of the world will not. Wear it like armor, and it can never be used to hurt you." — Tyrion Lannister
I work in a highly technical field, and after being in the field for more than 30 years, I have had many roles, but they have always involved in using software and hardware to solve problems. Over the years I have used my professional expertise for projects in my personal life, and from each of these projects I have come away with more knowledge than I brought. So, it was no surprise to anyone in my circle of family and friends when I volunteered to connect a remote island to the internet for my brother-in-law’s family. It has been a fun project that has helped the family with management of their island as well as leading me to give a presentation for technical conferences and the writing several blog posts, including this one for my employer, Marathon Consulting.
“We’re not a large house, but we are a proud one. And every man from Bear Island fights with the strength of 10 mainlanders.” - Lyanna Mormont
My brother-in-law’s family has owned a small island in Canada for the past 40 years. It was formerly a fishing camp and sold to the family as a Summer retreat. It has been a labor of love for the entire family over the years and for good reason; it is one of the most beautiful places in the world. I first started visiting the island about 20 years ago when my son (then 5) and I flew up and joined the group on the island for a week of fishing, projects and generally enjoying the serene beauty of the place. I have continued to visit the island over the years, and it is one of my favorite places.
Sunrise and a fisherman returning to the dock
The island is rustic, there are no commercial services to it, power is generated by solar panels or a generator, water is pumped from the surrounding bay, gasoline and liquid propane is carried to the island in a boat. When visiting, we bring all the food and take all the trash back with us. But over the years the family and friends have worked hard to make it livable and comfortable. There is a main cabin with the kitchen, dining and living rooms, several sleeping cabins with running water (some with toilets) and a bathhouse with toilet and shower. All the cabins have electricity.
The island from the air showing some of the cabins
"The man who passes the sentence should swing the sword." — Ned Stark
During a visit a few years ago, I mentioned to Oma and Opa (My brother-in-law’s parents) that it looked like the solar power system needed to be reconfigured or updated; and so, as is the way on the island, I was asked to diagnose the problem and see if it could be fixed. There were only two options; move the existing solar panels to a better location or purchase new, more powerful solar panels, mount them where they would not be obstructed by trees and add a sophisticated solar charge controller. The family decided on new panels and system since they would result in much more power for the island. The new equipment was delivered to the island and, with help from the vendor, we installed the system. The increase in power was immediate and significant. In Summer on a clear day, the old system could generate between 250 and 300 watts of power for about 2 hours a day; the new, between 1200 and 1300 watts for 6 to 7 hours per day. The new solar controller includes a feature that allows it to report the solar power statistics to the manufacturer via a data link. The manufacturer hosts a website that allows a user to review how the solar system and associated battery bank are performing. In order to utilize this capability, however, the system needed to be able to reach the internet so it could post the data. The amount of data is small and even with it reporting every 5 minutes, only uses about 500k bytes a day, or about 15 Mbytes a month. But getting this small amount of data off the island would take some engineering.
“The night is dark and full of terrors.” — Melisandre
To say the island is “off-grid” is an understatement, it is 16 miles from the nearest town and takes a 45-minute boat ride to get there. As mentioned, the gasoline and liquid propane is ferried to the island by boat. The LP gas runs the refrigerators, freezer and hot water heaters while the gasoline is used to run the generator when needed, as well as the boats. Communications on the island are helped with a cellular amplifier for cellular phones but being so far from the nearest cell tower (16 miles) causes problems. In order to be able to report the conditions of the solar system all year, we needed a consistent and reliable data link to the internet. I started pondering this problem almost as soon as I knew we would be installing the new solar system. After some research and testing, I came up with a simple system that would link the island to the internet.
"You know nothing, Jon Snow." — Ygritte
The initial was designed to allow the following devices to transmit data to/from the internet:
- Solar Charge Controller
- Wireless Camera (Solar Powered)
- Weather Station
- Raspberry Pi
Initial system component diagram
The system was built around a small USB cellular modem plugged into a Raspberry Pi. A mini Wi-Fi router connected to the cellular modem over Wi-Fi. The solar charge controller connected to the mini Wi-Fi router using an ethernet cable and the solar powered camera connected to the mini router using Wi-Fi. Power to the Raspberry Pi and the mini Wi-Fi router was supplied by a 24V to 5V power converter connected to a low voltage cut-off device to prevent the system from running the batteries to zero.
"Winter is coming." – Any member of the Stark Clan
When deployed, everything worked well, but as one or more of the components started to succumb to the -20 F weather in mid-winter, there was trouble. In late November, the system stopped sending data and we were unable to connect to the Raspberry Pi or the camera. I figured the system was dead, but in January, I was able to connect to the camera and get pictures and video back. A little while later I was able to get stunning pictures of the dock area after a snowfall. The scenes of the snow were beautiful, and the pictures from the island that the camera sent were the first that the family had ever seen of the island during Winter.
A shot of the docks area showing the snow and tracks of an unknown animal
About the same time, the solar charge controller resumed sending data. At this point, I knew that there was power to the Raspberry Pi because the cellular modem is powered by the USB port on the Pi, but I could not connect to the Pi. Just days before I lost contact with the system in November, a colleague of mine mentioned that I should program the watch-dog timer on the Pi so that if it ever locked up, the watch dog would reboot it. That was good advice not taken and brings us to our lessons.
“What we don’t know is what usually gets us killed.” – Little Finger
Lesson #1
When creating an unattended system, employ watchdogs or other techniques to force restarts on systems that lock up. Since there will be nobody to do a manual restart, this is an important feature that should be utilized.
If we had programmed the watchdog, the Raspberry Pi might have come back online.
Since the Raspberry Pi was locked up, and it runs a program that sends the weather data, the weather data stopped being sent too. Furthermore, if the Raspberry Pi had suffered a power-off failure, the cellular modem would have stopped operating, and that would have stopped all communications.
Lesson #2
When creating an unattended system utilize loosely coupled systems so that a failure in one does not impact the others. The failure of the Raspberry Pi caused the weather data to stop.
“When the snow falls and the white winds blow, the lone wolf dies, but the pack survives.” – Sansa Stark
On January 21st, 2019 there was a significant snowfall at the island and the solar panels were covered with enough snow that they no longer produced any power. The entire system was running on the stored energy in the battery pack. The available power in the battery pack is about 10kW. The solar charge controller, cellular modem, mini wi-fi router and Raspberry Pi used about 230 Watts per day, so the pack could power the devices with no solar input for about 43 days. It turns out that the system started producing power again on March 15th, 2019 or 52 days later. The batteries were just below the limit (24 V) to which they should be depleted when the panels started generating again. The batteries were rapidly charged back to full capacity after the snow melted off them. This chart below shows the time period when the panels were covered.
Battery volts and kilowatt hours during the snow cover
Lesson #3
Employ redundancy. Another set of panels positioned in such a way as to not get covered with snow would have allowed the system to continue operating indefinitely. An additional Raspberry Pi could have been used to access the Primary Raspberry Pi and possibly fix it.
"I read it in a book." — Samwell Tarly
When I designed the system to access the internet, I figured I knew more than the average person about cellular communications, I am a HAM radio operator after all. I was wrong. In the initial system I discounted the need for low loss, high quality antenna cables, a quality modem and the fact that cellular communication is designed for multiple antennas. I simply bought a directional antenna in the correct frequency ranges and deployed it with a thin cheap cable. Even though the cellular modem had 2 antenna connectors, I connected them together and ran it up to the single antenna. These were all decisions that impacted the quality of the link to the island. Even with all those mistakes, I was able to get a solid 3G connection to the internet most of the time.
Lesson #4
When working with antennas, antenna cables, and radios, quality components are worth the extra money.
“I’m a slow learner, it’s true. But I learn. Thank you for all your many lessons, Lord Baelish. I will never forget them.” – Sansa Stark
I believe I learned the lessons of the Winter of 2018/2019 well. During the following Summer and Fall, with the help of some of the people visiting the island, I was able to deploy several upgrades the system. They have proven themselves to be very effective. Each of the modifications were in part or to totally to address the lessons learned.
Current system component diagram
Hold the door!" — Wylis/Hodor
The new system centers around a Cellular Modem/Router by Cradlepoint. This device is industrial grade and designed for use in less than ideal conditions. I picked up an older model on eBay and it has been working very well since it was installed. This device is not fragile, it is intended for hard use. It is all metal with solid antenna and power connections, and includes an internal watchdog and a programmable automatic reboot. This device is a single point of failure, but utilizing a high-quality device designed for continuous use helps to mitigate that risk. In addition to that, if the proper license is purchased, the device sends data back to Cradlepoint’s NetCloud System so the user can logon to the network and see how the unit is performing, the quality of the connection, and even what devices are connected to the device and how much data they are using. The Cradlepoint NetCloud Interface looks like this:
Signal Quality Reported to NetCloud
Client Data Usage from NetCloud
Along with adding this device, we upgraded the antenna by adding an additional one and replacing the thin cables with serious low loss cables. This upgrade has allowed the island to have a high-speed LTE connection about 95% of the time. When not in LTE speeds, the unit will drop down to 4G and to 3G. We have occasional outages, but the connected time since October is 96.9%. When in LTE mode we typically get about 21.0/6.0 Mbps down/up to 13.0/3.0 Mbps. In the picture below we are on the low side but that may be due to the rainstorm that was occurring on the island at the time.
Raspberry Pi desktop using VNC and the results of a recent speed test to the island
When adding the Cradlepoint Modem/Router, I was able to get rid of the mini Wi-Fi router, which allowed better communication between the devices on the network.
I added a spare Raspberry Pi. This additional Pi has already saved the day. When the main Raspberry Pi failed to allow remote access with VNC, I was able to access it from the spare and get it back communicating. The spare Raspberry Pi also has a relay shield attached. In the future, I will be able to route the power cables for devices though these relays to allow me to do a power down reset to the powered devices when needed. Of course, the default for the relays will be to leave power on, in case that Raspberry Pi fails.
In order to mitigate the risk of the solar panels on the roof getting covered with snow again for a long time, I added a set of panels on the wall of the main cabin under the eaves and a separate charge controller. In this position they get sunshine on them, but they will not get covered with snow. The angle of the panels is not optimal, but they do not need to generate much power to keep the systems running the batteries charged after a heavy snow.
Lastly, to remove the dependence on the Raspberry Pi for the Weather Station, a new Weather Station has been purchased that will connect to the Wi-Fi network itself to push weather data, thus eliminating the dependence on the Raspberry Pi.
“A bruise is a lesson… and each lesson makes us better.” – Arya Stark
From learning about solar power systems to understanding some of the more sophisticated aspects of LTE Cellular communications this project has, and continues to be, a learning experience for me. I am deeply indebted to Oma and Opa Glaser and the entire Glaser family for supporting my project, purchasing the lion’s share of the equipment and having the vision to purchase this oasis of beauty and tranquility 40 years ago. Watching the sunset on Beaverstone Bay will never get old.
Gathering to watch the sunset