With the recent revelations about the Obama administration’s ongoing surveillance program many folks are wondering about where to store their data. The near certainty that tech firms are complicit with both friendly and unfriendly governments raises questions for all of us.
Here is Ted’s dictum: Big Data + Big Government = Big Problems.
Many of us have used Dropbox or Google Drive to share files across computers. There are great services but each has its own sketchy security past. For those of who have some tech savvy there are solutions which allow you keep your files on your own machines. I went to this not long ago and am loving it. I guess it is one small way to get a little more “off the grid” than I would without it.
In my case, I have my own server, which sits in my house, and I have access from all of my devices (desktop computers, laptop, phone, and tablet) with relative ease. Like Dropbox, my system has version control, automatic updates when a file or folder is updated or changed, and automated backups. In this document, I am going to tell you how I set it up in general terms. My system is not going to be exactly like yours so I can’t give you a keystroke-by-keystroke instruction set.
I have a number of different systems in place to provide this homespun Dropbox service. They are:
- A hacked router that gives me VPN capability. You don’t have to have VPN access to do this, but it makes me feel better.
- An IP forwarding service (aka “dynamic DNS”) to make my router available to me from anywhere I connect to the Internet.
- An old computer running linux. I use Ubuntu desktop, 12.04 but you could use just about any version of linux that runs SparkleShare.
- SparkleShare – an open source program that replaces Dropbox. I run this on a number of different computers.
- Unison – another free program that syncs files. This is for a different sort of backup / file sharing system.
What follows is a bit more detail on each component.
Note that I don’t consider this a “secure solution.” I don’t consider anything a “secure solution.” I take all the steps I can to make things secure but each component has its own vulnerabilities. I think it’s more secure than a commercial solution, though.
1. The Router
I use a Linksys router (WRT54G) which is an older router. The beauty of this router is that you can replace the stock software on it with something a bit more powerful. I use DDR-WRT. Your first step is to get a router that will run DDR-WRT or, as an alternative, Tomato will work.
I install DDR-WRT and then setup the VPN solution (PPTP). You can also setup OpenSSL on this router. I have them both working but I tend to find PPTP a bit easier to use. Note that PPTP has been cracked and I doubt that it’s terribly secure. Keep in mind that you can use SSH within the PPTP tunnel, though, so you have an extra layer of security if you do that.
I also use the router’s “NAT” capabilities to route incoming traffic to my computer.
2. An IP Forwarding Service
To find your router while you are out on road requires an IP forwarding service (also called a dynamic DNS service or DDNS). You won’t need this if you pay the big bucks for a dedicated IP but I don’t do that. You can choose any service you want. A common service is www.noip.com. Set it up and get your router – DDR-WRT has direct support for a number of different services.
3. A Linux Box
You don’t have to have to use linux, but why not? It’s free, pretty secure, and allows you to use an older computer. I have an old machine that was being tossed by somebody. It had 2 Gb of RAM, which is sufficient to run Ubuntu. I put in a pair of 1 terabyte drives and have them mirror each other. The hard drives are the biggest expense of this setup.
I setup a static IP address so that the router can be directed to send all traffic to it that is on certain ports (SSH, FTP, SFTP, a port for VNC).
You must install git on the computer or at least be able to create a git repository on the computer (via telnet or SSH).
Setup a git repository according to the directions on the SparkleShare.org website.
Now install SparkleShare on your other computers. I also run a copy of the SparkleShare client on the box so that it always has a set of files on it – otherwise the files would only exist within the git repository and I want them backed up on my mirrored hard drive.
SparkleShare is great for Dropbox style usage. You can drop files into the folder and automagically all the computers you setup get that file. I don’t recommend it for really huge repositories, though. There is quite a bit of overhead since the files are stored in a git repository. This makes it powerful (you can roll back to previous versions and all of the git commands work on the SparkleShare repository) but it also makes it a bit heavy on disk space. For this reason I would not store things like photos or video in a SparkleShare repository.
This is where Unison comes in. I have a folder of Archived information. This is stuff that changes less often but I want to hang onto. I have a copy on my main desktop and I like to have a copy on the Ubuntu machine (again, it’s a mirrored backup system). The hardest part about Unison is that you must have the exact same version number running on all machines. I had to downgrade my linux version to match the OS X version I run on my macs.
You will need to setup SSH keys to get Unison to work securely over the Internet.
I have Unison cron job that runs every night (it uses a shell script to kick off Unison with some parameters). Unison places anything I add to the large archive on my desktop into the archive each evening. For my laptop (a MacBook Air with only 250 Gb of SSD space) I don’t want the archive. If I need something from it I ftp into the Ubuntu server and get it. If I stick something onto the Ubuntu Archive, the desktop computer will get it that night when the sync happens.
By the way, the desktop computer sits in my office, a few miles away. Physical separation between these machines adds another layer of protection.
Some people like to use rsync instead of Unison but I don’t know if that’s better or not. It could be investigated if you can’t Unison to work.
Other Types of Access
Sometimes I like to check on the Ubuntu box while I am away. I use a VNC app for that (I like CyberDuck on OS X).
I also can get to the files using my phone. I have tried to compile the SparkleShare client for my Android phone but it’s not a great solution (it requires the SparkleShare dashboard which is out of date and impossible to setup at this point). Instead, I just ftp into the Ubuntu machine. I find this to be very fast and I can run it over the VPN and use SFTP to make sure the connection is “more secure.”
Of course I can also SSH into the Ubuntu machine if I want to. I have used SSH on the router as well but only to clear out caches that were eating up memory.
I also have a small “network attached storage” device. I have a backup going to this device as well just in case something goes wrong with the Ubuntu machine. This happens via a cron job that kicks off weekly and uses Unison.
All of the files are stored with the convenience of a cloud service that I own and operate. I have backups going in an automated way and have version control on the SparkleShare repository.
It probably sounds very complicated but it has evolved over time and thus seems pretty easy to me.