Running Sandstorm on Google Compute Engine

I've become intrigued with Sandstorm lately. The way I like to think of it, is a "Mobile OS" for your cloud server. Imagine if running a server was as easy as using your phone. Want a new app? Click a button. No installing and configuring databases or web servers. No fiddling with startup scripts or cli options. It just works. You can run any number of Sandstorm apps and not worry about things like conflicting version dependencies. The magic is based on containerization technology built into the Linux kernel. The same tech that is very much abuzz in other contexts, but it's not too important how it works, just that it does.

But, why do I need a server?

The thought of who owns your data is just barely starting to become a thing. There were always the conspiracy theorists, but it's grown a bit from there. Apps like GitLab and ownCloud are gaining interest. Then there are the more traditional reasons, hosting your own blog or site. Bitnami has done a great job making it really easy to get started with one of the above. But that is just it, only one at a time. Sandstorm lets you decouple the server from the app, You just need a Sandstorm server, and you can add or remove apps as you please.

So, how do I get one?

In the spirit of owning your data, Sandstorm (the project) is open source (Apache 2), and a primary goal is ensuring you can run it yourself on any server you wish. That said, Sandstorm (the company) will offer a hosting service in the future, letting you get started with minimal hassle. You can preorder now to get in on the beta, which I have. However, right now, you'll need to roll your own.

Google Compute Engine

It makes sense to try to find the cheapest server to run Sandstorm that meets a minimum performance expectation. I work on Google Cloud Platform, so GCE was my obvious choice, but I wanted to find the optimal solution. Yes, this is my personal Sandstorm server that I fund, and that this blog is hosted on. Looking at the smallest types, I saw:

Type CPU Memory Price/Month
f1-micro Shared 600MB $4.03
g1-small 1/2 core 1.7GB $13.68
n1-standard-1 1 core 3.75GB $25.20

The micro instance is by far the cheapest, so I gave it a shot. Using one app at a time was not a problem. I should mention here, that an app instance in Sandstorm is called a "grain," and if you have multiple documents for a single app, they are separate grains, and run in separate instances of the app. Having more than one grain open at the same time would sometimes not work, usually if the app used mysql. =) The second grain I tried to open would just not load, and I would never see mysql start in top. Why? 600MB was just not enough. Otherwise, the micro instance seemed good. Yes, some apps were a bit slow to load, but the tradeoff seems worth it to me for the price.

Swap

Swap is not set up by default in GCE. It would usually be very bad if your cloud server started swapping due to the extreme difference in performance. For example, a 100GB persistent disk only reaches 30 read and 150 write IOPS. However, I thought swap space might be the right answer for me. The small instance is $9.65 more a month and still only comes with 1.7 GB of RAM. I've heard chatter on IRC that that might still not be sufficient for 2 or 3 grains.

The clear problem is the disk performance. But, how about SSDs? GCE has remote SSDs available for as small as 10GB, which reach 300 read and write IOPS, per the calculations. This would only add $1.70 a month. I gave it a shot, and it seems to work ok! So, this is what I'm running now.

Total Cost

With the configuration I have, running your own server with the current pricing will cost:

Item Price/Month
f1-micro $4.03
100GB Disk $4.00
10GB SSD $1.70
Total $9.73

Setup

To get started, follow the setup instructions here. Get your GCP project setup with billing. Install the gcloud cli tool and authenticate and set your project.

We'll set the zone to us-central1-c, but use whichever you want.

$ gcloud config set compute/zone us-central1-c
$ gcloud config set compute/region us-central1

Next, create the instance, and open the Sandstorm port.

$ gcloud compute addresses create sandstorm
$ gcloud compute instances create my-sandstorm --image ubuntu-15-04 --address sandstorm --boot-disk-size 100GB --machine-type f1-micro
$ gcloud compute firewall-rules create all-6080 --allow tcp:6080

Create the SSD, attach the disk to the instance and give it the name swap

$ gcloud compute disks create swap --type pd-ssd --size 10GB
$ gcloud compute instances attach-disk my-sandstorm --disk swap --device-name swap

Now ssh into your machine. Run the rest of the commands on the machine.

$ gcloud compute ssh my-sandstorm

Partition the ssd, remember we named it swap

$ sudo fdisk /dev/disk/by-id/google-swap

Create a single partition by pressing n <enter> <enter> <enter> <enter> t 82 w

jeffmendoza@my-sandstorm:~$ sudo fdisk /dev/disk/by-id/google-swap

Welcome to fdisk (util-linux 2.25.2).  
Changes will remain in memory only, until you decide to write them.  
Be careful before using the write command.

Device does not contain a recognized partition table.  
Created a new DOS disklabel with disk identifier 0x32f1b55c.

Command (m for help): n  
Partition type  
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): 

Using default response p.  
Partition number (1-4, default 1):  
First sector (2048-20971519, default 2048):  
Last sector, +sectors or +size{K,M,G,T,P} (2048-20971519, default 20971519): 

Created a new partition 1 of type 'Linux' and of size 10 GiB.

Command (m for help): t  
Selected partition 1  
Hex code (type L to list all codes): 82  
Changed type of partition 'Linux' to 'Linux swap / Solaris'.

Command (m for help): w  
The partition table has been altered.  
Calling ioctl() to re-read partition table.  
Syncing disks.

jeffmendoza@my-sandstorm:~$  

Edit /etc/fstab and add the below line

/dev/disk/by-id/google-swap-part1 swap swap defaults 0 0

Go ahead and enable it now

$ sudo mkswap /dev/disk/by-id/google-swap-part1
$ sudo swapon /dev/disk/by-id/google-swap-part1

That is it for the GCE setup. Now go ahead and install sandstorm like you would on any server.

Persistent Disk Size

Firstly, as we saw with the SSD, the disk performance is a function of the disk size. In the calculations above, I'm only using 100GB as that is what I'm running. This is mostly due to size and cost for me, not performance.

Apps for music and pictures are great on Sandstorm, but then you need to size your disk appropriately for your music and picture collection. This is quite tough, as you should expect these to grow, but you don't want to pay for a huge disk that you are not using. If you outgrow your server, I see two options

  1. Resize your disk. I might tackle this with a later post.
  2. Migrate to a new server. All your sandstorm grains can easily be saved and uploaded to a new server.

Cleanup

Migrating to the beta, or somewhere else?

$ gcloud compute instances delete my-sandstorm
$ gcloud compute addresses delete sandstorm
$ gcloud compute disks delete swap
$ gcloud compute firewall-rules delete all-6080

I hope this is helpful. Find me on IRC in #sandstorm on freenode.