Last November, we moved Toggl from our previous hosting provider to Google Cloud.
By mid 2016 we were aware of the growing need to switch the hosting provider. The home for our platform – Rackspace hosting – was not able to meet our demands and we were looking for an alternative.
What we were looking for
Our way how to deal with the infrastructure created several requirements we wanted to address:
- Scaling & Flexibility – with the predictable load we didn’t require massive scaling but rather wanted to be able to scale on our terms. One of the internal goals – not to use networked storage for the master database server(s) – effectively ruled out cloud instances at Rackspace and manual provisioning of the dedicated servers would seriously hurt us on flexibility level. We prefer to be able to quickly use host-based storage.
- Simplicity – keeping things simple is in our DNA. We want the same from our suppliers and the hosting provider should not create overhead for us. This goes all the way from the contract management, the provider-specific knowledge and the day to day operations.
- Cost optimization – we are not shy to pay for the quality service – but we want to have a fair deal. With the growing cost of the inter-data center data replication for the high availability/recovery purposes, we were looking to save on the bandwidth used for internal purposes. The plus point was to be able to pay in euros to avoid currency conversion fees.
- Transparency – facing the hard truth is always better than being kept in the dark. We looked into different providers from their track record in the communication of their failures, both in time to first response and detail of the information provided.
Choosing the right platform
Internally we looked into a number of providers, including (but not limited to) Amazon AWS (which is already used by our other product, the Teamweek planning tool), Softlayer, Microsoft Azure, and a number of VPS-based hosting solutions. At the end, we decided to try out Google Compute Engine (GCE).
The main motivation was the fact it met all our requirements, plus added couple benefits on top of it (good internal networking, tooling, simple pricing structure, and good sales and technical support).
Before we could start thinking about switching providers we had to extend BOFT (our in-house configuration management suite). Luckily there we did not face any major problems as the tooling is written in Bash and Ruby and the libraries we use (like fog ) have already solid GCE support.
Next step was to get accustomed to the Google Compute Engine itself. There was a lot to learn – inventory checking via API, key management, internal network characteristics, machine characteristics, etc. We started the practice with our off-shore database replica. It was an important component of the move, as we needed it for a promotion to the new master DB server.
With it safely moved, it was time to setup the backbone of our services – Consul which we use both for a service discovery and monitoring purposes. It allowed us to start re-building the full stack to get our API running. As the move was also an exercise to verify how well is our setup automated we also did several changes to improve the infrastructure in general.
Once we were comfortable with the new environment we started planning the switch itself. The easiest way was to shut down Toggl for a limited time, make everything read-only, restore the service at GCE and redirect the traffic from the old load-balancers to the new destination. We exchanged the temporary higher latency caused by the cross-data center traffic handling during the DNS switch to ease up the migration for us.
Now, almost six months after the migration we can’t be happier. But it’s only the beginning. We are slowly looking around to understand additional tools/technologies we can use Google Cloud.
The general view is that the whole transition was painless. So we can almost say we do at least something right at the way how we manage infrastructure at Toggl.
Are you interested to help us to take both product and the infrastructure to the next level? We are always looking for good people.