Toggl Blog

Moving to Go

During the last few months we have systematically improved both front-end user interface, and backend server code for the Toggl main time tracking page. It was motivated by the fact that we found it increasingly hard to cope with the steady growth of users and traffic, resulting in serious slowdowns of the system and in some cases even downtime.

The majority of Toggl program code originated from the time when our user base was 10x smaller, so the system needed a major overhaul. There were also several functional shortcomings, for example entries were not refreshed automatically when changes were made elsewhere (e.g. mobile phone). Also we had no offline support of any kind on the web.

Our situation in May this year

  • It took 5-7 seconds to load a time tracking page in Toggl, quite often even up to 20 seconds and more. Most of the time was spent by the server to compile the necessary dataset of time entries, but also related data.
  • The whole backend was based on Ruby on Rails, we used Ruby 1.8 at that time.
  • User interface could not be used offline.
  • Javascript code was bloated and contained a lot of unnecessary code (for example we had multiple date parsing/formatting libraries, etc).
  • The loading and parsing of Javascript started to block our UI because of its size.
  • Backend API calls were not optimized for the purpose, we requested too much data which strained both the database and bandwidth.

New implementation

After careful consideration we decided to re-use the offline-enabled time tracking code that is also used in our mobile apps and in Toggl Desktop, only retaining the visual of the existing time tracking page. So basically we decided to replace the whole backend code of that page.

We had run some experiments with the Go programming language (http://golang.org/) before, and decided that we should re-implement some parts of our backend with this new platform. So far, it has paid off, as the development process was fast, and deployment surprisingly simple. The resulting code is a big improvement in terms of speed. We’ll continue to replace our backend code with Go.

Secondly we implemented a Redis-backed (http://redis.io/) WebSocket server in Go to enable realtime synchronization between different Toggl clients – so your time entries, projects etc. would be updated automatically if you use Toggl on multiple devices.

Thirdly we added HTML5 manifest and local storage to support offline usage of Toggl. This served also as the speed enabler. Offline was already implemented with our mobile interface m.toggl.com, so we reused a lot of that.

In frontend Javascript code, we’re moving to Backbone.js (http://backbonejs.org/). As the amount of Javascript code is increasing fast, this library enables to structure it better. For parsing, formatting and manipulating dates in Javascript, we’ve moved to Moment.js (http://momentjs.com/) It has simplified our code a lot.

Following Google Page Speed (https://developers.google.com/speed/pagespeed/) tips, we’ve started to use a Javascript loader to reduce resource blocking. At the moment, we’re using LABjs (http://labjs.com/).

Another update we made was to upgrade to Ruby 1.9. The upgrade gave the system another speed boost.

Finally we spent time measuring HTML/CSS/JS load times and optimizing the milliseconds there. Our goal was to get the load time to under 1 second, or even faster. While the tracking page now loads faster, it’s still a work in progress as we’re doing too many API requests when loading the timer. Also, we’re still using Javascript libraries that are quite bloated – for example for the sidebar report charts.

We still like Ruby on Rails a lot, and at the moment, will continue using it for serving user interface. Go will be used together with PostgreSQL and Redis for backend data crunching and efficient API calls.

Incremental launch

These changes encompassed several risks, as there were a lot of potential critical bugs associated. That’s why we decided to roll the update out incrementally. We started in early July, and slowly and cautiously added new users until we had approximately 10% using it. This amount of users gave us enough feedback on system stability and bugs. After 4 weeks it had stabilized enough to start rolling it out more aggressively. By now all users have been converted to the new version.

As mentioned before, speed is an important feature in Toggl. Robustness and speed is something you, our users, keep telling us if we ask what are the most important things you need from Toggl. We have gained some valuable lessons with the latest upgrade, and will continue implementing those also in other parts of Toggl.

Alari Aho

CEO
Alari is the Father of Toggl and Teamweek. In fact, he used to ask us to address him as such. To his dismay nobody ever did. He's a good guy, though. For an engineer.
  • Show 34 Comments

    1. September 18, 2012 | Permalink

      Thanks for being so transparent. This openness buys you a lot of patience, from me at least.

    2. September 18, 2012 | Permalink

      Sure as a toggl user, it’s nice to hear that you guys are working on this stuff. But as a fellow web developer, one of the first places I look for ideas on how to make awesome web apps is toggl.com. Looks like I’ve now got a bunch of new libraries/technologies to check out now. Thanks!

    3. September 18, 2012 | Permalink

      Thanks for this post.
      I don’t know how much it is related to what I’m going to mention, but the tracking web page has been way below the usual level.
      Empty tasks being added as you load the page, NaN:NaN times, stuff that never happened before and differs from the smooth workflow that’s been customary.
      We at Niramekko are enthusiastic about Toggl and hope all the changes are successful.

    4. Hubert
      September 18, 2012 | Permalink

      Thanks for the detailed info, very much like it. And I agree, speed is paramount, take a look at google.

    5. September 18, 2012 | Permalink

      I don’t understand most of this stuff, but I love learning about what you are doing, and I appreciate you sharing the process of making a great product even better.

    6. Matt C
      September 18, 2012 | Permalink

      Neat! I like Go as a language, and it’s cool to see it spreading everywhere.

      Thanks also for mentioning moment.js, which I was not aware of, likely better than the js date library I’ve been using.

    7. Milton
      September 18, 2012 | Permalink

      I echo Gustavo Saiani’s comments. Thanks for the reassurance, but please just get Toggl back to its old glory.

    8. Alex Jamrozek
      September 18, 2012 | Permalink

      neato! I too am in the software development industry and reference toggl’s design for layout and feature implementation. I’m curious what type of ALM tools (VCS, Build&Deploy automation) you use.

    9. John H
      September 18, 2012 | Permalink

      Congratulations on having the balls to do this kind of restructuring. Most places I’ve worked would wet themselves at the thought of this kind of risk. I hope you land on your feet. And thank you for all the info on how you’re doing it. I, too, love the transparency.

    10. September 18, 2012 | Permalink

      Thanks for the post, it’s always great to know what’s going on behind the scenes.

    11. September 18, 2012 | Permalink

      Thanks for the info. I love Toggl and am encouraging the rest of our company to move over to it. I have been using it for about 3 months now and have had great success.

      We use backbone on most of our web projects for clients, but I have looked much into Redis. That sounds like it could be useful for some apps we are creating. Thanks.

    12. laoshe
      September 19, 2012 | Permalink

      Good!I like Toggl

    13. September 19, 2012 | Permalink

      I concur with the rest guys – I really like your openness with telling us what you are doing. Probably 98% of the time, I am trouble free w/ Toggl, however I’m having an issue right now that is new. On the main time logging page, the bar graph on the left is seeing M as Su, and T as M. Did you move to having M be the start of the week in that little graph?
      Thanks for all your hard work and a great tool!!!

    14. September 19, 2012 | Permalink

      Thanks so much for the detailed update! Your transparency wins you a lot of trust points with me, and hopefully others. As a developer, I can empathize too, and appreciate you providing the links to the new libraries and stuff you’re using.

    15. September 19, 2012 | Permalink

      It was interesting to see what languages you use for your software. I’ve never heard of Go before. I’ve also become a fan of Estonia because of you guys. Long Live Estonia! (where is it, by the way). I have never had problems with toggl. It’s so good, I feel guilty using it!

      • November 23, 2012 | Permalink

        Most powerful&cost efetcfive SEO and website traffic service in world get up to 100’000 forum backlinks now! Get amazing web traffic using superb backlink blast available. We are able post your marketing post up to 100’000 forums around the web, get thousands of backlinks and large targeted online web traffic in very short time. Most affordable and most powerful service for web traffic and backlinks in the world!!!! Your post will be published up to 100000 forums worldwide your website or blog will get instant traffic and massive increase in seo rankings just after few days or weeks so your site will get targeted long term traffic from search engines. Order now: backlink service

    16. September 19, 2012 | Permalink

      Good to know ^^. Thanks for the great job!

    17. September 19, 2012 | Permalink

      Here are more stories about experiences with moving to Go:
      https://gist.github.com/3731476

    18. Rob
      September 19, 2012 | Permalink

      I’m glad I use C. People like to make fun of me for doing so but I never have any such issues. Nothing is faster or smaller than the code my little company creates. There is nothing I can’t interface to at least more easily than other languages. There are no performance hurdles despite what anyone wants to believe. There is nothing I struggle with or can’t do. It’s fundamental to everything computing everywhere.

    19. September 19, 2012 | Permalink

      Your transparency moving things forward is just amazing. You guys take really seriously the idea of getting closer to the client, congratulations!

    20. David
      September 19, 2012 | Permalink

      care to explain why Go is faster than Ruby? what makes it faster? because of number crunching processing?

    21. Mark
      September 19, 2012 | Permalink

      Go has the potential to be faster because it compiles down to native machine code, like C and C++ do. Go is basically C with garbage collection, a nice modern standard library and some formatting tweaks. It will always be theoretically slower than C though because of the added garbage collection.

      Note I use “potential” and “theoretical” because real world use varies a lot. A poorly written Ruby/Python app can easily run faster than a poorly written C or Go one.

    22. Cody
      September 20, 2012 | Permalink

      I love Go and am glad to see toggl using it! I am transforming a lot of my restful services to use Go at work too!

      Keep us updated on how things perform at scale please, I’d love to see charts/comparisons.

      – Cody

    23. repla
      September 20, 2012 | Permalink

      thanks that was worth to read!

    24. Amanda
      September 24, 2012 | Permalink

      I absolutely LOVE your program. Keep up the good work and thanks for the update!

    25. September 24, 2012 | Permalink

      Thank you for being so frank about the performance challenges with Toggl and the steps you have taken to address it. This is refreshing in the software industry. Even more impressive, how this has virtually had no disruption to me using toggl everyday. Please continue with this approach and I will continue to support and let my clients know what great product Toggl is.

    26. September 25, 2012 | Permalink

      I would strongly recommend looking at using require.js with Backbone.js. They work really well together, remove namespace issues and also provide a great minification process (plus much more!). LMK if you have any questions or would like to see use-cases.

    27. September 25, 2012 | Permalink

      I am not a technical person, just a lawyer who finds Toggl to be a simple and inexpensive option for time tracking. I had been in an email conversation with a Toggl person about a problem I’ve been having with the system that has not been resolved, and I wonder if it’s related to this posting. I am no longer able to add a new client matter from the time tracking page. I need to go to the settings page, add the client, and then go back to the tracking page. If I add it on the tracking page, it simply disappears. Any information about this issue would be helpful – this problem is an irritation because it slows me down. Thanks!

    28. September 25, 2012 | Permalink

      Brilliant. Look forward to seeing it.

      On the topic of speed:
      I’ve noticed the sidebar charts sometimes take a while to load.

      If you made the chart display optional; allowing users to toggle (hehehe) the display on and off, that might save you some load time.

      I like the sidebar charts, but don’t always need to see them. There are many times I would trade charts for faster load times. Some users may not look at the charts at all. Might be worth a user poll.

      Anyhow, thanks for taking the time to communicate!
      Best of luck with your epic quest for speed.

    29. Toggl.com user
      September 25, 2012 | Permalink

      This presentation + transparency: quite professional, helpful. And am learning new tech (Go, Robis, etc) in process that might be used in some of my projects–especially good stuff. The back-end sync/speed/pageload across all client mechanisms (I use all of web, iPhone, and Desktop) is quite remarkable.

      What’s possibility of single-client updates “pushing refreshes” to other clients? (having to continually click the refresh button is sub-optimal…)

    30. Toggl.com user
      September 25, 2012 | Permalink

      er… s/Robis/Redis.io/g

      (sorry)

    31. Peter
      March 12, 2013 | Permalink

      I still have an issue with starting toggl on my Mac and stopping it on my iPhone.

      Invariably, when I come back to the Mac, toggl is still running… though if I check on http://www.toggl.com, the timer has stopped.

      Still love it though! :)

    32. Peter
      March 12, 2013 | Permalink

      Nope… just did a quick test.

      Started toggl on the Mac, then put it to sleep; stopped toggl on my iPhone.

      When I started my Mac back up, toggl was still running both on the desktop and at http://www.toggl.com.

      Any ideas?

    33. June 1, 2013 | Permalink

      I don’t know if it’s just me or if perhaps everybody else experiencing
      problems with your site. It looks like some of the written
      text within your posts are running off the screen.
      Can someone else please comment and let me know if this
      is happening to them too? This may be a problem with my internet browser
      because I’ve had this happen before. Cheers

    One Trackback

    • By Better Faster Stronger – The New Toggl API on March 8, 2013 at 2:48 pm

      […] months ago we wrote about moving to the Go programming language and replacing our Ruby on Rails backend. Step-by-step we have done that and now most of the ajax […]

  • Free time recording software with timesheet export.
    Check out Toggl