On the back-end, we run a slew of servers. When it comes to our servers, there was a bit of a grand plan when we started, but in reality they all came about very organically. As we needed to solve new problems and fill voids, we built new servers into the architecture. As it stands right now, we have the following:
- Front-end web servers, which serve most of our pages and community content;
- API & Design servers, which do most of the heavy lifting and allow for collaboration;
- DB servers, which hold the datums; and
- Background workers, which handle our background processing and batch jobs.
So let’s talk tech…
- We use a lot of Linux (ub) (arch), both on our development workstations and all over our servers.
- We use nginx (http://nginx.org/) for our reverse proxy, load balancing and SSL termination.
- We use Flask (http://flask.pocoo.org/) (which is a like Sinatra) for our Community and Front-end web servers. We started with Django, but it was just too full blown and we found ourselves rewriting it enough that it made sense to step a rung lower.
- We use Tornado () for our API and design servers. We chose Tornado because it is amazingly good at serving these type of requests at break neck speed.
- We do our internal communication through ZMQ (www.zeromq.org) on top of Google Protocol Buffers
- We used MySQL () for both relational and KV data through a set of abstracted custom datastore procedures until very recently, when we switched our KV data over to Kyoto Tycoon ().
- The client communicates with the servers via long polling through custom built RPC functions and server-side protocol buffers.
- We draw the user interface with HTML5 and canvas (), through a custom drawing library which handles collisions and does damage based redrawing.
- And we use soy templates for all of our DOM UI dialogs, prompts, pop-ups, etc.
- We host on EC2 and handle our deployment through puppet master ().
- Monitoring is done through a collection of OpsView/nagios, PingDom and Collectd.
Our development environment is very much a point of pride for us. We have a spent a lot of time making it possible for us to do some of the things we are trying to do from both the client and server sides and putting together a dev environment that allows our team to work efficiently within our architecture. We value testing, and we are fascists about clean and maintainable code.
- We use git (obviously).
- We have python unit tests built on top of nose ().
- We run closure linting () and compiling set to the “CODE FACIEST” mode
- We run a full suite of checks within buildbot () on every push to master
- We also do code reviews on every push using Rietveld ().
- We are 4-3-1 VIM vs. Text Edit vs. Text Mate.
- We are 4-2-2 Linux vs. OSX vs. Windows 7.
- We are 5-2-1 Android vs. iPhone vs. dumb phone.
If any of this sounds like we are on the right path, you should drop us a line. We are in Toronto, we’re solving very real-world, wicked problems, and we’re always hiring smart developers.