Developing PythonAnywhere on PythonAnywhere
Yo dawg, I heard you like dogfood…
https://en.wikipedia.org/wiki/Eating_your_own_dog_food
As we’re developing an online IDE, it made a lot of sense to us that we should “dogfood” as much as possible, so we’re actually using PythonAnywhere to develop PythonAnywhere. When it’s come up in conversation, people are occasionally surprised or impressed, and they seem to think we should make a bigger deal out of it, so here you go!
It’s an evolving process, but here’s an outline of how we work now, and some of our hopes for the future.
Servers, servers, servers…
At the heart of it is a cluster of “dogfood” servers that are essentially a clone of our production environment, with a few minor differences, and we use the ‘dogfood’ version of PythonAnywhere to do almost all our development: we can use bash consoles to SSH into our dev servers, and reboot apache for example. We use the browser-based editor, or maybe vim from another bash console, to make changes to the source code, and that ends up on the servers because they have it mounted via NFS – that’s the only bit you can’t currently do on the live production PythonAnywhere.
At the moment we still tend to run our functional (selenium) tests from a local PC, but even that can start to change now that we’ve released headless selenium testing support. So we’re really very close to doing all our development for PythonAnywhere, entirely on PythonAnywhere.
A day in the life…
Here’s how a typical day might go.
I can start by logging into my user account on pythonanywhere-dogfood – they’re normal user accounts, no admin access or anything special. I open up a bash console, and checkout the main pythonanywhere repo from github.
The repo contains everything I need to build a dev server from scratch. We recently switched from running our dev servers on VMs on a box under one of our desk, to running them as amazon EC2 micro instances. So, in the same bash console, I can run a single command, ./create-and-start-servers.py dev all, and it will spin up a PythonAnywhere dev cluster on amazon. We use fabric to automate the building and config of each server, and we save disk images with all the libraries we need on them.
Here’s one bit where we cheat: Our dogfood server allows sharing over NFS, so each of my dev servers actually has my source tree on the dogfood server mounted, to use as its own codebase.
Once my server is up and running, I can use a separate checkout of the source tree on my local PC to write functional tests, and run those FTs against my dev server. Since we do pair programming, by then I’ve recruited a colleague to help me out (we have a rota, so no-one is forced to work with me every single day. there is the Geneva convention to think about…).
Once we’ve written an FT and seen it run from our local PC against the dev servers, we can switch back to dogfood to start writing our unit tests. We’ve got several vim addicts in the office, so that will probably happen in a console session, or we might use the browser-based editor.
To run the unit tests, we’ll have an SSH session logged into the dev server, which can run a ./manage.py test, for example. Then we can get on with writing the actual code. The NFS share is a convenience here, it means that our edits are immediately reflected on the server, so we can see the effect of code changes straight away – but we could use…
When we’re happy all our tests pass, we can commit them, and push them back up to github, where they’ll be picked up in the next run of our integration loop. And then it’s off to the pub!
In summary:
| On PythonAnwhere | On local PCs (or cheating) |
|---|---|
| All source code editing (either in the editor or console vim) | Running functional tests |
| DVCS – git commits, pull/push to github | pushing changes to dev environments |
| VM control: spin up EC2 instances, reboot apache etc | project management (we use trac) |
| unit testing | |
| deploying new versions to live |
What works well, what we want to make work better:
We started to move to more and more dogfooding over the last two months, and it’s interesting to look at what’s been good and bad about that.
On the plus side, we get all the benefits of having a cloud-based dev environment. IRL, that translates into things like making it easier to work from home – I was off sick the other day, and being able to access the full dev environment from my home PC made it possible for me to get some real work done… with no need for any complicated VPN antics. I could also pick up from exactly where I left off, with my vim and bash shells just where I’d left them the previous day.
It also really helps for weekend support. We like to make sure we pair up for any kind of serious support task, so having a shared console on pythonanywhere, alongside a skype call, means it’s easy for two of us to dive into the servers, test out fixes and so on, even when we’re not in the room together.
In the office, having access to each others consoles means we don’t have to keep hopping between desks too. It’s only a minor thing, but it helps… And, before you ask, we do know about screen, but screen is far from hassle-free. Again, just yesterday, Giles was unable to log into a screen session I’d left running for him on one of our legacy servers because of a weird permissions issue – pythonanywhere doesn’t give us those sorts of annoyances.
Which is not to say PythonAnywhere doesn’t give us other annoyances!
The hope, with dogfooding, is that if you’re forced to use the same tool as your customers, you’ll get round to fixing irritations more quickly! So here’s what’s been frustrating us and is therefore getting pushed up our to-do list:
- better copy & pasting inside consoles. Although the Ctrl+Shift+V solution is a reasonable workaround in chrome, it really needs to be a lot better, work in all browsers, and support multi-line pastes
- improve console responsiveness. We’re forcing ourselves to use servers in the US, so there’s a fair bit of latency, and that means typing into consoles is not as pleasant as it should be. We’re looking into spreading more servers into more different places around the world, but also into whether some kind of local keystroke echo might help
- improve the browser-based editor. It’s got the basics, but it needs more, things like multi-file find & replace, and some kind of scripting ability
Aside from fixing those bugs, we want to start running selenium tests directly from PythonAnywere, if we can. The trouble with this is that it’s harder to debug when you can’t put a “time.sleep” and see what’s happening in the real browser window… but a judicious use of screenshots could potentially work just as well… We also want to stop “cheating” by using an NFS mount, and develop PythonAnywhere on the production PythonAnywhere servers – maybe some judicious use of github callbacks, or even Dropbox…
But, in the meantime, things are working pretty well… Of course, the danger with dogfooding is that you get so used to your tool, and especially you get used to workarounds for little bugs, that you forget what it’s like for newer users. So, beloved users, if you’re using PythonAnywhere for serious development, what’s your experience? What do you think we need to change?
Platform, PythonAnywhere | harry | 
27 April 2012 11:51 am
Subscribe
By Adriano, 27 April 2012 @ 8:20 pm
Having some of your Amazon instances at US West (Northern California) would definitely give a stronger impression of paw to the coding community around Silicon Valley. Tens of milliseconds after each command at the console really add up after awhile :-)
By harry, 30 April 2012 @ 11:53 am
pff, as if there were are any developers in Northern California!
we are looking into what we can do in terms of having more datacenters, or just improving console performance in general…