This post originated from an RSS feed registered with Java Buzz
by Bill de hÓra.
Original Post: Switching from TextDrive to Rackspace and Linode
Feed Title: Bill de hÓra
Feed URL: http://www.dehora.net/journal/atom.xml
Feed Description: FD85 1117 1888 1681 7689 B5DF E696 885C 20D8 21F8
Some notes and observations on moving hosting providers.
Why?
Well, Joyent said August last year that people on lifetime deals from TextDrive hosting, of whom I'm one, would come to be no longer be supported. Then they changed their minds somewhat, and their deadlines. That had me looking around. Not because Joyent don't want to support years old BSD servers rusting in some cage somewhere, I have a realistic definition of lifetime when it comes to commercial offerings. Over six years I've been using TextDrive it has been a great deal, all things considered, so no complaints there.
Then TextDrive was restarted and said they would honor the accounts. That's beyond admirable, but communication has been less than ideal - simply figuring out what to do to get migrated and what I'd be migrating to has been too difficult (for me). The new TextDrive situation should work itself out in time - people are working hard and trying to do right thing. But I wanted something with more certainty and clarity.
Aside from incidentals like old mercurial/subversion repositories and files, the two main services to move were this weblog and email.
Email
Email was hosted on a server called chilco, and had been creaking for some time. IMAP access had slowed down noticeably in the last 12 months with increasing instances of not being to connect to the server for a period of time. That the server certs haven't been updated is a constant irritation. I decided I'm past the point of running my own mail server and it was time to pay for a service, although Citadel was tempting. After whittling it down to Google and Rackspace, I went with Rackspace Email, for four reasons - pricing, solid IMAP capability, just email and not an office bundle, and their support. It probably helped that I don't use GMail much.
Setting up mail with Rackspace was quick. DNS Made Easy lost me when I couldn't see my settings after my account expired, so I moved DNS management over to Rackspace as well and routed the MX records to their domain. Rackspace have an option to auto-migrate email from existing servers. This didn't work, I strongly suspect the certificates being out of date on the old server were the cause. The answer was to copy over mail from one account to another via a mail client. That wasn't much fun, but it worked.
Rackspace webmail and settings are proving very usable. IMAP access is fast. The sync option via an Exchange server is good. There's plenty of quota. Rackspace support has been excellent so far, they clearly do this for a living. I'm much happier with this setup.
Cost: $2 per email account per month, $1 per sync account per month.
Linode
I wanted to avoid a shared host/jailed setup. There's a lot to be said for running on bare metal, but virtualization for hosting at personal scale makes more sense to me. I considered AWS and Rackspace but the pricing didn't suit, mainly due to the same elastic pay as go model that works for commercial setups. Linode offered a predictable model for personal use. Other VPS/Cloud providers were either more expensive and/or lacked surrounding documentation. Plus I got a nod from someone I trust about Linode which settled it.
In hindsight I should have done this years ago. Getting an Ubuntu 12.04 LTS set up was easy. Payment was easy as was understanding the utilization model. Using the online admin is easy - the green and gray look really ties the site together. Linode instances have been responsive from the shell, more so than the previous BSD hosted option I was using. Having full access to the server instance is great as it avoids workarounds that come with shared hosting (I had an 'etc' folder in my home directory for example).
Moving to Linode meant moving this weblog, a custom engine I wrote in 2006/7 based on Django and MySQL,. The last major work here was a pre 1.0.3 Django update in late 2008 that had trivial patches to get TinyMCE to work in the admin. It did occur to me to move to a non-custom engine like Wordpress, but that would involve a data migration. And I figured having originally wrote it to learn something, fixing it up four years on might teach also teach me something.
I upgraded Django to 1.4.3, breaking one of my cardinal rule of migrations - migrate don't modify. In my defense I had started the upgrade process on the old server, but due to the uncertainty with Joyent/TextDrive and my discovering there was a December 31st 'deadline' for shutting off the old TextDrive servers, I decided to get migration done quick (that the deadline was with no explanation pushed back to January 31st after I raised it on the TxD forum, is an example of lack of clarity I mentioned).
It took a while but was straightforward. Django 1.4 has a more modular layout for its apps - the main app can live side by side with other apps instead of being nested. You can see the difference here between the two layouts -
This meant changes to import paths that were worthwhile, as the new way of laying out projects is much better. Some settings data structures have changed, notably for the database and for logging (which now actually has settings). The Feed API left the impression it was the one that was most changed from earlier versions, and while it's significantly improved, I couldn't see a simple way to emit blog content. I like full text feeds as it avoids people having to click through to read a post. I had to use the Feed API's internal framework for extending data -
Comments are still disabled until I figure out how to deal with spam which is a severe problem here, although the comments framework looks much improved. That's probably ok because the comments form UX on the blog sucked. Also, comments seem to be going away in general in favour of links get routed to social networks and the discussion happens there. I might experiment with Discqus or routing discussions to Google Plus.
Now that django-tinymce exists, I was able to remove my own integration hack with the admin app. I used Django's cache api, which is much improved from what I remember, in conjunction with memcached.
The old blog deployment used fast CGI with lighty. This was never a very stable deployment for me and I had wanted to get off it for some time. Years ago, mod_python was the production default for Django, mod_python in the past has made me want to stab my eyes out with a fork, hence fcgi. I switched over to mod_wsgi, which is the 1.4.3 default option for deployment, and we'll see how that goes. Certainly it requires less init boilerplate
The only Linode documentation that didn't work first time was getting Django and mod_wsgi setup, but it was from 2010 and easy to fix up. I ended up with this
I was very happy a data migration wasn't needed. Not a single thing broke with respect to the database in this upgrade.I'm with Linus when it comes to compatability/regressions and in fact think breaking data is even worse. If you do it, provide migrations, or go home. Too many people seem to think that stored data is something other or secondary to the code, which is wrong-headed - data over time is often more important than code.
It's been four years give or take since I used Django, it was fun to use it again. Django has grown in that time - apis like feed, auth and comments look much better, modularization is improved - while retaining almost everything I like about the framework. Django has a large ecosystem with scores of extensions. Django was and still is the gold standard by which I measure other web site frameworks. Grails and Play are the closest contenders in the JVM ecosystem, with Dropwizard being what I recommend for REST based network services there. One downside to Django (and by extension, Python) worth mentioning is that the stack traces are less than helpful - more than once I had to crack open the shell and import some modules to find my mistakes.
Observations
The main lessons learned in the move were as much reminders as lessons.
First if it ain't broke don't fix it, is a bad idea that doesn't work for web services. You should as much as possible be tipping around with code and keeping your hand in so that you don't forget it. Most of the time I spent migrating this weblog was due to unfamilarity with the code and Django/Python rustiness (but see the fifth point as well). Never mind the implications of running on ancient versions, I simply wish I had checked in on this project a couple of times a year. Not doing so was stupid.
Second, when a framework comes to support a feature you added by hand, get rid of your code/workaround - if it was any good you'd have contributed it back anyway.
Third, divest of undifferentiated heavy lifting; in my case this was email. If you don't enjoy doing it, learn nothing from it, gain no advantage by it, let it go.
Fourth, excess abstraction can be a future tax. In my case the weblog code was intended to support multiple weblogs off a single runtime. Turns out after five years I don't need that, and it resulted in excess complexity in, the database model, view injections, url routing, and path layouts for files/templates. All these got in my way - for example dealing with complex url routing cost me hours with zero benefit. I'll have to rip all this nonsense out to ungum the code. I mention this also because I've been critical recently of handwaving around excess abstraction - claims of YAGNI and other simplicity dogma are ignorable without examples.
Fifth, and this is somewhat related to the first point. Online service infrastructure has come a long long way in the last number of years. I know this because it's part of my career to know this, many people reading this will know this, but when you experience moving a more or less seven year old setup to a modern infrastructure, it's truly an eye opener. My 2006 hosting setup was so much closer to 1999 than 2013.
Finally - hobby blog projects are for life. I'm looking forward to expanding Linode usage and experimenting with the code more. When you have complete control of instances you also get more leverage in your infrastructure choices - for example I now have the option to use Cassandra and/or Redis to back the site. I don't see myself going back to a shared hosting model.
Previous Topic
