This post originated from an RSS feed registered with Java Buzz
by Marc Logemann.
Original Post: CI Server (TeamCity) in the Amazon Cloud
Feed Title: Logemann Blog
Feed URL: http://feeds.feedburner.com/LogemannBlog
Feed Description: Marc Logemann's thoughts on java and other stuff
We are using TeamCity as Continuous Integration Server. I still believe that its the best of the class. Installation of agents is so damn simple and the overall architecture is brilliant. I know of CI server which still use an agent-less approach which makes really no sense at all. You definitely want test different platforms, JDK versions, databases, etc and this only works out in an agent environment. But today i will focus on the cloud capabilities of TeamCity:
real On-Demand resources triggered by TeamCity
TeamCity is able to speak with the Amazon EC2 cloud via their webservice API. This sounds little exciting on first thought but on second its really amazing. I will picture our current scenario and explain the detailed steps later. Here is how our CI process works:
1) developer committs (and pushes) code into our GIT repository
2) TeamCity watch out for new committs in our main GIT repository and starts queing a new job for new committs.
3) TeamCity checks what agents are available for processing the job (with job i mean compiling, testing, etc.)
4) TeamCity picks a "cloud agent" from the availbale agents and tells Amazon to start an Instance
5) The Instance is started for the duration of the whole job
6) After finishing the CI job, the EC2 instance is shut down by TeamCity
7) user(s) get email notification of the test results
lets do the math with cloud hosting and TeamCity
So this is REAL on-demand and all this without user interaction. Of course the math should be done by yourself and depends heavily on the number of developers and your job triggering strategy in TeamCity. Lets suppose you have 3 developers and they push changes once a day to the company main GIT repository:
- Developer 1 pushes changes at 2pm.
- Developer 2 pushes changes at 3:40pm
- Developer 3 pushes changes at 4:00pm
This behaviour results with our current job trigger strategy in the following Amazon instance startups (assuming our build/test takes 20 minutes and init time is 5 minutes):
- Instance will be started at 2pm and starts working after initialization
- Instance will be stoped at 2:25pm (billed as 1 hour)
- Instance will be restarted at 3:40pm and stopped at 4:20pm (billed as 1 hour - but 2 build were done)
The last 2 builds were executed in one strech because the 3rd job was queued before the 2nd is finished. The instance wont be stopped after job 2 finishes. This saves a third restart of the instance. At the end we payed 2 hours of Amazon EC 2 computing. Compare this to the standard way of doing this.... 24x7 up and running where most the time the server is just idle but defnitely costs money (and its not exactly green-it either).
the future of spot instances with continuous integration
Amazon also sells "spot instances". There you can bid for a price you want and you get the resources if Amazon hits your price tag. From amazon perspective it makes sense to sell resources during non-peak-times at lower rates. The drawback from user perspective is that you dont know when your instance will be up because you dont know when the price tag matches. Think of it as a stock-order with maximum price definition. The order gets executed as soon as the stock will be equals or lower your max price. The same goes for Amazon. So if you have workloads where its not necessary to know exactly when this workload gets processed, this may be for you. One great example of that are "nightly builds". To me it really doesnt matter if our nightly build runs at 11pm or 5am.
TeamCity is not quite there when it comes to supporting spot instances but from what they already have, it should not be a big deal to implement that too.
bottom line
TeamCity has really a lot to offer. We at LOGENTIS love using it for years now. With IntelliJ IDEA and TeamCity, they really offer two great tools in the Java world. I cant imagine stop using any of these two. This doesnt mean that Eclipse and Jenkins are a terrible choice, its just not mine.
Disclaimer: I dont get money or anything else for this suggestion. And not each and every product in their offering is that convincing. I really cant get into YouTrack for instance. Not my prefered way of doing tickets. And of course i cant comment on their non-java offerings. But as a developer i like being vocal about good products and believe me, i am as vocal too when it comes to crap products.
Previous Topic
