This post originated from an RSS feed registered with Java Buzz
by Marc Logemann.
Original Post: I destroyed my linux inernet server
Feed Title: Logemann Blog
Feed URL: http://www.logemann.org/blojsom/blog/default/?flavor=rss2
Feed Description: Marc's thoughts on Java and more
As some might allready noticed, my blog and a lot more important services were down the last 1,5 days. Just to give you an idea what was down so that you get an idea how scrary such a situation is: Subverson repo, Mail, HTTP, bugtracking, blogserver. Since all these services are quite critical for my daily business (except for the blog of course :)), what happened? To make it short, i totally sucked in regard to linux skills. I made the error to use YAST (yep, its SUSE) for updating my kernel. Despite of the fact that i dont use standard kernels, but one supplied by the rootserver provider, i also dont have a clue about kernels at all. But this didnt hinder me to update it in the morning of a busy weekday (dont know what the hell was in my mind).
After the reboot nothing worked. Kernel panic on mounting file systems. Then i started reading in the hope to repair this damn thingy but even after some hours of investigation (downloading new provider kernel, editing symlinks in /boot, etc.), i didnt get it to work because LILO insisted on loading the wrong kernel. To make things even worse, the only person i know with some significant linux skills is on vacation. The provider only supports trivial questions and so i was on my own. I booted in a rescue system and backuped the complete filesystem, put it on a backup space and re-initialized the whole server. The provider has ready-to-run images whose installation can be triggered via webinterface (its UnitedInternet, the biggest provider in germany). I am writing while the server gets re-initialized and after that i will have the goddam task to bring the machine back to the old status in terms of mailaddresses, virtual hosts, ssh keys, users, just everything. I think my "update vision" costs me about 8 hours of my life.
So what do i learned today? Never do something that possibly can break where you dont have a clue how to fix it. Besides this, my next week off is spent for a Linux LPI course or at least some linux course which teaches me some internals. I know linux quite well but as it seems, not well enough. When you read these lines, i just sucessfully installed mod_jk and applied the necessary modifications to Apache2 :-)
Previous Topic
