Yesterday evening, the UFies server expanded to two servers (yay) which will give us a server to use for the database and for backup. However at the same time (well, a couple of hours later) the main server suffered some stunningly catastrophic hardware failures. The RAID array lost 2 of the 4 drives, and the hot spare never worked. Luckily there is a nightly backup which Fred and I managed to get access to (after some heart stopping "no such device" messages).
I spent today and this evening transferring over settings and data and reconfiguring all the old system stuff (from a gentoo install) to work on the new server (a ubuntu install).
Probably 90% of that is done now. User accounts have been moved over, along with data, mail appears to be flowing, DNS, ftp and some websites are back online. Please email me at alan@ufies.org if you see oddness with your stuff.
Sadly not the controlled transfer from old server to new we were wanting, and no chance to clean up dead user accounts and websites, but better (way better) than losing everything (again).
Remember folks, backup regularily!
Things still needing to be fixed: