I woke up this morning, got online through my new UMTS/HSPDA modem, and discovered that one of my servers had a load average of 239+
Not the best way to start the day.
Turns out an rsync backup job between two servers had gone nuts and was spinning them up through the roof. A couple
sudo killall rsync
commands, and everything started settling down.
What’s important to note is that while the CPU was pegged, and the load average was over 200, I was still able ssh in, run top, ps, netstat, and navigate around looking at log files, with very little delay. It was just a little slow, but no more than a second wait for anything.
Also, all of my apps which used MySQL as their backend were all dead, with “database refused connection” errors. All of my apps which used PostgreSQL as their backend were a little slow, but were all still up and functioning without errors.
So, Debian and PostgreSQL for the win!