You may have noticed 10MinuteMail was unavailable for a few minutes over the last couple of days. 10MinuteMail recently came under a DDOS attack which locked up the site a few times. Most of the malicious traffic came from the Netherlands, Germany, and to a lesser extend other European countries and the USA. Initially I dealt with it by generating a list of the malicious IPs and adding them to my block list. However, the DDOS kept spreading (botnet?) so I finally did what I should have done ages ago, and tuned my CSF/IPTables firewall to block DDOS patterns. So far so good:)

I have NO IDEA why anyone would be attacking 10MinuteMail. It’s very odd.

Share on Facebook

Automating Linux Anti-Virus Using ClamAV and Cron

Thankfully Linux isn’t a platform which has a significant problem with Viruses, however it is always better to be safe than sorry. Luckily ClamAV is an excellent free anti-virus solution for Linux servers. However, at least on RedHat Enterprise 5 (RHEL5) the default install doesn’t offer any automated scanning and alerting. So here is what I’ve done:

The following steps assume you are using RHEL5, but should apply to other Linux distributions as well.

First, you’ll want to install ClamAV:

yum install clamav clamav-db clamd
/etc/init.d/clamd start

On RHEL5 at least this automatically sets up a daily cron job that uses freshclam to update the virus definitions, so that’s good.

Next I recommend removing the test virus files, although you can save this until after you test the rest of the setup:

rm -rf /usr/share/doc/clamav-0.95.3/test/

Now we want to setup our automation. I have a daily cron job that scans the entire server which can take several minutes, and then an hourly cron job that only scans files which were created or modified within the last hour. This should provide rapid notification of any infection without bogging your server down for 5 minutes every hour. The hourly scans run in a couple of seconds.

Each scanning script then checks the scan logs to see if there were any infected files found, and if so immediately sends you a notification e-mail (you could set this address to your mobile phone’s SMS account if you wanted).

The Daily Scan:

emacs /etc/cron.daily/clamscan_daily

Paste in:

#!/bin/bash

# email subject
SUBJECT="VIRUS DETECTED ON `hostname`!!!"
# Email To ?
EMAIL="me@domain.com"
# Log location
LOG=/var/log/clamav/scan.log

check_scan () {

	# Check the last set of results. If there are any "Infected" counts that aren't zero, we have a problem.
	if [ `tail -n 12 ${LOG}  | grep Infected | grep -v 0 | wc -l` != 0 ]
	then
		EMAILMESSAGE=`mktemp /tmp/virus-alert.XXXXX`
		echo "To: ${EMAIL}" >>  ${EMAILMESSAGE}
		echo "From: alert@domain.com" >>  ${EMAILMESSAGE}
		echo "Subject: ${SUBJECT}" >>  ${EMAILMESSAGE}
		echo "Importance: High" >> ${EMAILMESSAGE}
		echo "X-Priority: 1" >> ${EMAILMESSAGE}
		echo "`tail -n 50 ${LOG}`" >> ${EMAILMESSAGE}
		sendmail -t < ${EMAILMESSAGE}
	fi

}

clamscan -r / --exclude-dir=/sys/ --quiet --infected --log=${LOG}

check_scan

chmod +x /etc/cron.daily/clamscan_daily

The Hourly Scan:

emacs /etc/cron.hourly/clamscan_hourly

Paste in:

#!/bin/bash

# email subject
SUBJECT="VIRUS DETECTED ON `hostname`!!!"
# Email To ?
EMAIL="me@domain.com"
# Log location
LOG=/var/log/clamav/scan.log

check_scan () {

	# Check the last set of results. If there are any "Infected" counts that aren't zero, we have a problem.
	if [ `tail -n 12 ${LOG}  | grep Infected | grep -v 0 | wc -l` != 0 ]
	then
		EMAILMESSAGE=`mktemp /tmp/virus-alert.XXXXX`
		echo "To: ${EMAIL}" >>  ${EMAILMESSAGE}
		echo "From: alert@domain.com" >>  ${EMAILMESSAGE}
		echo "Subject: ${SUBJECT}" >>  ${EMAILMESSAGE}
		echo "Importance: High" >>; ${EMAILMESSAGE}
		echo "X-Priority: 1" >> ${EMAILMESSAGE}
		echo "`tail -n 50 ${LOG}`" >> ${EMAILMESSAGE}
		sendmail -t < ${EMAILMESSAGE}
	fi

}

find / -mmin -61 -type f | grep -v '/proc/' | grep -v '/sys/' | xargs -r clamscan --exclude-dir=/proc/ --exclude-dir=/sys/ --quiet --infected --log=${LOG}
check_scan

find / -cmin -61 -type f | grep -v '/proc/' | grep -v '/sys/' | xargs -r clamscan --exclude-dir=/proc/ --exclude-dir=/sys/ --quiet --infected --log=${LOG}
check_scan

chmod +x /etc/cron.hourly/clamscan_hourly

Protected System

You should now have a well protected system with low impact to system performance and rapid alerting. Anti-Virus is only one piece of protecting a server, but hopefully this makes it easy to implement for everyone.

Share on Facebook

The Monster.com job board database was illegally accessed and large amounts of user data were stolen.

As is the case with many companies that maintain large databases of information, Monster is the target of illegal attempts to access and extract information from its database. We recently learned our database was illegally accessed and certain contact and account data were taken, including Monster user IDs and passwords, email addresses, names, phone numbers, and some basic demographic data. The information accessed does not include resumes. Monster does not generally collect – and the accessed information does not include – sensitive data such as social security numbers or personal financial data.

The fact that the database was accessed illegally (no word on if it was an internal or external access) is a huge deal. However the fact that they stored passwords in either plaintext, or in a weak enough hash that they feel all the user passwords are compromised, is the most disturbing part of this news in my opinion.

There is no excuse for either of those security failures. Especially after the highly public loss of 1.3 million users’ data in 2007.

Assume that your database will be accessed at some point by someone with nefarious intent. If it can happen to Monster.com it can happen to you. Therefore you should not store passwords in plaintext or weakly hashed.

Use a salted SHA-256 or bcrypt hashing algorithm to protect your users’ accounts.

If you use ATG please check out the open source SecurePassword ATG module. It replaces the default insecure password hashing algorithm with a salted SHA-256 hashing mechanism. (as a side note I will develop a bcrypt version shortly, but SSHA-256 is very secure).

Share on Facebook

Recently there has been a lot of press about AT&T DNS servers being hit with a DNS Cache Poisoning attack.

Some new easier exploits were recently published, and many DNS servers are still vulnerable. And up until the new exploits were published publicly, the majority of DNS servers were vulnerable. This situation is worse once you realize that “safe” DNS servers can be poisoned second hand by transitive trust relationships, allowing one compromised DNS server to effectually poison the caches of other un-compromised DNS servers.

DNS Cache Poisoning has been a serious issue for years. The recent flurry of press regarding the compromised AT&T DNS servers is just the tip of the iceberg. It is only reasonable to assume that over the past several years a large number of DNS server have been serving compromised results at some point, either by direct poisoning or indirect poisoning. It is also reasonable to assume that this will continue for the foreseeable future.

In light of this, please re-read my post on 3rd party Javascript.

If I were a malicious hacker, let’s say working for the Russian Mob, or for myself, here is the easiest way to make some money:

1. Create javascript files designed to mask Google Adwords, Google Analytics, Doubleclick Ads, Overture Ads, and maybe a couple others. These scripts would have their cache related response headers set to be cached on the browser for 1 year. These scripts would call back to the real versions of themselves (so that ads show up, etc…). They would also intercept any form submission events and would look for form fields with names like “creditcard” or “ssn” or “password” or “accountnumber”, etc… If any are found, it would essentially clone the form and send the form data, the site hostname and page, the client’s IP address and cookies, etc… to a server I control.

2. Start cache poisoning as many DNS servers as I could find that are vulnerable to point the REAL domains for those scripts to my malicious copies.

3. Sit back and watch the Credit Card numbers roll in.

The best part is that by getting the browser to cache the script locally, I only have to have a computer hit the poisoned cache once to control it for a whole year. On most IE6 installations it’s also easy to actually install a javascript application on the user’s computer.

Personally I use Google javascripts on my site. However I don’t capture credit card numbers here, so the risk is low. If you run an e-commerce site, please do not underestimate the risks involved here.

Share on Facebook

If you have a server with a world facing ssh server, you’ve probably seen brute force attacks in your logs. Some machine starts hammering your ssh server, trying all sorts of logins (staff, root, a, admin, etc…) over and over and over again.

This is bad on a lot of fronts.

I use two simple iptables rules to block any IP address who has made more than 3 ssh connections or attempted connections within the past 3 minutes. So your would-be brute force attacker, gets three tries, and then is locked out for a minimum of three minutes. However, since 99% of the attacks are run by an automated bot, it will either: give up after the connection is refused multiple times, or it will keep hammering away on the closed door, which keeps the running count of attempted connections in the past 3 minutes over 3, keeping the door closed.

First:

iptables -I INPUT -i eth1 -p tcp -m tcp --dport 22 -m state --state NEW -m recent --set --name DEFAULT --rsource

Then run:

iptables -I INPUT -i eth1 -p tcp -m tcp --dport 22 -m state --state NEW -m recent --update --seconds 180 --hitcount 4 --name DEFAULT --rsource -j DROP

I’d also recommend using the script in my post on blocking IP addresses using iptables to deal with any persistent folks, or people poking too hard on your web site, or other services.

Share on Facebook