email article printer friendly more resources

Secure Your Servers Against Attack
Get familiar with the various types of attacks that threaten your servers and take the necessary steps to protect them.
by Harry J. Segal

August 2003 Issue

You don't have to look any further than the headlines that followed the SQL Slammer outbreak to realize that keeping your Microsoft servers patched against known vulnerabilities is a tall order. The self-propagating malicious code (also know as the W32.Slammer and Sapphire worm) exploits a vulnerability in the Resolution Service of Microsoft SQL Server 2000 and Microsoft Desktop Engine (MSDE) 2000. Within moments of the infamous worm's launch, more than 35,000 unpatched servers were infected (see Resources). However, more than six months before the outbreak, Microsoft had issued a patch that fixed the vulnerability the worm exploited. So why were so many servers, including some of Microsoft's own internal servers, infected?

As the SQL Slammer and other well-publicized outbreaks prove, relying on network administrators to install critical patches is an ineffective security model. Reduced IT staffs and the sheer volume of patches make it difficult for organizations to track and install all the necessary patches. And because more and more companies use the Internet to do business 24 hours a day, seven days a week, it's also tough to schedule downtime to patch the servers. Despite these challenges, there's no question that installing patches is critical to maintaining a secure network. In this article I'll outline the various threats to your servers and explain how to combat them to protect your environment from attacks.

Let me start by offering a little history. By the mid 1990s the Internet had transformed the way companies did business. Many companies connected their businesses to the Internet each day, often with little understanding of the risks involved. Those who were aware of the risks sought out firewalls to provide an important layer of security between the Internet and their corporate networks. This protected them against many network-level attacks or threats, but not against application- and OS-level threats. Because communications on the Internet employ a standard set of defined protocols, firewalls are configured to allow connections over ports defined by these protocols. For example, when you connect to a Web site such as www.acme.com, the firewall protecting Acme's public Web server allows your machine to open a session using the HTTP protocol on port 80 to the Acme Web server.

Firewalls slowed down hackers for a short time by restricting traffic at the network level. However, once firewalls became standard, hackers began picking apart the applications commonly found running on corporate servers. Exploiting weaknesses at the application level brought about the next generation of attacks. Because firewalls are configured to allow traffic to communicate to the application, attacks which exploit an application's or OS's vulnerability flow unimpeded through the firewall. The end result is that once a vulnerability is discovered in an application, the host needs to be patched or reconfigured immediately to mitigate the problem. It becomes a race to see who will get to the servers first: the network administrator who installs patches or a hacker who exploits the vulnerabilities. The deck is stacked against the network administrator because he relies on the vendor to create and publish the patch. These patches often aren't available until weeks or months after the vulnerability is discovered, and the admin must test the patch first on his servers to make sure it doesn't effect the servers in his environment adversely.

Explore Next-Generation Security Measures
To deal with the threat of these attacks, the next generation of security products was devised: Intrusion Detection Systems (IDS). As the name implies, these systems attempt to detect attacks and alert the network or security administrator. In some cases, IDS can protect against attacks by stepping into the TCP/IP connection and resetting the connection before the full attack reaches the target machine. However, in most cases the administrator needs to take action to stop the attack and assess the damage that has been already done. Traditional IDS take a network- or host-based approach to recognizing and alerting administrators to attacks. In both cases, IDS look for attack signatures or specific patterns that usually indicate malicious or suspicious intent. When looking for these patterns in network traffic, IDS are network-based. Alternately, when IDS look for attack signatures in log files, they're host-based. Each approach has strengths and weaknesses, and a truly effective IDS solution employs both technologies. Although IDS technology is useful, it doesn't prevent or stop most attacks; the administrator must still step in, interpret the data, then take the appropriate action. It also places the burden on the administrator to sort out false positive information and discern the true threats represented in the alerts. Another important shortfall is IDS' reliance on the signatures of known attacks to recognize an attack is underway. This leaves the network vulnerable to new or unknown attacks, because IDS can't recognize threats to the network or systems they're monitoring if the signatures aren't included in their current signature databases.

Fortunately, there's a new generation of security technology that addresses some of these issues: Intrusion Prevention Systems (IPS). IPS also use the host- and network-based models. The most common and effective IPS today are host-based and use a combination of signatures of known attacks and, more important, an intimate knowledge of the operating system and specific applications running on the host. With this programmed knowledge, known as behavioral rules, IPS know what activity is abnormal or out-of-bounds and prevent the host from executing commands that don't fit the correct behavior of the operating system or application. For example, a behavioral rule that allows only the HTTP server process to access the Web files on a Web server triggers a response if any other process attempts to access a Web file. Because host-based IPS inspect the data after it's been decrypted and before it's executed, they aren't "blind" to Secure Sockets Layer (SSL)-encrypted exploits. And because IPS don't rely solely on signatures of known attacks to provide protection, they also protect servers against unknown attacks. For example, the more mature IPS products on the market have been able to prevent Code Red, NIMDA, and SQL Slammer before the attacks have been well understood and signatures developed (see Figure 1).