A penetration assessment will ensure your network is secure from attackers as well as malicious employees, partners and customers. We evaluated three products that take decidedly different routes to the same end. Which solution will leave you feeling bulletproof?
By Mike Fratto
Secure Enterprise
CORE IMPACT awarded Tester's Choice
You've deployed all the right security products--firewalls, VPNs, intrusion-detection or intrusion-prevention systems, authentication software, secure wireless gear. You've configured access control lists on routers and firewalls, and locked down access as best you could. But is it good enough? Securing any network and its resources properly is complex, and grows exponentially more complex the larger and more distributed your network is.
Overconfidence can be a killer here. Just one oversight and someone will blow a gaping hole through the best defenses. Consider a three-tier Web application with a Web server on a DMZ network accepting connections from the Internet. If you let the Web server pull content from a database that resides in the internal data center, you've created an access path from the Internet to the data center through the Web and database servers. If the Web server has an exploitable vulnerability, an attacker can get his foot in the DMZ. If the database also has a vulnerability that's exploitable from the Web server or lax access controls, the attacker can waltz right into your data center.
Why My Network?
Think that can't happen with your tight control of inbound traffic? If your network security posture is like most, outbound traffic is just as big a culprit. And once an attacker gains a foothold, any foothold, it's just a matter of time before your network is compromised. Granted, sound security strategies can mitigate the risk, but the threat of a mistake looms large. You'd rest easier if you knew your security implementation were functioning as it should.
Watching the Watchers
You can test the security of your network using an in-house tool or the services of a security firm. Penetration tests performed by consultancies are only as good as the team doing the work--the more experienced and varied their skills, the more thorough their tests. We've all heard horror stories of 500-page vulnerability scans presented as penetration tests that left many avenues of attack undiscovered, and there's nothing we hate more than a false sense of security. Adding insult to injury, penetration tests can cost thousands of dollars. And even if you address the problems found in the tests, how do you know if they're truly fixed and that other problems weren't introduced during the remediations?
Vulnerability-scanning products, whether network- or host-based, are often used in outside audits and penetration tests and by internal IT and security administrators. But don't mistake a vulnerability scan for a penetration assessment--vulnerability scans are marginally helpful for detecting vulnerabilities based on banner checking. They're more useful when the VA (vulnerability assessment) scanner can check for the presence of the vulnerability. For example, a banner check showing that you're running IIS on Windows 2000 means that server may be vulnerable to a slew of exploits. Actually finding, say, the printer ISAPI vulnerability tells you whether the server is vulnerable. Without local access to the target host to discover which applications and patches are installed, a VA scan tells only half the story.
The resulting data-management problem is evident. Scanning even a single Class C network can generate hundreds of alerts with varying degrees of severity. Processing that data--determining which alerts are real and which are false positives--is a formidable challenge.
Vulnerability-management suites like those we recently reviewed (see "Search and Defend,") provide structure to the mountain of data, but they won't give you insight into the context of the vulnerability. Riddle us this: Which is more critical, a Web server vulnerability that gives a user shell access as a guest on an Internet-facing Web server, or a vulnerability that gives database administrator and root access on a relational database containing employee records in the data center? It depends on context. If the RDBMS vulnerability is exploitable only inside of the data center, but the Web server vulnerability is exploitable from the Internet, we'd argue that the Web server is more important because it's accessible externally, and once an attacker has shell access as a guest user, elevating privileges and jumping to other parts of the network are often trivial. Yes, you want to fix both problems, but given finite resources, which do you tackle first? Without context, you may be setting the wrong priorities.
Ideally, you'll bring to the table knowledge of both your vulnerabilities and the context to get a holistic picture of your exploitable vulnerabilities viewed from different parts of the network. Only then can you prioritize remediation, assess the exploitable risk posed by nodes on the network, compare your existing security stance to what it should be, and perform a number of other monitoring functions.
The scope of your network dictates how difficult it will be to get this big-picture view. The scope of your network dictates how difficult it will be to get this big-picture view. Sure, you could meticulously map out and document your network and all the network access device rule sets, but there are likely to be mistakes initially and during maintenance. Network-management suites like Computer Associates' Unicenter and Hewlett-Packard's OpenView can map out Layer 3 networks, but they don't analyze router access controls or firewall rule sets. Yet, it's vital to see your vulnerabilities as an attacker would (see "Think Like an Attacker,").
There are two ways to model the vulnerability posture of hosts in the context of your network: Do your own penetration test or use a penetration-assessment simulator. Each method has benefits and drawbacks, so we decided to examine both approaches. We asked vendors for penetration-assessment software that can model and analyze a network topology, including routing, ACLs, firewall placement and firewall rules; add vulnerability data from a vulnerability scanner, then produce information about attack paths; and finally, model the effects of making changes, such as modifying firewall rules, patching vulnerabilities and using other hardening methods. For this portion of the review, we invited Skybox Security and Secure Elements to submit products to our Syracuse University Real-World Labs. Skybox sent us its Skybox View, but Secure Elements couldn't ship a product in time for our tests.
For the penetration-test tools portion, we asked Core Security and Immunity for suites that exploit system services on common platforms and applications and attempt to gain access through open vulnerabilities. We wanted tools that could successfully attack targets with known vulnerabilities, execute code on the target and employ evasive techniques in an attempt to slip past IDSs and firewalls. Both vendors stepped up with their current offerings.
Which Way Is for Me?
When we sent our invitations, we told the penetration-assessment vendors that our tester would have both networking and security skills; we promised the pen-testing vendors pure security chops. Likewise, the method you choose will depend on your network and your people.
Penetration simulation, the core functionality of Skybox View, let us explore multiple attack paths noninvasively. With a knowledge of our network topology, access controls and vulnerabilities, we could plot out all the possible attack routes from one point to another. The model is based on the assumption that if the vulnerability exists, it will be exploited. Simulations may be used to configure what-if scenarios--if a new worm broke out that took advantage of Windows file sharing, for example, we could quickly model how rule-blocking access would affect the worm's impact; the simulation also could indicate the optimal topology and access controls, compared with current controls.
Although this approach is undeniably useful, it has drawbacks. The simulation is only as accurate as the data you feed in. GIGO applies. False positives and negatives in vulnerability data and missing hosts, routers or ACLs will all skew the simulation. Equally important is what devices are modeled. Routers, firewalls and VPN servers are simple to model compared with IPSs, application firewalls, SSL VPN gateways and access points based on allowed application access. So the picture is never complete, just nearly so.
On the other side of the coin, penetration tests are difficult to do well--you need both broad and deep knowledge. Given enough time and access to sites like SecurityFocus, PacketStorm and the Full-disclosure and BugTraq mailing lists that carry vulnerability information and exploit code, you could build a current exploit toolset based on publicly available code. Or, if you have skilled coders at your disposal, you could build exploits from public information. Either way, plan on spending time testing exploit code in a lab environment before deploying.
This is where Core Security's Impact and Immunity's Canvas penetration products come in--they take much of the pain out of finding and building working exploits and interesting shell code, but the onus of figuring out which exploits to use still rests on your shoulders. And need we mention, working exploits can take down legitimate services and leave systems in an unstable state, so deploy with care.
These were fun products to play with, and they provided helpful information about our security. However, they may not make the best use of your scarce security dollars. In either case, they'll only tell you about server and service vulnerabilities. Application security problems, like SQL injection, form/field manipulation, cookie tampering and a host of other application nightmares aren't supported, so don't expect to test Web applications.
(Report Card)
If your organization hasn't built and implemented a security plan, focus your resources there. But if you can articulate how IT assets are protected and show working processes and procedures, we recommend taking one of these products for a test drive. Our split report card will help you weigh each product's strengths and weaknesses. For the lowdown on our test bed, see "Methodology," below.
Skybox ran unopposed, but its Tester's Choice for penetration assessment simulator is well-earned, thanks to Skybox View's thorough and varied analysis, clear and concise reporting, and robust interface that includes task scheduling and automation.
Skybox Security Skybox View Standard Edition 2.0
Skybox View creates a network map based on Layer 3 connectivity, models router and firewall ACLs, and populates a network diagram with host information provided through network and vulnerability scans.
The standard edition starts at $60,000 per year. For our as-tested scenario of 1,000 nodes (100 servers and infrastructure nodes and 900 workstations), the software license lists for $85,000 plus $17,000 for annual support and a vulnerability dictionary. That's a good chunk of change--more than a staff position per year. But compared with the cost to regularly assess your network, consistently gather assessment and audit reports, analyze a network during an acquisition, or perform other data-analysis-intensive processes, it could be a worthwhile investment.
Armed with a topology, ACLs and vulnerabilities, Skybox View let us simulate attacks from anywhere to anyplace on our network. Add tuning features, such as attacker skill, asset grouping and value, and compliance requirements, and Skybox View is a reasonably powerful analysis tool--once we learned how to use it. We say reasonably because there are limitations to the types of devices and software Skybox View can model. For example, our SSL VPN gateways, desktop firewalls and IPSs were out of luck. And Skybox View couldn't model access based on authentication and application access control.
Skybox sent a system engineer to help with the installation, as it would for any customer. One of the first hurdles is getting the topology and firewall rules imported and modeled correctly. Skybox View officially supports a few common devices, including Cisco routers and Pix, Check Point, Juniper NetScreen and Iptables firewalls, but the company says it's expanding this support. Skybox successfully modeled the SonicWall 3060 firewall and an older version of a NetScreen firewall that we had in our test bed. Because our test network was small, Skybox manually imported the configurations and Nessus vulnerability scan data we provided.
(Penetration
Assessment
Features)
Skybox View uses vulnerability data from many sources, including VA scanners and patch-management systems. Skybox Security normalizes the vulnerability data and assigns weights to skill level, access, likelihood and other qualitative measures, and builds a vulnerability dictionary that Skybox View uses to determine the severity of an attack. Interestingly, Skybox View accounts for dependencies like requiring authentication by lowering the risk of an attack. Skybox View also models local exploits.
(Penetration
Simulation
Features)
We configured automated device updates for our Pix firewall and Cisco routers. We modeled other devices by hand. Skybox lets you integrate nonsupported products in one of three ways: You can add objects to the map manually, as we did with our NetScreen firewall. You can use Perl scripts and an API to extract information from the devices and format it into the Skybox View schema. The scripts can run as tasks for automatic updating. Or you can request that Skybox fully support the device, which it says typically takes two to three months.
Once the model was in place, we set up Threat Origins, which define attacker characteristics, such as skill and likelihood to strike. Threat Origins also define where attackers are located, such as the Internet or an internal or partner network.
After completing our initial configuration, we ran the attack simulation against an internal Web server. The attack map showed all the attack paths from our Threat Origins to the target, including intermediary hops. Clicking on map elements offered more detail, like access path and vulnerabilities. The map is fully interactive. When we selected specific attacks or deleted nodes, the map reflected the resulting attack paths. Skybox View also displays automatically what it considers the most likely attack paths based on Threat Origin, vulnerability type and other data.
Attack simulations are fun, but diagramming attack paths is only a small part of what Skybox does. It was easy to build what-if scenarios that showed the effect of changes. For instance, we marked a vulnerability as fixed, and the resulting path and all its dependencies disappeared. Using a what-if scenario, we could see the effects of remediations without touching a device and, more important, we could ensure that configuration changes wouldn't open other holes in our network. Finally, we defined some network assets, assigned risk levels and values to them, and had Skybox View report on the most vulnerable.
Skybox View has some useful interactive features associated with what-if scenarios. For example, if we changed a vulnerability to fixed, Skybox View gave us the option to mark the problem as fixed permanently. However, if the next scan showed the vulnerability again, Skybox View would re-instate it. In addition, firewall rules can be modified and attacks simulated to see changes. There are three workspaces within Skybox View, each of which can have three different models, but unfortunately there's no automated way to compare models. For example, we'd have liked to compare a saved model of how the network should look against how it actually does look.
There are plenty of reports and opportunities to view network nodes based on system properties. For example, we found all our network nodes that have telnet enabled. Running reports and analyses based on groups or hosts, asset groups, or other aggregates lets you answer complex questions easily. Reports, like other system tasks, including updating the network map, can be scheduled through a robust scheduling mechanism.
Take advantage of the training offered by Skybox Security. The product has lots of details and features that, when fully explored, are powerful. That said, there are a few problems the vendor must address. The first is that Skybox View is really a network and transport layer analysis tool, but more enterprise applications are becoming "Webified," with HTTP/HTTPS as the conduit for enterprise applications. We'd like to see visibility into HTTP/HTTPS data paths. Device support could be broader as well. There is a wide range of devices on enterprise networks, and the more of them that are natively supported, the better.
For more on Skybox View, check out our December 2004 Test Run.
Skybox View 2.0, starts at $60,000; as tested: $85,000 plus $17,000 for annual support and vulnerability dictionary. Skybox Security, (866) 6SKYBOX, (650) 565-8060. www.skyboxsecurity.com
Penetration-Testing Tools
Got r00t? Applications like Core Security's Impact and Immunity's Canvas are penetration-testing frameworks that provide discovery tools, exploit code for remote and local vulnerabilities, remote agents, and other handy-dandy gadgets for exploring and exploiting a network. We didn't include Metasploit in this review because our expectations for support and exploit currency are higher for commercial products, but budget-conscious organizations may want to consider the open-source option (see "The Metasploit Framework," for details).
Be aware that these tools are invasive: If the product can exploit a vulnerability, it will, and it will give the operator access. Canvas and Impact have different features, but the general-use case is to exploit a remote computer. The application injects shell code into the vulnerable process, essentially installing a remote agent in memory. The agent is controlled by the attacker. Depending on which process was exploited, we sometimes had to run a local exploit to elevate privileges. Once done, we continued the discovery process on the local host, then looked for other hosts to attack in a similar manner.
After owning our network seven ways to Sunday, we give our Tester's Choice award to Core Security Impact. Its interface offers a true persistent workspace, flexible agents, regular updates to exploits and some useful built-in automation features. However, we missed having a command-line interface, which Canvas offers.
We asked for one-year license pricing based on our 1,000-node test scenario. Canvas' licensing is based on a one-time purchase of a number of seats and updates, and Immunity recommended a 10-seat license for $3,101, compared with Core Security's Impact at $25,000! Both licenses include unlimited nodes and all updates for a year.
Impact is a fully functional, GUI-based penetration test tool that's about as close to point and shoot as you can get. Its breadth of tools and exploits and advanced automation helped it overcome a vast price disparity with Canvas. With very little knowledge of how exploits work or even basic networking chops, a user can launch attacks and own a machine within minutes of installing the software by using Rapid Penetration Tests (RPTs). Core Security updates the attack database when new attacks are available. Exploits and tools are written in Python and compiled at run-time, so Impact can be customized and extended by experienced developers. Impact's workspaces retrieve and store data over multiple runs, so knowledge for each host is retained.
The workspace is laid out logically and provided access to all the exploits and tools we needed, as well as links to research on vulnerabilities. Each workspace is self-contained and lays out, in Explorer-like fashion, discovered nodes and which ones have agents running. Exploits can be chained together by adding actions to a list and setting requirements.
We started out using RPTs like big clubs to do initial information gathering on the local network. RPTs can be launched against a single host or a group of hosts. We began with the former. Within a few clicks, we defined our network address space and let the test run. There's nothing magic about this testing--the RPT ran a ping sweep and TCP port sweeps of the target network, then discovered hosts were port scanned and services ID attempts were made. Finally, the RPT tried to determine the OS using nmap-style fingerprinting, banner grabbing or SMB (Server Message Block) ID. Using the data gathered, Impact's attack and penetration RPT identified possible exploits and ran them on each target. The default action is to stop after the first successful attack and "infect" the target with a Level 0 agent, which is a piece of shell code running in the vulnerable processes memory space. Once the Level 0 agent is installed, local attacks for more information gathering and privilege escalation can be executed.
On a local network or on networks where Impact has complete access, the RPTs work well, with all the usual caveats about the inexactness of OS detections--you should confirm each OS guess. However, once we started poking at systems behind security devices that restrict or obfuscate access, both Impact and Canvas merely automate and simplify the break-in. For example, we had a Red Hat 9 Linux server, a Windows 2003 Server, a Windows 2000 Server SP0 and a Windows 2000 Server SP4 running SQL Server 2000 SP0 behind a Cisco PIX firewall. We configured the inbound rule set to allow only ICMP, SSH to the Linux server, and TCP 1433 and UDP 1434 to the SQL Server, and we set the outbound rule set to allow all--a common, though relatively insecure, state of affairs. The information-gathering RPT found our hosts via ICMP, but little after that. It identified the Windows 2000 computer, but not the type or Service Pack or what was running on Port 1433.
Because we were pretending that we didn't know anything about the host, we assumed that the computer was a server and that it was running SQL Server. We selected the SQL Server 2000 Resolution exploit and, because we assumed there was a firewall, we set the agent to connect back to Impact, making a reverse shell. Once that was successful and we had a Level 0 agent running, we upgraded to a Level 1 agent, which offers enhancements such as persistence between connections, mutlitasking and an encrypted channel. From our perch on the SQL server, we proceeded to discover and break in to the other hosts behind the firewall.
We jumped between individual attacks and RPTs depending on what we were trying to accomplish. The SQL Server 2000 Resolution exploit got us administrator access on the target, but we wanted system access. We used the Privilege Escalation RPT to get to the system. Then we used the Clean Up RPT to remove all the remote agents--it just removed agents and plug-ins; it didn't restart services.
Impact provides a few simple reports about activities. To test this. we generated a report of our findings in XML and HTML formats.
Core Impact, $25,000 per seat. Core Security Technologies, (617) 399-6980. www.coresecurity.com
Where Impact is a fully functional discovery and exploit product, Canvas is streamlined to provide exploitation and little else. In the hands of an experienced penetration tester, Canvas can be a powerful tool, but unlike Core Security, Immunity expects users to have considerable knowledge of penetration testing, exploits and system insecurity. We did enjoy having a CLI; we often prefer to use the CLI for remote work and for automation via scripting, and all Canvas' tasks can be executed from the command line or inserted in scripts using the language of your choice. The licensing is attractive, but it's not point-and-attack. You should take advantage of the training Immunity provides to get up to speed on Canvas.
Canvas doesn't do any initial host discovery, so we added our hosts to the interface manually, then initiated a port scan and OS detection. The discovered information goes into the hosts' "knowledge," which covers anything learned by Canvas. Using this information, we selected appropriate exploits based on our knowledge. If the exploit was successful, a new node signifying a remote shell server, or an agent in Canvas parlance, would populate the node tree. We selected that node as the first node, and all activity interacted with it. Nodes can be chained together through hosts, similar to Impact, so that attacks would ferret their way deeper into the network. A node offered options to interact with the host machine by getting directory listings, uploading and downloading files, taking screenshots and running commands.
Events, such as elevating privileges, opened new nodes with the new privileges, so we selected and reselected nodes as they appeared. Unfortunately, we didn't visually understand the Node Tree layout. We'd like to see this arranged in a true tree format, where the host is at the root and subsequent nodes are on lower branches. As with Impact, discovered host information populates the nearest matching host in the tree, so after a bit of work, the information is spread out. Both Core Security and Immunity should find a way to consolidate that information into a single object rather leaving a trail of bread crumbs.
Canvas doesn't have multilevel agents. Rather, proprietary mosdef (for most definitely, in Immunity-speak) shell server code is injected into the target process. In most cases, the exploit will recycle the network socket used to exploit the target, so a new connection doesn't need to be established. In some cases, a reverse shell will be initiated. Canvas, rather than using Impact's syscall proxy method to run remote code, compiles shell code locally and sends the shell code to the target node for execution in the mosdef shell server. This results in less network chatter.
Before walking away, we tried to save our work, but there isn't a way to do that! Immunity says it is adding a save feature in a future release, but that really should have been there from the start.
Canvas, $3,101 for a 10-seat license. Immunity, (212) 534-0857. www.immunitysec.com
Mike Fratto is editor of Secure Enterprise. He was previously a senior technology editor for Network Computing and an independent consultant in central New York. Write to him at mfratto@secureenterprisemag.com
If a vital switch is not operating correctly, the networking guys will likely hear about it in short order. But security is different. Lax access controls, exposed services and misconfigured security products are all hidden hazards. You may think everything is functioning normally until it's too late. You've been 0wned.
The problem is exacerbated by the interdependencies found in large networks. The more nodes, the more sites, the more complex the access controls and the more applications supported, the more difficult it is to make sure everything's buttoned up properly. That's where penetration testing comes in.
Product Category: We tested two types of products that can help perform in-house penetration testing: penetration assessment simulators that model and analyze networks looking for weaknesses, and penetration test tools that actually attempt to gain access through open vulnerabilities.
Products Tested: Skybox Security Skybox View, Core Security Technologies Core Impact and Immunity Canvas
Products Not Tested: Secure Elements couldn't deliver in time for this review
Who Won and Why: We gave out two Tester's Choice awards, one per category. Skybox ran unopposed but definitely earned kudos for its excellent analysis, full-featured UI and reporting that let us know exactly what was going on.
In the testing tools category, Impact beat out Canvas despite a huge price disparity and lack of a CLI. We think Impact's flexibility and usability make it worth the premium.
We used a mix of test and production networks and hosts. Our network contains a mixed infrastructure, with gear from Cisco Systems, Extreme Networks, Hewlett-Packard, SonicWall and 3Com. Our servers run e-mail, DNS, Web sites and other production software on a mix of Windows, Linux and Solaris. Our test workstations run Windows 2000 and XP.
Our test bed was protected by a Cisco Pix firewall with a restrictive rule set. The Pix network was our data center. We had a Windows 2000 server running Microsoft SQL Server 2000 SP0; a Red Hat 9 Linux server running DNS, SSH and postfix; and a Windows 2000 Active Directory. Our rule set allowed DNS, SSH, SMTP and SQL services through the firewall; everything else was denied in both directions. Our goal was to get into various parts of our network from the outside to infiltrate the data center.
(Methodology)
For Skybox View, we downloaded the router and firewall configurations and used Nessus to perform a vulnerability scan. We supplied Skybox Security with our SonicWall configuration file so it could import it; we provided the rest of the details.
When evaluating penetration-testing programs, we erred on the side of being uninformed about our network topology. We used only the information we could glean from the tools themselves or from other reconnaissance tools like nmap and Nessus. We did have to make some reasonable assumptions, such as seeing ports UDP 1434 and TCP 1433 open would indicate that Microsoft SQL server is the target. We moved methodically step-by-step through the network toward our nefarious goal.
All Secure Enterprise product reviews are conducted by current or former IT professionals in our own Real-World Labs, according to our own test criteria. Vendor involvement is limited to assistance in configuration and troubleshooting. Secure Enterprise schedules reviews based solely on our editorial judgment of reader needs, and we conduct tests and publish results without vendor influence.
The Metasploit Project recently released Metasploit Framework 2.4. Metasploit is a penetration framework similar to Canvas and Impact, with the notable exception that it is open source and licensed over GPL and Artistic License. Metasploit offers many of the same features as its commercial cousins. Granted, it doesn't have automated tests or remote commands in the shell, but as a tool to break in to hosts, Metasploit's CLI and module execution for scripting can't be beat.
With the latest distribution come 75 exploits, many covering vulnerabilities made public in the past year, and there are several payloads available to inject into the vulnerable host process. Most of the payloads bind to a network port or reuse a connection and spawn a shell on the target. A few payloads also act as stage loaders that can execute larger payloads once they are resident.
Metasploit has payloads for Windows, Solaris, Linux, BSD and BSDi, a somewhat broader set of OSs than Canvas or Impact provides. Unless you're up for developing your own shell code, the provided payloads work extremely well, especially if your sole purpose is to test for a vulnerability. Depending on the exploited process, you many need to elevate your privileges, but you will have to get local exploits to your target through another method, such as SCP, FTP or TFTP. This is where the cost of a commercial product may be justified to gain the functionality to run both remote and local exploits.
Using Metasploit, scripting languages, a network scanner and other common tools, you could cobble together a flexible customized automated penetration test suite, complete with host database, scheduling and reporting. Talk about a fun project.
Source: Network COmputing's Security Pipeline - CMP Media (Secure Enterprise Magazine)
http://nwc.securitypipeline.com/shared/article/pri
