DailyDDoSe: Stress-testing your network

Stress-testing your network

BreakingPoint, Mu Dynamics and other high-end tools help you know whether your network can stand up to real-world attacks

Neil Roiter, CSO

July 19, 2010

Contemporary IT infrastructure and applications operate in an extreme environment barely envisioned a decade ago, pushing networks to the limit and challenging the security industry to keep pace.

A handful of high-end testing products have had to evolve quickly to meet those challenges and evaluate how network and security devices perform under stress, and isolate and repair flaws.

Here's why, in a nutshell: Service provider and enterprise networks are performance-challenged, being called upon to support enormous high-speed traffic loads. That traffic is increasingly complex, comprising a growing array of protocols and applications supporting converged IP services—voice, video, data—and performance-sensitive online transactions. Throw in plenty of malicious-attack traffic and see how networks, network devices and network-based security products, from firewalls to intrusion-prevention systems (IPS), perform under stress.

See the companion article "Network stress testing tools: Dos and don'ts"

How do carriers know if their infrastructure will support their service-level agreements with demanding enterprise customers? How do enterprises know if their networks and data centers can support their business requirements and whether their network and security vendors' gear is really up to the job? And how do network and security vendors know that their products can deliver what they claim in their data sheets?

In this Toolbox, we'll explain how products from BreakingPoint Systems, Ixia, Mu Dynamics and Spirent Communications can be used to test networking and security gear and the applications they support to the limit, and how different types of organizations can leverage their unique capabilities.

The market
These are very expensive products that require permanent test-lab facilities and dedicated, expert staffing to deliver their full benefit.

Historically, the market has focused primarily on network equipment manufacturers and large service providers. Ixia and Spirent, which have specialized in generating heavy traffic, have dominated in testing load-bearing capabilities in the lower layers of the Open System Interconnection (OSI) stack.

The market has grown broader as the traffic mix has grown more complex, adding more and more protocols, high-performance applications, and attack traffic. Security vendors are important buyers now, and some government agencies "look a lot like a carrier or service provider," says Elisabeth Rainge, IDC program director for network software.

Security-sensitive agencies, especially in defense and cybersecurity, are also good customers, especially for those products emphasizing security.

Mu and BreakingPoint have entered the competition in recent years, emphasizing power security and application testing. Ixia and Spirent, which are well known for load testing, are moving up the stack as well, augmenting application and security capabilities.

In general, Rainge says, telecommunications companies and network equipment providers tend to still be focused on performance—though there are, of course, exceptions—while enterprises and service providers with a strong IT heritage tend to focus more on the application layers, in addition to security.

"If you're coming from an IT perspective as opposed to telecom or network, you're thinking more in terms of what application is involved or what is the end-user experience or how is this technology fitting with how our business is really doing," she says. "You don't necessarily have end-use case in mind; there's a reasonable chance you're looking at the network as more of a dumb pipe. It's performance rather than what kind of business a company is in."

Very large enterprises are becoming more important as customers. The enterprise buyers are generally large financial institutions—which can lose millions of dollars in an hour's downtime—and very large, complex companies.

But potential enterprise buyers are a relatively short list of large organizations that have the money, talent and commitment to testing to justify the purchase.

"Very high-end enterprises, major financials, anyone where it's mission-critical and not running standard traffic—guys like an eBay and Amazon—are a good fit for these kinds of tools," says Vik Phatak, chairman and CTO at NSS Labs. "Start talking lower than that, and it becomes problematic. Cost justification doesn't make sense. They're expensive and complicated."

Sorting out the tools
The common denominator among these four companies' products is that they are ubertools designed to throw a phenomenal volume and/or assortment of traffic at the target systems. To some extent, they're competitors; in some cases, they're complementary.

"Mu is more about security. BreakingPoint is aligned with security but leans a little more toward the network conversation," says Rainge. "Ixia and Spirent are casually referred to as 'packet blasters'; that's a very casual way to refer to load generation. As companies, they are competitors."

Ixia and Spirent have the longer pedigrees and are especially well known for load-testing networking equipment.

Spirent's Avalanche, which focuses on testing the capacity, performance and security of Layers 4 through 7, is available on its own or on Spirent's flagship network testing platform, TestCenter. Its primary security capability is vulnerability assessment, testing for thousands of known attacks and variants under a heavy application load of normal and malicious traffic.

Spirent recently released Avalanche Virtual, which can be loaded as a virtual instance to test the performance and security of virtual network and security appliances.

Ixia says it will be expanding its security testing with a new suite in a future release that expands on the fuzzing capabilities of IxDefend, adding some 6,000 different vulnerabilities and dozens of evasion techniques that can be thrown at the target device under heavy traffic loads. (Editor's note: Corrected from earlier version, which suggested that IxDefend would be replaced.)

Mu is a very different sort of product. It can design and launch an endless variety of attack and malformed traffic at its target. Mu uses its fuzzing technology on packet captures from the customer network to produce unpredictable and unexpected traffic for both functional and security testing, so the test traffic reflects the environment in which the tested device will work. Users can develop their own tests and leverage test profiles from the Mu user community and Mu's own library. Where Ixia and Spirent torture systems under an incredible volume of traffic, Mu tortures them with the sheer variety of possible traffic permutations.

For that reason, it's not unusual for Mu to be used together with Ixia or Spirent. In fact, the Mu website includes instructions for integrating Ixia and Spirent, so users can deliver Mu's smorgasbord of traffic under real-world loads. That may be an attractive option for organizations that are already heavily invested in Ixia or Spirent but like Mu's application and security technology.

Also see Joseph Guarino's "3 open-source options for network security"

BreakingPoint's Storm Cyber Tomography Machine appliance combines higher-layer testing and load tolerance, generating high volumes of stateful application traffic. It blends legitimate types of traffic from some 140 global applications, from Oracle database traffic to Skype to World of Warcraft, with a library of about 4,500 known attacks and 80 evasion techniques. It also uses fuzzing techniques to stress systems.

"Mu excels in the mutation part and fuzzing; it's really good at simulating both traffic and in its fuzzing capability. They do a little better than BreakingPoint," says Avishai Avivi, senior director of high-end security and services at Juniper Networks, which uses both products. "But MU doesn't have scale and ability to test networks at the speed we need them."

Juniper also uses Ixia and Spirent products for network testing lower on the OSI stack.

Use cases
These tools have a number of use cases for vendors, service providers and large enterprises.

Network equipment testing. Network equipment vendors have long used Ixia and Spirent, in particular, to test networking equipment before it's released on the market. The emphasis is on performance under load, incorporating a wide range of network and higher layers to approach contemporary network conditions.

Security product testing. Security vendors are now using them as well, to test their products' detection capabilities under stress.

"Security-testing devices don't traditionally measure performance," says Avivi, who uses BreakingPoint to test Juniper's SRX Series Services Gateway appliances. "We started with BreakingPoint because of security and very quickly learned it could help us do stress testing under real traffic conditions."

Vendor evaluation. Both service providers and large enterprises will use these tools in vendor evaluation, either on a case-by-case basis or in a bake-off among, say, three IPS vendors. They might test for performance or the ability to detect attacks, or both. It's no secret that intrusion-prevention vendors struggle at times to avoid creating a bottleneck if performance begins to lag while still providing the ability to detect attacks and evasion techniques.

"Vendors are not testing as rigorously as should be, especially from a security standpoint," says NSS Labs' Phatak. "Performance, they tend to get; security, not so much. Part of it is that people can see when something slows down, but they don't know if the IPS misses something."

In addition to making purchasing decisions, organizations can test before they buy to detect flaws they want the vendor to fix.

Data centers. Large organizations can test performance and security for data center upgrades, creation, expansion or, increasingly, consolidation to take advantage of virtualization.

New and upgraded applications. These tools can be valuable for performance, security and interoperability. Organizations will want to see how new or modified applications will behave on their networks: Will they be stable? Will they create problems for other applications? Will performance degrade?

"Say you know that you need to support 20,000 customers per hour and know how they behave—how many transactions per second, certain types of traffic, maintaining state," says Phatak. "These tools can replicate that. If performance goes down to 18,000 customers per hour, you may need to add servers; or maybe you can now do 25,000 and can scale back."

Virtualization. Virtualization reduces equipment costs, power consumption, space requirements and management overhead. But right-sizing the number of virtual machines and types of applications on a particular physical host gets tricky. (See Tom Olzak's "Server virtualization and control contexts") Phatak notes that virtualization vendors tend to talk about capacity in terms of the number of VMs you can install on a box, but not the amount of work it can actually support. Power testing tools allow you to evaluate the performance of combinations of servers and their applications on a particular server.

Cloud services. Organizations can use these products to evaluate a cloud provider's SLA against actual performance for their users, or test in advance to determine the level of services they need to purchase.

Consulting and integration services. Consultants can evaluate prospective products for their clients; integrators and IT architecture service providers can test their planned and implemented projects.

"In some cases, we test new services for clients, or new software they've written in-house, before going into production, to see if it can hold up," says Ed Skoudis, a founder and senior security consultant at InGuardians, which uses Mu products. "Humans can focus on desired business functionality and look for strange cases of business logic errors. Mu can automate security testing by throwing a lot of garbage, known attacks and custom attacks."

Specialized testing. Mu, in particular is valuable for crafting and delivering traffic to test particular environments.

For example, it's being used to test security products—network firewalls, intrusion detection and prevention, application firewalls—for IPv6 certification at the University of New Hampshire's InterOperability Laboratory.

"Otherwise, we would have to create our own test infrastructure and create all these vulnerability attacks and traffic patterns," said Tim Winters, a senior manager at the lab. "You could send one packet and trigger on it, but to do a whole stream or whole stateful firewall is much more complicated, and it's a lot easier to have an off-the-shelf solution we can manipulate."

And InGuardians is using it to test smart grid equipment.

"We created some test tools," Skoudis says. "We'll use Mu to generate traffic and our tools to deliver it to the targets—smart meters, SCADA systems, stuff that's in the grid."

© CXO Media Inc.