Protocol analyzers are often used to capture, decode, and evaluate traffic flows and packets for network debugging, troubleshooting, and optimization. But did you know that a protocol analyzer can also be indispensable for security incident investigation?
Perhaps the best-known open source protocol analyzer is Wireshark (nee Ethereal), capable of decoding scads of protocols, captured from wired or wireless networks using nearly any laptop, desktop, or dedicated “shark appliance.” Wireshark is freely-available and community-supported by plug-ins (dissectors) for new protocols.
But, even though Wireshark is free and flexible, there are times when it could use an assist – or as CACE Technologies might put it, a pilot to guide this large, complex “fish” through a narrow passage. That’s the purpose of CACE Pilot ($1295), a product that cuts large-volume traffic captures down to size through visualization, drill-down, reporting, and more – eventually kicking off Wireshark when and if necessary to complete a task.
Security depends on knowledge
We’ve been using CACE Pilot to watch live traffic and dig into capture files for several months. Pilot can be handy for many different tasks – especially those that benefit from large-volume traffic visualization and statistical analysis, such as performance reporting.
But we focused on using CACE Pilot for network security tasks, such as spotting unexpected protocols on a WLAN or determining which infected hosts are DoS-ing a LAN. After all, you can’t know that a network is really secure if you can’t see who’s using it and how.
Of course, there are many ways to monitor traffic, from router and firewall logs to network intrusion detection and forensics appliances. These and other tools can save capture files for future use. Where protocol analyzers excel is by interpreting those captured bits and bytes to deliver insight into sources/destinations, conversations, applications, and user activity.
If you’re only interested in history, you can drill into saved captures with a protocol analyzer. If you’re responding to an incident, you can use a protocol analyzer directly for live capture. Either way, protocol analysis is a fast way to get a grip on network activity by drilling down until you find what you’re looking for (or hoping that you wouldn’t find).
But it’s far too easy to get lost in packet details. Browsing a long list of decodes is an inefficient way to understand who is talking to whom in a large active LAN. With an analyzer like Wireshark, you can filter on most protocol fields/values – but constructing long nested filters to drill-down is tedious. Wireshark can also reconstruct TCP sessions or conversation lists, letting you work your way back from selected packet details to deliver some higher-level perspective.
You can do a lot with a good protocol analyzer and a small, focused capture file. But when you start with a huge file, much of it unrelated to the task at hand, this process can be slow, labor-intensive, and yield results that are hard to communicate to less technical folk. In our view, this is where CACE Pilot adds value—by reducing the time it takes to focus on what’s important, and making it easier to recall and share what you found.
Pilot fundamentals
So what does it take to buy and use Pilot? This software package starts at $1295/seat. We tested a Pilot + AirPcap NX bundle ($1923) – the latter is a USB stick that can scan 802.11a/b/g/n on Windows PCs. Updates are included with Pilot for one year and $300/year thereafter.
Each license lets Pilot run on a single Windows 7 (32 or 64-bit), Vista, or XP PC with min 1024×768 display. CACE recommends a dual-core 2.0 GHz CPU, 2GB RAM and 300 MB storage. We installed Pilot on an XP laptop with min specs and a slightly more powerful 64-bit Win7 PC. Pilot was responsive and reliable on both, capturing 10/100/1000 Ethernet and 802.11a/b/g/n Wi-Fi frames, except for one repeatable XP crash that CACE is investigating.
Although you don’t have to use Wireshark to use Pilot, the installer also includes Wireshark and its faithful companion WinPcap. CACE participates in community efforts surrounding Wireshark, but Wireshark updates are released independently. Fortunately, we had no trouble upgrading Wireshark to a new version released after we’d installed Pilot v2.2.
After installation, Wireshark can be conveniently launched from within Pilot as needed. For example, when drilling into a large capture file, a filtered subset can be sent to Wireshark to view per-packet decodes. Alternatively, Pilot can launch Wireshark to capture live packets, with or without filtering. Here, Pilot serves as a GUI-driven shortcut to opening captures in Wireshark; you don’t need to know filter syntax or fiddle with NIC parameters to do so.
But Pilot does require one or more traffic sources. When Pilot opens, available sources appear in a control panel as Devices or Files. To perform historical analysis, just add one or more .pcap files (or folders) to the Files list. However, Pilot can only handle .pcap files (including Radiotap files); other file formats (including raw 802.11 files) must be converted to .pcap first.
For live analysis, just choose a source NIC from Devices. Pilot supports any Ethernet NIC, but due to Windows RFMON limitations cannot scan with ordinary Wi-Fi NICs. To get around this, CACE sells RFMON-capable AirPcap USB sticks, starting with the b/g Classic ($198). Using an AirPcap, Pilot can scan a configurable set of channels or inject Wi-Fi packets. However, Pilot can still only capture packets from one Wi-Fi channel at a time. Capturing from several channels requires multiple AirPcaps. If you need to dig into non-802.11 RF as well, purchase sibling product WiFi Pilot instead, which includes the MetaGeek Wi-Spy spectrum analyzer.
Visualizing traffic
Pilot decodes, filters, and performs statistical analysis on supplied packets, displaying graphical results using “Views.” To watch live traffic, drag and drop desired View(s) onto a capture Device. To crank through previously-captured traffic, drag View(s) onto File(s). To narrow any View’s scope, select/edit a Wireshark-formatted filter (e.g., “port 80 or port 443”). Although you can’t change a filter applied to an active View, any filtered (or otherwise modified) View can be saved for later reuse.
Pilot groups dozens of predefined Views into seven categories:
- Generic
- 802.11 Troubleshooting
- LAN/networks/ Troubleshooting
- Bandwidth Usage
- Top Talkers and Conversations
- Performance and Errors
- Web/VoIP User Activity
This list is growing as CACE develops new Views based upon customer requests. For example, v2.2 included new VoIP SEER, Wi-Fi roam time, and TCP RTT Views. Unfortunately, there is no SDK to develop your own View – for example, if you wanted to dig into streaming video sessions at the same level of detail now supported for Web and VoIP.
But what the heck IS a View? Let’s illustrate a few simple Views from the Generic category: a table that sums up bits/bytes/packets analyzed and a graph of frame sizes, plotted over time. We only used these Views as launch points for something else (e.g., to watch for traffic bursts). Other frequently-used launch point Views include IP Conversations, Network Usage by Traffic Type, Protocol Distribution, and Bandwidth Over Time.
Each View category contains a collection of bar/pie/strip charts, real-time graphs, scatter diagrams, conversation rings, and/or “watches.” For example, the following Views are grouped under 802.11 to support layer 2 Wi-Fi troubleshooting tasks.
- The Discovery View presents an AP/Station table, organized by channel and SSID – similar to typical “Wi-Fi stumbler” output – along with a pie chart of management, control, and data frame distribution. From this View, it takes one click to drill into packets for any single channel/SSID or type of frame.
- Traffic Analysis Views enumerate “top talkers” per channel, SSID, or station; chart traffic distribution by frame type, transmission rate, encryption type, and vendor; and graph associate or probe requests by host and channel. For example, from here you can drill into frames sent by one top talker.
- Scatter Diagram Views use circles of varying sizes to emphasize the most active transmitters and channels – for example, depicting retransmissions, channel utilization, signal strength, and disassociations per Station or AP (identified by their MAC addresses).
- More 802.11 Views depict host/SSID/channel utilization, FCS errors, retransmissions, signal, noise, and inter-AP roam time.
- Finally, a set of 802.11 Watches are available to trigger actions whenever Pilot hears an unknown or unencrypted AP, or a low data rate/weak transmitter (see Live Monitoring).
These Views – and dozens in other categories – help you visualize traffic, eyeballing live or captured packets in different ways. But note that Views only include packets supplied by a source. As you drill down, Pilot constructs nested filters to suppress extraneous packets while homing in on those relevant to you. Moreover, instead of making you type long cryptic filters, Pilot lets you click on graphs and context-aware menus to pivot through potentially large files.
Pilot also lets you zoom in/out on periods of interest by selecting parts of a graph or using time control slider ribbons. When zooming, Pilot adjusts statistic intervals – that is, it doesn’t just make a flat line bigger; it displays dips and spikes not otherwise apparent. You can even “replay” data by clicking forward/backward through a capture – we found this useful to notice longer trends.
Finally, Pilot can export any or all Views as Reports in .PDF, Excel, Word, HTML, or text format. This isn’t just handy to print whatever appears on your desktop. Layouts can be customized to communicate professional results to managers, customers, etc, and may include checksums to correlate Views with saved captures.
Trading free for faster and easier
Could you create these selections and filters in Wireshark? Sure, with enough protocol savvy and imagination. Could you apply them directly to large capture files in Wireshark? Yes, if you had a whole lot of RAM and patience. Could you create Pilot’s colorful graphs and reports on your own? Perhaps with considerable effort, but probably not on the fly.
In short, Pilot’s value proposition is saving time and effort in exchange for a software license. Pilot isn’t for those who rarely perform protocol analysis or who do so only for small, familiar tasks. But Pilot could be a godsend for anyone who spends hours or days trying to make sense of unfamiliar traffic – especially when searching for the proverbial straw in the haystack.
That said, we still had to learn to use Pilot wisely. View summaries can be seen by hovering over them. When drilling down, Pilot nests refined Views and recaps the sequence that you got there. These make it tempting to create many Views fast. But filtered View tabs have the same name as their unfiltered counterparts, leading to confusion. Unused control panels and menu bars can be hidden, but it doesn’t take much to produce a busy, cluttered desktop.
Furthermore, Pilot does not make some I/O intensive tasks fast. IP/MAC addresses can be resolved, but we didn’t spot a “names table” to speed resolution in familiar networks. Web user activities can be broken down into per-Client URLs and objects, but that’s not a good idea with too many HTTP packets. Just because you can do something still doesn’t mean you should.
Live monitoring
Views applied to capture files crank out graphs and tables for a static set of packets, but Views applied to live sources only display real-time graphs/tables for newly-arrived packets. You can pause a live View and step back through it, but drilling down to a new View uses future packets.
Drill-downs are easy when you have already received traffic of the desired type from a network, IP, or MAC of interest. But this is not so when launching your first live View. You can wait for traffic from a suspicious node before drilling down, but then your new View will miss some traffic. Or you can manually construct a filter, thereby missing benefits. Hint: Save past Views as future “templates.”
Live Views are great for brief periods, but who wants to watch a graph for hours? Fortunately, CACE added Watches to Pilot v2: configurable events caused by triggers that cause actions. To create a Watch, you must select something in an existing View – an IP or MAC or protocol. Each Watch contains a Pilot-generated filter (what Pilot is “watching” for), configurable severity, trigger condition(s) (e.g., bps > N), and one or more predefined actions (e.g., display event, send event to e-mail/syslog/Twitter, start a packet capture).
?
Watches make live Views far more useful. For example, we’d rather capture all packets for ten minutes after a rogue AP is detected than be clueless about what it did. Available trigger conditions depend on Watch type. For example, we defined Watches to escalate the same filtered event by using different trigger values to cause increasingly significant actions.
However, Watches could be easier to maintain. We’d love to create a Watch without drilling into a View. Although Events appear in a searchable list, Pilot does not have a dashboard to roll-up events by type or severity. We could not find a way to see all defined Watches at once or to apply an edit to several Watches (e.g., change an e-mail address). And although Watches can post events to Twitter (why?), they can’t fire off a local script. Such features are network surveillance staples.
Putting Pilot to work
Given this overview of Pilot capabilities, let’s illustrate how one might use Pilot to detect or investigate security incidents.
- Assessments often turn up unexpected nodes – “dumb” devices churning out traps to nowhere or listening to unpublished Web or telnet ports. To spot these with Pilot, connect your laptop to a switch mirror port or tap – a location that can sniff packets from everyone. (Or sniff packets at several points throughout your network and load capture files into Pilot.) Launch the IP Conversations View: a conversation ring that shows who is talking to whom. Resolve all hostnames; are there any that don’t belong? Select that endpoint and drill down to Protocol Distribution – what is that suspect doing? Select a protocol it’s sending – for example, choose HTTP and drill down to Served Web Pages. Or send all packets of that type to Wireshark.
?
- Network intruders often perform reconnaissance that may be detected with protocol analysis. For example, use Pilot to create a Bandwidth Over Time View and then watch for Specific Traffic Types used for recon scan (e.g., ARP, ICMP). Or better yet, create an ARP Overview or ICMP Overview, then watch for excessive bit rates to not only detect when a scan occurs, but identify top sources.
?
- In WLANs, unexpected traffic can have big security implications. If your WLAN requires data encryption, the presence of unencrypted data might indicate a misconfigured or unauthorized AP – Pilot can detect these by applying a Watch to an AirPcap device. But what about unexpected protocols sent on your encrypted WLAN? Pilot can do that too, if your WLAN uses WEP or WPA/WPA2-PSK and you configure keys/SSIDs into Pilot. A Pilot report can be a good way to enlighten a systems’ owner about what they’re sending.
?
- Hosts infected by worms, Trojans, or spambots can send heavy traffic. Even if packets are blocked by egress filters, they can cause DoS conditions or put other hosts at risk. Of course, other security systems may detect this incident, but as you start investigating, Pilot offers a quick and easy way to see where most of your network’s traffic is coming from. Launch a Top IP Talkers or Top IP Sources View. Select a troublesome host and drill down to a Local vs. Non-Local View to see if other inside hosts are involved, or create a Detect Conversation Watch to be alerted when suspicious protocols are exchanged between hosts. And so on.
?
These are just a few simple examples of how Pilot can support security-related tasks. To be sure, there are limitations, like capturing on a single channel and recon activities that are too low-and-slow to trigger a Watch. And some Wireshark users are a whiz with filters and comfortable using a protocol analyzer to do much of this.
However, we hope these examples illustrate how Pilot can make it easier to visualize traffic that could be associated with a security vulnerability or incident. By quickly looking at live or captured packets in many different ways, filtering out unrelated noise, producing smaller focused capture files, and communicating statistical results through graphics and reports, Pilot builds on Wireshark’s foundation to deliver a more powerful tool.
Lisa Phifer owns Core Competence, a consulting firm focused on business use of emerging network and security technologies. Lisa has been involved in design, implementation, and testing of network protocols since 1982. She’s a long-time fan of Wireshark and got hooked on wireless LAN analysis back in 2002.
