If you are looking for an advanced NFAT (Network Forensics Analysis Tool) you should definitely check out Unsniff Network Analyzer. It has a fast and intuitive GUI but what sets it apart is its comprehensive automation interface. Analysts who know or are willing to learn a bit of Ruby can automate processing PCAP files. Why is that goog ? Because the worst job in the world has to be performing repetitive tasks on PCAPs.

Lets take an example.

The task

Here is an example of batch processing

Task : You have a PCAP file and want to save all user objects into a directory.

And here are four ground rules :

Not allowed to use a GUI. This process should be hands free (no clicking)
Must retain complete control of what to save, what filenames to use, post process files, etc.
Must be able to access network data like HTTP headers, source / dest IPs, cookie information all the way to protocol details.
Generally be able to do whatever you want to the output via simple Ruby scripting.

Automation objects

The plan of attack is always the same :

Import packets into Unsniff format (*.USNF)
Get hold of one of the top level collection objects (Packets, Flows, PDUs, UserObjects). Consult the scripting guide – currently only available as PDF for a list of objects/ methods/ properties. We are working to put out a HTML version of this guide.
Iterate over the collection – calling methods and properties of members as you go

For this task, we going to import a PCAP file, grab the UserObjects collection and iterate over each object calling the SaveToFile method on each.

The meat of the script is these 3 steps.

Step 1 Import your PCAP file into Unsniff format

This fragment converts the input pcap file stored in InputTCPD variable into the Unsniff format file temp_cap.usnf. We use a temporary file because we are going to toss it out once we extract the user objects into their own files. You can of course keep it in that format which opens in Unsniff Network Analyzer instantly.

UnsniffDB = WIN32OLE.new("Unsniff.Database")
UnsniffDB.New("temp_cap.usnf")
UnsniffDB.Import("libpcap",InputTCPD)

UnsniffDB = WIN32OLE.new("Unsniff.Database")

UnsniffDB.New("temp_cap.usnf")

UnsniffDB.Import("libpcap",InputTCPD)

Step 2 Grab the UserObjects collection, iterate and save

Once the data is available in Unsniff format, you can simply call the db.UserObjectsIndex property to get the collection. Here are the other available collections.

PacketIndex – Iterate over each packet, dive into protocol fields
PDUIndex – Messages such as TLS records, SMB records, etc
StreamIndex – TCP sessions
UserObjectsIndex – Extracted objects (our focus for today)

You can use the ruby enumeration methods to iterate or even a for-loop. Finally we call the SaveToFIle method with the PreferredFileName. There are about 15 other properties of a user object (see guide) you can play with.

UnsniffDB.UserObjectsIndex.each do |uo|
	next unless uo.State == "Complete"
	uo.SaveToFile uo.PreferredFileName
end

UnsniffDB.UserObjectsIndex.each do |uo|

next unless uo.State == "Complete"

uo.SaveToFile uo.PreferredFileName

end

Thats it !

Step 3 Run the code

Run this code over your pcap file

ruby xuo.rb  bigcapture.pcap output-dir

1	ruby xuo.rb bigcapture.pcap output-dir

Check the output directory where all your files are

xuo-output — All content extracted with correct filenames

Full code

The ready to run code is shown below. It adds a check for duplicate output filenames. Save it as myfile.rb and run as shown above.

# ------------------------------------------------------------------
# xuo.rb	Extracts all user objects into a directory
#		You can extend this script for NFAT use
#
# usage 	xuo.rb pcap-file output-dir
# ------------------------------------------------------------------
require 'win32ole'
require 'pathname'

raise "Usage : xuo.rb pcap-file output-dir" unless ARGV.length == 2

InputTCPD, DirName  = ARGV.shift, ARGV.shift

File.delete("temp_cap.usnf") if File.exists? "temp_cap.usnf"
Dir.mkdir(DirName) unless File.directory? DirName

UnsniffDB = WIN32OLE.new("Unsniff.Database")
UnsniffDB.New("temp_cap.usnf")
UnsniffDB.Import("libpcap",InputTCPD)
print "Successfully imported pcap " + InputTCPD

dup_cnt = 0
UnsniffDB.UserObjectsIndex.each do |uo|
	next unless uo.State == "Complete" 

	opath = Pathname.new(DirName) + uo.PreferredFileName
	if File.exists? opath
		opath = Pathname.new(DirName) + "U_#{dup_cnt}_#{uo.PreferredFileName}"
		dup_cnt = dup_cnt+1
	end

	p "Writing " + opath.to_s
	uo.SaveToFile opath.to_s

end

UnsniffDB.Close

# ------------------------------------------------------------------

# xuo.rb Extracts all user objects into a directory

# You can extend this script for NFAT use

# usage xuo.rb pcap-file output-dir

# ------------------------------------------------------------------

require 'win32ole'

require 'pathname'

raise "Usage : xuo.rb pcap-file output-dir" unless ARGV.length == 2

InputTCPD, DirName = ARGV.shift, ARGV.shift

File.delete("temp_cap.usnf") if File.exists? "temp_cap.usnf"

Dir.mkdir(DirName) unless File.directory? DirName

UnsniffDB = WIN32OLE.new("Unsniff.Database")

UnsniffDB.New("temp_cap.usnf")

UnsniffDB.Import("libpcap",InputTCPD)

print "Successfully imported pcap " + InputTCPD

dup_cnt = 0

UnsniffDB.UserObjectsIndex.each do |uo|

next unless uo.State == "Complete"

opath = Pathname.new(DirName) + uo.PreferredFileName

if File.exists? opath

opath = Pathname.new(DirName) + "U_#{dup_cnt}_#{uo.PreferredFileName}"

dup_cnt = dup_cnt+1

end

p "Writing " + opath.to_s

uo.SaveToFile opath.to_s

end

UnsniffDB.Close

More to come with Trisul

There are quite a few code samples of this kind at http://www.unleashnetworks.com/devzone/unsniff/script-library.html

We are also excited about the next upcoming release of Trisul (Release 2.4). It is going to feature very strong support for scripting. Trisul is a 24×7 system that does traffic monitring and tucks away flows, counters, and packets. You can combine the high level capabilities of Trisul with the deep analysis of Unsniff to create very powerful scripts.

Here is a sample :

Search all flows in the past week for a particular IP
Get all packets for those flows
Extract all content in those flows
Run them past malware scans
Automatically update an internal project page with status of check
Send an email to your team if anything was found

You can do all these tasks today by clicking around – but the idea behind Trisul and Unsniff is that you focus on creating repeatable scripts and tweaking them. I.E do the fun stuff – leave the heavy lifting to us.

—-

Links:

Trisul Web Site

Free Unsniff Download

Author: info

New Trisul builds released – fixes CPU usage issues

Awesome new flow features in Unsniff

Detect packet loss in captured stream

Export a flow to a separate capture file with a single click

Several more

Network forensics with Unsniff and Ruby