In Part-1, we used the Unsniff Scripting API to read a PCAP file and print the Certificate Chain for all HTTPS connections in it.

What if you did not have a PCAP (Packet Capture) file ?

In the real world, getting a sufficiently narrow PCAP file is the problem. If you have an NSM system with a strong API – you can conjure up whatever analysis you want on previously captured data. Using the Trisul API  (a.k.a Trisul Remote Protocol or TRP) you can write Ruby scripts to :

• securely connect to a Trisul Probe
• search for various types of data (traffic stats, flows, alerts, URLs, DNS, and packets)
• pull out required PCAPs for further deep processing by Unsniff or Wireshark

Task for Part 2

We have a TRP Server running on demo2-dot-trisul-dot-org – your task is to connect to this server, search for all HTTPS activity from a suspicious host 192.168.1.105 over the past 1 month and print out the certificate chain of each connection. This will help you cut through several gigabytes of packets.

The setup for TRP

Try it out first

Before we explain the code, lets gratify ourselves by running the sample code and getting some output.

1. Install Ruby and the trisulrp gem (see the tutorial for help)
2. Install Unsniff Network Analyzer (free) from the downloads page. You need this to do the deep analysis. Sorry this is a Windows MSI. If you are running Linux just comment out the print_cert_stack function.
3. Download the csx.rb script from the samples page
4. Download the demo client cert and key from  and place them in the same directory

Note: You dont need to install Trisul or the Web Interface. We already have a probe running on demo2trisulorg. You are just setting up a script client environment.

Run as below (password for the private key file is ‘client’ )

The csx.rb code

The code is quite straightforward.

Step 1. We connect to TRP and retrieve 20 HTTPS flows for IP 192.168.1.105 for the entire time interval available. The message used here is KeySessionActivity (give me all flows by IP and/or Port)

Step 2 : For each flow in the response, pull the packets out of Trisul. The message used here is FiltereredDatagramsRequest for each flow. Note we have capped the :max_bytes at 20,000. We use a trick here, we only retrieve the first 20K bytes of each flow because the Server Certificate is usually exchanged at the very beginning of a SSL session. This dramatically reduces the data transferred.

The full code is available as csx.rb from the TRP Samples Page.

Have fun !

—

We just released Trisul 2.4. The major new feature in it is the API (Trisul Remote Protocol).  Download it and let it watch your network. You never know when you may need its data.

 

In this two part post, we are going to see how we can utilize the scripting capabilities of Unsniff and Trisul to build our own automated analysis tools. The task here is to scan all HTTPS traffic and print the certificate chain for each session seen.

In Part 1 (this post) : We will use a standalone Unsniff script in Ruby to extract this information from a packet capture.

In Part 2 (next post) – We will see how we can use a Ruby script to connect to a Trisul sensor, pull out all HTTPS certificate chains accessed by a particular IP over 24 hours.

In the following example, the session between 212.149.50.181 and 192.168.1.5 is authenticated by the chain shown below it. The chain is :  www.commerzbaking.de is signed by TC Trust Centre which is in turn signed by Cybertrust Global Root which is in turn signed by GTE Cybertrust Global Root.

Where do we get this information ? As part of the SSL/TLS handshake the remote server sends its certificate chain in a protocol message called Server Certificate. Our Ruby script will look for these messages and print out the chain in the following format..

Using the Unsniff Scripting API

We will write a tiny Ruby script and the Unsniff Scripting API to accomplish this task.  (full code available here)

1. Pull out all the PDUs containing a Server Certificate.
2. For each cert in chain; navigate and print the commonName and organizationName of the subject and issuer.

Key methods in the script

How to pull out all reassembled SSL/TLS PDU records which contain a Server Certificate?

A quick note : Users of Wireshark maybe a bit confused here. In Wireshark the unit of analysis is the link layer packet, i.e Ethernet or Wireless frames. Typically the final packet in the stream contains a link to reassembled content. Unsniff monitors PDUs as top level units. What you see in the PDU sheet are reassembled messages without regard for packet boundaries. TLS is a message layer built on top of a bytestream layer TCP. So we need to work with PDU’s for this example.

Luckily for us, Unsniff sets the Description field of each PDU to contain the names of handshake messages.  So we can just select the PDUs which contains “Server Certificate” anywhere in its description..

Collect all the certificates in the stack

Frequently a Server Hello + Server Certificate + Server Hello Done are packed intoa single PDU. We only need to work with the “Server Certificate” its easy to select this as the code shows below.

In the above example, we are wrapping the pdu.Fields method in an Enumerable wrapper . This allows us to mix-in methods like Find and Select to the Unsniff Scripting objects which are backed by C++ classes.

How to pull out issuer and subject names ?

At this point, we now have a handle to each certificate in the chain. Our next and final task is to print the issuer and subject details. Our friend is the FindField method

Running the script

The complete script (xcert.rb) is available at the Unsniff Scripting Samples pages on our new Wiki.

To run this script.

1. Download Unsniff Network Analyzer (its a free download). Note that Unsniff is a Windows app.
2. Download and install the latest Ruby Windows One Click Installer 
3. Download the xcert.rb script
4. Capture some packets and save it as USNF format. You can also work with PCAP files directly, but you have to modify the script to import the PCAP file into USNF format first. See samples in Import / Export section for hints.
5. Run the script

For the analyst with some scripting skillz ?

The philosophy of both Unsniff and Trisul is to put powerful tools in the hands of the analyst. With mid-level skills in Ruby (or even VBScript) you can do amazing things automatically. Take out the tedium of clicking through to perform repeatable tasks. As an exercise you can extend this script to do the following:

• download the root certificates included with Firefox and compare the CAs in your chain for validity

 

Part 2 : Add Trisul scripting

We have seen how you can do such deep analysis with Unsniff scripting. But this requires you to have a capture file of a manageable size. What if you wanted to

• check all of your traffic during 9AM to 11AM yesterday and print the cert stack of each SSL session
• analyze all SSL sessions during the last week from a workstation 192.168.1.22 – Say you found malware in this machine and want to audit its past activity
• flag all TLS sessions which did not have a valid root certificate, yet the user overrode the Firefox warning and proceeded to complete his/her transaction

For this kind of analysis involving long time ranges and multi-gigabytes of data, Trisul must be used in conjunction with Unsniff. Our latest release of Trisul 2.4 features a very powerful remote scripting API called TRP (Trisul Remote Protocol). Part – 2 will extend this sample to show how you can perform this analysis over large time ranges.

Till next time. Happy packet hunting.

 

 

 

If you are looking for an advanced NFAT (Network Forensics Analysis Tool) you should definitely check out Unsniff Network Analyzer. It has a fast and intuitive GUI but what sets it apart is its comprehensive automation interface. Analysts who know or are willing to learn a bit of Ruby can automate processing PCAP files. Why is that goog ? Because the worst job in the world has to be performing repetitive tasks on PCAPs.

Lets take an example.

The task

Here is an example of batch processing

Task : You have a PCAP file and want to save all user objects into a directory.

And here are four ground rules :

1. Not allowed to use a GUI. This process should be hands free (no clicking)
2. Must retain complete control of what to save, what filenames to use, post process files, etc.
3. Must be able to access network data like HTTP headers, source / dest IPs, cookie information all the way to protocol details.
4. Generally be able to do whatever you want to the output via simple Ruby scripting.

Automation objects

The plan of attack is always the same :

1. Import packets into Unsniff format (*.USNF)
2. Get hold of one of the top level collection objects (Packets, Flows, PDUs, UserObjects).  Consult the scripting guide – currently only available as PDF for a list of objects/ methods/ properties. We are working to put out a HTML version of this guide.
3. Iterate over the collection – calling methods and properties of members as you go

For this task, we going to import a PCAP file, grab the UserObjects collection and iterate over each object calling the SaveToFile method on each.

The meat of the script is these 3 steps.

Step 1  Import your PCAP file into Unsniff format

This fragment converts the input pcap file stored in InputTCPD variable into the Unsniff format file temp_cap.usnf. We use a temporary file because we are going to toss it out once we extract the user objects into their own files. You can of course keep it in that format which opens in Unsniff Network Analyzer instantly.

Step 2  Grab the UserObjects collection, iterate and save

Once the data is available in Unsniff format, you can simply call the db.UserObjectsIndex property to get the collection. Here are the other available collections.

• PacketIndex – Iterate over each packet, dive into protocol fields
• PDUIndex – Messages such as TLS records, SMB records, etc
• StreamIndex – TCP sessions
• UserObjectsIndex – Extracted objects (our focus for today)

You can use the ruby enumeration methods to iterate or even a for-loop. Finally we call the SaveToFIle method with the PreferredFileName.  There are about 15 other properties of a user object (see guide) you can play with.

Thats it !

Step 3 Run the code

Run this code over your pcap file

Check the output directory where all your files are

Full code

The ready to run code is shown below. It adds a check for duplicate output filenames. Save it as myfile.rb and run as shown above.

More to come with Trisul

There are quite a few code samples of this kind at http://www.unleashnetworks.com/devzone/unsniff/script-library.html

We are also excited about the next upcoming release of Trisul (Release 2.4). It is going to feature very strong support for scripting. Trisul is a 24×7 system that does traffic monitring and tucks away flows, counters, and packets. You can combine the high level capabilities of Trisul with the deep analysis of Unsniff to create very powerful scripts.

Here is a sample :

• Search all flows in the past week for a particular IP
• Get all packets for those flows
• Extract all content in those flows
• Run them past malware scans
• Automatically update an internal project page with status of check
• Send an email to your team if anything was found

You can do all these tasks today by clicking around – but the idea behind Trisul and Unsniff is that you focus on creating repeatable scripts and tweaking them. I.E do the fun stuff – leave the heavy lifting to us.

—-

Links:

Trisul Web Site

Free Unsniff Download