Tag: network forensics

Network forensics of P2P with Unsniff

Network forensics of P2P networks with Unsniff. Software used was Limewire on a Gnutella network.

Network based forensics of P2P traffic has many applications, particularly in law enforcement. Recently, I added preliminary support for Gnutella in Unsniff Network Analyzer as a proof of concept. Wireshark has a rather rudimentary support for this protocol.

We would like to :

  1. Monitor what various subjects are searching for on the P2P network
  2. Monitor query responses including the filenames and their respective SHA1 hash
  3. Identify the file actually downloaded
  4. Reconstruct the file actually downloaded

This blog post is a summary of what I found and how you can put this feature to use.

Setup

This was the test setup.

  • Limewire 5.5.9 which connects using the Gnutella Protocol 0.6
  • Connected to the Gnutella network using the default setttings
  • Searched for “flotilla” videos
  • Downloaded one of them

All traffic from startup to shutdown of Limewire was captured using Unsniff.  You can convert these files to libpcap format by opening them in Unsniff and selecting File > Export > TCPDUMP

Limewire_Complete.usnf Contains complete traffic dump including bootstrapping, punching hole via UPnP, handshakes, downloading the file, and shutdown
Limewire_Handshake.usnf A single TCP session which contains a Gnutella handshake. Recommend you start here first to understand how the compression works

Notes on protocol

A quick note about the P2P protocol elements viewed from a forensics viewpoint.

Bootstrapping

When Limewire first starts up, it uses plain HTTP to download a list of peers from one of many well known hosts. Next it attempts to punch a hole in your firewall using UPnP.  It then selects a suitable peer based on your Geo location and starts a handshake process with it.More details about the bootstrapping process are available at the Gnutella Development Forum. You can observe the bootstrapping process in sessions 1,2,3 in the capture file Limewire_Complete.usnf .

Bootstrap + Punch hole via UPnP + Direct peer handshakes
Bootstrap + Punch hole via UPnP + Direct peer handshakes

Use of compression

The handshake takes place over a single TCP connection. The initial part of the handshake is in plaintext, but if both sides signal support for compression, the same TCP connection switches over to stream compression. Unsniff creates a “synthetic” uncompressed TCP stream corresponding to each compressed TCP handshake stream. You can view the uncompressed stream by opening the second capture file Limewire_Handshake.usnf .

fCompressed original stream + uncompressed synthetic stream created by Unsniff
Compressed original stream + uncompressed synthetic stream created by Unsniff

Use of encryption

Many messages including the actual file transfers are encrypted using AES-128 whose keys are exchanged using Anonymous-Diffie Hellman. This is bad news because our forensics ability effectively ends here. But traffic analysis can give us the some clues about what might have been transferred across the encrypted session.

Actual content encrypted using AES128
Actual content encrypted using AES128

Here is the complete stream that was used to transfer the selected file (click for bigger picture). You can view the TLS handshake settling upon the Anon-DH key exchange protocol and AES128 encryption.

Content encypted via TLS : Anon-DH and AES-128
Content encypted via TLS : Anon-DH and AES-128

Messages

Gnutella defines a handful of messages and extensions. The relevant protocol messages for this exercise are QUERY and QUERY HIT.  To view these messages,

  • switch to the PDU sheet (we are far removed from link layer packets)
  • click on a message to view its breakout
QUERY HIT message decoded
QUERY HIT response for "flotilla" videos

Analyzing Gnutella with Unsniff

Viewing Gnutella messages

1. With USNF files, you just have to open the capture file.

2. With PCAP files or live captures, you need to set up Unsniff to recognize Gnutella.

Since Gnutella uses random TCP ports, just assign Port 65535  to Gnutella. This is the special ‘catch all’ port in Unsniff.  All unrecognized TCP traffic will be treated as Gnutella. To do this :  Select Manage > Access Points > Click on TCP > Add Access Point > Enter 65535 + Gnutella.

3. Import the PCAP file or start a live capture

4. Look at the PDU sheet, it contains all the Gnutella messages.

Scripting to automate this task

To really use this in forensics applications you need to be able to script this stuff. Luckily Unsniff’s scripting API allows us to ditch the GUI altogether and run scripts to extract just what we want.

Here is a useful sample script in Ruby.

  • This prints a list of  peers, the filenames, and the SHA1 hash.

Running this code as

Produces this report containing the peer, files available, and their SHA-1 hash.

The SHA1 hash is in Base32.

Conclusions

It is possible to conduct network based forensics on P2P traffic, but it requires tools with advanced reconstruction abilities.

  • You CAN conduct surveillance of location of content
  • You CAN conduct surveillance of content being searched for
  • You CANNOT reconstruct the file actually transferred due to encryption and use of Anon-Diffie-Hellman
  • You CAN however get a fair idea, not of forensic evidence quality about what was transferred

—-


Download Unsniff Network Analyzer

Follow me on Twitter

Forensics Fun : YouTube videos from packet captures

Reconstruction of videos from packet captures. How to extract videos from a youtube session and to script the whole thing.

Who says people who look at raw packets cant have fun ? Let’s see how we can reconstruct YouTube videos from raw packets and play them back directly from Unsniff. We also show you a nifty Ruby script to automate this.

Playback

1. Fire up Unsniff Network Analyzer and start a packet capture

2. Go watch some YouTube video(s)  to completion. We will explain how to deal with half played videos later.

3. Once you are done,  switch over to the User Objects tab where all the action is. You will see a list of “user objects” extracted by Unsniff from the stream. Click on the Type column to sort by Type and locate the FLV objects. Right click to Save or Play (To playback you need the free VLC Player installed and FLV files associated with VLC).

Right click the user object of type FLV and select Play or Save
Right click the user object of type FLV and select Play or Save

Feeling adventurous ?  Lets try some more tricks.

Pruning the packet capture

When you interact with a site like YouTube there is a ton of content exchanged, not all of it is the video bits. You have packets containing images, HTML, Flash, and Javascript. The actual FLV is exchanged in a single TCP stream. If you can save these streams alone, you will be able to reconstruct the videos.

Here is how you do it.

1. Switch over to the “Sessions”, here is where you will see a list of TCP sessions updated in real time (every packet).

Only the TCP Stream containing the media is required ! Cut and paste it
Only the TCP Stream containing the media is required ! Cut and paste it

2. Locate the TCP Stream carrying FLV traffic. These are typically the really large ones. Select the stream, copy the stream (Edit->Copy) and Paste it as a new file (Edit -> Paste as New). Voila ! Switch to the User Objects in the new file and you will find the FLV without all the other clutter.

Script the whole thing using Ruby

The right clicking novelty quickly wears out after a while. You really want to automate this type of forensic stuff.

Here is a sample which shows you how you can use Ruby to script this stuff.

Task : Extract all the videos in a capture file as separate playable files. (including the ones that were aborted in the middle due to the use losing interest)

1. Save the capture as USNF format (this is the scriptable format used by Unsniff).

2. Run this script like so

We are still working on tweaking the experience for our upcoming Unsniff 2.0 release. In the meantime, we invite you to  try out the Unsniff Beta 1.8 and give us feedback on what you’d like to really see.