Unbrowse Automation 2 : Adding SNMP Agents

Some of our users manage dozens of network devices. Unbrowse SNMP features a scripting interface that allows you to automatically import SNMP agents from any format. You have to write a tiny bit of code in a scripting language like VBScript or Ruby.

This is the second and concluding part of the article. In the first part, we looked at the object model for SNMP Agents in Unbrowse.

The task

You are an organization running CiscoWorks to manage your dozens or hundreds of routers and switches.  You do not want to input the agent information (such as name, ip address, community) by hand – this is just too painful. If you have a flat file with agent information, you can write a simple script to import them into Unbrowse.

In this example, we will import a file in CiscoWorks DCRv3 format into Unbrowse SNMP. This is meant only to illustrate the scripting interface, you can adapt the script to your own file format quite easily.

Without much ado here is the script (in VBScript) :

To run the script, type “cscript impagent.vbs dcrtestfile.txt”

' ------------------------------------------------------------
' IMPORT agents in Ciscoworks DCRv3 format into Unbrowse SNMP
'
'  Usage: impagent
' ------------------------------------------------------------
Const ForReading=1

Set Stdout = WScript.Stdout
Set FSO = CreateObject("Scripting.FileSystemObject")

' -----------------------
' Check usage & arguments
' -----------------------
if WScript.Arguments.Count 1 then
 Stdout.WriteLine "Usage: impagent "
 WScript.Quit
end if
DCRFileName = WScript.Arguments.Item(0)
' ----------------------------------
' Open input file and read all lines
' ----------------------------------
Set InputFile = FSO.OpenTextFile( DCRFileName, ForReading)

InputFileContents = InputFile.ReadAll
If Not Err.Number = 0 Then
  If Err.Number = 424 Then StdOut.WriteLine "Input DCR File not found" & DCRFileName & vbCRLF
  StdOut.WriteLine "Error : " & Err.Description
  WScript.Quit
End If

InputFileLines = Split(InputFileContents,vbCRLF)

' ----------------------------------
' Create the Unbrowse Agent Manager
' ----------------------------------
Set AgentMgr =  CreateObject("UnbrowseSNMP.AgentManager")
AgentMgr.Init()

validLinesBegin = False
dcrCheck = False
For Each sLine In InputFileLines

 Trim(sLine)
 ' Skip comments and zero length
 If Len(sLine) > 0 AND Mid(sLine,1,1)   ";"  Then

  If validLinesBegin Then
   ProcessLine (sLine)
  Else
   If dcrCheck Then
    If InStr(sLine,"management_ip_address") Then
     validLinesBegin = True
    End If
   Else
    If InStr(sLine,"Type=DCRCSV") Then
     dcrCheck = True
    End If
   End If

  End If
 End If
  
Next

If Not dcrCheck Then
 StdOut.WriteLine "The input file is not in CiscoWorks DCRv3 format : filename " & DCRFileName & vbCRLF
End If
' ------------------------------------------------------------------------------
' Process a single line
' Create an agent and set attributes as specified in the input line (CSV format)
' ------------------------------------------------------------------------------
Sub ProcessLine (Line)

 LineFields = Split(Line,",")

 If UBound(LineFields) > 3 Then

  ipaddr = LineFields(0)
  hostname = LineFields(1)
  rcomm = LineFields(8)
  wcomm = LineFields(9)

  Set OneAgent = AgentMgr.NewAgent
  OneAgent.Name = hostname
  OneAgent.IPAddress = ipaddr
  OneAgent.ReadComm = rcomm
  OneAgent.WriteComm = wcomm

  StdOut.WriteLine "Added agent " & hostname & " to Unbrowse repository" & vbCRLF

 End If

End Sub
 

A sample test file is shown below

; This file is generated by DCR Export utilityCisco Systems NM Data import, Source=DCR Export; Type=DCRCSV; Version=3.0

;

;Start of section 0 - Basic Credentials

;

;HEADER:
management_ip_address,host_name,domain_name,device_identity,display_name,sysObjectID,dcr_device_type,mdf_type,snmp_v2_ro_comm_string,snmp_v2_rw_comm_string,user_defined_field_0,user_defined_field_1

;

10.77.202.40,Switch6009,cisco.com,,Switch2,1.3.6.1.4.1.9.1.281,0,268438100,public,private,field0,field1

10.77.202.10,Router7000,cisco.com,,Router1,1.3.6.1.4.1.9.1.8,0,278464493,public,private,field0,field1

10.77.202.30,Switch4006,cisco.com,,Switch1,1.3.6.1.4.1.9.5.46,0,268438086,public,private,field0,field1

10.77.202.20,Router6400,cisco.com,,Router2,1.3.6.1.4.1.9.1.180,0,269214543,public,private,field0,field1
;End of CSV file
 

[tags] SNMP, MIB Browser, Agents, VBScript [/tags]

Analyze web traffic traces

One of our favorite tech bloggers Richard Bejtlich has put up a new post about web traffic analysis using a new open source tool. This prompted me to write this post about using Unsniff Network Analyzer to perform similar analysis. I hope our users will find this article useful. Also see this article  for a detailed overview of HTTP analysis.

Unsniff allows you to :

  • Extract content from HTTP flows. We call this feature “User Objects”. Examples of user objects are images, css, flash, video, etc.
  • Put together separate flows into a web page. This is a powerful feature that allows you to reconstruct web pages completely. This way CSS styles are applied, and inline images and flash content appear in the reconstructed web pages. See picture below.
  • Script for automating analysis. For example: If you want to automatically extract all images from a lengthy capture into individual JPEG files, you can write a VBScript or Ruby program to do that. The Unsniff Scripting API allows such macro operations. See here for samples.
  • Support for HTTPS. If you are a system admin with access to the server certificate, you can perform the same type of analysis for HTTPS. This is quite valuable for debugging website problems at the server. This powerful feature is enabled by Unsniff’s top notch support for decrypting SSLv3 and TLS traffic.

Advantages of correlating flows into a single page.

A single webpage is rendered by the browser by putting together several individual HTTP responses (flows). Sometimes these object come from different hosts. For example, a CSS stylesheet, a flash SWF file, a set of JPEG files, and the master HTML document must all be combined to produce a webpage.

The sample webpages below have been reconstructed from a tcpdump.

Sample 1: Technorati website contains embedded images, complex stylesheet. http://technorati.com

trati.jpg

Sample 2: Richards website with CSS and embedded images. http://www.bejtlich.net/

httpcap.jpg

Notes about this feature.

  1. Switch to the “User Objects” sheet to see the individual flows.  
  2. Click on the HTML object to view reconstructed web page.
  3. Click on the image to see it rendered.
  4. Right click an item and select “View Data” to see raw contents (eg, HTML text, CSS text, Image bytes, Flash etc)
  5. Capture live traffic or select “File -> Import -> From TCPDUMP” to select a pre-captured file in libpcap (Ethereal or Wireshark native) format.

– – – – –

Here are solutions to some common problems emailed to us by users of this feature.

Step 1 : Enable HTTP reconstruction

  1. Go to menu “Plugins -> Customize” and enable the “Reconstruct HTTP Session” option.  This is not the default.
  2. Also check if the “Extract User Objects” option is enabled. This is the default.

Step 2 : Windows Firewall Warning

If you are running a personal firewall you may get a warning about a port. You can ignore that warning, that port is used internally by the HTTP reconstruction software.

Any questions ? Please leave a comment here or use our forum.

[tags] tcpdump, sniffer, website analysis, http analysis, unsniff [/tags]

Unbrowse Automation 1 : Adding SNMP Agents

This is the first of a two part article about Unbrowse SNMP scripting.

It is not uncommon for network administrators to deal with hundreds of devices scattered over a large geographical area. The device details are usually stored in a network management system such as HP Openview or CiscoWorks (both trademarks of their respective organizations). This article explains how you can use scripting to import SNMP agents into the Unbrowse database.

scriptagent.JPGUnbrowse SNMP is a MIB Browser and Trap Receiver that is intended to be used as a powerful troubleshooting and an all around utility. Unbrowse keeps track of agents (devices) and stores information such as IP addresses, community strings, timeouts, SNMP versions, V3 usernames, auth and priv passwords, and so forth. This allows the administrator to quickly manage devices without having to enter these details each time. This will work as long as the number of devices are few, when you are dealing with hundreds of devices it is not feasible to enter them manually into a tool like Unbrowse. Dont worry, help is at hand. You can write simple scripts in VBScript or Ruby to add devices automatically into Unbrowse.

Part 1 : The Automation objects

In this section, we will look at the methods and properties of the automation interface.

The Agent Manager object

This object is createable via the “UnbrowseSNMP.AgentManager” id. To create this object in

VBScriptSet Mgr = CreateObject("UnbrowseSNMP.AgentManager")Ruby

agentMgr = WIN32OLE.New("UnbrowseSNMP.AgentManager")

 

 

Name Purpose
Init Initialize the manager (not necessary to call this)
NewAgent Create a new agent object. You can then set the properties of this object
AddToRepository Parameter: an agent object. Add this agent to the database
RemoveFromRepository Parameter: an agent object. Remove this agent
FindAgent Find an agent by name. Returns the agent object or null

The SNMP Agent object

The following table show the supported properties on the object

Name Purpose
IPAddress The IP Address of the SNMP agent (can be an IPv6 address)
Name Name of agent
Name Purpose
SNMPVersion Version used to communicate (0=v1, 1=v2c, 3=v3)
ReadComm The community used for reading from agent (v2 only)
WriteComm The community used for SET requests (v2 only)
SecurityName The v3 USM user name
ContextName The v3 context name
SecurityLevel The type of v3 security (0 = noAuthNoPriv, 1 = authNoPriv, 2= authPriv)
AuthProtocol v3 authentication protocol (0=MD5, 1=SHA)
PrivProtocol v3 privacy protocol (0=DES, 1=AES)
Port The UDP port to which requests are sent
Timeout Timeout in milliseconds
Retries Number of times to retry communication before declaring error

 

Thats all. You only have to now write scripts to manipulate these two objects. It is really easy. In the next part we will write a script in Ruby to import devices from a CiscoWorks DCR v3 file.

 

[tags] MIB Browser, SNMP Agent, VBScript SNMP, Unbrowse, CiscoWorks, DCRv3 format, Ruby SNMP [/tags]