WAND Network Research Group University of Waikato Crest Of Arms

libtrace

libtrace is a library for trace processing. It supports multiple input methods, including device capture, raw and gz-compressed trace, and sockets; and mulitple input formats, including pcap and DAG.

The latest version is 3.0.21
The latest version can always be retrieved from here
We also maintain a detailed ChangeLog

Libtrace is now on GitHub. If you add a feature to libtrace that you think would benefit the libtrace community, please send us a pull request.

We have a libtrace-announce mailing list for announcements of new versions of libtrace, and a libtrace-users mailing list for discussions about libtrace.

Nevil Brownlee has kindly written excellent libtrace bindings for both the Ruby and Python languages, allowing for rapid prototyping of passive measurement applications. You can download ruby-libtrace and python-libtrace from here. Comprehensive documentation and examples are also available for both the ruby and python bindings.

Christoph Dwertmann has created libtrace RPM packages for both Centos and Fedora. The RPM packages can be downloaded from this location.

Libtrace packages are also available in the official Debian and Ubuntu repositories. Stable versions of these packages often lag behind the latest release by several versions, however. The package names are libtrace3 (the main library), libtrace3-dev (the development headers), libtrace-tools (the tool binaries), libwandio1-dev (libwandio) and libpacketdump3 (libpacketdump). The libtrace package is maintained by Matt Brown.

A libtrace package is also available for Gentoo - thanks to Tim Harder for creating and maintaining the Gentoo package.

Citing Libtrace

If you have used libtrace in work that you intend to publish, the best way to reference libtrace is to cite the paper "Libtrace: A Packet Capture and Analysis Library" by S. Alcock, P. Lorier and R. Nelson.

Usage

The libtrace wiki is the definitive source of information regarding how to use libtrace, including descriptions of the tools that come bundled with libtrace and instructions on how to perform a variety of common tasks. Doxygen generated documentation is also available online.

Libtrace 3 is not backwards compatible with libtrace 2. See our Porting to libtrace 3 guide.

Supported Trace Formats

FormatReadWrite
PCAP trace fileYesYes
PCAP interfaceYesYes
ERF trace fileYesYes
DAG live captureYesYes
Native Linux interfaceYesYes
Native BSD interfaceYesYes
Legacy ATM trace fileYesNo
Legacy Ethernet trace fileYesNo
Legacy POS trace fileYesNo
ATM cell header fileYesNo
TSH/FR+ trace fileYesNo

Trace file I/O

Libtrace can natively read and write compressed trace files. The supported compression formats are gzip, bzip2, lzma and lzo (lzo support is write only). To speed up file operations, compression and decompression is performed using a separate thread, resulting in significant performance gains for analysis tasks where the I/O is the limiting factor (most simple analysis tasks are I/O-limited).

NEW in 3.0.14 - the I/O functionality used by libtrace is now available as the libwandio shared library (included with libtrace). Now users can easily use libwandio to perform threaded reading and writing of any type of compressed file, not just trace files.

Format Agnostic

Programs written using libtrace are capture format agnostic. This means that they can be run against an input from any supported capture source without requiring any code modification. The same program will work just as well when reading from a PCAP trace file as it would reading from a live DAG capture.

Libtrace can also seamlessly convert packets from one format to another without any special effort on the part of the user.

Programming API

The libtrace API allows users to directly access the protocol header for any layer at or below the transport layer, automatically decoding and skipping any intermediate headers. There are both functions for accessing the header that is present at a given layer (e.g. the transport layer) and functions for finding the header for a specific protocol (if present).

Supported headers include:

  • Ethernet
  • 802.11
  • VLAN (802.1q)
  • MPLS
  • PPPoE
  • IPv4, IPv6
  • ICMP, ICMP6
  • TCP
  • UDP
  • OSPF

Libtrace also detects and handles edge-cases that occur when decoding protocol headers such as fragmentation, header truncation and tunneling.

Tools

Libtrace comes bundled with a series of tools that perform most common trace manipulation tasks. These include:

  • traceanon - anonymises trace files
  • traceconvert - converts a trace from one format to another
  • tracediff - reports differences between two trace files
  • traceends - summarises traffic sent and received by endpoints
  • tracefilter - applies a BPF filter to a trace
  • tracemerge - merges multiple trace inputs into a single trace
  • tracepktdump - displays packet contents in a readable format, similar to tcpdump
  • tracereplay - replays a trace file using original timing
  • tracereport - produces a variety of reports on a trace
  • tracertstats - produces stats about an input trace in real time
  • tracesplit - splits trace files
  • tracesplit_dir - splits trace files based on packet direction
  • tracestats - summarises number of bytes and packets matching BPF filters
  • tracesummary - summarises the basic stats for a trace
  • tracetop - reports the busiest flows over time, similar to ntop
  • tracetopends - reports the busiest endpoints in a trace

Libpacketdump

Libtrace includes a library called libpacketdump which can parse packets and display the packet contents in a nice human-readable form. The output is similar to that produced by tcpdump, although the formatting is somewhat more verbose. Libpacketdump forms the core of the tracepktdump tool, but can also be used in libtrace programs (for example, to dump a packet that has unexpected values in a header).

Installation instructions

Download the tarball, unpack, and run

./configure ; make ; make install
libtrace will install into /usr/local by default. Bearing that in mind, you will probably need to update your library search path to point at /usr/local/lib, if it doesn't already:
echo /usr/local/lib >> /etc/ld.so.conf
ldconfig

There are some configure-time options you can use, aside from the usual configure options for prefix and so on:

libpacketdump: this is a library to enable rapid debugging of network packets. It will dump out the contents of a libtrace_packet_t to stdout, parsing any layers (such as ethernet or TCP) that it knows about. It is built in a module fashion, so adding support for newer protocols is trivial. We have a tcpdump-like replacement called tracepktdump that uses libpacketdump.

Some of the tools have external dependancies such as libgdchart - if these libraries are not present on your system, the parts of the tool that depend on it will not be included.

DAG support: If you have a DAG card, and wish to compile libtrace with support for native reads from DAG cards, use the --with-dag option:

  --with-dag=DIR        include DAG support (located in directory DIR, if
		                            supplied)

Note: Older Distributions may have trouble reading large pcap files using the pcap: file parser. Use the internal "pcapfile:" parser instead.

Feedback

We are very interested in hearing feedback on libtrace. If you have any requests or comments, or wish to report a bug, please email contact@wand.net.nz