libtrace
libtrace is a library for trace processing. It supports multiple input methods, including device capture, raw and gz-compressed trace, and sockets; and multiple input formats, including pcap and DAG.
Libtrace 4, which adds support for parallelism in both packet capture and processing, is now out of beta and officially released.
Learn more about the new parallel API and how to use it from the HOWTO on the libtrace wiki.
Note that libtrace 4 is intended to be entirely backwards-compatible with libtrace 3, so you should be able to switch to libtrace 4 without any of your existing code breaking. However, you won't gain any benefits from parallelism unless you write your code to use the new parallel API.
The latest stable version is 4.0.13
The latest stable version can always be retrieved from here
The last libtrace 3 release can still be accessed here
We also maintain a detailed ChangeLog
Libtrace is now on GitHub. If you add a feature to libtrace that you think would benefit the libtrace community, please send us a pull request.
We have a libtrace-announce mailing list for announcements of new versions of libtrace, and a libtrace-users mailing list for discussions about libtrace.
Nevil Brownlee has kindly written excellent libtrace bindings for Python, allowing for rapid prototyping of passive measurement applications. These bindings are also available on GitHub. Comprehensive documentation and examples are also available for the python bindings.
We maintain our own packages for Debian and Ubuntu (versions 3.0.22 and later only, including libtrace 4). Packages for versions 4.0.6 and later can be obtained from the wand/libtrace repository on Bintray. Packages for older versions can still be reached via packages.wand.net.nz, at least for now.
Starting from libtrace 4.0.7, we also build and release our own RPMs for libtrace (Centos and Fedora). More details on how to install the libtrace RPMs is available here.
Libtrace 3 packages are also available in the official Debian and Ubuntu repositories. Stable versions of these packages often lag behind the latest release by several versions, however. The package names are libtrace3 (the main library), libtrace3-dev (the development headers), libtrace-tools (the tool binaries), libwandio1-dev (libwandio) and libpacketdump3 (libpacketdump). The libtrace package is maintained by Matt Brown.
A libtrace package is also available for Gentoo - thanks to Tim Harder for creating and maintaining the Gentoo package.
Citing Libtrace
If you have used libtrace in work that you intend to publish, the best way to reference libtrace is to cite the paper "Libtrace: A Packet Capture and Analysis Library" by S. Alcock, P. Lorier and R. Nelson.
Usage
The libtrace wiki is the definitive source of information regarding how to use libtrace, including descriptions of the tools that come bundled with libtrace and instructions on how to perform a variety of common tasks. Doxygen generated documentation is also available online.
Libtrace 3 is not backwards compatible with libtrace 2. See our Porting to libtrace 3 guide.
Supported Trace Formats
| Format | Read | Write |
|---|---|---|
| PCAP trace file | Yes | Yes |
| PCAPNG trace file | Yes | Yes |
| PCAP interface | Yes | Yes |
| ERF trace file | Yes | Yes |
| DAG live capture | Yes | Yes |
| Native Linux interface | Yes | Yes |
| Native BSD interface | Yes | Yes |
| Legacy ATM trace file | Yes | No |
| Legacy Ethernet trace file | Yes | No |
| Legacy POS trace file | Yes | No |
| ATM cell header file | Yes | No |
| TSH/FR+ trace file | Yes | No |
| ETSI LI live stream | Yes | No |
Trace file I/O
Libtrace can natively read and write compressed trace files. The supported compression formats are gzip, bzip2, lzma and lzo (lzo support is write only). To speed up file operations, compression and decompression is performed using a separate thread, resulting in significant performance gains for analysis tasks where the I/O is the limiting factor (most simple analysis tasks are I/O-limited).
NEW in 3.0.14 - the I/O functionality used by libtrace is now available as the libwandio shared library (included with libtrace). Now users can easily use libwandio to perform threaded reading and writing of any type of compressed file, not just trace files.
Format Agnostic
Programs written using libtrace are capture format agnostic. This means that they can be run against an input from any supported capture source without requiring any code modification. The same program will work just as well when reading from a PCAP trace file as it would reading from a live DAG capture.
Libtrace can also seamlessly convert packets from one format to another without any special effort on the part of the user.
Programming API
The libtrace API allows users to directly access the protocol header for any layer at or below the transport layer, automatically decoding and skipping any intermediate headers. There are both functions for accessing the header that is present at a given layer (e.g. the transport layer) and functions for finding the header for a specific protocol (if present).
Supported headers include:
- Ethernet
- 802.11
- VLAN (802.1q)
- MPLS
- PPPoE
- IPv4, IPv6
- ICMP, ICMP6
- TCP
- UDP
- OSPF
Libtrace also detects and handles edge-cases that occur when decoding protocol headers such as fragmentation, header truncation and tunneling.
Tools
Libtrace comes bundled with a series of tools that perform most common trace manipulation tasks. These include:
- traceanon - anonymises trace files
- traceconvert - converts a trace from one format to another
- tracediff - reports differences between two trace files
- traceends - summarises traffic sent and received by endpoints
- tracefilter - applies a BPF filter to a trace
- tracemerge - merges multiple trace inputs into a single trace
- tracepktdump - displays packet contents in a readable format, similar to tcpdump
- tracereplay - replays a trace file using original timing
- tracereport - produces a variety of reports on a trace
- tracertstats - produces stats about an input trace in real time
- tracesplit - splits trace files
- tracesplit_dir - splits trace files based on packet direction
- tracestats - summarises number of bytes and packets matching BPF filters
- tracesummary - summarises the basic stats for a trace
- tracetop - reports the busiest flows over time, similar to ntop
- tracetopends - reports the busiest endpoints in a trace
Libpacketdump
Libtrace includes a library called libpacketdump which can parse packets
and display the packet contents in a nice human-readable form. The output is
similar to that produced by tcpdump, although the formatting is somewhat more
verbose. Libpacketdump forms the core of the tracepktdump tool, but can also be
used in libtrace programs (for example, to dump a packet that has unexpected
values in a header).
Installation instructions
Download the tarball, unpack, and run
./configure ; make ; make installlibtrace will install into /usr/local by default. Bearing that in mind, you will probably need to update your library search path to point at /usr/local/lib, if it doesn't already:
echo /usr/local/lib >> /etc/ld.so.conf ldconfig
There are some configure-time options you can use, aside from the usual configure options for prefix and so on:
libpacketdump: this is a library to enable rapid debugging of network packets. It will dump out the contents of a libtrace_packet_t to stdout, parsing any layers (such as ethernet or TCP) that it knows about. It is built in a module fashion, so adding support for newer protocols is trivial. We have a tcpdump-like replacement called tracepktdump that uses libpacketdump.
Some of the tools have external dependancies such as libgdchart - if these libraries are not present on your system, the parts of the tool that depend on it will not be included.
DAG support: If you have a DAG card, and wish to compile libtrace with support for native reads from DAG cards, use the --with-dag option:
--with-dag=DIR include DAG support (located in directory DIR, if supplied)
Note: Older Distributions may have trouble reading large pcap files using the pcap: file parser. Use the internal "pcapfile:" parser instead.
Feedback
We are very interested in hearing feedback on libtrace. If you have any requests or comments, or wish to report a bug, please email contact@wand.net.nz