Motorcycle with Sidecar

About Sidecar

TCP Sidecar is a technique and associated API for embedding measurement probes into non-measurement TCP streams. Much like a sidecar attaches to a motorcycle, TCP Sidecar attaches to TCP streams and is just "along for the ride". By embedding probes into already existing TCP streams, Sidecar avoids abuse reports and allows measurements to tunnel through NATs and behind firewalls.

Our current experiments involve probing the core of the network with small TCP data segments. We are doing everything we can think of to limit the intrusiveness of these probes including avoiding endhosts, rate limiting the probes, and sending few probes to any one destination. If your network is receiving these probes, please contact me at capveg's email address to be removed from our experiments, and please accept our appologies in advance for any trouble caused.

a fragment of the output of sidecar

Published Data

Raw data from IMC06 "Touring the Internet in a TCP Sidecar" paper

This data is organized by PlanetLab source node. The directory contains two entries per host. The first is the log file of copying the data off of the host, and the second is a directory containing a series of tgz (tar'ed and gzip'ed) files representing the data generated in a given time period. Each tgz file contains individual trace files named:


Representing a trace from source_ip:source_port to dest_ip:dest_port (the id is only for internal use). For each src/dst pair, Passenger performs the following algorithm:
  • Calculate safettl as the min(11,hopcount from dst to src minus 3)
  • For iteration=0 to 5
    • For i=1 to safettl
      • send probe with TTL=i, set RR option if iteration is even
    • Wait RTO time for all probes to return
  • For i=safettl+1 to 30
    • For iteration=0 to 5
      • send probe with TTL=i
      • Wait RTO time for single probe to return
    • STOP if get response from endhost
All lines that do not match /RECV/ are for debugging and can be ignored when performing data processing. Each RECV line denotes the receipt of a single data probe. A typical RECV line looks like:
- RECV TTL 6 it=0 from (250)   ROUTER   rtt=0.002477 s t=1147905384.045258 RR, hop 1 , hop 2 , hop 3 , hop 4 , hop 5 , hop 6 ,  Macro
Which indicates that (as read left to right):
  1. The probe for TTL=6, iteration=0 generated an ICMP response from
  2. The TTL of the ICMP response when it reached Sidecar is 250
  3. We classify as a ROUTER. Other possible values here are NAT or ENDHOST.
    • A machine is a NAT if dest_ip is the same as the ICMP source address
    • A machine is an ENDHOST if we receive a redundant ACK from the machine instead of an ICMP TTL-Exceeded message.
  4. The round trip time of the probe was 0.002477 seconds or 2.477ms
  5. "t=1147905384.045258" means that this probe was received at 1147905384.045258 seconds past the time Epoch, 00:00:00 UTC, January 1, 1970, as returned from gettimeofday().
  6. "RR" indicates the probe returned with the RR options set
    • hop 2 indicates the 2nd RR entry is,etc.
  7. the "Macro" parameter is currently unused and can be ignored

Download Sidecar API and Tools

The most recent version of the code is sidecar-0.2.tar.gz
This includes the work from our Sigcomm08 DisCarte paper.


Unfortunately the documentation is largely UTSL at this point, but better documentation will be Coming Soon. The file "sidecar.h" in the codebase lists the entire sidecar api, and there are numerous examples of the uses in the example tools sideping and sidetrace.

Published Papers

  1. Touring the Internet in a TCP Sidecar
    Rob Sherwood, Neil Spring
    Internet Measurement Conference (IMC) 2006
    Sidecar IMC'06 Talk Slides
  2. A Platform for Unobtrusive Measurements on PlanetLab
    Rob Sherwood, Neil Spring
    Workshop on Real, Large Distributed Systems (WORLDS) 2006
    Sidecar WORLDS'06 Talk Slides


Valid HTML 4.01 Transitional