A Linux TCP implementation for NS2

(Part of the NS-2 enhancement project)

David X. Wei         Prof. Pei Cao
Netlab @ Caltech         CS @ Stanford

May 2006

NOTE: This page is for legacy support for NS-2 TCP-Linux with 2.6.16.3 kernel. For latest NS-2 TCP-Linux information, please visit the NS-2 TCP-Linux main website.

This is a patch that can run Linux TCP congestion control algorithms on NS2, with similar simulation speed and memory usages as other NS2 TCPs (e.g. Sack1). The implementation loosely follows the Linux TCP implementation, and can produce results comparable to Linux experimental results. The patch is for NS-2.29. It is confirmed to be compatible with NS-2.30. You may need some modification if you want to install it for other versions.

News (complete update records):

Patch for NS-2.29:

The patch can be downloaded from here. It is for NS-2.29. It is also confirmed to be compatible with NS-2.30. You may need some modification if you want to install it for other versions. To install the patch, you need to take the following steps, assuming you have successfully installed and run NS-2.29 in your computer:

  1. Copy the patch to ns-allinone-2.29/ns-2.29
  2. In the directory of ns-allinone-2.29/ns-2.29 , run: make clean
  3. In the directory of ns-allinone-2.29/ns-2.29 , run: patch -p1 < ns-linux.patch
  4. (Optional) delete the patch file by rm ns-linux.patch
  5. recompile the ns2 by make and make install

The patch changes the following files:

Usage:

This section serves as a quick reference for users who want to run different Linux TCP algorithms in the TCP Linux patch.

There is a mini-tutorial for TCP Linux . Please read the mini-tutorial for details if you want to design your own algorithms or port new algorithms from Linux to NS-2.

If you find some performance problem of some Linux algorithms, please check the known Linux bugs page to make sure it is really the problem of the algorithm, not a bug in Linux implementation.

1. Normal Usage:

The TCP-Linux module is call "Agent/TCP/Linux".  If you have an existing script that runs TCP and want to change to TCP-Linux, what you need to do is:

  1. Change the TCP Agent's name (e.g. Agent/TCP/Sack1) to Agent/TCP/Linux.
  2. Make sure the TCP Sink has Sack1 support. That is, either you are using Agent/TCPSink/Sack1 or Agent/TCPSink/Sack1/DelAck . Currently, TCP-Linux does not support receivers without SACK.
  3. Optional but recommended: change the window_ option in tcp agent to be large enough. This option is the upper bound of congestion window. It is 20 by default in NS-2. Most congestion control algorithms work the same if the congestion window is bounded by 20. The recommended value is at least bandwidth-delay-product.
  4. Before starting the simulation, add one command:
$ns at 0 "$tcp select_ca $TCP_Name"
where $ns is the NS-2 scheduler you are using in the script, and $TCP_Name is the congestion control algorithm you want to select. The patch comes with 12 different congestion control algorithms from Linux-2.6.16.3, as listed in the following table:

$TCP_Name
Congestion Control Algorithm
bic
Binary Increase Congestion control for TCP
cubic
TCP CUBIC: Binary Increase Congestion control for TCP v2.0 , an extension of BIC-TCP
highspeed
Sally Floyd's High Speed TCP (HS-TCP RFC 3649) congestion control
htcp
Hamilton TCP (H-TCP) congestion control
hybla
TCP-HYBLA Congestion control algorithm
reno
TCP NewReno
scalable
Tom Kelly's Scalable TCP
vegas
TCP Vegas congestion control
westwood
TCP Westwood+
veno
TCP Veno
compound
TCP Compound (C-TCP)
lp
TCP Low-Priority (TCP-LP)

2. Advanced Usage

You can add your own congestion control module once you develop the module in Linux.

If you decide to do so, take the following steps:

  1. Make sure your implementation is compliant to the congestion control structure in Linux (struct tcp_congestion_ops);
  2. Make sure this patch has been applied to your NS2 code base
  3. copy the Linux module files of the new congestion control algorithm to tcp/linux/ directory in your NS2 code base
  4. add a record in tcp/linux/ns-linux-util.h to export the congestion control structure's name to NS2:
    extern struct tcp_congestion_ops YourCongestionControlStructure;
  5. add a record (&YourCongestionControlStructure) in struct tcp_congestion_ops* static_list[] of tcp/tcp-linux.cc to register your algorithm, also increase CONGESTION_CONTROL_OPS_NUM by 1.
  6. add a record in Makefile by adding an item to let compiler know syour code:
    tcp/linux/YourCode.o
  7. compile, run and compare the simulation results with Linux experiments results

You might encounter one of the following problems in the last step:

  1. If your algorithm requires access to many new fields in Linux TCP structure, you might need to add more fields to struct tcp_sock in tcp/linux/ns-linux-util.h .
  2. If your algorithm has some variables with the names identical to some other existing algorithms, you will see duplicate definition error. In this case, change the duplicated names to others.
  3. If you algorithm has some actions in module register function, let the module init function to call the register function, since this NS2 implementation does not call module register function.

WARNING: After adding a new congestion control algorithm, please verify the simulation results extensively and carefully to make sure the simulation is running as expected.

Known Problems:

Here is a list of known problems:

  1. The implementation does not change receiver part. The delayed ack implementation in Linux might be different from the one in NS-2. This may results in some performance difference
  2. D-SACK: This may leads to performance difference in scenarios with packet reordering.
  3. F-RTO: Not yet implemented. Will be included very soon.
  4. ECN: The implementation is not extensively verified.
  5. TCP Segmentation: Not implemented. Will NOT be covered in the near future.
  6. Buffer tuning: Not implemented. Will NOT be covered in the near future.

Paper

NS-2 TCP-Linux: An NS-2 TCP Implementation with Congestion Control Algorithms from Linux ; D. X. Wei and P. Cao; in proceedings of ValueTool'06 -- Workshop of NS-2, Oct, 2006.
PDF Bibtex

Result comparison between TCP-Linux simulations and Linux experiments:

The following table shows the setup of the comparison simulations and experiments:

Setup of Linux experiments

Setup of NS2 simulations

A single flow is running from the sender side (right) to the receiver side (left). The congestion window size and the sending rate is recorded every 0.5second.

In experiment, the application is Iperf with large enough buffer. (See kernel tuning script). We read the /proc/net/tcp file every 0.5 second to get the congestion window and measure throughput at the receiver side's iperf output. (Dummynet setup script in the experiments can be found here . Also note that e1000 card's driver has set the RxDescriptors number to be 4096.)

In simulation, the application is infinite FTP flows, with large enough buffer. (See TCL script and CSH scripts ).

The following figures report the congestion window trajectory and the rate trajectory over time. Red curves are the results of NS-2 simulations. Green curves are the results of Linux-Dummynet experiments. For comparison, we also show the NS-2 Sack1 results (in blue curves) for the cases of Reno and HighSpeed TCP. (See TCL script and CSH scripts for NS-2 Sack1 simulations.)

You can click on a figure to get the full-size version.

TCP Options
Congestion Window Trajectory
(Y axle: packet; X axle: sec)
Rate Trajectory
(Y axle: bps; X axle: sec)
Remark
bic



cubic


The difference of cubic results is the most significant one among all the results. We still need to understand why.
highspeed


The blue curve is the NS-2 TCP/Sack1 results, with TCPSink/Sack1/DelAck. We also have the TCP/Sack1 result with TCPSink/Sack1 ( cwnd, rate) which are less closer to the Linux results.
htcp



hybla


hybla sets AI parameters based on minimum observed RTT. NS-2 has a cleaner situation with a smaller minimum observed RTT. Hence AI parameter in simulation is smaller than the one in dummynet experiment. That's my explanation why the simulation result has a longer congestion epoch than the dummynet result.
reno


The blue curve is the NS-2 TCP/Sack1 results, with TCPSink/Sack1. We also have the TCP/Sack1 result with TCPSink/Sack1/DelAck ( cwnd, rate) which are less closer to the Linux results.
scalable



vegas

Interestingly, we found that NS-2's Vegas implementation is much better than TCP-Linux, which is very close to Linux results.
Both Linux and TCP-Linux do not fully utilize the bottleneck bandwidth. We studied the problem and it turned out to be a problem with Linux implementation with DelayAck. We need to set alpha to be larger than 1 to eliminate this problem when DelayAck exists.
westwood




Code Structure:

The following figure shows the code structure of TCP-Linux
.

The whole modules include four parts, corresponding to the four white blocks in the figure. Yellow blocks are from outside source codes such as NS-2 or Linux:

  1. TCPLinuxAgent (in tcp-linux.h and tcp-linux.cc): this is the main component which loosely follows the Linux implementation in packet receiving, ack processing and congestion control.
  2. ScoreBoard1 (in scoreboard1.h and scoreboard1.cc): this is a new packet SACK/Loss/Retransmission control module which combines Scoreboard-rq in NS-2 and Linux's ACK/SACK processes. It loosely follows the steps in tcp_clean_rtx_queue and tcp_sacktag_write_queue in tcp_input.c in Linux.
  3. Interface between NS-2 and Linux (in linux/ns-linux-util.h and .cc): this part redefines the data structure in Linux TCP and provide interfaces between the NS-2's C++ code and Linux's C code.
  4. Shortcuts for other Linux system calls (in linux/ns-linux-c.h and .c): this part redefines many system calls in Linux (to void) and allows Linux source code to be compiled with very minor changes.

Acknowledgment:

The Linux+Dummynet experiments were carried out with WAN-in-Lab (WIL) facilities and greatly helped by Dr. Lachlan Andrew at Caltech.
The work is inspired and greatly helped by Prof. Pei Cao at Stanford and by Prof. Steven Low at Caltech. Many thanks to them!