Testing Methodology
Testing is straightforward, so detailed instructions are not provided. Testing the client/server performance is very subjective and should be thought of in terms of:

1. Acceptable: Test does not significantly affect the application
2. Marginal: Some degradation detected, but probably okay to use
3. Unacceptable: Absolutely can't be deployed under these conditions

If more detailed test results are required (e.g., transaction times), another tool should be used to model the application.

Testing

• Insert the emulator as indicated in Figure 1.
• Configure the PC for bridge mode.
• Verify client/server connectivity.
• Enable QoS (See appendix A for details).
• Run client/server tests. Repeat.
  - Vary latency and bandwidth to obtain values for each cell in Table 3.
  - Vary packet loss using the values .01%, .1%, 1% and 3%

Network Emulation Details and Validation
Netem is used to emulate the properties of wide area networks such as delay, bandwidth restrictions, and packet loss. The code is available in recent Linux kernels; version 2.6 and higher will suffice. Netem was used to implement network delay (e.g., 10ms latency) on a laboratory PC running Fedora Core 4 Linux distribution with 100Mbps LAN ports.

To restrict bandwidth, Quality of Service (QoS) controls were implemented using the hierarchical token bucket. Extensive documentation is available on the Web at the following locations:

• http://linux-net.osdl.org/index.php/Netem - the netem wiki
• http://lartc.org/howto/lartc.qdisc.html - Linux advanced routing
• http://luxik.cdi.cz/~devik/qos/htb/ - HTB home page

Both capabilities are invoked using the "tc" utility included in the iproute2 package. Only outgoing packets on an interface are affected, which is an important note. To emulate a WAN link, QoS must be applied to both interfaces (e.g., Asymmetrical DSL line).

The emulator was configured as a bridge to eliminate the need for changing IP addresses on the client. Queuing parameters were applied to both interfaces (eth1 and eth0) to shape both incoming and outgoing traffic. Eth0 is the upstream interface, which is connected to the wide area network. Eth1 is the downstream interface, which is connected to the client.

Commands

# Set up Ethernet 1
tc qdisc del dev eth1 root
# Use netem to implement packet delay
tc qdisc add dev eth1 root handle 1: netem delay 16ms
# Now, set up the bandwidth limits
tc qdisc add dev eth1 parent 1: handle 10: htb deault 1
tc class add dev eth1 parent 10: classid 10:1 htb rate 768kbit ceil 768kbit

# Do the same thing for Ethernet 0
tc qdisc del dev eth0 root
tc qdisc add dev eth0 root handle 1: netem delay 16ms
tc qdisc add dev eth0 parent 1: handle 10: htb deault 1
tc class add dev eth0 parent 10: classid 10:1 htb rate 128kbit ceil 128kbit

trpr - Trace Plot Real-Time
This program calculates flow rates using tcpdump capture files and outputs the results to a file suitable for graphing using gnuplot. Trpr was written by the Naval Research Laboratory.

Collect data using tcpdump, then calculate the rates with trpr and display the results with gnuplot. Trpr requires the hex output. When working with a slower computer, two steps are needed: capture the binary data, then write the hex file for trpr.

tcpdump -l -n -w /tmp/test.dmp
tcpdump -n -x host 192.168.10.102 -r /tmp/test.dmp | trpr \
auto X output /tmp/plot.plt
gnuplot -persist /tmp/plot.plt

Several experiments were run to validate the performance of the emulator. The tests are by no means exhaustive, but they do show that the emulator performs as expected. Figure 3 shows the result of an iperf (bandwidth test) session over a 100Mbps LAN link, using two Pentium 400s with the Linux OS. The plot was produced using the tools described above: tcpdump, trpr, and gnuplot. The transfer rate is about 80 Mbps.

Next, two plots are made with the simulator between the sending and receiving station. The first plot (See Figure 4) shows the results of setting the emulator at 128Kbps (in/out), and the second plot (See Figure 5) shows the results for a T1 (1.54Mbps). The latency is 80ms.

Lab-grade Testing
We used a lab-grade (Spirent) network testing device to verify operations of a PC-based emulator. The tests showed the latency and speed emulations were acceptable for qualitative testing. Assigned values differed slightly from measured values. The PC used in this test was a highly portable Mini-ITX system with 10Mbps LAN ports using a Fedora Core 4 Linux operating system.

One-way latency was measured as the "netem" latency parameters were varied from 0ms to 1000ms. The results are listed in Table 3.

Except for 0ms latency, the actual performance was about 11ms slower than the delay parameter as set. For example, the "netem" delay parameter was set to 20ms and the measured response was 30.8ms.

Throughput was measured to determine if the set queuing speed matched. Using the same test configuration as above, measurements were taken at T1 speeds and 128Kbps speeds. (Figure 6)

At T1 speed (1.544M bits per second), the measured throughput was 194 frames per second. Each frame was 1000 bytes long, which is equal to a rate of 1.552Mbps [1000 bytes X 194 frames X 8 bits/byte = 1552000 bits per second].

When the emulator was set for 128Kbps, the measured results were 15 frames per second, which corresponds to 120Kbps. (Figure 7) Again, this is probably close enough for qualitative measurements. (Packet sizes could be varied to see if finer resolution may be obtained.)

A packet sniffer captured traffic through a bridge (i.e., br0), and there seemed to be a discrepancy between the number of packets measured versus the number of packets sent by the Spirent test equipment. Ethereal (sniffer) indicated the packet was 996 bytes long, not 1000, as configured on the Spirent test equipment. The Ethernet header was 14 bytes, the IP header was 20, and the rest of the packet was 942. This is moot for the purposes of this test, but further work should be conducted to account for the "missing" 4 bytes.

References

y.1541 class of service IETF draft: http://tools.ietf.org/html/draft-ietf-nsis-y1541-qosm-02 .

The netem wiki: http://linux-net.osdl.org/index.php/Netem .

Linux advanced routing: http://lartc.org/howto/lartc.qdisc.html .

HTB home page: http://luxik.cdi.cz/~devik/qos/htb/ .

Naval Research Lab, Network and Communications System Branch: http://mgen.pf.itd.nrl.navy.mil/.