Methodology for Testing High-Speed Network Devices with Predicted Traffic

       P. Sagmeister     G. Dittmann     A. Herkersdorf     D. Webb
           Phone: +41-1-724 8912     e-mail: psa@zurich.ibm.com
                 IBM Research, Zurich Research Laboratory
                               Säumerstr. 4
                       8803 Rüschlikon, Switzerland

Performance validation of next generation network devices in the
multigigabit domain is a difficult issue. One of the major problems is the
lack of knowledge about future high-speed traffic characteristics and the
difficulties in predicting them. Today either synthetic or real traffic
traces are used for performance validation. Synthetic traces are generated
based on a variety of models, which employ concepts such as selfsimilarity,
Poisson, or user interaction. It is possible to generate traces which
resemble full link load or bursts of very short packets, i.e. certain
worst-case situations for specific network devices. However, all of the
characteristics seen in real traffic traces cannot be considered at the
same time. An alternative is the use of existing real traces, but they are
very short and monitored links in general do not carry full load.
Furthermore, only traces taken from today's OC-3  or OC-12 links are
available. They have the advantage, however, of being able to preserve the
original traffic shape.
The presented method attempts to gain traffic traces for validation
purposes, which resemble the traffic load of a full high-speed link, e.g.
40 Gbps, without losing the original traffic characteristics. The mechanism
is based on real OC-3 traffic traces. In a first step, the original OC-3
traces are load compressed by eliminating all time gaps between packets.
This produces traces of a fully loaded OC-3 link. In a second phase, a time
compression of the resulting traces takes place. Depending on the
compression factor, traffic traces of a fully loaded high-speed link, e.g.
40 Gbps, are created. In a third step, the resulting traces can be combined
to achieve larger test traces where the following mechanisms are used: (a)
packet interleaving of different traces on a one-by-one basis or randomly
selected (b) concatenation of different or equal traces.
While observing fully loaded links, phenomena depending on link load
distribution over time, such as selfsimilarity, have been eliminated.
Functional traffic analysis, therefore, was carried out to gain information
about the effects of the described three-step modification method. Common
analysis, e.g. packet, flow or protocol distribution, of the modified
traces compared to the original OC-3 traces showed no significant
differences in traffic shape.
Additional information was gathered to characterize a fully loaded link,
e.g. flows with a maximum number of packets or bytes within the trace, load
share of source/destination hosts, occurrence of large packets, or the
number of concurrently active flows. This analysis detected different kinds
of burstiness: rhythmic fluctuations in packet size within a flow,
fluctuations in the occurrence of hosts and in their load share. As these
phenomena might present serious challenges for a network device, they need
to be included in future models for synthetic traffic generation.
The functional analysis has shown that combining traces by means of
interleaving reduces the burstiness compared to individual traces before
interleaving. This scenario should represent high-speed backbone traffic
consisting of many multiplexed low-speed links quite well. In contrast,
concatenation of traces does not smooth the traffic behavior. However,
combining multiple copies of the same trace roughly doubles the maximum
number of simultaneously active flows. This can be a severe challenge, e.g.
for those networking devices that must maintain per-flow state information.
Based on the results of the functional traffic analysis, a large test suite
was composed to represent a variety of different traffic characteristics.
Used to validate the concept of a 40 Gbps network device,  the test suite
was based on real traces and revealed several performance limitations,
which were not detected by synthetic traces generated with traditional
models. These results led to considerable design improvements.