Sunday, February 21, 2010

Networking: Bandwidth, Latency, Errors

There are 3 ways to measure "network quality": bandwidth, latency, and network errors.

Raw transfer speed in bytes per second.

Latency: Elapsed time for a single byte to reach its destination.
Errors: Number of dropped or corrupted data packets.

Bandwidth, or throughput, describes how much data can be sent over the network, measured in bytes per second, such as downloading a large file. Network links are often described by bandwidth, from 28.8 baud modems to 100Mbit Ethernet.

Latency is time it takes, usually measured in milliseconds, for a single byte or packet to travel from one host to the other. The best way to measure this is sending a "ping" packet to a host, who immediately responds with a "pong". Divide the round trip time by two to calculate the average one-way latency. Where bandwidth describes how much data a link supports, latency measures how fast a given packet moves from point A to point B.

Finally, network communications are unreliable. Electrical interference can corrupt packet data, a busy router may drop packets, or a working link may suddenly fail and cut off parts of the network. Packets may also arrive safely, but out of order, by travelling different paths.

It's convenient to examine these attributes using archery as a metaphor. A skilled archer may release many arrows quickly, even before previous arrows have hit the target, because a given arrow can only travel so fast. In this example, bandwidth is the time it takes the archer to empty his quiver. Latency is the measurable (but short) time an arrow is airborne. Obviously, network errors are just arrows that miss the target.

Most network applications are concerned with transferring large amounts of data, such as web pages or email. Unused bandwidth is wasted, but flooding a slow link with too much data can impact network performance for everyone on the wire. For this reason, most network software and hardware is optimized for throughput -- even at the cost of latency.

For example, analog modems have relatively little bandwidth and rely on compression to squeeze more data over the link. To work well, compression requires enough data to find and reduce patterns, so the modem may "hold" onto outgoing data (for up to 50ms) until there is enough to compress effectively. While this improves overall bandwidth, the latency for a given packet is much higher.

In arcade and real-time strategy games, latency is the enemy, because faster communication more important than sending alot of data. When a player looks at his radar or shoots at his enemy, he wants instantaneous response. On a single machine, these updates are performed quickly (on the order of a few microseconds). Between two or more machines, however, the communication depends on the network connecting them (on the order of tenths of second).

Network games use common tricks to hide delays caused by latency, such as reducing how much data is sent or reducing the number of hosts that rely on that data. Dead reckoning uses basic physics to predict the path of of an object in motion over a short period of time; the host only transmits a new position and velocity to its peers when the momentum changes, due to collision or course correction. Similarly, a player doesn't need to know every detail about players on the other side of the world -- so he only receives status updates about nearby players.

Network errors, unfortunately, are the biggest cause of poor performance. A reliable protocol must hold onto outgoing data until it knows the other side has received it, because packets may be dropped or corrupted. Incoming packets must be examined so that data is received correctly and in order. If a packet doesn't arrive in a reasonable timeframe, it must be retransmitted. On the other hand, resending a lost packet uses more bandwidth and doubles its latency.

The following chart illustrates some typical network connections and how they contribute to the network "quality":



(one way)


28.8 Analog Modem

3.0 KB/Sec

120 ms


28.8 No Compression

2.0 KB/Sec

70 ms


56K Analog Modem

5.0 KB/Sec

100 ms


ISDN Digital Modem

4-8 KB/Sec

20 ms


Direct Serial

10-115 KB/Sec

20 ms


Ethernet (0-1 hops)

2-12 MB/sec



Ethernet (2-3 hops)

1-4 MB/sec

10-30 ms


Ethernet (4-6 hops)

0.5-1 MB/sec

50-100 ms


Ethernet (distant)


>100 ms


Remember that bandwidth is limited by the poorest link, but latency and network errors are cumulative. The values above are simply guidelines for "good" links -- a poor router or link may adversely affect one or more measures. The further data has to travel between hosts, the more traffic it must compete with and the more resources it uses.