Understanding Latency versus Throughput

One of the effects of adopting a High Level Synthesis design methodology is that the barrier between "Systems designers" and "Hardware designers" is substantially reduced if not totally eliminated. Suddenly, both "Systems designers" and "Hardware designers" are using not only the same input language to specify their models (C++ / System C) but they are also exposed to the same terminology. For this reason, "Hardware designers" are suddenly exposed to two terms to which they have had little or no exposure in the past.

The purpose of this post is to clarify two "systems" terms that are usually confused and sometimes used interchangeably: latency and throughput.

Definition of terms

Let us attempt to define those two terms:

Latency is the time required to perform some action or to produce some result. Latency is measured in units of time -- hours, minutes, seconds, nanoseconds or clock periods.

Throughput is the number of such actions executed or results produced per unit of time. This is measured in units of whatever is being produced (cars, motorcycles, I/O samples, memory words, iterations) per unit of time. The term "memory bandwidth" is sometimes used to specify the throughput of memory systems.

A simple example

The following manufacturing example should clarify these two concepts:

An assembly line is manufacturing cars. It takes eight hours to manufacture a car and that the factory produces one hundred and twenty cars per day.

The latency is: 8 hours.

The throughput is: 120 cars / day or 5 cars / hour.

A design example

Now that these two concepts are clear, let us apply these concepts to a problem "closer to home." 

A designer is given the task to create hardware for a communications device that has the following characteristics:

Clock frequency:                                          100MHz

Time available to perform the computation:     1000ns

Throughput of the device:                              640 Mbits / second

Word width of each output:                            64 bits

Let us translate these requirements into latency and throughput measurements that are more meaningful from the point of view of the hardware designer.

Latency: 1000 ns = 1000 ns * (1 s / 10^9 ns) * (100 * 10^6 clock periods/ 1s) = 10^11/10^9 = 100 clock periods.

Throughput = 640 Mbits / s = (640 * 10^6 bits/s) * (1 word / 64 bits) * (1 s / 100 * 10^6 clock periods) =  640 * 10^6 / 64 * 100 * 10^6 = 10 * 10 / 100 = 1 / 10 = 0.1 words / clock period.

The throughput could be read more conveniently as follows: "one word every 10 clock periods"

Latency expressed in clock periods, and throughput expressed in number of available clock cycles between words, are parameters that a designer can use to create the desired hardware according to the performance specifications.

A final clarification

Some tools do not express the throughput in units per unit of time but in clock periods. This is incorrect but commonly used because of convenience. Therefore some tools would report the throughput of our communication algorithm as 10.

Related Resources:

• How the Productivity Advantages of High-Level Synthesis Can Improve IP Design, Verification, and Reuse
• Genus Synthesis Solution
• Stratus High-Level Synthesis