By Eric Seppanen
Monday, May 7, 2001

The question was recently asked on the linuxbios mailing list,

> are there recommended algorithms for setting the timers? just pick some
> nice number? I've never been sure what to do with those timers.

And since I knew a bit about them but they weren't terribly well documented, I wrote this response. In general this is meant for people writing PCI configuration software. It probably won't help you fix your broken PC.

PCI latency timers are a mechanism for PCI bus-mastering devices to share the PCI bus fairly. "Fair" in this case means that devices won't use such a large portion of the available PCI bus bandwidth that other devices aren't able to get needed work done.

How this works is that each PCI device that can operate in bus-master mode is required to implement a timer, called the Latency Timer, that limits the time that device can hold the PCI bus. The timer starts when the device gains bus ownership, and counts down at the rate of the PCI clock. When the counter reaches zero, the device is required to release the bus. If no other devices are waiting for bus ownership, it may simply grab the bus again and transfer more data.

If the latency timer is set too low, PCI devices will interrupt their transfers unnecessarily often, hurting performance. If it's set too high, devices that require frequent bus access may overflow their buffers, losing data.

So in theory there's a compromise that can be reached somewhere in between, where all devices can get good performance plus a reasonable guarantee that they will get bus access on a timely basis.

Here's my best impression of the various ways you can handle the timers:

The Bad Way: Don't set them at all (in all cases I've looked at this results in a setting of zero).

The Better Way: just pick a number (I use 48 in my code) and set all devices and bridges to that.

The Best Way, in theory, though I doubt anybody does it:

For each card, read the MIN_GNT and MAX_LAT values, and calculate the following:

• latency_multiplier = (8 for 33MHz) (this is the number of PCI clocks in one- quarter microsecond.)
• min_grant_clocks= latency_multiplier * min_gnt
• max_latency_clocks= latency_multiplier * max_lat

Then, calculate the desired latency timer values for each device such that:

1. each card's latency timer should be greater than that card's min_grant_clocks.
2. each card's latency timer should be set as high as possible. This is because the latency timer will truncate pci busmaster cycles when it expires, so a too-low setting will hurt performance.
3. all latency timers should be set low enough that if boards A, B, and C all start doing maximum length transfers, board D will still get the bus before max_latency_clocks has elapsed. So,

   A.latency_timer + B.latency_timer + C.latency_timer < D.max_latency_clocks

   Of course, the device manufacturer might have included themselves in the calculation, and in that case the requirement would be:

   A.latency_timer + B.latency_timer + C.latency_timer + D.latency_timer < D.max_latency_clocks

   which is easier to calculate anyway, since A+B+C+D could simply be called system_max_latency. If it's greater than any card's max_latency_clocks, you might have trouble.

Whew.

Needless to say, factory BIOSes are all over the board on this, and I suspect that most simply choose the "better" option (fixed values).

In truth, rule number 1 seems to get violated all the time without any ill effects. Since everybody else ignores rule #1 and I doubt anybody even tries to satisfy rule #3, the "better" option seems like an acceptable compromise.

Eric

(in section 6.2.4, Miscellaneous Registers)

Latency Timer

This register specifies, in units of PCI bus clocks, the value of the Latency Timer for this PCI bus master (refer to section 3.5.). This register must be implemented as writable by any master that can burst for more than two data phases. This register may be implemented as read-only for devices that burst two or fewer data phases, but the hardwired value must be limited to 16 or less. A typical implementation would be to build the five high-order bits (leaving the bottom three as read-only), resulting in a timer granularity of eight clocks. At RST#, the register must be initialized to 0 (if programmable).

...

MIN_GNT and MAX_LAT

These read-only byte registers are used to specify the device's desired settings for Latency Timer values. For both registers, the value specifies a period of time in units of 1/4 microsecond. Values of 0 indicate that the device has no major requirements for the settings of Latency Timers.

MIN_GNT is used for specifying how long a burst period the device needs assuming a clock rate of 33MHz. MAX_LAT is used for specifying how often the device needs to gain access to the PCI bus.

Devices should specify values that will allow them to most effectively use the PCI bus as well as their internal resources. Values should be chosen assuming that the target does not insert any wait states.

Implementation Example: Choosing MIN_GNT and MAX_LAT

A fast Ethernet controller (100Mbps) has a 64-byte buffer for each transfer direction. Optimal usage of these internal resources is achieved when the device treats each buffer as two 32-byte ping-pong buffers. Each 32-byte buffer has eight DWORDS of data to be transferred, resulting in eight data phases on the PCI bus. These eight data phases translate to 1/4 microsecond at 33MHz, so the MIN_GNT value for this device is "1". When moving data, the device will need to empty or fill a 32-byte buffer every 3.2 µs (assuming a throughput of 10 MB/s). This would correspond to a MAX_LAT value of 12.

/*
 *  If we set up a device for bus mastering, we need to check the latency
 *  timer as certain crappy BIOSes forget to set it properly.
 */

I recently picked up a book that I'd recommend for anyone that has to deal with these sorts of issues on a regular basis. It's PCI System Architecture by Tom Shanley and Don Anderson, and published by Mindshare, Inc.

The book does the best job I've seen of covering PCI latency in a precise and very readable way. It also pointed out some interesting things:

• 33MHz is assumed when calculating the latency timer value.
• Devices are permitted to force less significant bits to zero, in effect rounding the value written down to a multiple, of, for example, 8 (which the authors say is typical). Unless you're going to read back the latency values to see if the hardware mangled them, it's probably best to just make your code only write multiples of 8. The PCI spec also mentions this.
• It is possible that the bus arbiter might be configured to favor devices with low MAX_LAT register values. In the text above, I'm assuming a fairly stupid arbiter that deals out bus grant (GNT) signals in round-robin fashion. If the arbiter were smarter, as could certainly be the case, my calculation of a "system_max_latency" would be useless because the system may be able to service a demanding device more often. This complicates the situation, of course, but my original description should still be useful as a worst-case boundary.
• The book implies that MAX_LAT isn't really intended to calculate an upper bound for latency timer values, but rather is supposed to provide hints for configuring the bus arbiter to favor demanding devices. If this is the case, it's difficult to determine the upper limit for the latency timers. I am unsure what to make of this implication, since the PCI spec explicitly states that MIN_GNT and MAX_LAT "are used to specify... desired settings for Latency Timer values."

Eric

..