Re: Gigabit Linux Server Bottlenecks

From: Aman Singla (aman@cthulhu.engr.sgi.com)
Date: Tue Feb 08 2000 - 14:28:57 EST


> There's a latency tradeoff there, most coalescing algorithms basically
> do wait for X packets and generate an interrupt or generate the
> interrupt inbetween if there is more than Y usecs between packets. You
> do not want an interrupt for each packet coming in.

The Scheduled Transfer Protocol (STP) addresses some of these issues
with high performance NICs with small on-the-wire frame sizes. STP
uses a data hierarchy of transfers->blocks->STUs (Scheduled Transfer
Units), where STUs correspond to the on-the-wire frame size, while
blocks correspond to the level at which the protocol itself does data
accounting. By putting some protocol state in the NIC, it is possible
to dma the STUs (where 1500 bytes or 9K jumbo) directly to the
receive buffers, but have the host be interrupted only once per block
(64K-256K). Since there is some protocol state in the NIC, latency
sensitive packets don't have to be subjected to indiscriminate interrupt
coalescing => there isn't a latency tradeoff.
Please lookout for an initial developmental source code release of STP
on Linux from SGI later in the week.

Our experiments with running STP on GbE, and the sensitivity
of the b/w performance to the interrupt coalescing factor on the NIC,
in fact seem to strongly suggest that an appropriate IP fragmentation
and reassembly mechanism running in the NIC might be a good idea.

thanks,

:a

----------------------

Aman Singla
MTS - Core Design, SGI

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Tue Feb 15 2000 - 21:00:13 EST