Re: Direct I/O

From: Stephen C. Tweedie (sct@redhat.com)
Date: Fri Mar 10 2000 - 12:59:15 EST


Hi,

On Thu, 9 Mar 2000 19:56:46 +0100, Alexander S A Kjeldaas
<Alexander.Kjeldaas@fast.no> said:

> I'm looking for opinions about how direct I/O can be implemented on
> Linux. I am interested in a design that does not touch the
> page-tables of the process, and where the data is DMAed directly to
> user-space visible pages.

Sorry. You can't access user space from DMA _without_ touching page
tables at some point. The only two ways of doing it are to create
memory in kernel space and then mmap() it into user space (and faulting
those in requires direct page table access), or by following user page
tables to convert user virtual addresses into physical memory
addresses (which is what map_user_kiobuf() does in 2.3).

> My goal is to have an interface that lets me DMA directly from disk
> into the CPU-cache

You can't. You can only DMA to/from main memory, not cache.

> using a combination of direct I/O on a static buffer (held in the CPU
> cahce), and bus-snooping catching the new data as it flies past the
> CPU during the DMA (if the CPU is able to get to the data on the bus
> during snooping).

It can't. Bus snooping can invalidate the existing contents of CPU
caches, but it doesn't update the caches with the new values on the
bus.

> The idea might be ugly, but it might be better than nothing :-)

> o An mmap hint (MAP_BLOCK?) that requests that the mmap should block
> until the region requested has been read into memory.

Chuck Lever has been doing a lot of work on this sort of thing. He has
been posting patches for mincore() (to find out the residency of an area
of memory), and madvise() (to force pagein or discarding of memory
ranges).

> Could something like the above be an acceptable alternative to direct
> I/O to user-space-provided pages?

Direct IO is already possible anyway. The raw devices in 2.3 provide
it (at least unless you have user memory higher in memory than the
device drivers can access). We are working on a more flexible
infrastructure for this sort of thing, too: the kiobuf patches I posted
to l-k this week are a step towards separating out the management of
filesystem caches internally from the way we pass the data to and from
user space.

--Stephen

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Wed Mar 15 2000 - 21:00:18 EST