Re: Cache incoherencies (WAS: New resources - pls, explain :-( )

Russell King (rmk@arm.linux.org.uk)
Tue, 24 Aug 1999 20:51:51 +0100 (BST)


Benjamin Herrenschmidt writes:
> And I don't think those macros are enough. I may be wrong, but I beleive
> the way tulip.c is currently fixed for non cache coherent archs is wrong.
> Why ? Because we should also take care of the cache line granularity,
> especially when flushing/invalidating the ring descriptors themselves.

You are correct here. The current system and functions are not perfect.

> Another issue, which doesn't seem to be handled in tulip.c neither, is
> that for datas written by the chipset to memory, the cache must be
> invalidated before the chip begins writing (so when the buffers are added
> to the ring), not when the datas are read by the CPU. This is because you
> may still have stale dirty cache lines corresponding to this piece of
> memory that can be flushed at any time, corrupting the datas beeing
> written by the chip.

That is handled - dma_cache_inv is generally called before reading the
ring descriptors anyway. There was, however, one instance in my patches
(which as far as I know is the only place where they are in existance
at the moment) which it wasn't handled correctly (tx_fill_ring).

> The only way I see to make the fist issue safe is to have the ring
> descriptor entries non cachable. Non-cache coherent archs should probably
> define a kmalloc flag to allocate non cachable space. (I still don't know
> what is the cleanest way to get non-cachable space. ioremap ?).

AFAIK, you can't use (and shouldn't use) ioremap on the kernels image of
SDRAM.

> For the buffers themselves, we should use the inval routine when adding
> skbuffs to the ring, (and make sure skbuffs are cache-aligned, and their
> size a multiple of a cache line too).

This is actually an issue for the dma_cache_* functions. For instance,
if the area the dma_cache_inv function is asked to invalidate is in
the moddle of a cache line, then you *must* write it back. This means
that dma_cache_inv should be called even before the chip starts writing
to the space.

> In order to make something compilable on both coherent and incoherent
> archs, we should standardize the #define for the cache line size, add a
> routine/macro to query about the coherency of the bus, and provide a
> "magic" bit for kmalloc that gives you non cachable space.

The idea of getting a space which is non-cacheable appears good, but
I think this may be done in one of two ways. The first way would
be to have the SDRAM mapped in twice (bad, since you end up with two
large images of the same data). Second way would be to remap SDRAM as
required. I think this would be the cleaner solution, since you
could do something similar to vmalloc (but with kmalloc granularity).

(btw, do we need such a long CC: list?)
_____
|_____| ------------------------------------------------- ---+---+-
| | Russell King rmk@arm.linux.org.uk --- ---
| | | | http://www.arm.linux.org.uk/~rmk/aboutme.html / / |
| +-+-+ --- -+-
/ | THE developer of ARM Linux |+| /|\
/ | | | --- |
+-+-+ ------------------------------------------------- /\\\ |

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/