Re: [RFC PATCH] arm64: mm: Add invalidate back in arch_sync_dma_for_device when FROM_DEVICE

[Nanyong Sun]

On 2022/11/17 18:23, Robin Murphy wrote:

> On 2022-11-17 08:24, Ard Biesheuvel wrote:
> > On Thu, 17 Nov 2022 at 07:33, Nanyong Sun <sunnanyong@xxxxxxxxxx> wrote:
> > > The commit c50f11c6196f ("arm64: mm: Don't invalidate FROM_DEVICE
> > > buffers at start of DMA transfer") replaces invalidate with clean
> > > when DMA_FROM_DEVICE, this changes the behavior of functions like
> > > dma_map_single() and dma_sync_single_for_device(*, *, *, 
> > > DMA_FROM_DEVICE),
> > > then it may make some drivers works unwell because the implementation
> > > of these DMA APIs lose the original cache invalidation.
> > >
> > > Situation 1:
> > ...
> > > Situation 2:
> > > After backporting the above commit, we find a network card driver go
> > > wrong with cache inconsistency when doing DMA transfer: CPU got the
> > > stale data in cache when reading DMA data received from device.
> >
> > I suppose this means those drivers may lack dma_sync_single_for_cpu()
> > calls after the inbound transfers complete, and are instead relying on
> > the cache invalidation performed before the transfer to make the DMA'd
> > data visible to the CPU.
>
> It also commonly goes wrong the other way round when the drivers are 
> correct but DT/ACPI failed to indicate a coherent device as such. If 
> writes from the device actually snoop, they hit the still-present 
> cache line, which then gets invalidated by unmap/sync_for_cpu and the 
> new data is lost.
>
> Robin.
>
> .
Is it acceptable?  Does the coherent device MUST run with coherent 
identifier? At least they can run with no wrong before.