Re: [LSF/MM TOPIC] Discuss least bad options for resolving longterm-GUP usage by RDMA

[Ira Weiny]

On Fri, Feb 15, 2019 at 03:00:31PM -0700, Jason Gunthorpe wrote:
> On Fri, Feb 15, 2019 at 06:31:36PM +0000, Christopher Lameter wrote:
> > On Fri, 15 Feb 2019, Matthew Wilcox wrote:
> > 
> > > > Since RDMA is something similar: Can we say that a file that is used for
> > > > RDMA should not use the page cache?
> > >
> > > That makes no sense.  The page cache is the standard synchronisation point
> > > for filesystems and processes.  The only problems come in for the things
> > > which bypass the page cache like O_DIRECT and DAX.
> > 
> > It makes a lot of sense since the filesystems play COW etc games with the
> > pages and RDMA is very much like O_DIRECT in that the pages are modified
> > directly under I/O. It also bypasses the page cache in case you have
> > not noticed yet.
> 
> It is quite different, O_DIRECT modifies the physical blocks on the
> storage, bypassing the memory copy.
>

Really?  I thought O_DIRECT allowed the block drivers to write to/from user
space buffers.  But the _storage_ was still under the control of the block
drivers?

>
> RDMA modifies the memory copy.
> 
> pages are necessary to do RDMA, and those pages have to be flushed to
> disk.. So I'm not seeing how it can be disconnected from the page
> cache?

I don't disagree with this.

Ira

> 
> Jason