Re: [PATCH v3 02/14] dm: add ->copy_from_iter() dax operation support

[Kani, Toshimitsu]

On Wed, 2017-06-14 at 18:45 -0600, Toshi Kani wrote:
> On Fri, 2017-06-09 at 13:23 -0700, Dan Williams wrote:
> > Allow device-mapper to route copy_from_iter operations to the
> > per-target implementation. In order for the device stacking to work
> > we need a dax_dev and a pgoff relative to that device. This gives
> > each layer of the stack the information it needs to look up the
> > operation pointer for the next level.
> > 
> > This conceptually allows for an array of mixed device drivers with
> > varying copy_from_iter implementations.
> > 
> > Cc: Toshi Kani <toshi.kani@xxxxxxx>
> > Reviewed-by: Mike Snitzer <snitzer@xxxxxxxxxx>
> > Signed-off-by: Dan Williams <dan.j.williams@xxxxxxxxx>
> 
> I was worried about possible overhead with additional stub calls, but
> it looks fine with a single thread fio write test with direct=1.
> 
> Â92.62%ÂÂ[kernel.kallsyms]ÂÂÂ[k] __copy_user_nocache
> Â 0.04%ÂÂ[kernel.kallsyms]ÂÂÂ[k] entry_SYSCALL_64_fastpath
> Â 0.08%ÂÂlibpthread-2.22.soÂÂ[.] __GI___libc_write
> Â 0.01%ÂÂ[kernel.kallsyms]ÂÂÂ[k] sys_write
> Â 0.02%ÂÂ[kernel.kallsyms]ÂÂÂ[k] vfs_write
> Â 0.02%ÂÂ[kernel.kallsyms]ÂÂÂ[k] __vfs_write
> Â 0.02%ÂÂ[kernel.kallsyms]ÂÂÂ[k] ext4_file_write_iter
> Â 0.02%ÂÂ[kernel.kallsyms]ÂÂÂ[k] dax_iomap_rw
> Â 0.03%ÂÂ[kernel.kallsyms]ÂÂÂ[k] iomap_apply
> Â 0.04%ÂÂ[kernel.kallsyms]ÂÂÂ[k] dax_iomap_actor
> Â 0.01%ÂÂ[kernel.kallsyms]ÂÂÂ[k] dax_copy_from_iter
> Â 0.01%ÂÂ[kernel.kallsyms]ÂÂÂ[k] dm_dax_copy_from_iter
> Â 0.01%ÂÂ[kernel.kallsyms]ÂÂÂ[k] linear_dax_copy_from_iter
> Â 0.03%ÂÂ[kernel.kallsyms]ÂÂÂ[k] copy_from_iter_flushcache
> Â 0.00%ÂÂ[kernel.kallsyms]ÂÂÂ[k] pmem_copy_from_iter

I had bs=256k, which was too big for this test.  bs=4k result is not
this pretty at all, only 23% in __copy_user_nocache.  This change
accounts for approx. 1% with 4k.  Given we have larger overheads in
many other functions in the path, the change looks acceptable (I keep
my review-by).  I'd prefer to reduce code in the path, though.

Thanks,
-Toshi