Re: [RFC] extending splice for copy offloading

[Bernd Schubert]

On 09/30/2013 08:02 PM, Myklebust, Trond wrote:
> On Mon, 2013-09-30 at 19:48 +0200, Bernd Schubert wrote:
> > On 09/30/2013 07:44 PM, Myklebust, Trond wrote:
> > > On Mon, 2013-09-30 at 19:17 +0200, Bernd Schubert wrote:
> > > > It would be nice if there would be way if the file system would get a
> > > > hint that the target file is supposed to be copy of another file. That
> > > > way distributed file systems could also create the target-file with the
> > > > correct meta-information (same storage targets as in-file has).
> > > > Well, if we cannot agree on that, file system with a custom protocol at
> > > > least can detect from 0 to SSIZE_MAX and then reset metadata. I'm not
> > > > sure if this would work for pNFS, though.
> > >
> > > splice() does not create new files. What you appear to be asking for
> > > lies way outside the scope of that system call interface.
> >
> > Sorry I know, definitely outside the scope of splice, but in the context
> > of offloaded file copies. So the question is, what is the best way to
> > address/discuss that?
>
> Why does it need to be addressed in the first place?

An offloaded copy is still not efficient if different storage 
servers/targets used by from-file and to-file.

> What is preventing an application from retrieving and setting this
> information using standard libc functions such as fstat()+open(), and
> supplemented with libattr attr_setf/getf(), and libacl acl_get_fd/set_fd
> where appropriate?

At a minimum this requires network and metadata overhead. And while I'm 
working on FhGFS now, I still wonder what other file system need to do - 
for example Lustre pre-allocates storage-target files on creating a 
file, so file layout changes mean even more overhead there.
Anyway, if we could agree on to use libattr or libacl to teach the file 
system about the upcoming splice call I would be fine. Metadata overhead 
is probably negligible for large files.




Thanks,
Bernd

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/