Re: [openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspace verbsimplementation
From: Timur Tabi
Date: Thu Apr 21 2005 - 15:10:16 EST
Andy Isaacson wrote:
I'm familiar with MPI 1.0 and 2.0, but I haven't been following the
development of modern messaging APIs, so I might not make sense here...
Assuming that the app calls into the library on a fairly regular basis,
Not really. The whole point is to have the adapter DMA the data directly from memory to
the network. That's why it's called RDMA - remote DMA.
Therefore, cluster admins are going to do their
darndest to avoid this behavior, so we might as well just kill the job
and make it explicit.
Yes, and if it turns out that the same MPI application dies on Linux but not on Solaris
because Linux doesn't really care if the memory stays pinned, then we're going to see a
lot of MPI customers transitioning away from Linux.
*You* need to come up with a solution that looks good to *the community*
if you want it merged.
True, but I'm not going to waste my time adding this support if the consensus I get from
the kernel developers that they don't want Linux to behave this way.
Do you guys simply raise RLIMIT_MEMLOCK to allow apps to lock their
pages? Or are you doing something more nasty?
A little more nasty. I raise RLIMIT_MEMLOCK in the driver to "unlimited" and also set
cap_raise(IPC_LOCK). I do this because I need to support all 2.4 and 2.6 kernel versions
with the same driver, but only 2.6.10 and later have any support for non-root mlock().
If and when our driver is submitted to the official kernel, that nastiness will be removed
of course.
--
Timur Tabi
Staff Software Engineer
timur.tabi@xxxxxxxxxxx
One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
-- Ed Smylie, NASA engineer for Apollo 13
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/