Re: [PATCH 1/11] Add generic helpers for arch IPI function calls

From: Peter Zijlstra
Date: Tue Apr 22 2008 - 09:01:29 EST


On Tue, 2008-04-22 at 13:14 +0200, Jens Axboe wrote:
> On Tue, Apr 22 2008, Jens Axboe wrote:
> > On Tue, Apr 22 2008, Avi Kivity wrote:
> > > Jens Axboe wrote:
> > > >This adds kernel/smp.c which contains helpers for IPI function calls. In
> > > >addition to supporting the existing smp_call_function() in a more efficient
> > > >manner, it also adds a more scalable variant called
> > > >smp_call_function_single()
> > > >for calling a given function on a single CPU only.
> > > >
> > > >The core of this is based on the x86-64 patch from Nick Piggin, lots of
> > > >changes since then. "Alan D. Brunelle" <Alan.Brunelle@xxxxxx> has
> > > >contributed lots of fixes and suggestions as well.
> > > >
> > > >+int smp_call_function_single(int cpu, void (*func) (void *info), void
> > > >*info,
> > > >+ int retry, int wait)
> > > >+{
> > > >+ unsigned long flags;
> > > >+ /* prevent preemption and reschedule on another processor */
> > > >+ int me = get_cpu();
> > > >+ int ret = 0;
> > > >+
> > > >+ /* Can deadlock when called with interrupts disabled */
> > > >+ WARN_ON(wait && irqs_disabled());
> > > >+
> > > >+ if (cpu == me) {
> > > >+ local_irq_save(flags);
> > > >+ func(info);
> > > >+ local_irq_restore(flags);
> > > >+ } else {
> > > >+ struct call_single_data d;
> > > >+ struct call_single_data *data;
> > > >+
> > > >+ if (!wait) {
> > > >+ data = kmalloc(sizeof(*data), GFP_ATOMIC);
> > > >+ if (unlikely(!data)) {
> > > >+ ret = -ENOMEM;
> > > >+ goto out;
> > > >+ }
> > > >+ data->flags = CSD_FLAG_ALLOC;
> > > >+ } else {
> > > >+ data = &d;
> > > >+ data->flags = CSD_FLAG_WAIT;
> > > >+ }
> > > >+
> > > >
> > >
> > > Instead of introducing a rare error case, how about falling back to the
> > > wait case if the allocation fails?
> > >
> > > Of course, if the called function relies on the calling cpu doing
> > > something else, then this fails, but I don't think anybody would do
> > > that? On the other hand, there is at least one use of
> > > smp_call_function_single() with !wait, which doesn't check the error return.
> >
> > Sure, either failling back to waiting, or add a static call_single_data
> > like it exists for smp_call_function(). In reality it'll never happen,
> > so the fallback static structure appeals the most to me.
>
> We don't need any extra statically allocated data, we can just reuse the
> 'csd' element of the existing call_data_fallback. So that is what I did.
> Once all archs are converted, we can now change
> smp_call_function_single() to a void return, as it always succeeds now.

Introducing this fallback will make any usage from irq disabled context
deadlock prone.

I rather like the current interface.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/