Re: Memory-ordering recipes

[Paul E. McKenney]

On Fri, Sep 22, 2017 at 11:29:18AM +0200, Peter Zijlstra wrote:
> On Thu, Sep 21, 2017 at 09:40:46AM -0700, Paul E. McKenney wrote:
> > > Also, none of these cover 'simple' stuff like a ring-buffer.
> > 
> > Control dependencies are now 'simple'?  ;-)
> 
> They're not particularly harder than any other barrier I find.

Other than keeping the compiler from breaking them, I suppose.  ;-)

> But again, this comes back to the purpose of this recipes thing. I'm
> thinking its meant to be how-to crib sheet for people who really don't
> want to understand this stuff.

Other use cases include learning about memory ordering to begin with,
a convenient repository of the most common cases, and helping guide
people to the more maintainable concurrency designs.

> So it should include common, robust patterns and leave it at that. And
> explain them through the code, not through litmus tests -- because those
> are a royal friggin pain to learn to read in any case ;-)

Litmus tests are no harder to learn than is kernel code, and we do need
to allow people starting from scratch.  Plus the big advantage of the
litmus-test code is that you can check it with the tool, which is not yet
feasible for general kernel code.  Though I believe we will get there,
just not in the next 5 years.

But I agree that pointing to examples in the kernel is a good thing,
though left to myself, I would include the kernel version and not sign
up to change the examples as the kernel changes.

> So, 'how do I do a lockless ring-buffer' is a simple enough question
> with a fairly straight forward answer. The how do I prove that answer is
> correct is a whole other thing, but one I suspect most people really
> don't care about.

At least some of us had jolly better care about showing that it is
correct!  But yes, most people are going to cargo-cult known-good
patterns.  And these people are an important audience for the
recipes document.

And "how do I do a lockless ring buffer" might have a straightforward
answer now, but that code had at least its share of bugs along the way.
On the other hand, if your point is that the memory model would cover
more cases if it handled arrays, no argument here.  I would prioritize
efficiently handling locking higher, but arrays would be good.

> So, once again, what is the intended purpose of his document? A gentle
> introduction to memory ordering, or a crib sheet for the working code
> monkey?

Those are definitely two of the use cases.

> Because if people _want_ to understand this stuff, memory-barriers.txt
> should be their document. If its become too hard to read, we should look
> at fixing that, but adding another document doesn't seem like a solution
> to that problem.

There is no denying that memory-barriers.txt exceeded critical mass some
years ago (https://lwn.net/Articles/575835/), and that was before the
_acquire, _release, and _relaxed variants of atomic RMW operations,
among other things.  Plus it is incomplete, as it is essentially a
compendium of answers to specific questions that have come up over
the years.  For example, right now if you want to know how some
specific memory-ordering mechanism interacts with RCU, too bad, it
is not documented at all.  I suspect that most people do understand at
some level that synchronize_rcu() acts like a full memory barrier, and
there are the comment headers for the various grace-period functions,
but how does all this interact with (say) an atomic_add_return_acquire()
in the middle of some random RCU read-side critical section?  This sort
of thing transforms memory-barriers.txt into "Combinatorial Explosion
Ð Us", hence the formal model.

This whole memory-model effort is in fact the way we are looking to fix
this.  From what I can see, memory-barriers.txt (or a document derived
from it) would have some explanation of what the hardware gets up to,
discussion of I/O barriers, care and feeding of control dependencies,
and perhaps a few other things.

> For the people who really don't care and just want to get their thing
> done; we could have a document explaining common patterns.. maybe.

And that is another use case for this recipes document.

> So once again, what's the purpose of this new document?

Let's see...

1.	By-example introduction to the most common memory-ordering
	patterns for people starting out.

2.	Repository of common cases for easy reference.

3.	Repository of good examples for "don't make up some strange
	thing if these common cases work for you" purposes.

4.	Source of cargo-culting for people who just want to get their
	job done and get on with their lives, without necessarily
	understanding the full glory of memory ordering.

This document is not expected to stand alone.  More complex cases, up
to a point, can be handled by the memory model.  And comments in the
source code are very good things as well.

Does that help, or am I missing your point?

							Thanx, Paul