Re: Missing a write memory barrier in tls_init()

[Dae R. Jeong]

On Fri, Nov 10, 2023 at 07:22:48PM +0900, Dae R. Jeong wrote:
> On Wed, Nov 08, 2023 at 10:07:58AM +0100, Sabrina Dubroca wrote:
> > 2023-11-07, 18:53:24 -0800, Jakub Kicinski wrote:
> > > On Tue, 7 Nov 2023 23:45:46 +0100 Sabrina Dubroca wrote:
> > > > Wouldn't it be enough to just move the rcu_assign_pointer after ctx is
> > > > fully initialized, ie just before update_sk_prot? also clearer wrt
> > > > RCU.
> > > 
> > > I'm not sure, IIUC rcu_assign_pointer() is equivalent to
> > > WRITE_ONCE() on any sane architecture, it depends on address
> > > dependencies to provide ordering.
> > 
> > Not what the doc says:
> > 
> >     /**
> >      * rcu_assign_pointer() - assign to RCU-protected pointer
> >      [...]
> >      * Inserts memory barriers on architectures that require them
> >      * (which is most of them), and also prevents the compiler from
> >      * reordering the code that initializes the structure after the pointer
> >      * assignment.
> >      [...]
> >      */
> > 
> > And it uses smp_store_release (unless writing NULL).
> > 
> 
> I think Sabrina is right. We can rely on the release semantic implied
> in rcu_assign_pointer(). Simply moving rcu_assign_pointer() to the end
> of tls_ctx_create() should prevent a scenario what I thought (ie.,
> store-store reordering between ctx->sk_proto and sk->sk_prot).
> 
> diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
> index 1c2c6800949d..d20b823c68d4 100644
> --- a/net/tls/tls_main.c
> +++ b/net/tls/tls_main.c
> @@ -816,9 +816,9 @@ struct tls_context *tls_ctx_create(struct sock *sk)
>                 return NULL;
>  
>         mutex_init(&ctx->tx_lock);
> -       rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
>         ctx->sk_proto = READ_ONCE(sk->sk_prot);
>         ctx->sk = sk;
> +       rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
>         return ctx;
>  }
> 
> But what I also wonder is that, do we need to ensure that
> ctx->{tx,rx}_conf is visible before updating sk->sk_prot? If so, as
> Sabrina suggested, we may want to move rcu_assign_pointer() right
> before update_sk_prot().
> 
> 
> Best regards,
> Dae R. Jeong

I sent a patch by taking suggestions from Sabrina. The patches 1)
moves rcu_assign_pointer() after fully initializing ctx, and 2) gets
rid of tls_ctx_create().

I'm not sure whether removing tls_ctx_create() is a good idea or not,
but it still did not fully initialize ctx (i.e., ctx->{tx,rx}_conf).

Let me know if there is any issue, then I will rewrite a patch.


Best regards,
Dae R. Jeong