Re: [PATCH 2/4] io_uring/cmd: Introduce SOCKET_URING_OP_GETSOCKOPT

[Breno Leitao]

On Tue, Jul 25, 2023 at 09:56:50AM -0400, Willem de Bruijn wrote:
> Breno Leitao wrote:
> > On Mon, Jul 24, 2023 at 06:58:04PM -0400, Willem de Bruijn wrote:
> > > Breno Leitao wrote:
> > > > Add support for getsockopt command (SOCKET_URING_OP_GETSOCKOPT), where
> > > > level is SOL_SOCKET. This is leveraging the sockptr_t infrastructure,
> > > > where a sockptr_t is either userspace or kernel space, and handled as
> > > > such.
> > > > 
> > > > Function io_uring_cmd_getsockopt() is inspired by __sys_getsockopt().
> > > > 
> > > > Differently from the getsockopt(2), the optlen field is not a userspace
> > > > pointers. In getsockopt(2), userspace provides optlen pointer, which is
> > > > overwritten by the kernel.  In this implementation, userspace passes a
> > > > u32, and the new value is returned in cqe->res. I.e., optlen is not a
> > > > pointer.
> > > > 
> > > > Important to say that userspace needs to keep the pointer alive until
> > > > the CQE is completed.
> > > > 
> > > > Signed-off-by: Breno Leitao <leitao@xxxxxxxxxx>
> > > > ---
> > > >  include/uapi/linux/io_uring.h |  7 ++++++
> > > >  io_uring/uring_cmd.c          | 43 +++++++++++++++++++++++++++++++++++
> > > >  2 files changed, 50 insertions(+)
> > > > 
> > > > diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
> > > > index 9fc7195f25df..8152151080db 100644
> > > > --- a/include/uapi/linux/io_uring.h
> > > > +++ b/include/uapi/linux/io_uring.h
> > > > @@ -43,6 +43,10 @@ struct io_uring_sqe {
> > > >  	union {
> > > >  		__u64	addr;	/* pointer to buffer or iovecs */
> > > >  		__u64	splice_off_in;
> > > > +		struct {
> > > > +			__u32	level;
> > > > +			__u32	optname;
> > > > +		};
> > > >  	};
> > > >  	__u32	len;		/* buffer size or number of iovecs */
> > > >  	union {
> > > > @@ -79,6 +83,7 @@ struct io_uring_sqe {
> > > >  	union {
> > > >  		__s32	splice_fd_in;
> > > >  		__u32	file_index;
> > > > +		__u32	optlen;
> > > >  		struct {
> > > >  			__u16	addr_len;
> > > >  			__u16	__pad3[1];
> > > > @@ -89,6 +94,7 @@ struct io_uring_sqe {
> > > >  			__u64	addr3;
> > > >  			__u64	__pad2[1];
> > > >  		};
> > > > +		__u64	optval;
> > > >  		/*
> > > >  		 * If the ring is initialized with IORING_SETUP_SQE128, then
> > > >  		 * this field is used for 80 bytes of arbitrary command data
> > > > @@ -729,6 +735,7 @@ struct io_uring_recvmsg_out {
> > > >  enum {
> > > >  	SOCKET_URING_OP_SIOCINQ		= 0,
> > > >  	SOCKET_URING_OP_SIOCOUTQ,
> > > > +	SOCKET_URING_OP_GETSOCKOPT,
> > > >  };
> > > >  
> > > >  #ifdef __cplusplus
> > > > diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c
> > > > index 8e7a03c1b20e..16c857cbf3b0 100644
> > > > --- a/io_uring/uring_cmd.c
> > > > +++ b/io_uring/uring_cmd.c
> > > > @@ -166,6 +166,47 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw,
> > > >  }
> > > >  EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed);
> > > >  
> > > > +static inline int io_uring_cmd_getsockopt(struct socket *sock,
> > > > +					  struct io_uring_cmd *cmd)
> > > > +{
> > > > +	void __user *optval = u64_to_user_ptr(READ_ONCE(cmd->sqe->optval));
> > > > +	int optname = READ_ONCE(cmd->sqe->optname);
> > > > +	int optlen = READ_ONCE(cmd->sqe->optlen);
> > > > +	int level = READ_ONCE(cmd->sqe->level);
> > > > +	void *koptval;
> > > > +	int err;
> > > > +
> > > > +	err = security_socket_getsockopt(sock, level, optname);
> > > > +	if (err)
> > > > +		return err;
> > > > +
> > > > +	koptval = kmalloc(optlen, GFP_KERNEL);
> > > > +	if (!koptval)
> > > > +		return -ENOMEM;
> > > 
> > > This will try to kmalloc any length that userspace passes?
> > 
> > Yes, this value is coming directly from userspace.
> > 
> > > That is unnecessary ..
> > > > +
> > > > +	err = copy_from_user(koptval, optval, optlen);
> > > > +	if (err)
> > > > +		goto fail;
> > > > +
> > > > +	err = -EOPNOTSUPP;
> > > > +	if (level == SOL_SOCKET) {
> > > > +		err = sk_getsockopt(sock->sk, level, optname,
> > > > +				    KERNEL_SOCKPTR(koptval),
> > > > +				    KERNEL_SOCKPTR(&optlen));
> > > 
> > > .. sk_getsockopt defines a union of acceptable fields, which
> > > are all fairly small.
> > 
> > Right, and they are all I need for SOL_SOCKET level for now.
> > 
> > > I notice that BPF added BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN to
> > > work around the issue of pre-allocating for the worst case.
> > 
> > I am having a hard time how to understand how
> > BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN gets the MAX_OPTLEN. Reading this
> > function, it seems it is conditionally get_user().
> > 
> > 
> > 	#define BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN(optlen)
> > 	({
> > 		int __ret = 0;
> > 		if (cgroup_bpf_enabled(CGROUP_GETSOCKOPT))
> > 			get_user(__ret, optlen);
> > 		__ret;
> > 	})
> > 
> > That said, how is BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN being used to
> > workaroundthe pre-allocating for the worst case?
> > 
> > > But that also needs to deal woth other getsockopt levels.
> > 
> > Right, and we also have a similar kmalloc() on the setsockopt path
> > (SOCKET_URING_OP_SETSOCKOPT).
> > 
> > What about if I pass the userspace sockptr (USER_SOCKPTR) to the
> > {g,s}etsockopt callback directly, instead of kmalloc() in io_uring(), as
> > I was doing int the RFC[1]? It avoids any extra kmalloc at all.
> 
> That looks like a great solution to me.
> 
> Avoids the whole problem of kmalloc based on untrusted user input.
> 
> > Something as:
> > 
> > 	static inline int io_uring_cmd_getsockopt(struct socket *sock,
> > 						  struct io_uring_cmd *cmd)
> > 	{
> > 		void __user *optval = u64_to_user_ptr(READ_ONCE(cmd->sqe->optval));
> > 		int optlen = READ_ONCE(cmd->sqe->optlen);
> > 		int optname = READ_ONCE(cmd->sqe->optname);
> > 		int level = READ_ONCE(cmd->sqe->level);
> > 		int err;
> > 
> > 		err = security_socket_getsockopt(sock, level, optname);
> > 		if (err)
> > 			return err;
> > 
> > 		if (level == SOL_SOCKET) {
> > 			err = sk_getsockopt(sock->sk, level, optname,
> > 					    USER_SOCKPTR(optval),
> > 					    KERNEL_SOCKPTR(&optlen));
> > 			if (err < 0)
> > 				return err;
> > 			return optlen;
> > 		}
> 
> Do you have a plan to extend this to other levels?
> 
> No need to implement immediately, but it would be good to know
> whether it is feasible to extend the current solution when the
> need (inevitably) shows up.

Yes, I plan to extend getsockopt() to all levels, but it means we need
to convert proto_ops->setsockopt to uset sockptr_t instead of
userpointers. This might require some intrusive changes, but totally
doable.