Re: [PATCH v2 net] octeontx2-pf: fix page_pool creation fail for rings > 32k

[Alexander Lobakin]

From: Ratheesh Kannoth <rkannoth@xxxxxxxxxxx>
Date: Wed, 23 Aug 2023 08:23:25 +0530

> octeontx2 driver calls page_pool_create() during driver probe()
> and fails if queue size > 32k. Page pool infra uses these buffers
> as shock absorbers for burst traffic. These pages are pinned down
> over time as working sets varies, due to the recycling nature
> of page pool, given page pool (currently) don't have a shrinker
> mechanism, the pages remain pinned down in ptr_ring.
> Instead of clamping page_pool size to 32k at
> most, limit it even more to 2k to avoid wasting memory.
> 
> This have been tested on octeontx2 CN10KA hardware.
> TCP and UDP tests using iperf shows no performance regressions.

See now? I told ya you don't need that many to provide good recycling :>
But nice to see it was confirmed 2k is enough.

> 
> Fixes: b2e3406a38f0 ("octeontx2-pf: Add support for page pool")
> Suggested-by: Alexander Lobakin <aleksander.lobakin@xxxxxxxxx>
> Reviewed-by: Sunil Goutham <sgoutham@xxxxxxxxxxx>
> Signed-off-by: Ratheesh Kannoth <rkannoth@xxxxxxxxxxx>
> ---
> 
> ChangeLogs:
> 
> v1->v2: Commit message changes and typo fixes
> v0->v1: Commit message changes.
> ---
>  drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c | 2 +-
>  drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h | 2 ++
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
> index 77c8f650f7ac..3e1c70c74622 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
> @@ -1432,7 +1432,7 @@ int otx2_pool_init(struct otx2_nic *pfvf, u16 pool_id,
>  	}
>  
>  	pp_params.flags = PP_FLAG_PAGE_FRAG | PP_FLAG_DMA_MAP;
> -	pp_params.pool_size = numptrs;
> +	pp_params.pool_size = min(OTX2_PAGE_POOL_SZ, numptrs);
>  	pp_params.nid = NUMA_NO_NODE;
>  	pp_params.dev = pfvf->dev;
>  	pp_params.dma_dir = DMA_FROM_DEVICE;
> diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
> index ba8091131ec0..f6fea43617ff 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
> +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h

Isn't otx2_txrx.h a better place for this definition? For example, I'd
place it somewhere between CGX_CHAN_BASE and OTX2_DATA_ALIGN. Or after
MAX_XDP_MTU, doesn't matter much.

> @@ -30,6 +30,8 @@
>  #include <rvu_trace.h>
>  #include "qos.h"
>  
> +#define OTX2_PAGE_POOL_SZ 2048

Don't forget about alignment. Otx2 code has very inconsistent alignment
of definitions -- just one space in some places and proper tab alignment
in others. I still believe we prefer the second for netdev code.

> +
>  /* IPv4 flag more fragment bit */
>  #define IPV4_FLAG_MORE				0x20
>  

Thanks,
Olek