[PATCH 4/6] perf: Optimize get_recursion_context()

From: Peter Zijlstra
Date: Fri Oct 30 2020 - 11:21:54 EST


"Look ma, no branches!"

Cc: Jesper Dangaard Brouer <brouer@xxxxxxxxxx>
Cc: Steven Rostedt <rostedt@xxxxxxxxxxx>
Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
---
kernel/events/internal.h | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)

--- a/kernel/events/internal.h
+++ b/kernel/events/internal.h
@@ -205,16 +205,15 @@ DEFINE_OUTPUT_COPY(__output_copy_user, a

static inline int get_recursion_context(int *recursion)
{
- int rctx;
+ unsigned int pc = preempt_count();
+ unsigned int rctx = 0;

- if (unlikely(in_nmi()))
- rctx = 3;
- else if (in_irq())
- rctx = 2;
- else if (in_serving_softirq())
- rctx = 1;
- else
- rctx = 0;
+ if (pc & (NMI_MASK))
+ rctx++;
+ if (pc & (NMI_MASK | HARDIRQ_MASK))
+ rctx++;
+ if (pc & (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET))
+ rctx++;

if (recursion[rctx])
return -1;