Re: [PATCH][RT] netpoll: Always take poll_lock when doing polling

[Alison Chaiken]

 I asked on 2016-06-07 17:19:43 [-0700]:
>>cpsw_rx_poll() is called even when there is essentially no network
>>traffic, so I'm not sure how to tell if NAPI is working as intended.

On Thu, Jun 9, 2016 at 5:37 AM, Sebastian Andrzej Siewior
<bigeasy@xxxxxxxxxxxxx> wrote:
> You should see an invocation of __raise_softirq_irqoff_ksoft() and then
> cpsw's poll function should run in "ksoftirqd/" context instead in the
> context of the task it runs now.

The attached patch uses a kprobe to detect when Ethernet switches to
NAPI on a Freescale i.MX6 board.   Thanks to Sebastian for the
suggestion about the method.   As expected, there are no messages when
I ping-flood the board from another host.   However, if I also spawn
multiple scp's of large files at the same time, then the messages
appear.     I tested with 4.4.4-rt11, but the virtually identical
patch is against 4.1.18-rt17.   I'm posting it here in case it's
useful to someone else.   It seems to me that if the various IRQs that
can invoke the net_rx_action() are pinned to different cores, that the
use of smp_processor_id() to identify the device that spawns the IRQ
is therefore robust.

The RT scheduling problem we had (namely, system falls over under
ping-flood) was solved by my colleague Brian Silverman, who pinned our
userspace application that ran the critical event loop and adjusted
its priority.   Doing so prevented a ping-flood from causing the event
loop to miss cycles.

Thanks again to everyone for your advice, and I hope to meet some of
you in Berlin next month.

-- Alison
From 1c83b4ee5d572bc1ede630fc72d01228ff2338e2 Mon Sep 17 00:00:00 2001
From: Alison Chaiken <alison@xxxxxxxxxxxxxxxx>
Date: Sat, 3 Sep 2016 15:51:41 -0700
Subject: [PATCH] kprobes: detect ethernet and CAN NAPI

Inserting this module will induce the core network driver to print a
message whenever handling of CAN or Ethernet packets is performed via
NAPI in ksoftirqd rather than in the context of the invoking hard IRQ
thread.  Tested on Boundary Devices Nitrogen board with i.MX6Q SOC
with 4.4.4-rt11 kernel.

Signed-off-by: Alison Chaiken <alison@xxxxxxxxxxxxx>
---
 samples/kprobes/kprobe_example.c | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/samples/kprobes/kprobe_example.c b/samples/kprobes/kprobe_example.c
index 366db1a..ef3b5ee 100644
--- a/samples/kprobes/kprobe_example.c
+++ b/samples/kprobes/kprobe_example.c
@@ -16,7 +16,8 @@
 
 /* For each probe you need to allocate a kprobe structure */
 static struct kprobe kp = {
-	.symbol_name	= "do_fork",
+/*	.symbol_name	= "do_fork", */
+	.symbol_name	= "__raise_softirq_irqoff_ksoft",
 };
 
 /* kprobe pre_handler: called just before the probed instruction is executed */
@@ -51,6 +52,8 @@ static int handler_pre(struct kprobe *p, struct pt_regs *regs)
 static void handler_post(struct kprobe *p, struct pt_regs *regs,
 				unsigned long flags)
 {
+  unsigned id = smp_processor_id();
+
 #ifdef CONFIG_X86
 	printk(KERN_INFO "post_handler: p->addr = 0x%p, flags = 0x%lx\n",
 		p->addr, regs->flags);
@@ -67,6 +70,25 @@ static void handler_post(struct kprobe *p, struct pt_regs *regs,
 	printk(KERN_INFO "post_handler: p->addr = 0x%p, ex1 = 0x%lx\n",
 		p->addr, regs->ex1);
 #endif
+
+	/* change id to that where the eth IRQ is pinned */
+	if (id == 0) {
+		pr_info("Switched to ethernet NAPI.\n");
+#ifdef CONFIG_ARM
+		pr_info("post_handler: p->addr = 0x%p, pc = 0x%lx,"
+			" lr = 0x%lx, cpsr = 0x%lx\n",
+			p->addr, regs->ARM_pc, regs->ARM_lr, regs->ARM_cpsr);
+#endif
+	}
+	/* change id to that where the CAN IRQ is pinned */
+	if (id == 1) {
+		pr_info("Switched to CAN NAPI.\n");
+#ifdef CONFIG_ARM
+		pr_info("post_handler: p->addr = 0x%p, pc = 0x%lx,"
+			" lr = 0x%lx, cpsr = 0x%lx\n",
+			p->addr, regs->ARM_pc, regs->ARM_lr, regs->ARM_cpsr);
+#endif
+	}
 }
 
 /*
-- 
2.1.4