Re: [PATCH 1/2] coresight: trbe: Fix TRBE potential sleep in atomic context

From: hejunhao
Date: Tue Aug 15 2023 - 07:40:58 EST


Hi Suzuki


On 2023/8/15 6:57, Suzuki K Poulose wrote:
On 14/08/2023 14:32, hejunhao wrote:
Hi Suzuki


On 2023/8/14 18:34, Suzuki K Poulose wrote:
Hi Junhao

On 14/08/2023 10:38, Junhao He wrote:
smp_call_function_single() will allocate an IPI interrupt vector to
the target processor and send a function call request to the interrupt
vector. After the target processor receives the IPI interrupt, it will
execute arm_trbe_remove_coresight_cpu() call request in the interrupt
handler.

According to the device_unregister() stack information, if other process
is useing the device, the down_write() may sleep, and trigger deadlocks
or unexpected errors.

arm_trbe_remove_coresight_cpu
coresight_unregister
device_unregister
device_del
kobject_del
__kobject_del
sysfs_remove_dir
kernfs_remove
down_write ---------> it may sleep

Add a helper arm_trbe_disable_cpu() to disable TRBE precpu irq and reset
per TRBE.
Simply call arm_trbe_remove_coresight_cpu() directly without useing the
smp_call_function_single(), which is the same as registering the TRBE
coresight device.

Fixes: 3fbf7f011f24 ("coresight: sink: Add TRBE driver")
Signed-off-by: Junhao He <hejunhao3@xxxxxxxxxx>
---
drivers/hwtracing/coresight/coresight-trbe.c | 35 +++++++++++---------
1 file changed, 20 insertions(+), 15 deletions(-)

diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c
index 7720619909d6..ce1e6f537b8d 100644
--- a/drivers/hwtracing/coresight/coresight-trbe.c
+++ b/drivers/hwtracing/coresight/coresight-trbe.c
@@ -1225,6 +1225,17 @@ static void arm_trbe_enable_cpu(void *info)
enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE);
}
+static void arm_trbe_disable_cpu(void *info)
+{
+ struct trbe_drvdata *drvdata = info;
+ struct trbe_cpudata *cpudata = this_cpu_ptr(drvdata->cpudata);
+
+ disable_percpu_irq(drvdata->irq);
+ trbe_reset_local(cpudata);
+ cpudata->drvdata = NULL;
+}
+
+
static void arm_trbe_register_coresight_cpu(struct trbe_drvdata *drvdata, int cpu)
{
struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu);
@@ -1326,18 +1337,12 @@ static void arm_trbe_probe_cpu(void *info)
cpumask_clear_cpu(cpu, &drvdata->supported_cpus);
}
-static void arm_trbe_remove_coresight_cpu(void *info)
+static void arm_trbe_remove_coresight_cpu(struct trbe_drvdata *drvdata, int cpu)
{
- int cpu = smp_processor_id();
- struct trbe_drvdata *drvdata = info;
- struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu);
struct coresight_device *trbe_csdev = coresight_get_percpu_sink(cpu);
- disable_percpu_irq(drvdata->irq);
- trbe_reset_local(cpudata);
if (trbe_csdev) {
coresight_unregister(trbe_csdev);
- cpudata->drvdata = NULL;
coresight_set_percpu_sink(cpu, NULL);

I am a bit concerned about "resetting" the sink from a different CPU.
Could we instead, schedule a delayed work to unregister the trbe_csdev?

Yes, I will try to do that.
Sorry for my following questions.
As you mean, do we need to take the same care when setting the percpu sink
in the register trbe_csdev ?

Apologies, having taken another look, we set the percpu_sink for
a cpu outside smp_call_function(). So, I think your patch is fine.



Best regards,
Junhao.



}
}
@@ -1366,8 +1371,12 @@ static int arm_trbe_remove_coresight(struct trbe_drvdata *drvdata)
{
int cpu;
- for_each_cpu(cpu, &drvdata->supported_cpus)
- smp_call_function_single(cpu, arm_trbe_remove_coresight_cpu, drvdata, 1);
+ for_each_cpu(cpu, &drvdata->supported_cpus) {
+ if (cpumask_test_cpu(cpu, &drvdata->supported_cpus))
+ smp_call_function_single(cpu, arm_trbe_disable_cpu, drvdata, 1);
+ if (cpumask_test_cpu(cpu, &drvdata->supported_cpus))
+ arm_trbe_remove_coresight_cpu(drvdata, cpu);

Do we need to test the cpu here in both places ? We already check that
in the loop entry. The reason why we repeat the check during the probe,
is to skip any CPUs that may have a TRBE not accessible.

Suzuki


Ok, Will fix in next version.

Best regards,
Junhao.


+ }
free_percpu(drvdata->cpudata);
return 0;
}
@@ -1406,12 +1415,8 @@ static int arm_trbe_cpu_teardown(unsigned int cpu, struct hlist_node *node)
{
struct trbe_drvdata *drvdata = hlist_entry_safe(node, struct trbe_drvdata, hotplug_node);
- if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) {
- struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu);
-
- disable_percpu_irq(drvdata->irq);
- trbe_reset_local(cpudata);
- }
+ if (cpumask_test_cpu(cpu, &drvdata->supported_cpus))
+ arm_trbe_disable_cpu(drvdata);
return 0;
}


.




.