Re: [PATCH] serial8250: ratelimit "too much work" error

[Andrew Morton]

On Sat,  2 Oct 2010 12:04:38 +0100 (BST)
Daniel Drake <dsd@xxxxxxxxxx> wrote:

> Running a serial console, if too many kernel messages are generated within
> a short time causing a lot of serial I/O, the 8250 driver will generate
> another kernel message reporting this, which just adds to the I/O. It has
> a cascading effect and quickly results the system being brought to its knees
> by a flood of "too much work" messages.
> 
> Ratelimit the error message to avoid this.
> 
> Signed-off-by: Daniel Drake <dsd@xxxxxxxxxx>
> ---
>  drivers/serial/8250.c |    5 +++--
>  1 files changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/serial/8250.c b/drivers/serial/8250.c
> index 24110f6..d3c5855 100644
> --- a/drivers/serial/8250.c
> +++ b/drivers/serial/8250.c
> @@ -1606,8 +1606,9 @@ static irqreturn_t serial8250_interrupt(int irq, void *dev_id)
>  
>  		if (l == i->head && pass_counter++ > PASS_LIMIT) {
>  			/* If we hit this, we're dead. */
> -			printk(KERN_ERR "serial8250: too much work for "
> -				"irq%d\n", irq);
> +			if (printk_ratelimit())
> +				printk(KERN_ERR "serial8250: too much work for "
> +					"irq%d\n", irq);
>  			break;
>  		}
>  	} while (l != end);

printk_ratelimit() shares a common ratelimiting state between all
callers of printk_ratelimit().  This is pretty sucky because if one
printk_ratelimit() caller is going crazy then this can cause punishment
of other unrelated printk_ratelimit() callers who *aren't* going crazy.

So it's generally better to use printk_ratelimited(), which will
ratelimit this printkand no other printk, without affecting other
printk_ratelimit[ed]() users.

So, this:

--- a/drivers/serial/8250.c~serial8250-ratelimit-too-much-work-error-fix
+++ a/drivers/serial/8250.c
@@ -1606,9 +1606,8 @@ static irqreturn_t serial8250_interrupt(
 
 		if (l == i->head && pass_counter++ > PASS_LIMIT) {
 			/* If we hit this, we're dead. */
-			if (printk_ratelimit())
-				printk(KERN_ERR "serial8250: too much work for "
-					"irq%d\n", irq);
+			printk_ratelimited(KERN_ERR
+				"serial8250: too much work for irq%d\n", irq);
 			break;
 		}
 	} while (l != end);
_

which, interestingly, doesn't compile because someone stuck a
DEFINE_RATELIMIT_STATE() in include/linux/kernel.h and it ain't defined
anywhere.  Let me fix that up...

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/