Re: [PATCH 1/2] Resource usage threshold notification addition to res_counter (v3)

From: KAMEZAWA Hiroyuki
Date: Mon Jul 13 2009 - 20:49:31 EST

Next message: Paul Menage: "Re: [PATCH 8/9] [RFC] Example multi-bindable subsystem: a per-cgroup notes field"
Previous message: Hitoshi Mitake: "Re: [PATCH][RFC] Adding information of counts processes acquiredhow many spinlocks to schedstat"
In reply to: Paul Menage: "Re: [PATCH 1/2] Resource usage threshold notification addition to res_counter (v3)"
Next in thread: Paul Menage: "Re: [PATCH 0/2] Memory usage limit notification feature (v3)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Mon, 13 Jul 2009 17:36:40 -0700
Paul Menage <menage@xxxxxxxxxx> wrote:

> As I mentioned in another thread, I think that associating the
> threshold with the res_counter rather than with each individual waiter
> is a mistake, since it creates global state and makes it hard to have
> multiple waiters on the same cgroup.
>
Ah, Hmm...maybe yes.

But the problem is "hierarchy". (even if this usage notifier don't handle it.)

While we charge as following res_coutner+hierarchy

res_counter_A + PAGE_SIZE
res_counter_B + PAGE_SIZE
res_counter_C + PAGE_SIZE

Checking "where we exceeds" in smart way is not very easy. Balbir's soft limit does
similar check but it's not very smart, either I think.

If there are prural thesholds (notifer, softlimit, etc...), this is worth to be
tried. Hmm...if not, size of res_coutner excees 128bytes and we'll see terrible counter.
Any idea ?

Thanks,
-Kame

> Paul
>
> On Mon, Jul 13, 2009 at 5:16 PM, Vladislav
> Buzov<vbuzov@xxxxxxxxxxxxxxxxx> wrote:
> > This patch updates the Resource Counter to add a configurable resource usage
> > threshold notification mechanism.
> >
> > Signed-off-by: Vladislav Buzov <vbuzov@xxxxxxxxxxxxxxxxx>
> > Signed-off-by: Dan Malek <dan@xxxxxxxxxxxxxxxxx>
> > ---
> > ÂDocumentation/cgroups/resource_counter.txt | Â 21 ++++++++-
> > Âinclude/linux/res_counter.h Â Â Â Â Â Â Â Â| Â 69 ++++++++++++++++++++++++++++
> > Âkernel/res_counter.c Â Â Â Â Â Â Â Â Â Â Â | Â Â7 +++
> > Â3 files changed, 95 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/cgroups/resource_counter.txt b/Documentation/cgroups/resource_counter.txt
> > index 95b24d7..1369dff 100644
> > --- a/Documentation/cgroups/resource_counter.txt
> > +++ b/Documentation/cgroups/resource_counter.txt
> > @@ -39,7 +39,20 @@ to work with it.
> > Â Â Â ÂThe failcnt stands for "failures counter". This is the number of
> > Â Â Â Âresource allocation attempts that failed.
> >
> > - c. spinlock_t lock
> > + e. unsigned long long threshold
> > +
> > + Â Â Â The resource usage threshold to notify the resouce controller. This is
> > + Â Â Â the minimal difference between the resource limit and current usage
> > + Â Â Â to fire a notification.
> > +
> > + f. void (*threshold_notifier)(struct res_counter *counter)
> > +
> > + Â Â Â The threshold notification callback installed by the resource
> > + Â Â Â controller. Called when the usage reaches or exceeds the threshold.
> > + Â Â Â Should be fast and not sleep because called when interrupts are
> > + Â Â Â disabled.
> > +
> > + g. spinlock_t lock
> >
> > Â Â Â ÂProtects changes of the above values.
> >
> > @@ -140,6 +153,7 @@ counter fields. They are recommended to adhere to the following rules:
> > Â Â Â Âusage Â Â Â Â Â usage_in_<unit_of_measurement>
> > Â Â Â Âmax_usage Â Â Â max_usage_in_<unit_of_measurement>
> > Â Â Â Âlimit Â Â Â Â Â limit_in_<unit_of_measurement>
> > + Â Â Â threshold Â Â Â notify_threshold_in_<unit_of_measurement>
> > Â Â Â Âfailcnt Â Â Â Â failcnt
> > Â Â Â Âlock Â Â Â Â Â Âno file :)
> >
> > @@ -153,9 +167,12 @@ counter fields. They are recommended to adhere to the following rules:
> > Â Â Â Âusage Â Â Â Â Â prohibited
> > Â Â Â Âmax_usage Â Â Â reset to usage
> > Â Â Â Âlimit Â Â Â Â Â set the limit
> > + Â Â Â threshold Â Â Â set the threshold
> > Â Â Â Âfailcnt Â Â Â Â reset to zero
> >
> > -
> > + d. Notification is enabled by installing the threshold notifier callback. It
> > + Â Âis up to the resouce controller to communicate the notification to user
> > + Â Âspace tasks.
> >
> > Â5. Usage example
> >
> > diff --git a/include/linux/res_counter.h b/include/linux/res_counter.h
> > index 511f42f..5ec98d7 100644
> > --- a/include/linux/res_counter.h
> > +++ b/include/linux/res_counter.h
> > @@ -9,6 +9,11 @@
> > Â*
> > Â* Author: Pavel Emelianov <xemul@xxxxxxxxxx>
> > Â*
> > + * Resouce usage threshold notification update
> > + * Copyright 2009 CE Linux Forum and Embedded Alley Solutions, Inc.
> > + * Author: Dan Malek <dan@xxxxxxxxxxxxxxxxx>
> > + * Author: Vladislav Buzov <vbuzov@xxxxxxxxxxxxxxxxx>
> > + *
> > Â* See Documentation/cgroups/resource_counter.txt for more
> > Â* info about what this counter is.
> > Â*/
> > @@ -35,6 +40,19 @@ struct res_counter {
> > Â Â Â Â */
> > Â Â Â Âunsigned long long limit;
> > Â Â Â Â/*
> > + Â Â Â Â* the resource usage threshold to notify the resouce controller. This
> > + Â Â Â Â* is the minimal difference between the resource limit and current
> > + Â Â Â Â* usage to fire a notification.
> > + Â Â Â Â*/
> > + Â Â Â unsigned long long threshold;
> > + Â Â Â /*
> > + Â Â Â Â* the threshold notification callback installed by the resource
> > + Â Â Â Â* controller. Called when the usage reaches or exceeds the threshold.
> > + Â Â Â Â* Should be fast and not sleep because called when interrupts are
> > + Â Â Â Â* disabled.
> > + Â Â Â Â*/
> > + Â Â Â void (*threshold_notifier)(struct res_counter *counter);
> > + Â Â Â /*
> > Â Â Â Â * the number of unsuccessful attempts to consume the resource
> > Â Â Â Â */
> > Â Â Â Âunsigned long long failcnt;
> > @@ -87,6 +105,7 @@ enum {
> > Â Â Â ÂRES_MAX_USAGE,
> > Â Â Â ÂRES_LIMIT,
> > Â Â Â ÂRES_FAILCNT,
> > + Â Â Â RES_THRESHOLD,
> > Â};
> >
> > Â/*
> > @@ -132,6 +151,21 @@ static inline bool res_counter_limit_check_locked(struct res_counter *cnt)
> > Â Â Â Âreturn false;
> > Â}
> >
> > +static inline bool res_counter_threshold_check_locked(struct res_counter *cnt)
> > +{
> > + Â Â Â if (cnt->usage + cnt->threshold < cnt->limit)
> > + Â Â Â Â Â Â Â return true;
> > +
> > + Â Â Â return false;
> > +}
> > +
> > +static inline void res_counter_threshold_notify_locked(struct res_counter *cnt)
> > +{
> > + Â Â Â if (!res_counter_threshold_check_locked(cnt) &&
> > + Â Â Â Â Â cnt->threshold_notifier)
> > + Â Â Â Â Â Â Â cnt->threshold_notifier(cnt);
> > +}
> > +
> > Â/*
> > Â* Helper function to detect if the cgroup is within it's limit or
> > Â* not. It's currently called from cgroup_rss_prepare()
> > @@ -147,6 +181,21 @@ static inline bool res_counter_check_under_limit(struct res_counter *cnt)
> > Â Â Â Âreturn ret;
> > Â}
> >
> > +/*
> > + * Helper function to detect if the cgroup usage is under it's threshold or
> > + * not.
> > + */
> > +static inline bool res_counter_check_under_threshold(struct res_counter *cnt)
> > +{
> > + Â Â Â bool ret;
> > + Â Â Â unsigned long flags;
> > +
> > + Â Â Â spin_lock_irqsave(&cnt->lock, flags);
> > + Â Â Â ret = res_counter_threshold_check_locked(cnt);
> > + Â Â Â spin_unlock_irqrestore(&cnt->lock, flags);
> > + Â Â Â return ret;
> > +}
> > +
> > Âstatic inline void res_counter_reset_max(struct res_counter *cnt)
> > Â{
> > Â Â Â Âunsigned long flags;
> > @@ -174,6 +223,26 @@ static inline int res_counter_set_limit(struct res_counter *cnt,
> > Â Â Â Âspin_lock_irqsave(&cnt->lock, flags);
> > Â Â Â Âif (cnt->usage <= limit) {
> > Â Â Â Â Â Â Â Âcnt->limit = limit;
> > + Â Â Â Â Â Â Â if (limit <= cnt->threshold)
> > + Â Â Â Â Â Â Â Â Â Â Â cnt->threshold = 0;
> > + Â Â Â Â Â Â Â else
> > + Â Â Â Â Â Â Â Â Â Â Â res_counter_threshold_notify_locked(cnt);
> > + Â Â Â Â Â Â Â ret = 0;
> > + Â Â Â }
> > + Â Â Â spin_unlock_irqrestore(&cnt->lock, flags);
> > + Â Â Â return ret;
> > +}
> > +
> > +static inline int res_counter_set_threshold(struct res_counter *cnt,
> > + Â Â Â Â Â Â Â unsigned long long threshold)
> > +{
> > + Â Â Â unsigned long flags;
> > + Â Â Â int ret = -EINVAL;
> > +
> > + Â Â Â spin_lock_irqsave(&cnt->lock, flags);
> > + Â Â Â if (cnt->limit > threshold) {
> > + Â Â Â Â Â Â Â cnt->threshold = threshold;
> > + Â Â Â Â Â Â Â res_counter_threshold_notify_locked(cnt);
> > Â Â Â Â Â Â Â Âret = 0;
> > Â Â Â Â}
> > Â Â Â Âspin_unlock_irqrestore(&cnt->lock, flags);
> > diff --git a/kernel/res_counter.c b/kernel/res_counter.c
> > index e1338f0..9b36748 100644
> > --- a/kernel/res_counter.c
> > +++ b/kernel/res_counter.c
> > @@ -5,6 +5,10 @@
> > Â*
> > Â* Author: Pavel Emelianov <xemul@xxxxxxxxxx>
> > Â*
> > + * Resouce usage threshold notification update
> > + * Copyright 2009 CE Linux Forum and Embedded Alley Solutions, Inc.
> > + * Author: Dan Malek <dan@xxxxxxxxxxxxxxxxx>
> > + * Author: Vladislav Buzov <vbuzov@xxxxxxxxxxxxxxxxx>
> > Â*/
> >
> > Â#include <linux/types.h>
> > @@ -32,6 +36,7 @@ int res_counter_charge_locked(struct res_counter *counter, unsigned long val)
> > Â Â Â Âcounter->usage += val;
> > Â Â Â Âif (counter->usage > counter->max_usage)
> > Â Â Â Â Â Â Â Âcounter->max_usage = counter->usage;
> > + Â Â Â res_counter_threshold_notify_locked(counter);
> > Â Â Â Âreturn 0;
> > Â}
> >
> > @@ -101,6 +106,8 @@ res_counter_member(struct res_counter *counter, int member)
> > Â Â Â Â Â Â Â Âreturn &counter->limit;
> > Â Â Â Âcase RES_FAILCNT:
> > Â Â Â Â Â Â Â Âreturn &counter->failcnt;
> > + Â Â Â case RES_THRESHOLD:
> > + Â Â Â Â Â Â Â return &counter->threshold;
> > Â Â Â Â};
> >
> > Â Â Â ÂBUG();
> > --
> > 1.5.6.3
> >
> >
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Next message: Paul Menage: "Re: [PATCH 8/9] [RFC] Example multi-bindable subsystem: a per-cgroup notes field"
Previous message: Hitoshi Mitake: "Re: [PATCH][RFC] Adding information of counts processes acquiredhow many spinlocks to schedstat"
In reply to: Paul Menage: "Re: [PATCH 1/2] Resource usage threshold notification addition to res_counter (v3)"
Next in thread: Paul Menage: "Re: [PATCH 0/2] Memory usage limit notification feature (v3)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]