Re: inotify limits - thousands (tens of thousands?) of watches

From: Tvrtko Ursulin
Date: Wed May 20 2009 - 06:12:36 EST


On Wednesday 20 May 2009 10:26:57 Marcin Krol wrote:
> Hello everyone,
>
> First, apols for using up bandwidth, but I honestly found no other place
> where I can ask about this (and get meaningful reply).
>
> I'm not a kernel programmer, but I want to develop a program that would
> watch modifications in *all* user directories on a busy server using
> inotify.
>
> This is for high-availability purposes - events would be collected and
> once every several minutes changed dirs would be rsync'ed to failover
> server or smth like that would be done.
>
> As inotify watches particular directory and not its subdirs, I would
> have to watch all directories.
>
> This means I would have to create thousands or even tens of thousands of
> inotify watches.
>
> So my questions are:
>
> 1. is it safe? that is, will it not lock the kernel up, or cause
> excessive memory consumption?
>
> 2. is it economic in terms of CPU time and RAM? I have no idea how to
> even measure such a thing happening in the kernel..

Hi,

I don't know the specific answers apart my intuitive feeling, which is that it
probably would cause excessive memory use and that it is not elegant at all.

What may be interesting to you is the notification "engine" rewrite Eric Paris
is doing and new fanotify interface which will, hopefully, come soon on top
of it. With fanotify you will be able to monitor all changes much more easily
and efficiently.

Have a look at patch descriptions in:
http://people.redhat.com/~eparis/fsnotify/

Tvrtko
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/