I noticed that a "buffers eating" (sorry, I don't know how are called,
they are more that just IO constrained process) programs/processes such
as:
- grep'ing in large files,
- backup software (bru, etc),
- find'ing in large fs,
- extracting stats from large apache logs,
- updatedb (locate db)
eat a lot of cache pages and slowdown all the other IO processes due to
the lack of buffers/cache. I saw, for example, that a database server took
up to 13 seconds to finish a transaction on a medium size database when
there was a find running (to delete core files). The server is a
SMP and normally takes few milliseconds when the system workload is low.
The machine had a _lot_ of free memory, the output of free is something
like:
total used free shared buffers cached
Mem: 257904 243960 13944 20760 5420 200276
-/+ buffers/cache: 28264 229640 ^^^^ ^^^^^^
Swap: 129004 3616 125388 ^^^^ ^^^^^^
The "special feature" of these processes is that they read huge amount of
blocks but they are not going to be modified nor read again in a near
future.
I think that limiting in some way the amount of cache for this kind of
processes can improve a lot the behaviour of the system because is not
worth to waste ram in those blocks. Indeed, backups can be done more
frequently without affecting too much to the system (which is not true
presently).
My questions are:
1. Is my diagnostic correct?
2. Is my idea correct? Is it worth to play with it?
3. If both are right, can one of the vm system experts send me some
pointers of where to start with?
I realised that shrink_mmap and vmstat.c related functions are very
importants but I don't catch the big picture (organisation of
functions and files of buffers and cache management) yet.
Thanks.
-- Ricardo Galli
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/