2.4.10pre10aa1 (first spin to the vm rewrite included)

From: Andrea Arcangeli (andrea@suse.de)
Date: Mon Sep 17 2001 - 11:09:14 EST

Next message: Jean-Marc Saffroy: "[Q] Implementation of spin_lock on i386: why "rep;nop" ?"
Previous message: Eric W. Biederman: "Re: broken VM in 2.4.10-pre9"
Next in thread: Dipankar Sarma: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Maybe reply: Dipankar Sarma: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Reply: Olaf Zaplinski: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Some of the the main features of 2.4.10pre10aa1 are:

o vm rewrite (see below a description)
o do all the possible vma merging during mmap/mremap/mprotect and
        rewrite of the vma lookup engine using rb trees checkpointing for
        avoiding various browse of the tree during vma merging, mmap etc...
o blkdev in pagecache (you can also open a blkdev and mmap it,
        or you can open it with O_DIRECT and do rawio without using /dev/raw)
o rawio improvement with preallocation of the kiobuf in the f_iobuf
        provided by the O_DIRECT support (next step is to split the kiobuf
        for the contention case that doesn't happen during the DB workload
        and for the non I/O users like bttv)
o O_DIRECT support
o latest TUX 2 release developed by Ingo Molnar (RedHat Inc) (you need
        the tux 2.1.0 userspace package from redhat.com to use it)
o latest UML release developed by Jeff Dike
o rcu core included (thought it will be rewritten, I agreed with
        Dipankar that the per-cpu scheduling sequence number seems
        the best approch, such number can serve also as a statistic
        to usespace infact, it is very similar to Rusty's patch but it
        doesn't add any branch to the schedule fast path, he's rewriting
        the patch at the moment and I'll include it in the next release)

Only in 2.4.10pre8aa1: 00_gcc-30-extern-static-3
Only in 2.4.10pre8aa1: 00_nfs-delete-lock-1
Only in 2.4.10pre8aa1: 00_tgid-1

Merged into mainline.

Only in 2.4.10pre10aa1: 80_vm-aa-1

VM rewrite, should fix:

        - kswapd looping forever on DMA or NORMAL classzones
        - swap+ram will be almost all available address space
                (modulo when the swap cache serves to avoid swapins
                of shares anonymous memory after a fork)
        - slowdowns after multiple runs of the workloads
        - swapout storms
        - dbench runs twice faster with this patch applied

        I've several ideas on how to change the vm algorithms to
        be even better, this first one may be a little too agrressive, however
        I tried a few of them and I did a check of the dbench numbers after
        that and they were always slower than the algorithm in the 80_vm-aa-1
        patch. (OTOH I don't want a VM that works wonderfully on dbench
        but not with other normal workloads)

        This patch may still swap a little too much (but it shouldnt'
        generate the storms during heavy I/O), I need feedback from differnt
        workloads to be sure it's ok, at this point it is trival for me to
        change the behaviour radically after I get the feedback and numbers
        from users and developers.

So please test it so I will know if I must change something.

        About stability it should be rock solid, but of course there's
        not a very good statistic yet (my most critical desktop is running
        2.4.10pre10aa1 since last night under high load without problems yet).
        I probably have a bug in swapoff but let's ignore it for now, just
        try to run swapoff only before shutting down the machine. The fact
        is that the 2.4 VM is broken freeing physically dirty pages.
        The last owner of the page (usually the VM except in swapoff) has to
        clear the dirty flag before freeing the page, in swapoff it may
        be a little more complicate (we may need to grab the pagecache_lock
        to ensure nobody start using the page while we clear it). And swapoff
        is probably racy anyways as usual (swapoff in 2.2 is racy too). In
        short I didn't focused on swapoff yet, I just made an hack to make it
        to work while shutting down the machine so far.

        I know this patch could hurt some developer since I backed out
        lots of (broken) stuff that probably they liked, I'm sorry for that,
        but I will mainly listen to numbers, not to "voice/email" vapourware
        arguments (like the argument numa is broken, that's just a red herring,
        this patch flyes on the numa wildfire simulation with all the numa
        heuristics enabled as well as on uml with mem=4m, origin 2k or whatever
        x86/ia64 numa hardware will never ever have problems either with it and
        I will make sure it will work correctly). So in short I will love to
        be proved wrong on the _paper_ with real life issues.

Only in 2.4.10pre10aa1: 00_alpha-compile-1

Fix a few alpha compile troubles.

Only in 2.4.10pre8aa1: 00_alpha-fp-disabled-1
Only in 2.4.10pre10aa1: 00_alpha-fp-disabled-2

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 00_copy-user-lat-2
Only in 2.4.10pre10aa1: 00_copy-user-lat-3

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 00_module-gfp-2
Only in 2.4.10pre10aa1: 00_module-gfp-3

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 00_silent-stack-overflow-8
Only in 2.4.10pre10aa1: 00_silent-stack-overflow-9

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 10_prefetch-6
Only in 2.4.10pre10aa1: 10_prefetch-8

Part of it merged into mainline, rediffed the missing part.

Only in 2.4.10pre8aa1: 40_blkdev-pagecache-16
Only in 2.4.10pre10aa1: 40_blkdev-pagecache-17

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 50_uml-patch-2.4.9-4.bz2
Only in 2.4.10pre10aa1: 50_uml-patch-2.4.9-6.bz2

Picked last update with the move of the kernel address space from
sourceforge.

Only in 2.4.10pre8aa1: 60_tux-2.4.9-ac10-H2
Only in 2.4.10pre10aa1: 60_tux-2.4.9-ac10-H7

Picked last update from www.redhat.com/~mingo/ .

Only in 2.4.10pre8aa1: 70_mmap-rb-8
Only in 2.4.10pre10aa1: 70_mmap-rb-9

Rediffed due trivial rejects.

URL:

        ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.10pre10aa1.bz2
        ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.10pre10aa1/
        ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.10pre10aa1/80_vm-aa-1

Some diffstats:

diffstat of the whole patch:

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Next message: Jean-Marc Saffroy: "[Q] Implementation of spin_lock on i386: why "rep;nop" ?"
Previous message: Eric W. Biederman: "Re: broken VM in 2.4.10-pre9"
Next in thread: Dipankar Sarma: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Maybe reply: Dipankar Sarma: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Reply: Olaf Zaplinski: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

This archive was generated by hypermail 2b29 : Sun Sep 23 2001 - 21:00:20 EST