Re: Are there kernel testing suites out there? We need them.

Chuck Lever (cel@monkey.org)
Sun, 4 Jul 1999 14:07:46 -0400 (EDT)


On Sun, 4 Jul 1999, Dan Kegel wrote:
> Rik van Riel <riel@nl.linux.org> wrote:
> > Stresstesting a Linux kernel on a known-good piece of hardware
> > will only show up the most obvious bugs and not the typical
> > race conditions and marginal-hardware cases that bother us most.
>
> Let's leave NT out of this, shall we? Regression tests like the
> ones Miles are proposing are just sanity checks to make
> sure nothing's gone south lately in your implementation.

i think regression tests are a good idea. whatever happened to the
project to test kernel builds with random config options? that was a good
start.

i also regularly run Spec benchmark suites on Linux. as well as measuring
performance changes among releases, these suites exercise many parts of an
OS, including the file system, VM, standard programs like cp, mv, cpio,
and so on.

> The other main point about automated regression tests is that
> they take no work on the part of the kernel developers to run.

in most commercial software houses, regression testing and quality
assurance are usually carried out by different people than the developers.
that leads to more scientific results. besides, some people are better
than others at breaking things. someone with little familiarity with a
piece of software is more likely to stumble across bugs (isn't that a
corollary to Moore's law? :-)

> Earlier warnings about problems are what
> corporations love. Red Hat, for instance, might well feel
> inclined to set up a mechanized regression test to give
> it a little added certainty that it's not about to ship a lemon.

this could be a great value-add for commercial Linux distributors like RH
and Caldera.

- Chuck Lever

--
corporate:	<chuckl@netscape.com>
personal:	<chucklever@netscape.net> or <cel@monkey.org>

The Linux Scalability project: http://www.citi.umich.edu/projects/linux-scalability/

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/