Re: New Feature Idea: Compress swap file

Richard B. Johnson (root@chaos.analogic.com)
Sun, 22 Feb 1998 01:37:54 -0500 (EST)


Remember y'all the kernel swaps PAGES. On Intel Machines, a PAGE is
8192 bytes. Suppose it got compressed to 14 bytes using RLE. What do
you do with it? --er where do you put it until you have accumulated
enough compressed pages to make a Disc write efficient? Then, suppose
you need to get back a compressed page that's been concatenated into
others to make an efficient write. There's a LOT of overhead extracting
that 14 bytes from an accumulated buffer. Now, you've got a hole..

I think someone should simulate the required page-swap alogrithms before
they actually try it. I think you will find that CPU time is not free,
copying buffers is not free, rehashing and filling holes is not free,
etc. Unless you are swapping to a floppy, I think you lose big time.

Digital spent a lot of time working on modified-page writers, and
other paging/swapping schemes while trying to optimize VMS. They had
small pages, 512 bytes. The fastest scheme remained, if you have
a dirtied page you have to free, write it, and write it now -- but
don't write a page unless you absolutely, positively have to. This
means stealing unwritten pages from others if necessary.

RLE is quite efficient for sparse data, blocks of 0s get compressed
to char(token), word(how-much), char(what kind). Therefore a long
will substitute for 64k of such sparse data. The problem remains
with what-to-do with it after you've compressed it!

Linux will never swap a page that has not been written so you don't
have the problem of "demand-zero" paging. Until a page is actually
dirtied, it may be "owned" by every process in the system!

And, no, an efficient compression scheme should never "know" about
the nature of the data. Data are just a string of... bits, bytes,
words, longwords.... depending upon the target machine. If you
analyze the data, you lose big time.

FYI, in the days of CP/M, 43k, and 3.3 MHz clocks, I made the first
compressed file utilities so one could download a whole "user-area", i.e.,
"directory" at 300 baud. Anybody remember FILENAME.LBR and FILENAME.LQR
(Library file, Compressed library file)?

This was quite a few years before ".ARC" and ".ZIP" on PCs.

Cheers,
Dick Johnson
***** FILE SYSTEM MODIFIED *****
Penguin : Linux version 2.1.87 on an i586 machine (66.15 BogoMips).
Warning : It's hard to remain at the trailing edge of technology.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu