re: Big mallocs, mmap sorrows and double buffering.

Bennie Venter (venb0113@sasol.com)
Wed, 12 Feb 1997 10:04:14 -0200 (GMT+2)


On Tue, 11 Feb 1997, John Carter wrote:

> On Mon, 10 Feb 1997, Mark Hahn wrote:
>
> > use mmap! it's _easier_ than malloc/read, and better yet, is also
> > _optimal_ when confronted with changing memory loads. and of course
> > it also happens to be significantly more efficient, since you get
> > buffer-cache blocks mapped directly into your address space:
> > your disk controller reads them into memory, and you get to see
> > them, no silly copying from kernel to user-space.
> >
> > let me know if you need some code on mmap. I use it extensively.
>
> Sounds like just what I need.
>
> Hmm. I remember using a facility like this on the old vaxen. The
> glitch was if I mapped an entire satellite image (+-400megabytes) the
> page tables used up so much physical memory that the thing started
> thrashing again. (There we just settled down to using two 32k buffers
> and doing NO_WAIT read/writes.)
>

I do not know how the linux kernel handles it, but did you ever consider
using memory mapped files - this way only the portions you work on is in
memory much like the demand-paged executables. I do assume you are
working on very big files here.

Tnx & Bi
Bennie Venter