Re: Hrm, _that_ was easy. Fix for binfmt_elf coredump with funny mmap's , doing state D.

John Carter (
Wed, 2 Apr 1997 10:28:02 +0200 (SAT)

On Tue, 1 Apr 1997, Greg Alexander wrote:

> Here's my solution. I think it still dumps all that is needed in your
> coredump, but if you have any probs don't use this. :)
> - if (!(vma->vm_flags & (VM_READ|VM_WRITE|VM_EXEC)))
> + if (!(vma->vm_flags & (VM_READ|VM_EXEC)))

Yip. It works. Didn't quite patch clean to the version I had, but I
did the patch by hand. But let me see if I understand this...

If the proverbial hits the fan, the kernel says 'stopit!' and does a
core dump. The coredump routine wombles through a list of vm areas
associated with that process, and calls "maydump" to determine whether
or not to write it out to the 'core' file. (Gives rise to rusty
childhood memories of seeing little wires threaded through tiny iron
rings, we really do drag the past with us...)

What is the difference between VM_EXEC and VM_EXECUTABLE? My wild
guess that one is an executable area of memory and the other is mapped
to an executable file.

Now the maydump algorithm before the patch was....
static inline int maydump(struct vm_area_struct *vma)
if (!(vma->vm_flags & (VM_READ|VM_WRITE|VM_EXEC)))
return 0;
#if 1
if (vma->vm_flags & (VM_WRITE|VM_GROWSUP|VM_GROWSDOWN))
return 1;
return 0;
return 1;

The patch just removes the |VM_WRITE in the first "if".

Now let me see if I understand this... After the patch the algorithm

If the vma is (readable or executable) and (writable or grows up or
grows down), dump it.

If the vma is (readable or executable) and (not (writable or grows up
or grows down)) and (readable or executable or mapped to an executable
or shared), don't dump it.

Otherwise dump it.

Now From: David Schwartz <>

> Do you get this bug even if you disable core dumps? I think it's an
> ELF core-dump bug. If memory serves me, the ELF core-dump routine
> tries to dump information from the mmap'ed file, but it's already
> been unmapped because the process is dead.

Hmm. If David is right, then my guess is this patch is the wrong fix,
it may cease to dump info that may well be needed. The core dump
should be done before the unmap'ing. Or the vma struct needs to
somehow carry info to say that this vma was mmap'd, hence don't dump it.

John Carter EMail:
Telephone : 27-12-808-0374x194 Fax:- 27-12-808-0338

Founder of the Council for Unnatural Scientists.