2.6.11 bug: unbacked private shared memory segments missing in core dump

[Jochen Suckfuell]

Hello!

Since 2.6.10, unbacked private shared memory allocated via shmget is not
included in core dumps.

This is a simple example code demonstrating the bug:

#include <sys/shm.h>

int main(int argc, char ** argv)
{
	int size = 1000;
	int id = shmget(IPC_PRIVATE, size, (IPC_CREAT | 0660));
	if(id < 0) return(1);
	int *buffer = (int *)shmat(id, 0, 0);
	int i;
	for(i = 0; i < 1000; i++)
		buffer[i] = i;

	// now dump core
	*((unsigned long *)1) = 0;

	// The private shared memory is not included in the core dump,
	// although it's not backed and cannot be accessed any more in any
	// way.
	return 0;
}

This bug was introduced in 2.6.10 by a patch to binfmt_elf.c that
resulted in:

static int maydump(struct vm_area_struct *vma)
{
	/* Do not dump I/O mapped devices, shared memory, or special mappings */
	if (vma->vm_flags & (VM_IO | VM_SHARED | VM_RESERVED))
		return 0;
...

(See the thread at
http://www.ussg.iu.edu/hypermail/linux/kernel/0410.2/1890.html)

Excluding all pages with VM_SHARED set is also excluding the unbacked
private mapping and should be replaced by a more specific criterion.

Bye
Jochen

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/