Re: [Xen-devel] [Patch V3 14/15] xen: allow more than 512 GB of RAM for 64 bit pv-domains

From: Juergen Gross
Date: Mon Jun 08 2015 - 01:09:45 EST


On 05/27/2015 07:05 PM, David Vrabel wrote:
On 27/05/15 17:25, David Vrabel wrote:
On 20/04/15 06:23, Juergen Gross wrote:
64 bit pv-domains under Xen are limited to 512 GB of RAM today. The
main reason has been the 3 level p2m tree, which was replaced by the
virtual mapped linear p2m list. Parallel to the p2m list which is
being used by the kernel itself there is a 3 level mfn tree for usage
by the Xen tools and eventually for crash dump analysis. For this tree
the linear p2m list can serve as a replacement, too. As the kernel
can't know whether the tools are capable of dealing with the p2m list
instead of the mfn tree, the limit of 512 GB can't be dropped in all
cases.

This patch replaces the hard limit by a kernel parameter which tells
the kernel to obey the 512 GB limit or not. The default is selected by
a configuration parameter which specifies whether the 512 GB limit
should be active per default for domUs (domain save/restore/migration
and crash dump analysis are affected).

Memory above the domain limit is returned to the hypervisor instead of
being identity mapped, which was wrong anyway.

The kernel configuration parameter to specify the maximum size of a
domain can be deleted, as it is not relevant any more.

Something in this patch breaks the hvc console in my test domU.

kernel BUG at /local/davidvr/work/k.org/tip/drivers/tty/hvc/hvc_xen.c:153

Which suggests the hvc driver mapped the wrong console ring frame.

Sorry, it's patch #13 (xen: move p2m list if conflicting with e820 map)
that seems to be bad.

I think I've found the reason: the console frame isn't being marked as
"reserved" any more. With moving the p2m list I had to change the call
of memblock_reserve() for it. Before that patch this call covered the
p2m list, the start_info page, the xenstore page and the console page.
I added a memblock_reserve() for start_info, but failed to do so for
xenstore and console.

I'll modify the patch and respin.

I have to check why I didn't hit this issue. Maybe my test machine was
too large and the memory in question didn't get reused until my test
was finished.


Thanks for testing,

Juergen

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/