[PATCH] hugetlb: check pte with page table lock before handing to userfault

[Mike Kravetz]

In hugetlb_no_page we decide a page is missing if not present in the
page cache.  This is perfectly valid for shared/file mappings where
pages must exist in the page cache.  For anon/private mappings, the page
table must be checked.  This is done early in hugetlb_fault processing
and is the reason we enter hugetlb_no_page.  However, the early check is
made without holding the page table lock.  There could be racing updates
to the pte entry, so check again with page table lock held before
deciding to call handle_userfault.

Signed-off-by: Mike Kravetz <mike.kravetz@xxxxxxxxxx>
---
 mm/hugetlb.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 60e077ce6ca7..4cb44a4629b8 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -5560,10 +5560,29 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm,
 		if (idx >= size)
 			goto out;
 		/* Check for page in userfault range */
-		if (userfaultfd_missing(vma))
+		if (userfaultfd_missing(vma)) {
+			/*
+			 * For missing pages, the page cache (checked above) is
+			 * the 'source of truth' for shared mappings.  For anon
+			 * mappings, the source of truth is the page table.  We
+			 * already checked huge_pte_none_mostly() in
+			 * hugetlb_fault.  However, there could be racing
+			 * updates.  Check again while holding page table lock
+			 * before handing off to userfault.
+			 */
+			if (!(vma->vm_flags & VM_MAYSHARE)) {
+				ptl = huge_pte_lock(h, mm, ptep);
+				if (!huge_pte_none_mostly(huge_ptep_get(ptep))) {
+					spin_unlock(ptl);
+					ret = 0;
+					goto out;
+				}
+				spin_unlock(ptl);
+			}
 			return hugetlb_handle_userfault(vma, mapping, idx,
 						       flags, haddr, address,
 						       VM_UFFD_MISSING);
+		}
 
 		page = alloc_huge_page(vma, haddr, 0);
 		if (IS_ERR(page)) {
-- 
2.37.3