[PATCH] hugetlbfs: Fix integer overflow check in hugetlbfs_file_mmap()

[Linke Li]

From: Linke Li <lilinke99@xxxxxxxxx>

	vma_len = (loff_t)(vma->vm_end - vma->vm_start);
	len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
	/* check for overflow */
	if (len < vma_len)
		return -EINVAL;

The existing code includes an integer overflow check, which indicates
that the variable len has the potential to overflow, leading to undefined
 behavior according to the C standard. However, both GCC and Clang
compilers may eliminate this overflow check based on the assumption
that there will be no undefined behavior. Although the Linux kernel
disables these optimizations by using the -fno-strict-overflow option,
there is still a risk if the compilers make mistakes in the future.

To address this potential issue, this patch introduces a new check
that effectively prevents integer overflow. This check ensures the
safe operation of the code even when the Linux kernel is compiled
without the -fno-strict-overflow option, providing an added layer
of protection against potential issues caused by compiler behavior.

Signed-off-by: Linke Li <lilinke99@xxxxxxxxx>
---
 fs/hugetlbfs/inode.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 7b17ccfa039d..1b4648a8e296 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -157,7 +157,7 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma)
 	vma_len = (loff_t)(vma->vm_end - vma->vm_start);
 	len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
 	/* check for overflow */
-	if (len < vma_len)
+	if (vma_len > LLONG_MAX - ((loff_t)vma->vm_pgoff << PAGE_SHIFT))
 		return -EINVAL;
 
 	inode_lock(inode);
-- 
2.25.1