Re: [PATCH] hugetlbfs: Fix integer overflow check in hugetlbfs_file_mmap()

From: linke li
Date: Thu Jul 13 2023 - 03:56:43 EST


> So we're adding code to handle eventual future compiler bugs? That sounds
> wrong, but maybe I misunderstood the problem you are trying to solve?

Sorry for not making it clear. My focus is the presence of undefined
behavior in kernel code.
Compilers can generate any code for undefined behavior and compiler
developers will not
take this as compiler bugs. In my option, kernel should not have
undefined behavior.

I double check this patch, this patch can not solve this issue well. I
am considering a new
patch below. The new patch do overflow check before the addition operation.
```
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -155,10 +155,10 @@ static int hugetlbfs_file_mmap(struct file
*file, struct vm_area_struct *vma)
return -EINVAL;

vma_len = (loff_t)(vma->vm_end - vma->vm_start);
- len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
/* check for overflow */
- if (len < vma_len)
+ if (vma_len > LLONG_MAX - ((loff_t)vma->vm_pgoff << PAGE_SHIFT))
return -EINVAL;
+ len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
```