[34/44] Limit sysctl_tcp_mem and sysctl_udp_mem initializers to prevent integer overflows.

From: Greg KH
Date: Tue Dec 07 2010 - 19:34:54 EST


2.6.27-stable review patch. If anyone has any objections, please let us know.

------------------

From: Robin Holt <holt@xxxxxxx>

[ Problem was fixed differently upstream. -DaveM ]

On a 16TB x86_64 machine, sysctl_tcp_mem[2], sysctl_udp_mem[2], and
sysctl_sctp_mem[2] can integer overflow. Set limit such that they are
maximized without overflowing.

Signed-off-by: Robin Holt <holt@xxxxxxx>
To: "David S. Miller" <davem@xxxxxxxxxxxxx>
Cc: Willy Tarreau <w@xxxxxx>
Cc: linux-kernel@xxxxxxxxxxxxxxx
Cc: netdev@xxxxxxxxxxxxxxx
Cc: linux-sctp@xxxxxxxxxxxxxxx
Cc: Alexey Kuznetsov <kuznet@xxxxxxxxxxxxx>
Cc: "Pekka Savola (ipv6)" <pekkas@xxxxxxxxxx>
Cc: James Morris <jmorris@xxxxxxxxx>
Cc: Hideaki YOSHIFUJI <yoshfuji@xxxxxxxxxxxxxx>
Cc: Patrick McHardy <kaber@xxxxxxxxx>
Cc: Vlad Yasevich <vladislav.yasevich@xxxxxx>
Cc: Sridhar Samudrala <sri@xxxxxxxxxx>
Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxx>
---
net/ipv4/tcp.c | 4 +++-
net/ipv4/udp.c | 4 +++-
net/sctp/protocol.c | 4 +++-
3 files changed, 9 insertions(+), 3 deletions(-)

--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2754,12 +2754,14 @@ void __init tcp_init(void)

/* Set the pressure threshold to be a fraction of global memory that
* is up to 1/2 at 256 MB, decreasing toward zero with the amount of
- * memory, with a floor of 128 pages.
+ * memory, with a floor of 128 pages, and a ceiling that prevents an
+ * integer overflow.
*/
nr_pages = totalram_pages - totalhigh_pages;
limit = min(nr_pages, 1UL<<(28-PAGE_SHIFT)) >> (20-PAGE_SHIFT);
limit = (limit * (nr_pages >> (20-PAGE_SHIFT))) >> (PAGE_SHIFT-11);
limit = max(limit, 128UL);
+ limit = min(limit, INT_MAX * 4UL / 3 / 2);
sysctl_tcp_mem[0] = limit / 4 * 3;
sysctl_tcp_mem[1] = limit;
sysctl_tcp_mem[2] = sysctl_tcp_mem[0] * 2;
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -1722,11 +1722,13 @@ void __init udp_init(void)

/* Set the pressure threshold up by the same strategy of TCP. It is a
* fraction of global memory that is up to 1/2 at 256 MB, decreasing
- * toward zero with the amount of memory, with a floor of 128 pages.
+ * toward zero with the amount of memory, with a floor of 128 pages,
+ * and a ceiling that prevents an integer overflow.
*/
limit = min(nr_all_pages, 1UL<<(28-PAGE_SHIFT)) >> (20-PAGE_SHIFT);
limit = (limit * (nr_all_pages >> (20-PAGE_SHIFT))) >> (PAGE_SHIFT-11);
limit = max(limit, 128UL);
+ limit = min(limit, INT_MAX * 4UL / 3 / 2);
sysctl_udp_mem[0] = limit / 4 * 3;
sysctl_udp_mem[1] = limit;
sysctl_udp_mem[2] = sysctl_udp_mem[0] * 2;
--- a/net/sctp/protocol.c
+++ b/net/sctp/protocol.c
@@ -1179,7 +1179,8 @@ SCTP_STATIC __init int sctp_init(void)

/* Set the pressure threshold to be a fraction of global memory that
* is up to 1/2 at 256 MB, decreasing toward zero with the amount of
- * memory, with a floor of 128 pages.
+ * memory, with a floor of 128 pages, and a ceiling that prevents an
+ * integer overflow.
* Note this initalizes the data in sctpv6_prot too
* Unabashedly stolen from tcp_init
*/
@@ -1187,6 +1188,7 @@ SCTP_STATIC __init int sctp_init(void)
limit = min(nr_pages, 1UL<<(28-PAGE_SHIFT)) >> (20-PAGE_SHIFT);
limit = (limit * (nr_pages >> (20-PAGE_SHIFT))) >> (PAGE_SHIFT-11);
limit = max(limit, 128UL);
+ limit = min(limit, INT_MAX * 4UL / 3 / 2);
sysctl_sctp_mem[0] = limit / 4 * 3;
sysctl_sctp_mem[1] = limit;
sysctl_sctp_mem[2] = sysctl_sctp_mem[0] * 2;


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/