[PATCH 1/1] ipc: sem: fix used_sems overflow in newary()

[PATCH 1/1] ipc: sem: fix used_sems overflow in newary()

[Ren Wei]

From: Ruide Cao <caoruide123@gmail.com>

newary() checks namespace-wide semaphore usage before creating a new
array, but the current accounting uses a plain signed addition.

If the accumulated semaphore count overflows, the limit check can fail
open and allow allocations past sc_semmns, breaking namespace semaphore
resource enforcement and potentially leading to resource exhaustion.

Fix this by using check_add_overflow() before comparing the new total
against sc_semmns, and reject overflow the same way as a true limit
exceed.

Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Cc: stable@kernel.org
Reported-by: Yuan Tan <yuantan098@gmail.com>
Reported-by: Yifan Wu <yifanwucs@gmail.com>
Reported-by: Juefei Pu <tomapufckgml@gmail.com>
Reported-by: Xin Liu <bird@lzu.edu.cn>
Signed-off-by: Ruide Cao <caoruide123@gmail.com>
Tested-by: Ren Wei <enjou1224z@gmail.com>
Signed-off-by: Ren Wei <n05ec@lzu.edu.cn>
---
 ipc/sem.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/ipc/sem.c b/ipc/sem.c
index 6cdf862b1f5c..00c45de33c46 100644
--- a/ipc/sem.c
+++ b/ipc/sem.c
@@ -535,11 +535,13 @@ static int newary(struct ipc_namespace *ns, struct ipc_params *params)
 	key_t key = params->key;
 	int nsems = params->u.nsems;
 	int semflg = params->flg;
+	int total_sems;
 	int i;
 
 	if (!nsems)
 		return -EINVAL;
-	if (ns->used_sems + nsems > ns->sc_semmns)
+	if (check_add_overflow(ns->used_sems, nsems, &total_sems) ||
+	    total_sems > ns->sc_semmns)
 		return -ENOSPC;
 
 	sma = sem_alloc(nsems);
-- 
2.34.1

Re: [PATCH 1/1] ipc: sem: fix used_sems overflow in newary()

[Andrew Morton]

> On Mon, 11 May 2026 18:42:53 +0800 Ren Wei <n05ec@lzu.edu.cn> wrote:

Slight delay, sorry!

> From: Ruide Cao <caoruide123@gmail.com>
> 
> newary() checks namespace-wide semaphore usage before creating a new
> array, but the current accounting uses a plain signed addition.
> 
> If the accumulated semaphore count overflows, the limit check can fail
> open and allow allocations past sc_semmns, breaking namespace semaphore
> resource enforcement and potentially leading to resource exhaustion.
> 
> Fix this by using check_add_overflow() before comparing the new total
> against sc_semmns, and reject overflow the same way as a true limit
> exceed.

LLM review tells me

: The fix looks correct. The existing writer lock covers the check and
: the later used_sems increment, so that increment needs no further
: change.

and

: Please qualify the changelog. Default SEMMNS and SEMMSL cannot trigger
: this overflow. It requires raised limits and substantial existing
: allocations: sem_alloc() separately bounds each array's size, so one
: enormous request isn't sufficient to demonstrate a successful bypass.
:
: With sufficient memory, semget() can then succeed instead of returning
: ENOSPC, exceed the namespace quota and wrap the semaphore usage reported
: by SEM_INFO negative. Additional allocations can increase memory
: pressure.

So I propose this changelog.  Please check:

: newary() checks the namespace-wide semaphore limit using a signed
: addition of used_sems and nsems. If that addition overflows, the
: comparison can fail to reject a request which exceeds sc_semmns.
: 
: The default semaphore limits cannot trigger this overflow. It requires
: raised limits and substantial existing allocations. sem_alloc()
: separately bounds each array's allocation size, so a single enormous
: request is insufficient to demonstrate a successful quota bypass.
: 
: With enough memory, semget() can succeed instead of returning ENOSPC,
: allowing allocations beyond the namespace semaphore quota and increasing
: memory pressure. The wrapped used_sems count can also cause SEM_INFO to
: report a negative semaphore usage count.
: 
: Use check_add_overflow() before comparing the new total against
: sc_semmns, rejecting overflow with ENOSPC. The existing writer lock
: covers both the check and the later accounting update.


> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
> Cc: stable@kernel.org
> Reported-by: Yuan Tan <yuantan098@gmail.com>
> Reported-by: Yifan Wu <yifanwucs@gmail.com>
> Reported-by: Juefei Pu <tomapufckgml@gmail.com>
> Reported-by: Xin Liu <bird@lzu.edu.cn>
> Signed-off-by: Ruide Cao <caoruide123@gmail.com>
> Tested-by: Ren Wei <enjou1224z@gmail.com>
> Signed-off-by: Ren Wei <n05ec@lzu.edu.cn>
> ---
>  ipc/sem.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/ipc/sem.c b/ipc/sem.c
> index 6cdf862b1f5c..00c45de33c46 100644
> --- a/ipc/sem.c
> +++ b/ipc/sem.c
> @@ -535,11 +535,13 @@ static int newary(struct ipc_namespace *ns, struct ipc_params *params)
>  	key_t key = params->key;
>  	int nsems = params->u.nsems;
>  	int semflg = params->flg;
> +	int total_sems;
>  	int i;
>  
>  	if (!nsems)
>  		return -EINVAL;
> -	if (ns->used_sems + nsems > ns->sc_semmns)
> +	if (check_add_overflow(ns->used_sems, nsems, &total_sems) ||
> +	    total_sems > ns->sc_semmns)
>  		return -ENOSPC;
>  
>  	sma = sem_alloc(nsems);
> -- 
> 2.34.1
>