From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755558AbcEZVVI (ORCPT ); Thu, 26 May 2016 17:21:08 -0400 Received: from mail-wm0-f68.google.com ([74.125.82.68]:33162 "EHLO mail-wm0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755426AbcEZVUL (ORCPT ); Thu, 26 May 2016 17:20:11 -0400 From: Nicolai Stange To: Herbert Xu Cc: David Howells , Tadeusz Struk , Michal Marek , linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Nicolai Stange Subject: [PATCH 2/5] lib/digsig: digsig_verify_rsa(): return -EINVAL if modulo length is zero Date: Thu, 26 May 2016 23:19:52 +0200 Message-Id: <1464297595-24032-3-git-send-email-nicstange@gmail.com> X-Mailer: git-send-email 2.8.2 In-Reply-To: <1464297595-24032-1-git-send-email-nicstange@gmail.com> References: <1464297595-24032-1-git-send-email-nicstange@gmail.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Currently, if digsig_verify_rsa() detects that the modulo's length is zero, i.e. mlen == 0, it returns -ENOMEM which doesn't really fit here. Make digsig_verify_rsa() return -EINVAL upon mlen == 0. Signed-off-by: Nicolai Stange --- lib/digsig.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/lib/digsig.c b/lib/digsig.c index a121cbc..55b8b2f 100644 --- a/lib/digsig.c +++ b/lib/digsig.c @@ -114,13 +114,15 @@ static int digsig_verify_rsa(struct key *key, datap += remaining; } - err = -ENOMEM; - mblen = mpi_get_nbits(pkey[0]); mlen = DIV_ROUND_UP(mblen, 8); - if (mlen == 0) + if (mlen == 0) { + err = -EINVAL; goto err; + } + + err = -ENOMEM; out1 = kzalloc(mlen, GFP_KERNEL); if (!out1) -- 2.8.2