[PATCH] vhost-scsi: Prevent OOM from invalid protection SGL count

[PATCH] vhost-scsi: Prevent OOM from invalid protection SGL count

[Jia Jia]

The protection SGL path passes the result of vhost_scsi_calc_sgls()
directly to sg_alloc_table_chained(). The helper returns a negative
errno when the iterator is invalid or the request exceeds the segment
limit. The negative errno is then treated as a very large unsigned count
and sends the request into the SGL allocation path with an invalid size.

Repeated malformed T10-PI submissions from a host-side application caused
memory usage to rise sharply. MemAvailable fell to about 200 MB, and PSI
full avg10 reached about 1.46. The OOM killer terminated several userspace
processes before the endpoint cleanup completed. The kernel log included:

    [17036.451028] Out of memory: Killed process 2345 (systemd)
    [17036.493325] Out of memory: Killed process 2349 (sd-pam)
    [17078.265127] Out of memory: Killed process 1793 (networkd-dispat)

Return the calculation error before setting up the protection SGL. This
keeps the protection path consistent with the data SGL path and prevents
the invalid count from entering the allocation path.

Fixes: bca939d5bcd0 ("vhost-scsi: Dynamically allocate scatterlists")
Cc: stable@vger.kernel.org
Signed-off-by: Jia Jia <physicalmtea@gmail.com>
---
 drivers/vhost/scsi.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 9a1253b9d..8486652fd 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -972,6 +972,9 @@ vhost_scsi_mapal(struct vhost_scsi *vs, struct vhost_scsi_cmd *cmd,
 	if (prot_bytes) {
 		sgl_count = vhost_scsi_calc_sgls(prot_iter, prot_bytes,
 						 VHOST_SCSI_PREALLOC_PROT_SGLS);
+		if (sgl_count < 0)
+			return sgl_count;
+
 		cmd->prot_table.sgl = cmd->prot_sgl;
 		ret = sg_alloc_table_chained(&cmd->prot_table, sgl_count,
 					     cmd->prot_table.sgl,
-- 
2.43.0

Re: [PATCH] vhost-scsi: Prevent OOM from invalid protection SGL count

[Mike Christie]

cc'ing Linfeng.

Linfeng also reported this bug and provided a patch that has your check 
and an extra check to avoid calling into the function in the first 
place. It's been stuck in some other list.

I think it might be best for Linfeng to re-submit their patch now so it 
can get merged. For that patch, I think there was one outstanding 
question left where Michael had asked if the issue was found with AI so 
the proper tag could be added. I don't think that question was responded to.


On 7/17/26 9:22 AM, Jia Jia wrote:
> The protection SGL path passes the result of vhost_scsi_calc_sgls()
> directly to sg_alloc_table_chained(). The helper returns a negative
> errno when the iterator is invalid or the request exceeds the segment
> limit. The negative errno is then treated as a very large unsigned count
> and sends the request into the SGL allocation path with an invalid size.
> 
> Repeated malformed T10-PI submissions from a host-side application caused
> memory usage to rise sharply. MemAvailable fell to about 200 MB, and PSI
> full avg10 reached about 1.46. The OOM killer terminated several userspace
> processes before the endpoint cleanup completed. The kernel log included:
> 
>      [17036.451028] Out of memory: Killed process 2345 (systemd)
>      [17036.493325] Out of memory: Killed process 2349 (sd-pam)
>      [17078.265127] Out of memory: Killed process 1793 (networkd-dispat)
> 
> Return the calculation error before setting up the protection SGL. This
> keeps the protection path consistent with the data SGL path and prevents
> the invalid count from entering the allocation path.
> 
> Fixes: bca939d5bcd0 ("vhost-scsi: Dynamically allocate scatterlists")
> Cc: stable@vger.kernel.org
> Signed-off-by: Jia Jia <physicalmtea@gmail.com>
> ---
>   drivers/vhost/scsi.c | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
> index 9a1253b9d..8486652fd 100644
> --- a/drivers/vhost/scsi.c
> +++ b/drivers/vhost/scsi.c
> @@ -972,6 +972,9 @@ vhost_scsi_mapal(struct vhost_scsi *vs, struct vhost_scsi_cmd *cmd,
>   	if (prot_bytes) {
>   		sgl_count = vhost_scsi_calc_sgls(prot_iter, prot_bytes,
>   						 VHOST_SCSI_PREALLOC_PROT_SGLS);
> +		if (sgl_count < 0)
> +			return sgl_count;
> +
>   		cmd->prot_table.sgl = cmd->prot_sgl;
>   		ret = sg_alloc_table_chained(&cmd->prot_table, sgl_count,
>   					     cmd->prot_table.sgl,
> -- 
> 2.43.0
> 

Re: [PATCH] vhost-scsi: Prevent OOM from invalid protection SGL count

[Michael S. Tsirkin]

On Fri, Jul 17, 2026 at 10:13:36AM -0500, Mike Christie wrote:
> cc'ing Linfeng.
> 
> Linfeng also reported this bug and provided a patch that has your check and
> an extra check to avoid calling into the function in the first place. It's
> been stuck in some other list.
> 
> I think it might be best for Linfeng to re-submit their patch now so it can
> get merged. For that patch, I think there was one outstanding question left
> where Michael had asked if the issue was found with AI so the proper tag
> could be added. I don't think that question was responded to.

Right, was waiting for that.

> 
> On 7/17/26 9:22 AM, Jia Jia wrote:
> > The protection SGL path passes the result of vhost_scsi_calc_sgls()
> > directly to sg_alloc_table_chained(). The helper returns a negative
> > errno when the iterator is invalid or the request exceeds the segment
> > limit. The negative errno is then treated as a very large unsigned count
> > and sends the request into the SGL allocation path with an invalid size.
> > 
> > Repeated malformed T10-PI submissions from a host-side application caused
> > memory usage to rise sharply. MemAvailable fell to about 200 MB, and PSI
> > full avg10 reached about 1.46. The OOM killer terminated several userspace
> > processes before the endpoint cleanup completed. The kernel log included:
> > 
> >      [17036.451028] Out of memory: Killed process 2345 (systemd)
> >      [17036.493325] Out of memory: Killed process 2349 (sd-pam)
> >      [17078.265127] Out of memory: Killed process 1793 (networkd-dispat)
> > 
> > Return the calculation error before setting up the protection SGL. This
> > keeps the protection path consistent with the data SGL path and prevents
> > the invalid count from entering the allocation path.
> > 
> > Fixes: bca939d5bcd0 ("vhost-scsi: Dynamically allocate scatterlists")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Jia Jia <physicalmtea@gmail.com>
> > ---
> >   drivers/vhost/scsi.c | 3 +++
> >   1 file changed, 3 insertions(+)
> > 
> > diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
> > index 9a1253b9d..8486652fd 100644
> > --- a/drivers/vhost/scsi.c
> > +++ b/drivers/vhost/scsi.c
> > @@ -972,6 +972,9 @@ vhost_scsi_mapal(struct vhost_scsi *vs, struct vhost_scsi_cmd *cmd,
> >   	if (prot_bytes) {
> >   		sgl_count = vhost_scsi_calc_sgls(prot_iter, prot_bytes,
> >   						 VHOST_SCSI_PREALLOC_PROT_SGLS);
> > +		if (sgl_count < 0)
> > +			return sgl_count;
> > +
> >   		cmd->prot_table.sgl = cmd->prot_sgl;
> >   		ret = sg_alloc_table_chained(&cmd->prot_table, sgl_count,
> >   					     cmd->prot_table.sgl,
> > -- 
> > 2.43.0
> > 

Re: [PATCH] vhost-scsi: Prevent OOM from invalid protection SGL count

[Jia Jia]

Before sending my patch, I did a duplicate check against the upstream
linux-next tree, upstream history, publicly searchable mailing-list archives,
and Patchwork. I searched using the vhost-scsi and scatterlist function names,
T10-PI, invalid protection SGL counts, and the OOM/resource-exhaustion
symptoms. I did not find Linfeng's earlier patch.

Could you please point me to Linfeng's patch, or confirm what the additional
check covers? In particular, does it also validate:

    prot_bytes <= exp_data_len

before subtracting prot_bytes from exp_data_len and entering the SGL
mapping path? I had identified this as a related PI length-mismatch path, but
did not include it in the patch I sent.

With a one-byte combined payload and
pi_bytesout=2, KGDB stopped at the following call stack:

    #0  sg_alloc_table_chained(
            table=0xffff88810bbc5108,
            nents=0,
            first_chunk=0xffff888110050000,
            nents_first_chunk=2048)
        at lib/sg_pool.c:117

    #1  vhost_scsi_handle_vq+2295
    #2  vhost worker task
    #3  srso_alias_return_thunk

The stop was at the `BUG_ON(!nents)` instruction in
`sg_alloc_table_chained()`, and the kernel log recorded `kernel BUG at
lib/sg_pool.c:117!`.  This confirms that the PI length check is needed before
the iterator adjustment and SGL mapping path.

If Linfeng's patch covers both the invalid protection SGL count and this PI
length check, I will not submit duplicate work. 

Regarding the AI question: AI assistance was used during the source analysis
and test development. I also checked the relevant code path and the
host-side behavior.