From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 390F2285C8B for ; Wed, 24 Jun 2026 14:08:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.41 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782310133; cv=none; b=F03zdfy6vFSIfQI+O/TlCW4Q08aFEPDyQPdbiBe9g4q9REks9ttzOZ5K7K/+elZ8wj9E6pRNz3AquOncMeK33Ub9AMDWqsUiXYPPYNlp6VLO2dH3q6CltQ2q8o7l6RUMUGfZh+Y5PfNSF3bmRyGIhYKPI0nCIxYdqiiILR+Qu4U= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782310133; c=relaxed/simple; bh=ls5x7/h6PkyMwjIK7m+WptLGXeW9x7K+JqAeCp80rAA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=V6J6Qs43mZQAu8Mc5XyyNAsNSuOu6ZyBiopU8T64dxn9Vr1i61dBaZ+5HzMLM4fi48IA1opvOy7BVF4BkfyRXPZ2C14nyssPUKMLOHEIdkwstsTtSnh1ZKEN3Fa+VOyAYbXqgyJawuoXlqXu8ze6V/RZwQBD+G+5viwxpFvOtMU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=openvz.org; spf=pass smtp.mailfrom=openvz.org; dkim=pass (2048-bit key) header.d=openvz.org header.i=@openvz.org header.b=hv7rhSbY; arc=none smtp.client-ip=209.85.221.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=openvz.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=openvz.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=openvz.org header.i=@openvz.org header.b="hv7rhSbY" Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-46019edc13dso676514f8f.1 for ; Wed, 24 Jun 2026 07:08:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1782310129; x=1782914929; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=gHAT42RXInTAMI9ksXMYSe7Z9GutkddjZOFYn2zaAnw=; b=hv7rhSbY0UgFtLgbuUnEWaWZhUZM5+UYfIfYEDSV4vwC5wDH0WPkJ6CEYK75N/J8tK +jEj92nqMKhoZpAtrG2rckgxqoVAdKbY8FaA6ZPaPfcUWjdpL4UBBeqXSLXaPCcBo8cn SLFm3X6wpaFYqjcLfSb4bigU4XaDrUcP9IL0FgUHVLhAxf65J1YdUErDJzKBxYWwxwNQ ptmye9NJx6zEA/QejZva5aTGjIzlo4iw4r6P0ojfdUPtuf9fADNLCbTLb3wI2itwo0hU zEWhpQISQTJ25f/4WF9weQR4adx4Ec6QIY+w1GDJ/gq2Onf30wVQa+f4arYNsIz4tA86 Rpkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782310129; x=1782914929; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=gHAT42RXInTAMI9ksXMYSe7Z9GutkddjZOFYn2zaAnw=; b=SPay71Rx1sJv6L4iMx9uaO8/9spb3DU2rcX5fttiFpZxK5Fd3pxQ3h57UxMSJnSuXl +3jlj3J+x1ZvgFvEoIQtmRXXZIdbHbH5GUjrrp0NaELNLso/ygbdsRgUWJpgo2Ocr8xR BrWKChAPjXTbvzaq7+3cVsQu3w+B+2w/l20/0R/m00+GqpoKag3NjiIQQ7VsYevcrTZo XVPUMKWviV39foh0aWH4HvGm2wLB0/eSfMdnESYISTndpRwMS4TQ6fmad3OFHY1An5RC C2uyV0YI7wd8X6/li/Q68j+v4N0GFYRHpdj2cEFT/htbwJypndpjqeaF+XJ7Tq6o3fct LE1A== X-Forwarded-Encrypted: i=1; AFNElJ+CZIX/BrhWXbOe4dQY0p4iEDPWXbvgDnlYK7LeUyfyKOZ771L4gdqkan/EJj7HFF7UwSLgLh2jpB9ke9w=@vger.kernel.org X-Gm-Message-State: AOJu0YyYjxUb/g5f6+v9arg3LB50xySVGedx1Z9ObBkB1AwVBh9o7c1k iH2yq4LPuwXSNPLycPAFZYdQyFpa4uezrH5LEuilnugdHraFo+AOm0BKiTn2zJOECh0= X-Gm-Gg: AfdE7clPqfvn06BFoSNFfzLwkb1h43ILWOre64zoxng3BUb+40aLf+utrnyo5WwyB+/ 5sZ9rI5Y6d8jm6Xms02/fXZ/iDcyWnPBu8qqWdhw+GPdVeGFecOxmFBImT9mD9rx4UDj98K8LLf QWEnDGptF5fIxfdv7DK+TMRqh15VFuSJdY01I8knrGNE9scIz2zwuLQRKb7FpF8WbuCRwqYgbNj sqGrbXkmUXvt76JrPuFmOfT1fXM/vw+LrQ653Lfnkzc4iF/U1GlTVfORls8CK2CiErrO96/nN34 OfCJgQ2LOLXgK1EQJHF3dMvx782fVrASFwhOrEdhfChhVcSlyo/oiQIIQK/7uOGW90stC9jwR0/ 6yG+Wtz7v1iAlb/mxrvZxyPxPdGUWBMdHHOfHH5Mlt9vBQB9flL7YrwFoZfCjODLb0bVzlhVHfM T2dF2i X-Received: by 2002:a05:600c:c8c:b0:492:3291:9011 with SMTP id 5b1f17b1804b1-4925b389fdemr121858035e9.30.1782310129541; Wed, 24 Jun 2026 07:08:49 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:306c:c2d:85d5:28d5]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4923fd1fa34sm519680275e9.5.2026.06.24.07.08.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2026 07:08:49 -0700 (PDT) From: "Denis V. Lunev" To: mst@redhat.com, david@kernel.org Cc: virtualization@lists.linux.dev, linux-kernel@vger.kernel.org, "Denis V. Lunev" Subject: [PATCH v2 0/4] virtio_balloon: quiesce balloon work on device shutdown Date: Wed, 24 Jun 2026 16:08:42 +0200 Message-ID: <20260624140846.2616797-1-den@openvz.org> X-Mailer: git-send-email 2.53.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Since commit 8bd2fa086a04 ("virtio: break and reset virtio devices on device_shutdown()") the virtio bus breaks and resets every virtio device during device_shutdown(), i.e. on reboot and kexec. virtio_balloon has no .shutdown of its own, so that generic path runs while the balloon's asynchronous work is still armed: the free page reporting worker, the inflate/deflate and stats workers, the OOM notifier and the free page shrinker. Once the device has been broken, virtqueue_add_inbuf() in virtballoon_free_page_report() returns -EIO and trips its WARN_ON_ONCE(). On a kernel booted with panic_on_warn that turns an ordinary reboot into a fatal panic in the middle of device_shutdown(), so the machine never reaches the new kernel. The inflate/deflate and OOM paths do not warn but are no better off: they call wait_event(vb->acked, ...) and would block forever on a queue that can no longer complete. This was hit in the field as an intermittent failure of a virtualization cluster upgrade: guest storage nodes were rebooted via kexec into the new kernel, and the ones whose free page reporting happened to run during device_shutdown() panicked (the guests run with panic_on_warn) and never came back, stalling the rolling upgrade. The crash dump showed the WARN at virtio_balloon.c:216 in a page_reporting kworker, with all the balloon virtqueues already broken. Validated by churning balloon inflate/deflate from the host while kexec-rebooting the guest in a loop under panic_on_warn: the unpatched kernel reproduces the WARN within a couple of cycles, while the patched kernel survives many consecutive kexec cycles cleanly (12/12 in the final run, 0 WARNs). checkpatch is clean across the series. Changes in v2: - Add a virtio_device_shutdown() core helper and call it from the balloon .shutdown handler instead of open-coding break + synchronize_cbs + reset (David Hildenbrand). - New patch: make tell_host() warn and bail instead of hanging if a buffer add ever fails (David Hildenbrand); kept as a separate patch (Michael S. Tsirkin). v1: https://lore.kernel.org/all/20260622133715.3707707-1-den@openvz.org Denis V. Lunev (4): virtio: add virtio_device_shutdown() helper virtio_balloon: factor out virtballoon_quiesce() virtio_balloon: quiesce balloon work before device shutdown virtio_balloon: warn on failed buffer add in tell_host() drivers/virtio/virtio.c | 41 ++++++++++++++++++++++----------- drivers/virtio/virtio_balloon.c | 40 ++++++++++++++++++++++++-------- include/linux/virtio.h | 1 + 3 files changed, 59 insertions(+), 23 deletions(-) -- 2.53.0