From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 479D3C433FE for ; Mon, 28 Nov 2022 13:45:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232045AbiK1NpG (ORCPT ); Mon, 28 Nov 2022 08:45:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231983AbiK1Nor (ORCPT ); Mon, 28 Nov 2022 08:44:47 -0500 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6D90BC13 for ; Mon, 28 Nov 2022 05:44:45 -0800 (PST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 7FA6B5C0136; Mon, 28 Nov 2022 08:44:43 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 28 Nov 2022 08:44:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm1; t=1669643083; x= 1669729483; bh=me2xJR9KwKVEE8ktVSM+nRFnWYDYu52Yxlt4XSskicY=; b=U TTF3+0y5QEh9eH+LuP/VNDq2iKlI7mS5r74wvlOCIjiPzgyU6/nT7420hiJ6t4ma EV6YNmoy4n4KAy/P3svfsw/q7OX0rTxX/toJpi4KCX4PZ/LiiAIpR7YN2toZLKC8 yOa56dF84TJqwt4+Q5L0ClwItRhUdRU3OxB4I/O3kPBhLqLV5DVReG82SEpOugkY jyvALZKIakZICYu1fIdYzXl2kASWWqIb6pVXnpX4sD5rTcaBdTw2vfeOlHb2GoPO 79bcI82YbOcYLsZNCMYd0mBSoEHH6A7wOeJS2VqLE36ugfOyGKQ18GZ82JK/yLWF 31MMyFby1Cl6HonQZKcnQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1669643083; x=1669729483; bh=me2xJR9KwKVEE8ktVSM+nRFnWYDY u52Yxlt4XSskicY=; b=xN2M/q4xDa1cSYUbAuKlhx+Uh4BZxc0Nulb5ZP7y49JN xFuBr9LtQybNzFW3ttFnH9nxvItGjdi8dwkcJFHeyDUpCyJxrosev52oL6faznJG fBsOfVgJUOUeP+OkQB3YTheC/z879+9Wr56wereGH/u8HlSk3qUYByTTOdJ+cP5a BhGTFdnE3IDIWsXBqZi1r+V2Pk4j9zvV3RamZrA3utQ6supI1oN1a4Q6DzeB6R8D iA6v8BorjyZMUidtu0I7/sznLffJ7pCJzMK4PPImUmZOA1Uomk13R/g4TldhC4JQ 1yvNwrU6uRJ6V1BRiYfsPrGAynpaNInKOiaFgX3y7w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrjedvgdehiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefgudel teefvefhfeehieetleeihfejhfeludevteetkeevtedtvdegueetfeejudenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 28 Nov 2022 08:44:42 -0500 (EST) Date: Mon, 28 Nov 2022 14:44:39 +0100 From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= To: Jason Andryuk Cc: linux-kernel@vger.kernel.org, Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Jan Beulich , "moderated list:XEN HYPERVISOR INTERFACE" Subject: Re: [PATCH v3] xen-pciback: Consider INTx disabled when MSI/MSI-X is enabled Message-ID: References: <20221118154931.1928298-1-marmarek@invisiblethingslab.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="aozQvSbj7iUV5V+i" Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --aozQvSbj7iUV5V+i Content-Type: text/plain; protected-headers=v1; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Date: Mon, 28 Nov 2022 14:44:39 +0100 From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= To: Jason Andryuk Cc: linux-kernel@vger.kernel.org, Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Jan Beulich , "moderated list:XEN HYPERVISOR INTERFACE" Subject: Re: [PATCH v3] xen-pciback: Consider INTx disabled when MSI/MSI-X is enabled On Mon, Nov 21, 2022 at 05:16:37PM +0100, Marek Marczykowski-G=C3=B3recki w= rote: > On Mon, Nov 21, 2022 at 10:41:34AM -0500, Jason Andryuk wrote: > > On Sat, Nov 19, 2022 at 11:33 AM Marek Marczykowski-G=C3=B3recki > > wrote: > > > > > > On Sat, Nov 19, 2022 at 09:36:54AM -0500, Jason Andryuk wrote: > > > > Hi, Marek, > > > > > > > > On Fri, Nov 18, 2022 at 4:46 PM Marek Marczykowski-G=C3=B3recki > > > > wrote: > > > > > > > > > > On Fri, Nov 18, 2022 at 03:46:47PM -0500, Jason Andryuk wrote: > > > > > > I was trying to test your xen-pciback v3 patch, and I am having > > > > > > assignment fail consistently now. It is actually failing to > > > > > > quarantine to domIO in the first place, which matches the failu= re from > > > > > > the other day (when I more carefully read through the logs). I= t now > > > > > > consistently fails to quarantine on every boot unlike the other= day > > > > > > where it happened once. > > > > > > > > > > Does this include the very first assignment too, or only after do= main > > > > > reboot? If the latter, maybe some cleanup missed clearing MASKALL? > > > > > > > > It's the quarantine during dom0 boot that fails. Later assignment > > > > during VM boot fails. I tried warm reboots and cold boots and it > > > > happened both times. > > > > > > > > I also modified my initrd to halt in there and checked the config > > > > space. MASKALL wasn't set at that time. I need to double check - > > > > MASKALL may have been unset after dom0 booted in that case. > > > > > > > > I'll test more to figure when and how MASKALL is getting set. > >=20 > > I'm testing with a laptop without a battery. It seems MASKALL remains > > set when rebooting or when left plugged in. > >=20 > > From unplugged, a cold boot doesn't have MASKALL set and the network vm= boots. > >=20 > > After that, rebooting the laptop leaves MASKALL set on the NIC when > > the laptop reboots. NIC assignment fails. > >=20 > > Shutdown and later boot while left plugged in keeps MASKALL set. NIC > > assignment fails. I have only tested this scenario for short periods > > of time, so I don't know if it would eventually clear after a longer > > time. >=20 > That's interesting, seems like firmware is not resetting the device > properly. Maybe related to enabled wake on lan? >=20 > Anyway, resetting the device at domain create/destroy is AFAIR normally > done by pciback (at the instruction by the toolstack). Should it maybe > be done when assigning to pciback initially too? Or maybe in this > specific case, device reset doesn't properly clear MASKALL, so pciback > should clear it explicitly (after ensuring the MSI-X enable is cleared > too)? Can you check if `echo 1 > /sys/bus/pci/devices/$SBDF/reset` clears MASKALL on this device? I'm tempted to add libxl__device_pci_reset() as part of libxl__device_pci_assignable_add(). --=20 Best Regards, Marek Marczykowski-G=C3=B3recki Invisible Things Lab --aozQvSbj7iUV5V+i Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmOEu0cACgkQ24/THMrX 1ywlKQgAjmX9f7oEMMKhPvTU9Cwer1pxYt/jaRfYJThGAZvH/4w7VkyNInSN2u6G 7obNR47Rju6OXU1j7JOO5uW6DDQ6oo4tyRzYUpJl3a0QVSRpTgSN32eisomrGSSB 1/TU7XFcKS+ZBXJaRqq6whhdiXwFRuPhuO96pd4JVcxRoL2ocGzS/3B01Vv4OF5N 0bamXgj9FFQvUcXHcQRWzwj/mjOShwe/dYeDjaAmCf4kKJBAX5TJg3mEu3e7yz3d MiKHnU0bLSOC38cW3/XMs16NH3cMYD48ickM7Eca5PmZAX7wQrAeCJiVBuH8IHPR ZlSWFUeLsMnImI4snXIT4aex+c/2Mw== =Yg9M -----END PGP SIGNATURE----- --aozQvSbj7iUV5V+i--