From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 303273B6370 for ; Mon, 27 Jul 2026 15:57:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.197 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785167863; cv=none; b=r37fiCwgMDZ2NpiV1Pc+WtKanXYIdG6+KS+PEfsIASe0m9hNMKIKO1Z2Be1VVhKZztfI3LIOtJZ5mQCP7GVIj6SbMkptY0eekWDhqcg3VqsEoOlpAx+qGRXsiNv0nGo7Y3o6dNaM7lxZe2uGW+QG9/hCIaVnBnT0ySnwnbGtRbY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785167863; c=relaxed/simple; bh=+STRKREutXOyFTtv+83VpeyUbfK8/iLir6ikDZIoTY8=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=j653jQI0rauIWdKhp0CBGn7d1kJBb0CoCuVm90++ioS9E5KaZexjC0XnLRR3xGMS3Byy3kHem3aPtZO4+o4eOqiUJRBDo0px4Fru+Mn1/Y3jjDcml5ysBr4lYtDiV61X8xhWJ84X/28b3zHR9dpzTfT3+kBDrgLGz5roGT19fL4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--loganodell.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Hd/kKBl7; arc=none smtp.client-ip=209.85.215.197 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--loganodell.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Hd/kKBl7" Received: by mail-pg1-f197.google.com with SMTP id 41be03b00d2f7-c96b4f58ddcso2171240a12.3 for ; Mon, 27 Jul 2026 08:57:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1785167860; x=1785772660; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:mime-version:date:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fYdmGeZqdcTZT2Ey5trUwgkVjKGLhBqyrb9UPgD7TCM=; b=Hd/kKBl74vbE3LH4V+areYiXOfo6cHuwbwT5JT0oavnrv9Kw4ng+lLEoqoQXMlTr2M lxD7c1m9PB/1tvqusPCo8Rh3AdYPgNryQqOVvDKm3okCBWiUWLs6Gre11BACytILH5EC qzu3TjxKbR1n4/txIg23Z+0sOPDwR/wD3cxAG7mdu4/oROzyhZ292FYJ7gexvKnuONxt itq/KmysTX/8die1dbcLMqxljo2gWagFytPgbuw/ynxRe63WWupy3F1djGeeTws+ncVg RqZnWaWjmcU+Gq2AXqDyKb9K8zRI/BBEm0cn7laY0vA0rCR2T64lBfuDEW/RTqrVenip Jilg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785167860; x=1785772660; h=content-type:cc:to:from:subject:message-id:mime-version:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=fYdmGeZqdcTZT2Ey5trUwgkVjKGLhBqyrb9UPgD7TCM=; b=qag5RFWeQkLE4GlaDWspZmT6tCu5jIUIf9Lic/pQ9+POuMhZwfBQcHKN8KUwtWUVO4 UkfA28Y/WGsAWFiOgkgVQZnDJpNJwVr/9us6+Nwjv4dhPXZ1eNRW2gRgkg4w4JQsJmvt V1MgOgIWtkaTz39bFv5rwFEf+qAlRwS1VYjoVgsbrCqZpJQ84HQByOhx8eCJJLvgvSpK hhJ//HOBBOfXIHgIS0CWYKuZdlR44kAy4wO3OuUfhof3uKOyvIQx08lLL3mGNC6bs2Zq 7GHJaM8SAAzqQjedN0TivBfZEifXXX2FVrCBHIjUZGTz7OIzRwpnQkTo/hdee8Mxmabu q82w== X-Forwarded-Encrypted: i=1; AHgh+Ro3OSEO2zUxRPRoT1wXi7tXno6EX+55AoS7Dxji8NeJ8mWEYHl9lgi3lO+dTmJ1PXVlvs2htHizDr3mSr0=@vger.kernel.org X-Gm-Message-State: AOJu0Yy0LWDVPAv/hmVQyF6DwZcoHKNFpLWz1bK90hqntvk52LTpRHWC POBPoJeBy9XNYdTBZCoUmRlqXbkBNB8sczf7vAUuqnAXimyrBDL0T12e/ZgcMxxJAtO47kR8R2X EhG8T85u5uvY/xaXOxdGa8g== X-Received: from dltt19.prod.google.com ([2002:a05:701a:c973:b0:13c:e20a:efb6]) (user=loganodell job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:3393:b0:3c3:b57b:628f with SMTP id adf61e73a8af0-3c67df285fcmr9621374637.48.1785167860167; Mon, 27 Jul 2026 08:57:40 -0700 (PDT) Date: Mon, 27 Jul 2026 08:57:20 -0700 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.55.0.229.g6434b31f56-goog Message-ID: <20260727155720.2692587-1-loganodell@google.com> Subject: [PATCH v3] iommu/iova: Move CPU magazine init to first insert From: Logan Odell To: robin.murphy@arm.com, joro@8bytes.org, will@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, mclapinski@google.com, skhawaja@google.com, Logan Odell Content-Type: text/plain; charset="UTF-8" A large amount of memory may be allocated for these magazines on machines with a lot of IOMMU groups and CPU cores. Not all may be used as some devices may be unused or be bound to drivers that do not use the DMA-API. Furthermore, some drivers may not use all levels or CPUs. Move the initialization of the loaded and prev magazines for each CPU on the first attempt to try to insert a freed IOVA to them. Signed-off-by: Logan Odell Signed-off-by: Michal Clapinski --- v2: only rebase v3: Combine the initilaization logic with new magazine logic in __iova_rcache_insert --- Change-Id: I987b6dfb2b1125d8da8618fff540f315b99bad13 --- drivers/iommu/iova.c | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 021daf6528de..82d52eb12e4c 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -621,6 +621,9 @@ iova_magazine_free_pfns(struct iova_magazine *mag, struct iova_domain *iovad) unsigned long flags; int i; + if (!mag) + return; + spin_lock_irqsave(&iovad->iova_rbtree_lock, flags); for (i = 0 ; i < mag->size; ++i) { @@ -739,12 +742,6 @@ int iova_domain_init_rcaches(struct iova_domain *iovad) cpu_rcache = per_cpu_ptr(rcache->cpu_rcaches, cpu); spin_lock_init(&cpu_rcache->lock); - cpu_rcache->loaded = iova_magazine_alloc(GFP_KERNEL); - cpu_rcache->prev = iova_magazine_alloc(GFP_KERNEL); - if (!cpu_rcache->loaded || !cpu_rcache->prev) { - ret = -ENOMEM; - goto out_err; - } } } @@ -777,19 +774,23 @@ static bool __iova_rcache_insert(struct iova_domain *iovad, cpu_rcache = raw_cpu_ptr(rcache->cpu_rcaches); spin_lock_irqsave(&cpu_rcache->lock, flags); - if (!iova_magazine_full(cpu_rcache->loaded)) { + if (cpu_rcache->loaded && !iova_magazine_full(cpu_rcache->loaded)) { can_insert = true; - } else if (!iova_magazine_full(cpu_rcache->prev)) { + } else if (cpu_rcache->prev && !iova_magazine_full(cpu_rcache->prev)) { swap(cpu_rcache->prev, cpu_rcache->loaded); can_insert = true; } else { struct iova_magazine *new_mag = iova_magazine_alloc(GFP_ATOMIC); if (new_mag) { - spin_lock(&rcache->lock); - iova_depot_push(rcache, cpu_rcache->loaded); - spin_unlock(&rcache->lock); - schedule_delayed_work(&rcache->work, IOVA_DEPOT_DELAY); + if (cpu_rcache->loaded && !cpu_rcache->prev) { + cpu_rcache->prev = cpu_rcache->loaded; + } else if (cpu_rcache->loaded) { + spin_lock(&rcache->lock); + iova_depot_push(rcache, cpu_rcache->loaded); + spin_unlock(&rcache->lock); + schedule_delayed_work(&rcache->work, IOVA_DEPOT_DELAY); + } cpu_rcache->loaded = new_mag; can_insert = true; @@ -831,9 +832,9 @@ static unsigned long __iova_rcache_get(struct iova_rcache *rcache, cpu_rcache = raw_cpu_ptr(rcache->cpu_rcaches); spin_lock_irqsave(&cpu_rcache->lock, flags); - if (!iova_magazine_empty(cpu_rcache->loaded)) { + if (cpu_rcache->loaded && !iova_magazine_empty(cpu_rcache->loaded)) { has_pfn = true; - } else if (!iova_magazine_empty(cpu_rcache->prev)) { + } else if (cpu_rcache->prev && !iova_magazine_empty(cpu_rcache->prev)) { swap(cpu_rcache->prev, cpu_rcache->loaded); has_pfn = true; } else { -- 2.55.0.229.g6434b31f56-goog