* [PATCH v5 1/2] PCI: endpoint: improve fixed_size bar handling when allocating space
2025-04-24 8:34 [PATCH v5 0/2] PCI: endpoint: space allocation fixups Jerome Brunet
@ 2025-04-24 8:34 ` Jerome Brunet
2025-04-24 8:34 ` [PATCH v5 2/2] PCI: endpoint: pci-epf-vntb: simplify ctrl/spad space allocation Jerome Brunet
2025-04-27 17:17 ` [PATCH v5 0/2] PCI: endpoint: space allocation fixups Manivannan Sadhasivam
2 siblings, 0 replies; 4+ messages in thread
From: Jerome Brunet @ 2025-04-24 8:34 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Lorenzo Pieralisi,
Jon Mason, Dave Jiang, Allen Hubbe, Christoph Hellwig,
Sagi Grimberg, Chaitanya Kulkarni
Cc: Marek Vasut, Yoshihiro Shimoda, Yuya Hamamachi, linux-pci,
linux-kernel, ntb, linux-nvme, Jerome Brunet, Niklas Cassel
When trying to allocate space for an endpoint function on a BAR with a
fixed size, the size saved in the 'struct pci_epf_bar' should be the fixed
size. This is expected by pci_epc_set_bar().
However, if the fixed_size is smaller that the alignment, the size saved
in the 'struct pci_epf_bar' matches the alignment and it is a problem for
pci_epc_set_bar().
To solve this, continue to allocate space that match the iATU alignment
requirement, saving it as .aligned_size, then save the size that matches
what is present in the BAR.
Fixes: 2a9a801620ef ("PCI: endpoint: Add support to specify alignment for buffers allocated to BARs")
Reviewed-by: Niklas Cassel <cassel@kernel.org>
Signed-off-by: Jerome Brunet <jbrunet@baylibre.com>
---
drivers/pci/endpoint/pci-epf-core.c | 21 ++++++++++++++-------
include/linux/pci-epf.h | 3 +++
2 files changed, 17 insertions(+), 7 deletions(-)
diff --git a/drivers/pci/endpoint/pci-epf-core.c b/drivers/pci/endpoint/pci-epf-core.c
index 394395c7f8decfa2010469655a4bd58a002993fd..982db6c1fbe77653f6a74a31df5c4e997507d2d8 100644
--- a/drivers/pci/endpoint/pci-epf-core.c
+++ b/drivers/pci/endpoint/pci-epf-core.c
@@ -236,12 +236,13 @@ void pci_epf_free_space(struct pci_epf *epf, void *addr, enum pci_barno bar,
}
dev = epc->dev.parent;
- dma_free_coherent(dev, epf_bar[bar].size, addr,
+ dma_free_coherent(dev, epf_bar[bar].aligned_size, addr,
epf_bar[bar].phys_addr);
epf_bar[bar].phys_addr = 0;
epf_bar[bar].addr = NULL;
epf_bar[bar].size = 0;
+ epf_bar[bar].aligned_size = 0;
epf_bar[bar].barno = 0;
epf_bar[bar].flags = 0;
}
@@ -264,7 +265,7 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar,
enum pci_epc_interface_type type)
{
u64 bar_fixed_size = epc_features->bar[bar].fixed_size;
- size_t align = epc_features->align;
+ size_t aligned_size, align = epc_features->align;
struct pci_epf_bar *epf_bar;
dma_addr_t phys_addr;
struct pci_epc *epc;
@@ -285,12 +286,17 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar,
return NULL;
}
size = bar_fixed_size;
+ } else {
+ /* BAR size must be power of two */
+ size = roundup_pow_of_two(size);
}
- if (align)
- size = ALIGN(size, align);
- else
- size = roundup_pow_of_two(size);
+ /*
+ * Allocate enough memory to accommodate the iATU alignment requirement.
+ * In most cases, this will be the same as .size but it might be different
+ * if, for example, the fixed size of a BAR is smaller than align.
+ */
+ aligned_size = align ? ALIGN(size, align) : size;
if (type == PRIMARY_INTERFACE) {
epc = epf->epc;
@@ -301,7 +307,7 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar,
}
dev = epc->dev.parent;
- space = dma_alloc_coherent(dev, size, &phys_addr, GFP_KERNEL);
+ space = dma_alloc_coherent(dev, aligned_size, &phys_addr, GFP_KERNEL);
if (!space) {
dev_err(dev, "failed to allocate mem space\n");
return NULL;
@@ -310,6 +316,7 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar,
epf_bar[bar].phys_addr = phys_addr;
epf_bar[bar].addr = space;
epf_bar[bar].size = size;
+ epf_bar[bar].aligned_size = aligned_size;
epf_bar[bar].barno = bar;
if (upper_32_bits(size) || epc_features->bar[bar].only_64bit)
epf_bar[bar].flags |= PCI_BASE_ADDRESS_MEM_TYPE_64;
diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
index 879d19cebd4fc6d8df9d724e3a52fa7fbd61e535..749cee0bcf2cc0600ad3be1582b0c497c5a21bed 100644
--- a/include/linux/pci-epf.h
+++ b/include/linux/pci-epf.h
@@ -114,6 +114,8 @@ struct pci_epf_driver {
* @phys_addr: physical address that should be mapped to the BAR
* @addr: virtual address corresponding to the @phys_addr
* @size: the size of the address space present in BAR
+ * @aligned_size: the size actually allocated to accommodate the iATU alignment
+ * requirement
* @barno: BAR number
* @flags: flags that are set for the BAR
*/
@@ -121,6 +123,7 @@ struct pci_epf_bar {
dma_addr_t phys_addr;
void *addr;
size_t size;
+ size_t aligned_size;
enum pci_barno barno;
int flags;
};
--
2.47.2
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v5 2/2] PCI: endpoint: pci-epf-vntb: simplify ctrl/spad space allocation
2025-04-24 8:34 [PATCH v5 0/2] PCI: endpoint: space allocation fixups Jerome Brunet
2025-04-24 8:34 ` [PATCH v5 1/2] PCI: endpoint: improve fixed_size bar handling when allocating space Jerome Brunet
@ 2025-04-24 8:34 ` Jerome Brunet
2025-04-27 17:17 ` [PATCH v5 0/2] PCI: endpoint: space allocation fixups Manivannan Sadhasivam
2 siblings, 0 replies; 4+ messages in thread
From: Jerome Brunet @ 2025-04-24 8:34 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Lorenzo Pieralisi,
Jon Mason, Dave Jiang, Allen Hubbe, Christoph Hellwig,
Sagi Grimberg, Chaitanya Kulkarni
Cc: Marek Vasut, Yoshihiro Shimoda, Yuya Hamamachi, linux-pci,
linux-kernel, ntb, linux-nvme, Jerome Brunet
When allocating the shared ctrl/spad space, epf_ntb_config_spad_bar_alloc()
should not try to handle the size quirks for underlying BAR, whether it is
fixed size or alignment. This is already handled by pci_epf_alloc_space().
Also, when handling the alignment, this allocate more space than necessary.
For example, with a spad size of 1024B and a ctrl size of 308B, the space
necessary is 1332B. If the alignment is 1MB,
epf_ntb_config_spad_bar_alloc() tries to allocate 2MB where 1MB would have
been more than enough.
Drop the handling of the BAR size quirks and let
pci_epf_alloc_space() handle that. Just make sure the 32bits SPAD register
are aligned on 32bits.
Signed-off-by: Jerome Brunet <jbrunet@baylibre.com>
---
drivers/pci/endpoint/functions/pci-epf-vntb.c | 26 +++-----------------------
1 file changed, 3 insertions(+), 23 deletions(-)
diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
index 874cb097b093ae645bbc4bf3c9d28ca812d7689d..e4da3fdb000723e3adad01f0ddf230ecc0e572a7 100644
--- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
@@ -408,11 +408,9 @@ static void epf_ntb_config_spad_bar_free(struct epf_ntb *ntb)
*/
static int epf_ntb_config_spad_bar_alloc(struct epf_ntb *ntb)
{
- size_t align;
enum pci_barno barno;
struct epf_ntb_ctrl *ctrl;
u32 spad_size, ctrl_size;
- u64 size;
struct pci_epf *epf = ntb->epf;
struct device *dev = &epf->dev;
u32 spad_count;
@@ -422,31 +420,13 @@ static int epf_ntb_config_spad_bar_alloc(struct epf_ntb *ntb)
epf->func_no,
epf->vfunc_no);
barno = ntb->epf_ntb_bar[BAR_CONFIG];
- size = epc_features->bar[barno].fixed_size;
- align = epc_features->align;
-
- if ((!IS_ALIGNED(size, align)))
- return -EINVAL;
-
spad_count = ntb->spad_count;
- ctrl_size = sizeof(struct epf_ntb_ctrl);
+ ctrl_size = ALIGN(sizeof(struct epf_ntb_ctrl), sizeof(u32));
spad_size = 2 * spad_count * sizeof(u32);
- if (!align) {
- ctrl_size = roundup_pow_of_two(ctrl_size);
- spad_size = roundup_pow_of_two(spad_size);
- } else {
- ctrl_size = ALIGN(ctrl_size, align);
- spad_size = ALIGN(spad_size, align);
- }
-
- if (!size)
- size = ctrl_size + spad_size;
- else if (size < ctrl_size + spad_size)
- return -EINVAL;
-
- base = pci_epf_alloc_space(epf, size, barno, epc_features, 0);
+ base = pci_epf_alloc_space(epf, ctrl_size + spad_size,
+ barno, epc_features, 0);
if (!base) {
dev_err(dev, "Config/Status/SPAD alloc region fail\n");
return -ENOMEM;
--
2.47.2
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH v5 0/2] PCI: endpoint: space allocation fixups
2025-04-24 8:34 [PATCH v5 0/2] PCI: endpoint: space allocation fixups Jerome Brunet
2025-04-24 8:34 ` [PATCH v5 1/2] PCI: endpoint: improve fixed_size bar handling when allocating space Jerome Brunet
2025-04-24 8:34 ` [PATCH v5 2/2] PCI: endpoint: pci-epf-vntb: simplify ctrl/spad space allocation Jerome Brunet
@ 2025-04-27 17:17 ` Manivannan Sadhasivam
2 siblings, 0 replies; 4+ messages in thread
From: Manivannan Sadhasivam @ 2025-04-27 17:17 UTC (permalink / raw)
To: Krzysztof Wilczyński, Kishon Vijay Abraham I, Bjorn Helgaas,
Lorenzo Pieralisi, Jon Mason, Dave Jiang, Allen Hubbe,
Christoph Hellwig, Sagi Grimberg, Chaitanya Kulkarni,
Jerome Brunet
Cc: Manivannan Sadhasivam, Marek Vasut, Yoshihiro Shimoda,
Yuya Hamamachi, linux-pci, linux-kernel, ntb, linux-nvme,
Niklas Cassel
On Thu, 24 Apr 2025 10:34:03 +0200, Jerome Brunet wrote:
> This patchset fixes problems while trying to allocate space for PCI
> endpoint function.
>
> The problems, and related fixups, have been found while trying to link two
> renesas rcar-gen4 r8a779f0-spider devices with the vNTB endpoint
> function. This platform has 2 configurable BAR0 and BAR2, with an alignment
> of 1MB, and fairly small fixed BAR4 of 256B.
>
> [...]
Applied, thanks!
[1/2] PCI: endpoint: improve fixed_size bar handling when allocating space
commit: df4b24b510c9ceac32aeb3ed5904fd93b06a51b8
[2/2] PCI: endpoint: pci-epf-vntb: simplify ctrl/spad space allocation
commit: b47493ff0d43608257a24d65f5b26a4a6376ff9b
Best regards,
--
Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
^ permalink raw reply [flat|nested] 4+ messages in thread