Re: [PATCH 4/6] irqchip: irq-mvebu-icu: new driver for Marvell ICU

[Marc Zyngier <marc.zyngier@arm.com>]

On 30/05/17 13:05, Thomas Petazzoni wrote:
> Hello,
> 
> On Tue, 30 May 2017 12:10:29 +0100, Marc Zyngier wrote:
> 
>> Thanks for that, looks pretty interesting. A couple of comments below.
> 
> Thanks again for the review!
> 
>>> +/* GICP registers */
>>> +#define GICP_SETSPI_NSR_OFFSET	0x0
>>> +#define GICP_CLRSPI_NSR_OFFSET	0x8  
>>
>> Shouldn't that live in some gicp-specific include file?
> 
> Would drivers/irqchip/irq-mvebu-gicp.h, included by both
> irq-mvebu-gicp.c and irq-mvebu-icu.c be fine for you?

Sure, that'd be fine, assuming that it is necessary (see below).

> 
>>> +/* ICU definitions */
>>> +#define ICU_MAX_IRQS		207
>>> +#define IRQS_PER_ICU		64
>>> +#define ICU_MAX_SPI_IRQ_IN_GIC	(2 * IRQS_PER_ICU)  
>>
>> What's the relationship between ICU_MAX_IRQS and
>> IRQS_PER_ICU/ICU_MAX_SPI_IRQ_IN_GIC, if any? Is there only a single ICU?
>> Or can you have multiple ones?
> 
> There is one ICU per CP. The Armada 7K SoC has one CP, the Armada 8K
> SoC has two CPs. Therefore on Armada 8K, you have two ICUs, one per CP.

OK. Is there any restriction on which SPI an ICU can generate?

> But I see your point: there is in fact no direct relation between the
> number of GICP SPI interrupts reserved and the number of ICUs and
> interrupts per ICU.

Indeed. And maybe we should have an instance of the ICU device per CP.

> The number of GICP SPI interrupts (128) should be moved into
> irq-mvebu-gicp.h, together with the GICP register offsets.
> 
>>> +#define ICU_GIC_SPI_BASE0	64
>>> +#define ICU_GIC_SPI_BASE1	288  
>>
>> My own gut feeling is that there will be another version of this IP one
>> of these days, with different bases. Should we bite the bullet right
>> away and put those in DT?
> 
> Where should these properties go? Under the gicp DT node, or under the
> ICU DT node?

If the ICU has no knowledge of the SPI it can generate, I'd rather put
that in the GICP node.

> 
>>> +static DEFINE_SPINLOCK(icu_lock);
>>> +static DECLARE_BITMAP(icu_irq_alloc, ICU_MAX_SPI_IRQ_IN_GIC);
>>> +
>>> +static unsigned int
>>> +mvebu_icu_icuidx_to_gicspi(unsigned int icuidx)
>>> +{
>>> +	if (icuidx < ICU_GIC_SPI_BASE0)
>>> +		return ICU_GIC_SPI_BASE0 + icuidx;
>>> +	else
>>> +		return ICU_GIC_SPI_BASE1 + (icuidx - IRQS_PER_ICU);  
>>
>> A small comment on how ICU indexes and GIC SPIs map would be good.
> 
> ACK.
> 
>> Correct me if I'm wrong, but it really looks like you're dealing with
>> two devices here, each with their own SPI window.
> 
> We in fact don't really care about how many ICUs we have here. We have
> 128 GICP SPI interrupts available, in ranges:
> 
>  - ICU_GIC_SPI_BASE0 ; ICU_GIC_SPI_BASE0 + 64
> 
>  - ICU_GIC_SPI_BASE1 ; ICU_GIC_SPI_BASE1 + 64
> 
> The icu_irq_alloc bitmap is a global one, which allows to allocate one
> GICP SPI interrupts amongst the available 128 interrupts, and this
> function simply allows to map the index in this bitmap (from 0 to 127)
> to the corresponding GICP SPI interrupt.

That makes a lot more sense now, thanks.

> 
>>> +	writel(icu_int, icu->base + ICU_INT_CFG(hwirq));  
>>
>> You can use a writel_relaxed here.
> 
> ACK.
> 
>>> +	/*
>>> +	 * The SATA unit has 2 ports, and a dedicated ICU entry per
>>> +	 * port. The ahci sata driver supports only one irq interrupt
>>> +	 * per SATA unit. To solve this conflict, we configure the 2
>>> +	 * SATA wired interrupts in the south bridge into 1 GIC
>>> +	 * interrupt in the north bridge. Even if only a single port
>>> +	 * is enabled, if sata node is enabled, both interrupts are
>>> +	 * configured (regardless of which port is actually in use).
>>> +	 * The ICU index of SATA0 = 107, SATA1 = 109  
>>
>> You can drop that last comment about the indexes, the #defines are good
>> enough.
> 
> Agreed.
> 
>>> +	 */
>>> +	if (hwirq == ICU_SATA0_IRQ_INT || hwirq == ICU_SATA1_IRQ_INT) {
>>> +		writel(icu_int, icu->base + ICU_INT_CFG(ICU_SATA0_IRQ_INT));
>>> +		writel(icu_int, icu->base + ICU_INT_CFG(ICU_SATA1_IRQ_INT));
>>> +	}  
>>
>> Aren't you wasting an SPI here?
> 
> No: we allocate a single SPI, icu_int. What we're doing here is that we
> have two different wired interrupts in the CP that are "connected" to
> the same GICP SPI interrupt.

But if both ports are enabled, you're going to allocate one SPI per call
to this function, and the last one wins (you never "remember" that you
have configured one port already, and always allocate a new interrupt).

>> If you configure SATA0 first, then SATA1, SATA0's allocated SPI will
>> be wasted (not to mention the corresponding Linux interrupt too).
>> Can't this be worked around at the AHCI level? It is not like we
>> don't have any quirk there already...
> 
> This is something I wanted to look at, but at a follow-up work, as I
> believe the proposed work around is reasonable, and does not affect the
> DT binding.

Sure.

>>> +	writel(0, icu->base + ICU_INT_CFG(irqd_to_hwirq(irq)));  
>>
>> writel_relaxed (everywhere in this file).
> 
> ACK.
> 
> 
>>> +	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>>> +	icu->base = devm_ioremap_resource(&pdev->dev, res);
>>> +	if (IS_ERR(icu->base)) {
>>> +		dev_err(&pdev->dev, "Failed to map icu base address.\n");
>>> +		return PTR_ERR(icu->base);
>>> +	}  
>>
>> Careful here, you're leaking the memory from kstrdup above. Consider
>> using devm_kstrdup or/and moving all the potential failures (including
>> hooking on GICP) before doign any memory allocation.
> 
> Well spotted, will fix.
> 
> 
>>> +	/* Set Clear/Set ICU SPI message address in AP */
>>> +	writel(upper_32_bits(gicp_res->start + GICP_SETSPI_NSR_OFFSET),
>>> +	       icu->base + ICU_SETSPI_NSR_AH);
>>> +	writel(lower_32_bits(gicp_res->start + GICP_SETSPI_NSR_OFFSET),
>>> +	       icu->base + ICU_SETSPI_NSR_AL);
>>> +	writel(upper_32_bits(gicp_res->start + GICP_CLRSPI_NSR_OFFSET),
>>> +	       icu->base + ICU_CLRSPI_NSR_AH);
>>> +	writel(lower_32_bits(gicp_res->start + GICP_CLRSPI_NSR_OFFSET),
>>> +	       icu->base + ICU_CLRSPI_NSR_AL);  
>>
>> I wonder if it wouldn't be better to have a proper function call into
>> the GICP code to retrieve this information.
> 
> I've never been a big fan of random cross function calls between
> drivers, but this can be done.
> 
>> Or move the whole GICP probing in here, because even if it is a
>> separate piece of HW, it serves no real purpose on its own.
> 
> So you suggest to merge the whole irq-mvebu-gicp.c stuff inside
> irq-mvebu-icu.c ?

Given how small that code is, it wouldn't be a big deal.

>>> +	/*
>>> +	 * Clean all ICU interrupts with type SPI_NSR, required to
>>> +	 * avoid unpredictable SPI assignments done by firmware.
>>> +	 */
>>> +	for (i = 0 ; i < ICU_MAX_IRQS ; i++) {
>>> +		icu_int = readl(icu->base + ICU_INT_CFG(i));
>>> +		if ((icu_int >> ICU_GROUP_SHIFT) == ICU_GRP_NSR)
>>> +			writel(0x0, icu->base + ICU_INT_CFG(i));  
>>
>> Erm... Does it mean that non-secure can write to the configuration of
>> a secure interrupt? If that's the case, that's pretty... interesting.
> 
> I'll let Hannah and Yehuda answer this question, they know the ICU
> details much better than I do.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...