Re: [PATCH 00/53] dyn_array/nr_irqs/sparse_irq support v10

["Yinghai Lu" <yhlu.kernel@gmail.com>]

On Thu, Aug 14, 2008 at 1:05 PM, Eric W. Biederman
<ebiederm@xmission.com> wrote:
> "Yinghai Lu" <yhlu.kernel@gmail.com> writes:
>
>> On Thu, Aug 14, 2008 at 6:26 AM, Ingo Molnar <mingo@elte.hu> wrote:
>>>
>>> (lkml Cc:-ed)
>>>
>>> * Yinghai Lu <yhlu.kernel@gmail.com> wrote:
>>>
>>>> v10: make the x86 32 bit support sparse_irq too.
>>>>     also make remove irqbalance in kernel for 32 bit
>>>>     and make 32 bit support per_cpu vector. so start merging or io_apic_xx.c
>>>>
>>>>     01-03 is some fix for current tree
>>>>     04: is revert for one patch hide in sched/ and tip/master
>>>>
>>>> based on tip/master
>>>>
>>>> to do:
>>>>     merge io_apic_xx.c
>>>
>>> ok, i've created a new tip/irq/sparseirq topic branch for this.
>>>
>>> Could you please send future updates against:
>>>
>>>   git-checkout tip/master
>>>   git-merge tip/irq/sparseirq
>>>
>>> Not yet integrated into tip/master, i've still got some other backlog.
>>>
>>> A couple of observations about the general structure of the sparse irqs
>>> code:
>>>
>>> - the new APIs should probably live in mm/bootmem.c not in init/main.c,
>>>  and in bootmem.h. Also, it should be outlined clearly why this new API
>>>  is needed as a wrapper ontop of existing bootmem mechanisms.
>>
>> move
>> pre_alloc_dyn_array, per_cpu_dyn_array_size, per_cpu_alloc_dyn_array
>> to mm/bootmem.c?
>>
>>>
>>> - the #ifdef complications, while fine for migration, should be
>>>  eliminated. Could we introduce some compatible form for the
>>>  definition/allocation APIs that work even if an architecture still
>>>  uses a flat irq array? That would remove most of the uglinesses.
>> # git grep CONFIG_HAVE_DYN_ARRAY | wc -l
>> 9
>>
>> #git grep CONFIG_HAVE_SPARSE_IRQ | wc -l
>> 39
>>
>> arch/x86/kernel/io_apic_32.c:#ifdef CONFIG_HAVE_SPARSE_IRQ_DEBUG
>> arch/x86/kernel/io_apic_32.c:#ifndef CONFIG_HAVE_SPARSE_IRQ
>> arch/x86/kernel/io_apic_64.c:#ifdef CONFIG_HAVE_SPARSE_IRQ_DEBUG
>> arch/x86/kernel/io_apic_64.c:#ifndef CONFIG_HAVE_SPARSE_IRQ
>> arch/x86/kernel/irq_32.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> arch/x86/kernel/irq_64.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> drivers/char/random.c:#ifndef CONFIG_HAVE_SPARSE_IRQ
>> drivers/char/random.c:#ifndef CONFIG_HAVE_SPARSE_IRQ
>> drivers/pci/intr_remapping.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> drivers/pci/intr_remapping.c:#else /* !CONFIG_HAVE_SPARSE_IRQ */
>> drivers/pci/intr_remapping.c:#ifndef    CONFIG_HAVE_SPARSE_IRQ
>> fs/proc/proc_misc.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> fs/proc/proc_misc.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> fs/proc/proc_misc.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> include/linux/irq.h:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> include/linux/irq.h:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> include/linux/irq.h:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> include/linux/irq.h:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> include/linux/irq.h:#if defined(CONFIG_INTR_REMAP) &&
>> defined(CONFIG_HAVE_SPARSE_IRQ)
>> include/linux/irq.h:#ifndef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/chip.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/chip.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/chip.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/chip.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/chip.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifndef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ_DEBUG
>> kernel/irq/handle.c:#ifndef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/handle.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/manage.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/manage.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>> kernel/irq/migration.c:#ifdef CONFIG_HAVE_SPARSE_IRQ
>>
>> because try to avoid more api to be changed to take irq_desc as parameter.
>>
>>>
>>> - some of the irq < 16 checks in x86 look a bit ugly, can we do anything
>>>  about them?
>>
>> is_legacy_irq(irq) ? or legacy_irq(irq)?
>
>
>>
>>>
>>> - i think as a final-ish commit we should just remove the dyn-array
>>>  define and make all architectures use this facility. You converted
>>>  most of them - how many are still missing? Sparse-irq is more
>>>  intrusive so that should probably stay a Kconfig knob.
>>
>> so when config_sparse_irq is not selected, need to use dyn_array to
>> allocate irq_desc and irq_cfg, or just static array?
>>
>>>
>>> - there was one aspect of NR_IRQS that was nice and useful: it acted as
>>>  a sanity check against BIOS bugs and driver bugs that pass in some
>>>  irrealistically high irq number. Now we'll just try to allocate some
>>>  really high IRQ and accept it. I _think_ there should be a single,
>>>  simple 'absolute maximum IRQ number' define which should set the
>>>  maximum possible IRQ number. Lets call it NR_IRQS_MAX or so, and set
>>>  it to NR_CPUS*NR_IO_APICS*224 or so?
>
> We may want an is_valid_isa_irq() for drivers.  Otherwise drivers
> should not be passing in irqs.  And the request_irq path should
> valid the irq number but it should not allocate it.  So we will
> have failures with invalid irq numbers.
>
> For acpi, mptables and the like we can easily have a check for some
> reasonably high value if there is value in that.  Last I looked parts
> of that code were using numbers like 1024 and 4096 already.
>
>> when sparse_irq is used, irq is [0, -1U]
>> driver could be irq_desc(irq) to check if irq is valid or not. and use
>> irq_desc_with_new(new) to get a new one.
>
> Note that request_irq should do this for free.
>
>> when generic_hardirqs is not used (s390, m68k, sparc), we need to
>> create some stubs in linux/interrupt.h for them
>
> s390 is weird.  I'm not even convinced they use linux/interrupt.h
>
> Yep.  If we need more than the proper checks in request_irq.
>
>>> - i'm a bit worried about linecount increase in general:
>>>
>>>     247 files changed, 3671 insertions(+), 2052 deletions(-)
>>>
>>>  could we work on reducing that somewhat? The new infrastructure bits
>>>  in mm/* and kernel/irq/* will be unavoidable, what we should
>>>  concentrate on is to make usage of the new facility just as
>>>  straightforward, easy and compact as the old irq_desc[] usages.
>> before that we could use get_irq_desc() and get_irq_desc_without_new()
>> and according to eric, i changed that to irq_desc_with_new and irq_desc().
>>
>> so the array irq_desc[] need to be changed to irq_descX[]
>> ( or *irq_desc to *irq_descX for dyn_array)
>
> My mistake.  irq_desc() is a bad name for the conversion method
> because it conflicts like that.  I really just did not want something
> wordy, or something that suggests you need to call put_irq_desc.  BothÜ
> of which get_irq_desc() are.

it took me one day to change them. anyway irq_desc() is better than get_irq_desc

>
> What I wound up using in my tree is a little different.
>
> I introduced an opaque/empty structure: struct irq to be used in places
> where we need pointers instead of an integer irq number in the interfaces.
>
> I have a version of the genirq api that works on struct irq instead of
> unsigned int irq.
>
> I have functions:
> struct irq * to_irq(unsigned int nr);
> struct irq_desc *to_idesc(struct irq *);
> unsigned int irq_nr(struct irq *irq);

struct irq {
  unsigned int nr;
  struct irq_desc *idesc;
}

?

>
> Is there any reason why the migration path for architectures can not be:
> Until they are converted:
> #define NR_IRQS and use the irq_desc array.
>
> If they are just using a dynamically allocated array.
> #define NR_IRQS nr_irqs
>
> Once we kill the array entirely.
> #undef NR_IRQS or
> #define NR_IRQS 0xfffff000
why not use -1U here?

YH