Topics

RFC: Remove dynamic interrupts / exceptions


Boie, Andrew P
 

Problem statement:
We currently have two separate implementations on how to connect
interrupts on all arches: static IRQs and dynamic IRQs. On x86 only we
also have static and dynamic APIs for connecting exception handlers.
Although static interrupts & exceptions impose some constraints on
developers, they save a nontrivial amount of RAM (generally 500b to 1K
depending on configuration) as certain data structures like the IDT do
not need to be writable and can remain in ROM. In the interest of
keeping the Zephyr codebase small and not having to maintain two
different ways of doing the same thing, I propose we deprecate and
eventually remove the dynamic implementations.

Details are below. In the discussion I only talk about interrupts, but
for x86 dynamic vs. static exceptions the same arguments apply. On ARC
and ARM all the exceptions are hard-coded to their handlers and there
is no API for them.

The static IRQ macro IRQ_CONNECT() imposes the constraint that all its
arguments be constants resolvable at build time. This includes the IRQ
line number and the context data pointer. This makes things a little
clumsy if you have a single driver that has multiple instances on the
system. Since you need to use build-time constants you cannot do
something like this:

int mydriver_init(struct device *dev)
{
    struct mydriver_config *config = dev->config->config_info;
    ...
    IRQ_CONNECT(config->irq, config->irq_prio, mydriver_isr, dev,      
                config->irq_flags);
    irq_enable(config->irq);
    ...
}

Instead, the idiom used in Zephyr drivers is to define a per-instance
config function and save a pointer to it in the config struct:

void mydriver_0_config_fn(void)
{
    IRQ_CONNECT(MYDRIVER_0_IRQ, MYDRIVER_0_PRI, mydriver_isr,
                DEVICE_GET(mydriver_0), MYDRIVER_0_FLAGS);
    irq_enable(MYDRIVER_0_IRQ);
}

struct mydriver_config mydriver_0_config {
    ...
    .config_func = mydriver_0_config_fn;
    ...
};

int mydriver_init(struct device *dev)
{
    struct mydriver_config
*config = dev->config->config_info;
    ...
    config->config_func();
   
 ...
}

IRQ_CONNECT is not actually a function; it does have a runtime
component (to program the interrupt controller) but it does linker or
assembly black magic to set up all the relevant data structures at
build time.

What about instances where the interrupt arguments are simply not known
at build time? I feel that for Zephyr this will never be a problem
given the classes of devices we need to support:

- PCI devices do announce what their interrupts are, but for
microcontrollers with PCI busses (like galileo) we already know this
information at build time anyway. Once you get to the point where the
hardware really does have hot-pluggable peripherals you might as well
run Linux on it; adding proper hotplug support to Zephyr is a much
larger thing than just having dynamic interrupts.

- Microkernel Task IRQs currently rely on dynamic interrupts, but this
is an implementation detail. We are considering removing these from
Zephyr as well, but if not, it's possible to refactor them to use
static.

- None of the drivers included in the Zephyr tree use dynamic
interrupts.

- A few of the footprint tests use dynamic interrupts, but again this
is an implementation detail.

The RAM savings come from a few areas which differ per arch.

x86:
- the IDT can remain in ROM. This is where most of the savings come
from.
- The _interrupt_vectors_allocated and dyn_irq_list data structures can
be omitted from RAM if no dynamic interrupts are used.
- The IRQ_CONNECT() macro as part of its job automatically generates
the assembly language stub. These stubs trampoline to common code which
fetches the context pointer associated with the interrupt; this is how
our ISRs can be passed parameters. For dynamic interrupts you need to
know how many dynamic IRQs you plan on using in advance and set
CONFIG_NUM_DYNAMIC_STUBS; if the number of calls to
irq_connect_dynamic() exceeds this you will crash.

ARC/ARM:
On these two arches both static and dynamic interrupts are tracked in
sw_isr_table, an array indexed by IRQ line with each element noting the
ISR and parameter for that IRQ. If no dynamic IRQs are used, this table
can remain in ROM.

In summary, I'm not currently seeing any scenarios where dynamic IRQs
are necessary, and would like to attach __attribute__((deprecated))
tags to their APIs and eventually remove them from the kernel.

Andrew


Benjamin Walsh <benjamin.walsh@...>
 

Hang on on this, I'm cleaning up a GDB server implementation that we
want to upstream, and it uses at least dynamic exceptions. I'll see if I
can refactor it to use static ones instead.

On Tue, Mar 08, 2016 at 09:48:36PM +0000, Boie, Andrew P wrote:
Problem statement:
We currently have two separate implementations on how to connect
interrupts on all arches: static IRQs and dynamic IRQs. On x86 only we
also have static and dynamic APIs for connecting exception handlers.
Although static interrupts & exceptions impose some constraints on
developers, they save a nontrivial amount of RAM (generally 500b to 1K
depending on configuration) as certain data structures like the IDT do
not need to be writable and can remain in ROM. In the interest of
keeping the Zephyr codebase small and not having to maintain two
different ways of doing the same thing, I propose we deprecate and
eventually remove the dynamic implementations.

Details are below. In the discussion I only talk about interrupts, but
for x86 dynamic vs. static exceptions the same arguments apply. On ARC
and ARM all the exceptions are hard-coded to their handlers and there
is no API for them.

The static IRQ macro IRQ_CONNECT() imposes the constraint that all its
arguments be constants resolvable at build time. This includes the IRQ
line number and the context data pointer. This makes things a little
clumsy if you have a single driver that has multiple instances on the
system. Since you need to use build-time constants you cannot do
something like this:

int mydriver_init(struct device *dev)
{
    struct mydriver_config *config = dev->config->config_info;
    ...
    IRQ_CONNECT(config->irq, config->irq_prio, mydriver_isr, dev,      
                config->irq_flags);
    irq_enable(config->irq);
    ...
}

Instead, the idiom used in Zephyr drivers is to define a per-instance
config function and save a pointer to it in the config struct:

void mydriver_0_config_fn(void)
{
    IRQ_CONNECT(MYDRIVER_0_IRQ, MYDRIVER_0_PRI, mydriver_isr,
                DEVICE_GET(mydriver_0), MYDRIVER_0_FLAGS);
    irq_enable(MYDRIVER_0_IRQ);
}

struct mydriver_config mydriver_0_config {
    ...
    .config_func = mydriver_0_config_fn;
    ...
};

int mydriver_init(struct device *dev)
{
    struct mydriver_config
*config = dev->config->config_info;
    ...
    config->config_func();
   
 ...
}

IRQ_CONNECT is not actually a function; it does have a runtime
component (to program the interrupt controller) but it does linker or
assembly black magic to set up all the relevant data structures at
build time.

What about instances where the interrupt arguments are simply not known
at build time? I feel that for Zephyr this will never be a problem
given the classes of devices we need to support:

- PCI devices do announce what their interrupts are, but for
microcontrollers with PCI busses (like galileo) we already know this
information at build time anyway. Once you get to the point where the
hardware really does have hot-pluggable peripherals you might as well
run Linux on it; adding proper hotplug support to Zephyr is a much
larger thing than just having dynamic interrupts.

- Microkernel Task IRQs currently rely on dynamic interrupts, but this
is an implementation detail. We are considering removing these from
Zephyr as well, but if not, it's possible to refactor them to use
static.

- None of the drivers included in the Zephyr tree use dynamic
interrupts.

- A few of the footprint tests use dynamic interrupts, but again this
is an implementation detail.

The RAM savings come from a few areas which differ per arch.

x86:
- the IDT can remain in ROM. This is where most of the savings come
from.
- The _interrupt_vectors_allocated and dyn_irq_list data structures can
be omitted from RAM if no dynamic interrupts are used.
- The IRQ_CONNECT() macro as part of its job automatically generates
the assembly language stub. These stubs trampoline to common code which
fetches the context pointer associated with the interrupt; this is how
our ISRs can be passed parameters. For dynamic interrupts you need to
know how many dynamic IRQs you plan on using in advance and set
CONFIG_NUM_DYNAMIC_STUBS; if the number of calls to
irq_connect_dynamic() exceeds this you will crash.

ARC/ARM:
On these two arches both static and dynamic interrupts are tracked in
sw_isr_table, an array indexed by IRQ line with each element noting the
ISR and parameter for that IRQ. If no dynamic IRQs are used, this table
can remain in ROM.

In summary, I'm not currently seeing any scenarios where dynamic IRQs
are necessary, and would like to attach __attribute__((deprecated))
tags to their APIs and eventually remove them from the kernel.

Andrew


Boie, Andrew P
 

On Tue, 2016-03-08 at 17:01 -0500, Benjamin Walsh wrote:
Hang on on this, I'm cleaning up a GDB server implementation that we
want to upstream, and it uses at least dynamic exceptions. I'll see
if I
can refactor it to use static ones instead.
Sure, let me know.
If it ends up you need a dynamic handler for one specific exception on
x86 we could treat it as a special case. irq_offload() is like this.

Andrew