On Wed, 13 Apr 2016, Luiz Augusto von Dentz wrote:

Hi Dmitriy,

On Wed, Apr 13, 2016 at 1:18 AM, Dmitriy Korovkin
<dmitriy.korovkin(a)windriver.com> wrote:

On Tue, 12 Apr 2016, Luiz Augusto von Dentz wrote:

Hi Dmitriy,

On Mon, Apr 11, 2016 at 9:32 PM, Dmitriy Korovkin
<dmitriy.korovkin(a)windriver.com> wrote:

On Fri, 8 Apr 2016, Luiz Augusto von Dentz wrote:

Hi,

For the network protocols is quite common to have timers/timeouts for
retrying, etc, and these cold be many in parallel depending on how
many protocols and connections are involved, for that reason the IP
stack actually contains a Timer Fiber to keep track of every timer, it
does basically something like this:

net/ip/net_core.c:666:
while (1) {
/* Run various timers */
next_wakeup = etimer_request_poll();
if (next_wakeup == 0) {
/* There was no timers, wait for fiber_wakeup */
next_wakeup = TICKS_UNLIMITED;
} else {
if (next_wakeup > MAX_TIMER_WAKEUP) {
next_wakeup = MAX_TIMER_WAKEUP;
}
...
fiber_sleep(next_wakeup);
}

This actually uses contiki etimer infra but that in the end is using
nano_timer as a backend.

In the other hand in the Bluetooth stack we actually use delayed
fibers, but that requires each and every timeout to create its own
stack to be run separately which adds more footprint to the stack, so
we would like to use the same approach of IP stack and share the same
fiber/stack but without having to resort to any contiki API.

I am not quite sure I understand the problem. Kernel keeps the track of
nanokernel timers and timeouts. If needed, each fiber can wait on a
timer (one fiber per timer). Not sure, what is the need for a separate
fiber
that runs through the timers.

By kernel I guess you mean that each fiber has a capability to run
timers, which is not useful in case of net subsystem since that
requires each and every timeout to have a dedicated stack.

In other words, the idea of several fibers does not work for you, I see.

With this in mind Id like to get some opinions regarding the design of
a Timer/Timeout API:

- Shall this be global/system wide or net specific? I think it could
be global enabled with a Kconfig option and perhaps make _nano_timeout
API public.

Depends on what is needed. If this is a global change (apility for
multiple
fibers to wait on one timer, for instance), this should be global.

I never said we want the ability for multiple fibers to wait on one
timer, it is actually the opposite that we are doing in IP since there
is a single fiber managing multiple nano_timers calling a callback
when they expire. Anyway I starting to think we would be better off
prototyping this under net so we get the ball rolling.

- Perhaps have the API flexible enough so private fiber could be used
instead in case one don't want to use the global one?

As the kernel keeps track of the timers, there may be something else is
needed.

As I understand the kernel has APIs to put a fiber/task to sleep for
an x amount of ticks, blocking them, it doesn't have any API that the
user provide a callback which gets called when the timer expire
without blocking or requiring a new fiber for each call.

Ok, now it gets clearer. You are asking for a nanokernel timer API that
executes a registered callback when it expires. I do not think it is a good
idea to execute callbacks in ISR context, which brings us to an idea
of a fiber that executes these callbacks. Is this what you are asking for?
For this we need to make sure that a fiber can sleep on multiple timers.
The stack size for this fiber should be configurable.
Does it look good?

Yes, it is actually what we already have in IP which maintain a fiber
to execute the callbacks but the API is based on contiki etimers which
is something we would not like to use in Bluetooth. We have actually
discussed about using ISR context and came to the same conclusion that
it is perhaps not a good idea, although the callback should not block
either since that would block other callbacks to be executed. This
concept in fact is very similar to Linux Workqueues, but in Linux it
is possible to create dedicated workqueues which perhaps is useful for
drivers, etc. Anyway I started prototyping for Bluetooth first to see
how it works in practice.

I would suggest starting with RFC, as it may be good to have it
implemented in the Zephyr OS kernel:
If we enable NANO_TIMER_CALLBACK config option, we start a fiber that
sleeps on all the nanokernel timers that have registered a callback
function. When this fiber starts, it may probably just call
_nano_fiber_swap() and wait. When we register a nano_timer callback,
we add the "callback fiber" tcs to the timer, as if the fiber is
waiting for this timer to expire. And do same things with all the other
timers when register a callback for them. When the timer expires, it
wakes up the callback fiber, fiber walks through the expired timers
(somehow) and executes their callbacks. Then it calls _nano_fiber_swap()
and waits for another timer to expire.
This is just a rough idea and it opens several questions:
- do we need both timer with a callback and a fiber (not the "callback
fiber") waiting for it.
- do we need an argument for the callback? Should the argument be the user
data pointer that we have for the nano timer?
- how is it better to keep the callback pointer? For each nanokernel
timer or within the "callback fiber" list?
- how to get the list of expired timers without walking through the list
of all the registered ones?

Regards,
/Dmitriy.

Hi Dmitriy,

On Wed, Apr 13, 2016 at 1:18 AM, Dmitriy Korovkin
<dmitriy.korovkin(a)windriver.com> wrote:

On Tue, 12 Apr 2016, Luiz Augusto von Dentz wrote:

Hi Dmitriy,

On Mon, Apr 11, 2016 at 9:32 PM, Dmitriy Korovkin
<dmitriy.korovkin(a)windriver.com> wrote:

On Fri, 8 Apr 2016, Luiz Augusto von Dentz wrote:

Hi,

For the network protocols is quite common to have timers/timeouts for
retrying, etc, and these cold be many in parallel depending on how
many protocols and connections are involved, for that reason the IP
stack actually contains a Timer Fiber to keep track of every timer, it
does basically something like this:

net/ip/net_core.c:666:
while (1) {
/* Run various timers */
next_wakeup = etimer_request_poll();
if (next_wakeup == 0) {
/* There was no timers, wait for fiber_wakeup */
next_wakeup = TICKS_UNLIMITED;
} else {
if (next_wakeup > MAX_TIMER_WAKEUP) {
next_wakeup = MAX_TIMER_WAKEUP;
}
...
fiber_sleep(next_wakeup);
}

This actually uses contiki etimer infra but that in the end is using
nano_timer as a backend.

In the other hand in the Bluetooth stack we actually use delayed
fibers, but that requires each and every timeout to create its own
stack to be run separately which adds more footprint to the stack, so
we would like to use the same approach of IP stack and share the same
fiber/stack but without having to resort to any contiki API.

I am not quite sure I understand the problem. Kernel keeps the track of
nanokernel timers and timeouts. If needed, each fiber can wait on a
timer (one fiber per timer). Not sure, what is the need for a separate
fiber
that runs through the timers.

By kernel I guess you mean that each fiber has a capability to run
timers, which is not useful in case of net subsystem since that
requires each and every timeout to have a dedicated stack.

In other words, the idea of several fibers does not work for you, I see.

With this in mind Id like to get some opinions regarding the design of
a Timer/Timeout API:

- Shall this be global/system wide or net specific? I think it could
be global enabled with a Kconfig option and perhaps make _nano_timeout
API public.

Depends on what is needed. If this is a global change (apility for
multiple
fibers to wait on one timer, for instance), this should be global.

I never said we want the ability for multiple fibers to wait on one
timer, it is actually the opposite that we are doing in IP since there
is a single fiber managing multiple nano_timers calling a callback
when they expire. Anyway I starting to think we would be better off
prototyping this under net so we get the ball rolling.

- Perhaps have the API flexible enough so private fiber could be used
instead in case one don't want to use the global one?

As the kernel keeps track of the timers, there may be something else is
needed.

As I understand the kernel has APIs to put a fiber/task to sleep for
an x amount of ticks, blocking them, it doesn't have any API that the
user provide a callback which gets called when the timer expire
without blocking or requiring a new fiber for each call.

Ok, now it gets clearer. You are asking for a nanokernel timer API that
executes a registered callback when it expires. I do not think it is a good
idea to execute callbacks in ISR context, which brings us to an idea
of a fiber that executes these callbacks. Is this what you are asking for?
For this we need to make sure that a fiber can sleep on multiple timers.
The stack size for this fiber should be configurable.
Does it look good?

Yes, it is actually what we already have in IP which maintain a fiber
to execute the callbacks but the API is based on contiki etimers which
is something we would not like to use in Bluetooth. We have actually
discussed about using ISR context and came to the same conclusion that
it is perhaps not a good idea, although the callback should not block
either since that would block other callbacks to be executed. This
concept in fact is very similar to Linux Workqueues, but in Linux it
is possible to create dedicated workqueues which perhaps is useful for
drivers, etc. Anyway I started prototyping for Bluetooth first to see
how it works in practice.

--
Luiz Augusto von Dentz

On Tue, 12 Apr 2016, Luiz Augusto von Dentz wrote:

Hi Dmitriy,

On Mon, Apr 11, 2016 at 9:32 PM, Dmitriy Korovkin
<dmitriy.korovkin(a)windriver.com> wrote:

On Fri, 8 Apr 2016, Luiz Augusto von Dentz wrote:

Hi,

For the network protocols is quite common to have timers/timeouts for
retrying, etc, and these cold be many in parallel depending on how
many protocols and connections are involved, for that reason the IP
stack actually contains a Timer Fiber to keep track of every timer, it
does basically something like this:

net/ip/net_core.c:666:
while (1) {
/* Run various timers */
next_wakeup = etimer_request_poll();
if (next_wakeup == 0) {
/* There was no timers, wait for fiber_wakeup */
next_wakeup = TICKS_UNLIMITED;
} else {
if (next_wakeup > MAX_TIMER_WAKEUP) {
next_wakeup = MAX_TIMER_WAKEUP;
}
...
fiber_sleep(next_wakeup);
}

This actually uses contiki etimer infra but that in the end is using
nano_timer as a backend.

In the other hand in the Bluetooth stack we actually use delayed
fibers, but that requires each and every timeout to create its own
stack to be run separately which adds more footprint to the stack, so
we would like to use the same approach of IP stack and share the same
fiber/stack but without having to resort to any contiki API.

I am not quite sure I understand the problem. Kernel keeps the track of
nanokernel timers and timeouts. If needed, each fiber can wait on a
timer (one fiber per timer). Not sure, what is the need for a separate fiber
that runs through the timers.

By kernel I guess you mean that each fiber has a capability to run
timers, which is not useful in case of net subsystem since that
requires each and every timeout to have a dedicated stack.

In other words, the idea of several fibers does not work for you, I see.

With this in mind Id like to get some opinions regarding the design of
a Timer/Timeout API:

- Shall this be global/system wide or net specific? I think it could
be global enabled with a Kconfig option and perhaps make _nano_timeout
API public.

Depends on what is needed. If this is a global change (apility for multiple
fibers to wait on one timer, for instance), this should be global.

I never said we want the ability for multiple fibers to wait on one
timer, it is actually the opposite that we are doing in IP since there
is a single fiber managing multiple nano_timers calling a callback
when they expire. Anyway I starting to think we would be better off
prototyping this under net so we get the ball rolling.

- Perhaps have the API flexible enough so private fiber could be used
instead in case one don't want to use the global one?

As the kernel keeps track of the timers, there may be something else is
needed.

As I understand the kernel has APIs to put a fiber/task to sleep for
an x amount of ticks, blocking them, it doesn't have any API that the
user provide a callback which gets called when the timer expire
without blocking or requiring a new fiber for each call.

Ok, now it gets clearer. You are asking for a nanokernel timer API that
executes a registered callback when it expires. I do not think it is a
good idea to execute callbacks in ISR context, which brings us to an idea
of a fiber that executes these callbacks. Is this what you are asking for?
For this we need to make sure that a fiber can sleep on multiple timers.
The stack size for this fiber should be configurable.
Does it look good?
Regards,
/Dmitriy.

Hi Dmitriy,

On Mon, Apr 11, 2016 at 9:32 PM, Dmitriy Korovkin
<dmitriy.korovkin(a)windriver.com> wrote:

On Fri, 8 Apr 2016, Luiz Augusto von Dentz wrote:

Hi,

For the network protocols is quite common to have timers/timeouts for
retrying, etc, and these cold be many in parallel depending on how
many protocols and connections are involved, for that reason the IP
stack actually contains a Timer Fiber to keep track of every timer, it
does basically something like this:

net/ip/net_core.c:666:
while (1) {
/* Run various timers */
next_wakeup = etimer_request_poll();
if (next_wakeup == 0) {
/* There was no timers, wait for fiber_wakeup */
next_wakeup = TICKS_UNLIMITED;
} else {
if (next_wakeup > MAX_TIMER_WAKEUP) {
next_wakeup = MAX_TIMER_WAKEUP;
}
...
fiber_sleep(next_wakeup);
}

This actually uses contiki etimer infra but that in the end is using
nano_timer as a backend.

In the other hand in the Bluetooth stack we actually use delayed
fibers, but that requires each and every timeout to create its own
stack to be run separately which adds more footprint to the stack, so
we would like to use the same approach of IP stack and share the same
fiber/stack but without having to resort to any contiki API.

I am not quite sure I understand the problem. Kernel keeps the track of
nanokernel timers and timeouts. If needed, each fiber can wait on a
timer (one fiber per timer). Not sure, what is the need for a separate fiber
that runs through the timers.

By kernel I guess you mean that each fiber has a capability to run
timers, which is not useful in case of net subsystem since that
requires each and every timeout to have a dedicated stack.

With this in mind Id like to get some opinions regarding the design of
a Timer/Timeout API:

- Shall this be global/system wide or net specific? I think it could
be global enabled with a Kconfig option and perhaps make _nano_timeout
API public.

Depends on what is needed. If this is a global change (apility for multiple
fibers to wait on one timer, for instance), this should be global.

I never said we want the ability for multiple fibers to wait on one
timer, it is actually the opposite that we are doing in IP since there
is a single fiber managing multiple nano_timers calling a callback
when they expire. Anyway I starting to think we would be better off
prototyping this under net so we get the ball rolling.

- Perhaps have the API flexible enough so private fiber could be used
instead in case one don't want to use the global one?

As the kernel keeps track of the timers, there may be something else is
needed.

As I understand the kernel has APIs to put a fiber/task to sleep for
an x amount of ticks, blocking them, it doesn't have any API that the
user provide a callback which gets called when the timer expire
without blocking or requiring a new fiber for each call.

--
Luiz Augusto von Dentz

On Fri, 8 Apr 2016, Luiz Augusto von Dentz wrote:

Hi,

For the network protocols is quite common to have timers/timeouts for
retrying, etc, and these cold be many in parallel depending on how
many protocols and connections are involved, for that reason the IP
stack actually contains a Timer Fiber to keep track of every timer, it
does basically something like this:

net/ip/net_core.c:666:
while (1) {
/* Run various timers */
next_wakeup = etimer_request_poll();
if (next_wakeup == 0) {
/* There was no timers, wait for fiber_wakeup */
next_wakeup = TICKS_UNLIMITED;
} else {
if (next_wakeup > MAX_TIMER_WAKEUP) {
next_wakeup = MAX_TIMER_WAKEUP;
}
...
fiber_sleep(next_wakeup);
}

This actually uses contiki etimer infra but that in the end is using
nano_timer as a backend.

In the other hand in the Bluetooth stack we actually use delayed
fibers, but that requires each and every timeout to create its own
stack to be run separately which adds more footprint to the stack, so
we would like to use the same approach of IP stack and share the same
fiber/stack but without having to resort to any contiki API.

I am not quite sure I understand the problem. Kernel keeps the track of
nanokernel timers and timeouts. If needed, each fiber can wait on a
timer (one fiber per timer). Not sure, what is the need for a separate
fiber that runs through the timers.

With this in mind Id like to get some opinions regarding the design of
a Timer/Timeout API:

- Shall this be global/system wide or net specific? I think it could
be global enabled with a Kconfig option and perhaps make _nano_timeout
API public.

Depends on what is needed. If this is a global change (apility for
multiple fibers to wait on one timer, for instance), this should be
global.

- Perhaps have the API flexible enough so private fiber could be used
instead in case one don't want to use the global one?

As the kernel keeps track of the timers, there may be something else is
needed.

Regards,
/Dmitriy

Hi Marcel,

Do you have anything to add? Or is this something that came out while
discussing with Daniel due to my MSR?

On Fri, Apr 8, 2016 at 8:48 PM, Kalowsky, Daniel
<daniel.kalowsky(a)intel.com> wrote:

+ Marcel, as he mentioned some changes to both areas (I think) at OpenIoT.

-----Original Message-----
From: Luiz Augusto von Dentz [mailto:luiz.dentz(a)gmail.com]
Sent: Friday, April 8, 2016 4:38 AM
To: devel(a)lists.zephyrproject.org
Subject: [devel] RFC: Timer/Timeout API

Hi,

For the network protocols is quite common to have timers/timeouts for
retrying, etc, and these cold be many in parallel depending on how
many protocols and connections are involved, for that reason the IP
stack actually contains a Timer Fiber to keep track of every timer, it
does basically something like this:

net/ip/net_core.c:666:
while (1) {
/* Run various timers */
next_wakeup = etimer_request_poll();
if (next_wakeup == 0) {
/* There was no timers, wait for fiber_wakeup */
next_wakeup = TICKS_UNLIMITED;
} else {
if (next_wakeup > MAX_TIMER_WAKEUP) {
next_wakeup = MAX_TIMER_WAKEUP;
}
...
fiber_sleep(next_wakeup);
}

This actually uses contiki etimer infra but that in the end is using
nano_timer as a backend.

In the other hand in the Bluetooth stack we actually use delayed
fibers, but that requires each and every timeout to create its own
stack to be run separately which adds more footprint to the stack, so
we would like to use the same approach of IP stack and share the same
fiber/stack but without having to resort to any contiki API.

With this in mind Id like to get some opinions regarding the design of
a Timer/Timeout API:

- Shall this be global/system wide or net specific? I think it could
be global enabled with a Kconfig option and perhaps make _nano_timeout
API public.
- Perhaps have the API flexible enough so private fiber could be used
instead in case one don't want to use the global one?
- Where would be the best location in the source tree?

--
Luiz Augusto von Dentz

--
Luiz Augusto von Dentz

+ Marcel, as he mentioned some changes to both areas (I think) at OpenIoT.

Hi,

For the network protocols is quite common to have timers/timeouts for
retrying, etc, and these cold be many in parallel depending on how
many protocols and connections are involved, for that reason the IP
stack actually contains a Timer Fiber to keep track of every timer, it
does basically something like this:

net/ip/net_core.c:666:
while (1) {
/* Run various timers */
next_wakeup = etimer_request_poll();
if (next_wakeup == 0) {
/* There was no timers, wait for fiber_wakeup */
next_wakeup = TICKS_UNLIMITED;
} else {
if (next_wakeup > MAX_TIMER_WAKEUP) {
next_wakeup = MAX_TIMER_WAKEUP;
}
...
fiber_sleep(next_wakeup);
}

This actually uses contiki etimer infra but that in the end is using
nano_timer as a backend.

In the other hand in the Bluetooth stack we actually use delayed
fibers, but that requires each and every timeout to create its own
stack to be run separately which adds more footprint to the stack, so
we would like to use the same approach of IP stack and share the same
fiber/stack but without having to resort to any contiki API.

With this in mind Id like to get some opinions regarding the design of
a Timer/Timeout API:

- Shall this be global/system wide or net specific? I think it could
be global enabled with a Kconfig option and perhaps make _nano_timeout
API public.
- Perhaps have the API flexible enough so private fiber could be used
instead in case one don't want to use the global one?
- Where would be the best location in the source tree?

--
Luiz Augusto von Dentz