Re: Problems managing NBUF DATA pool in the networking stack


Marcus Shawcroft <marcus.shawcroft@...>
 

On 8 February 2017 at 07:04, Jukka Rissanen
<jukka.rissanen@linux.intel.com> wrote:

One option would be to split the DATA pool to two so one pool for
sending and receiving. Then again this does not solve much as you might
still get to a situation where all the buffers are exhausted.
Running out of resources is bad, dead lock, especially undetected
deadlock, is worse. Avoiding the dead lock where the RX path starves
the rest of the system of resources requires that the resources the RX
path can consume are separate from the resources available to the TX
path(s). Limiting resource consumption by the RX path is straight
forward, buffers come from fixed size pool, when the pool is empty we
drop packets. Now we have a situation where RX cannot starve TX, we
just need to ensure that multiple TX paths cannot deadlock each other.

Dealing with resource exhaustion on the TX side is harder. In a
system with multiple TX paths either, there need to be sufficient TX
resources that all TX paths can acquire sufficient resources to
proceed in parallel or there need to be sufficient resources for any
one path to make progress along with a mechanism to serialize those
paths. The former solution is probably a none starter for a small
system because the number of buffers required is likely to be
unreasonably large. The latter solution I think implies that no TX
path can block waiting for resources unless it currently holds no
resources.... ie blocking to get a buffer is ok, blocking to extend a
buffer or to get a second buffer is not ok.

The timeout to buffer API helps a bit but still we might run out of
buffers.
For incremental acquisition of further resources this doesn't help, it
can't guarantee to prevent dead lock and its use in the software stack
makes reasoning about deadlock harder.

Cheers
/Marcus

Join devel@lists.zephyrproject.org to automatically receive all group messages.