Re: More elaborate k_fifo API?


Paul Sokolovsky
 

Hello Johan,

On Sat, 1 Apr 2017 15:44:43 +0300
Johan Hedberg <johan.hedberg@intel.com> wrote:

Hi Paul,

On Sat, Apr 01, 2017, Paul Sokolovsky wrote:
Motivation: BSD Sockets API work.

I already have code like:

+ // TODO: k_fifo accessor for this?
+ struct net_buf *last_buf = (struct
net_buf*)sys_slist_peek_tail(&socket->recv_q.data_q);

to peek tail element of the fifo.

And I get a next net_buf and store it in a pointer within a socket
structure, but instead I could just peek a head element - as long as
fifo would allow me to wait until it becomes non-empty (without
extracting first element, I'll peek it afterwards). Granted, this
can be done with adhoc code like peeking tail, but becomes more and
more fragile.
If your only use case is to wait until a k_fifo becomes non-empty
without actually getting an element from there, that's possible
already now using k_poll. It'll signal that there's data in the FIFO
but require you to separately call k_fifo_get to get the element.
Thanks for the hint, that seems to work!

Note that with net_buf you must always use net_buf_get/put and never
the k_fifo_get/put APIs directly to prevent corruption of the
fragment lists.
Yeah, so what happens is that net_pkt's (hopefully still a new name for
net_nbuf) gets extracted from global rx queue, gets fragment chain
attached, and passed to socket receive callback, just to be queued again
to a per-socket queue. Yes, that would smash the main fragment pointer,
so I already had to look for a random place in net_pkt to save it.
Fortunately, the place found wasn't that much of random - the user
token:
https://github.com/micropython/micropython/blob/master/zephyr/modusocket.c#L106

But with the work which prompted this email, it gets much worse. The
packet keeps being in fifo, until its data is fully processed. So, I
need to maintain duality of a packet both being in fifo and having a
fragment chain. That already sounds ugly, but I actually spent last
hour trying to get it all to work, almost giving up every 5 mins. All
the possibility of that at all depends on the knowledge that appending a
new packet to a non-empty fifo requires accessing just the tail
pointer (so head can be "hosed", i.e. pointing to fragments instead
of fifo elements). And then, as the processing loop is pretty
involved, I need to save-restore-set-reset ->frags pointer multiple
times depending on what function to be called next (in a loop, with few
branches). Whoa!

Fairly speaking, even after extensive discussion at the mini-summit, I
don't understand how comes that net_pkt object gets with a lot of
effort disguised as being a net_buf object, whereas they're clearly
wildly different types of objects, forming clear hierarchy in
relationship. The only explanation is desire to have fewer pools of
different element sizes, but I'd say it's already clear that the
current design isn't saving of memory, while makes everything *so*
convoluted...


The current evolution of TCP recv handling code can be seen at
https://github.com/pfalcon/micropython/commits/zephyr-socket (UDP is
already in MicroPython mainline).


Johan


--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

Join devel@lists.zephyrproject.org to automatically receive all group messages.