Re: RFC: BSD Socket (like) API


Paul Sokolovsky
 

Hello Luiz Augusto,

On Wed, 29 Mar 2017 14:18:04 +0300
Luiz Augusto von Dentz <luiz.dentz@gmail.com> wrote:

[]

As my initial RFC message pointed, the initial target is MicroPython
with its "usocket" (micro-socket) module. That's a single app, but
it wraps a large chunk of BSD Sockets API for Python language. And
of course, this RFC is to see what other projects people are
interested to port.
I suppose you need much more than sockets to make a python program to
work on top of zephyr.
For an existing Python program, right. I made a presentation at the
last OpenIoT Summit on the topic of usage of scripting languages in
(deeply embedded) IoT, and the argument was that most useful apps in
scripting languages (whatever one is used) would need to be written
from scratch, with attention to resource constraints to the target
platform.

Now I do agree having python would be very
useful to prototype, or js, but for products it might be a different
story.
I appreciate agreeing to this conclusion too. Indeed scripting
languages (JS, Python) can already deliver in the areas of rapid
testing/rapid prototyping. Whether they will be useful for developing
"production" apps is yet to be see.

Anyway, we're diverging from the topic of the discussion, how
MicroPython pops up here is that it needs a pretty complete BSD Sockets
API, so should be a good testcase for the work to do (i.e. it's not
limited to just IPv4, or UDP, or client-only API).

Casting a pointer to an integer as suggested might work, but will
most likely be an issue with the common pattern of checking for
error by just comparing the sign of the return value, instead of
the (technically correct) comparison with -1.
We can check if there are any paddings on net_context, if there is
it might not be a bad idea to add an id to them, or really make
them compatible with k_poll.

In general Im much more concern about the stack fragmentation,
though. Adding compatibility layers and offloading works against
the stack itself,
Well, I don't see how proper, configurable (on/off) layering works
against the stack. I'd say, vice-versa, it allows to leverage it for
more usecases. Offloading, let me skip that part ;-).
I was referring to APIs like net_buf or application layer protocols
like coap, while I agree we can make all configurable that doesn't
dismiss the fact these interfaces might be reimplemented on top of the
socket layer.
Yes, that's one of "paradoxes" or "logic reversals" of architecting
this things. We provide native (or natively ported) MQTT, OCF, etc.
high-level interfaces and think it'll cover all the users' needs, but
any conservative assessment based on history of the industry will tell
it won't work. That's because all those things are niche novelties
comparing with something such established, well-known, and generic as
BSD Sockets. So, it's better not to fight this tendency, but plan and
layer it well, then document clearly (that e.g. native protocol
support is much more efficient), and let users choose what they want to
do.

In fact the case of net_buf there is probably no
solution since the BSD Socket interface will most likely be using
plain pointers or iovec, so we lose zero copy and possible add yet
another buffer pool on top.
Yes, in one of the earlier emails I mentioned that we lose zero copy
with BSD Sockets API. But at the same time we gain memory separation of
system net buffers vs application memory, which may be just what we
need for network stack security in Zephyr.

[]

As a maintainer in other projects, I can easily relate to that
thought (that I, a maintainer, know better what contributors rather
be working on ;-) ). But here's how it works at Linaro: our aim is
to reduce/avoid fragmentation among ARM vendors (and not at the
expense of more fragmentation with other architectures). So if a
member comes to us and says "we'll use Zephyr if ...", we'd better
listen, because there's no lack of alternatives to Zephyr, and if
interested parties select something else instead, there won't be
anything good from that to Zephyr, or to the industry at all, which
will continue in a chaos of several dozens of adhoc (vs
full-featured) RTOSes. And yeah, BSD Sockets is a frequent request
we heard for a while. Other things you mention are in the plans
(ours including) too.
That is a fair point, but ultimately the memory constraint comes from
the very same vendors. So are there new boards coming with more
memory? 128 K or more ram and a fair bit more flash? Because I can
tell you in most boards memory is already quite tight and that is
without debug enable,
Well, so my account of that is that we should stick to the original
Zephyr promise of being able to support even the smallest
configuration, while adding proper layering and configurability on top
of that. And maintaining and even further optimizing this "the most
minimal configuration", as a concurrent mailing list thread suggests,
should be among top priorities.

in fact echo server don't even work with qemu
when all debugs options are enabled:

/opt/zephyr-sdk/sysroots/x86_64-pokysdk-linux/usr/libexec/i586-zephyr-elf/gcc/i586-zephyr-elf/6.2.0/real-ld:
zephyr_prebuilt.elf section `noinit' will not fit in region `RAM'
/opt/zephyr-sdk/sysroots/x86_64-pokysdk-linux/usr/libexec/i586-zephyr-elf/gcc/i586-zephyr-elf/6.2.0/real-ld:
region `RAM' overflowed by 5520 bytes
Well, we need to bump qemu_x86 RAM size (as it's used to host the .text
largely) now, and optimize "non-minimal" configurations like debug
build, TLS, etc. later (later, in contrast to the "most minimal" config,
which needs to be optimized and maintained every day, or we will lose
it).


--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

Join devel@lists.zephyrproject.org to automatically receive all group messages.