RFC: BSD Socket (like) API


Paul Sokolovsky
 

Hello,

Support for BSD Sockets API in Zephyr is one of the frequently asked
features from different parties. Possibility of such support was a topic
of discussions at the recent OpenIoT Summit Portland and Zephyr
Mini-summit Budapest. I'm happy to report that I didn't hear a single
"cons" vote, most people I heard or talked with were positive that they
either interested in it themselves, or it least OK with it if it's
properly layered and doesn't bloat existing networking API.

So, based on that, Linaro Zephyr team would like to proceed with
bootstrapping work on this, collecting initial requirements, and
starting prototyping. I submitted a Jira Epic
https://jira.zephyrproject.org/browse/ZEP-1921 for this feature, which
has a detailed, even if maybe somewhat unstructured discussion of
initial ideas/requirements.

I won't paste it here, instead inviting interested parties to read it
there. Instead, here's a summary of the initial ideas:

1. There's no talk about implementing complete 100% (or 99.9%) POSIX
compliant sockets API. We may get there eventually, but that would
require stakeholders for each expensive or hard to implement feature.
The current approach is that we value lightweight nature of Zephyr, and
looking towards finding a minimal set of changes (additions) to provide
BSD Sockets *like* API to Zephyr.

2. The specific featureset of the initial interest is as follows. The
current Z networking API is push-based, where the OS pushes data into
an application (via a callback). Whereas BSD Sockets API is pull-based,
where an application asks OS for new data to process, when an
application feels like. Implementing this pull-style API is the
initial focus of the effort.

3. The work is going to be guided by porting efforts of an actual
application which needs BSD Sockets API (MicroPython). That would serve
as an initial (out-of-tree) prototype, with relevant code to be
optimized and merged in-tree for wider reuse.


Consequently, questions to the Zephyr community:

1. Are you interested in BSD Sockets like API?
2. Does the plan above sound good? Any important points to take
care of right from the beginning?
3. Or would you do something differently?
4. Any other feedback is appreciated.



Thanks,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog


Sterling Hughes <sterling@...>
 

+1

https://github.com/apache/incubator-mynewt-core/tree/develop/net/ip/mn_socket/include/mn_socket

I’d like to point to Mynewt’s socket APIs as a reference here, a couple of things we considered:

1- Don’t keep POSIX names unless you are going to be POSIX compliant. When running the system simulated, it’s often helpful to use system sockets to communicate and do things, if you use POSIX names you have conflicts.
1a- This also allows you to have a “true” POSIX mapping layer on top, for people who have more memory and truly want socket-level compatibility.

2- FDs just waste memory, add locking and make things harder to debug, use socket structures.

3- Allow for multiple socket providers (https://github.com/apache/incubator-mynewt-core/blob/develop/net/ip/mn_socket/include/mn_socket/mn_socket_ops.h), that way it should be easy to “offload” the IP stack for cases (e.g. WINC1500) where the IP stack is not on-chip, or where somebody wants to use an existing commercial/industrial IP stack.

4- If you are interested in unifying this API with Mynewt, we’d be happy to talk about agreeing on a unified API for sub-embedded sockets.

Best,

Sterling

On 26 Mar 2017, at 9:06, Paul Sokolovsky wrote:

Hello,

Support for BSD Sockets API in Zephyr is one of the frequently asked
features from different parties. Possibility of such support was a topic
of discussions at the recent OpenIoT Summit Portland and Zephyr
Mini-summit Budapest. I'm happy to report that I didn't hear a single
"cons" vote, most people I heard or talked with were positive that they
either interested in it themselves, or it least OK with it if it's
properly layered and doesn't bloat existing networking API.

So, based on that, Linaro Zephyr team would like to proceed with
bootstrapping work on this, collecting initial requirements, and
starting prototyping. I submitted a Jira Epic
https://jira.zephyrproject.org/browse/ZEP-1921 for this feature, which
has a detailed, even if maybe somewhat unstructured discussion of
initial ideas/requirements.

I won't paste it here, instead inviting interested parties to read it
there. Instead, here's a summary of the initial ideas:

1. There's no talk about implementing complete 100% (or 99.9%) POSIX
compliant sockets API. We may get there eventually, but that would
require stakeholders for each expensive or hard to implement feature.
The current approach is that we value lightweight nature of Zephyr, and
looking towards finding a minimal set of changes (additions) to provide
BSD Sockets *like* API to Zephyr.

2. The specific featureset of the initial interest is as follows. The
current Z networking API is push-based, where the OS pushes data into
an application (via a callback). Whereas BSD Sockets API is pull-based,
where an application asks OS for new data to process, when an
application feels like. Implementing this pull-style API is the
initial focus of the effort.

3. The work is going to be guided by porting efforts of an actual
application which needs BSD Sockets API (MicroPython). That would serve
as an initial (out-of-tree) prototype, with relevant code to be
optimized and merged in-tree for wider reuse.


Consequently, questions to the Zephyr community:

1. Are you interested in BSD Sockets like API?
2. Does the plan above sound good? Any important points to take
care of right from the beginning?
3. Or would you do something differently?
4. Any other feedback is appreciated.



Thanks,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


Tomasz Bursztyka
 

Hi Hughes,


https://github.com/apache/incubator-mynewt-core/tree/develop/net/ip/mn_socket/include/mn_socket

I’d like to point to Mynewt’s socket APIs as a reference here, a couple of things we considered:

1- Don’t keep POSIX names unless you are going to be POSIX compliant. When running the system simulated, it’s often helpful to use system sockets to communicate and do things, if you use POSIX names you have conflicts.
1a- This also allows you to have a “true” POSIX mapping layer on top, for people who have more memory and truly want socket-level compatibility.

2- FDs just waste memory, add locking and make things harder to debug, use socket structures.
I don't know Mynewt, but if I want BSD socket API in Zephyr, I would like to see no prefix in front of types and functions.
Sure, it's not going to be 100% Posix compliant, and it's not the point here yes.

But I really want to use socket, bind, listen, as I mostly would in any other OS.

For the FD, I hope we can make any struct pointer to look alike.
Maybe there are limitations however, not sure.


3- Allow for multiple socket providers (https://github.com/apache/incubator-mynewt-core/blob/develop/net/ip/mn_socket/include/mn_socket/mn_socket_ops.h), that way it should be easy to “offload” the IP stack for cases (e.g. WINC1500) where the IP stack is not on-chip, or where somebody wants to use an existing commercial/industrial IP stack.
Offloading is already handled through net_context, so it will seamlessly work already.

Br,

Tomasz


Jukka Rissanen
 

Hi Paul,

On Sun, 2017-03-26 at 19:06 +0300, Paul Sokolovsky wrote:
Hello,

Support for BSD Sockets API in Zephyr is one of the frequently asked
features from different parties. Possibility of such support was a
topic
of discussions at the recent OpenIoT Summit Portland and Zephyr
Mini-summit Budapest. I'm happy to report that I didn't hear a single
"cons" vote, most people I heard or talked with were positive that
they
either interested in it themselves, or it least OK with it if it's
properly layered and doesn't bloat existing networking API.

So, based on that, Linaro Zephyr team would like to proceed with
bootstrapping work on this, collecting initial requirements, and
starting prototyping. I submitted a Jira Epic
https://jira.zephyrproject.org/browse/ZEP-1921 for this feature,
which
has a detailed, even if maybe somewhat unstructured discussion of
initial ideas/requirements.

I won't paste it here, instead inviting interested parties to read it
there. Instead, here's a summary of the initial ideas:

1. There's no talk about implementing complete 100% (or 99.9%) POSIX
compliant sockets API. We may get there eventually, but that would
require stakeholders for each expensive or hard to implement feature.
The current approach is that we value lightweight nature of Zephyr,
and
looking towards finding a minimal set of changes (additions) to
provide
BSD Sockets *like* API to Zephyr.
The definition of what is BSD Socket *like* system seems to differ from
person to person. For me the current net_context API in Zephyr is quite
BSD socket like, meaning that the API provides similar functions that
are found in BSD socket API like open, close, bind, connect, accept
etc. So it is quite easy to port the application in this respect.

The bigger difference between BSD socket API and Zephyr net_context API
is:
* net_context API uses net_buf to pass data. The net_buf does not
provide linear memory but data needs to be partitioned when sending and
read in chunks when receiving. We have helpers defined in nbuf.h for
handling reading/writing data in this case. The issue with linear
memory case is that it uses much more memory as we need to be prepared
to receive at least 1280 byte size chunks of data (IPv6 min data packet
size).

* The net_context is asynchronous and caller needs to have callbacks
defined. The BSD socket API is synchronous. The net_context can be used
in synchronous way so this is a smaller issue imho.

Having a BSD socket API on top of net_context will use more memory so
if one is concerned about memory consumption, then using native API
should be preferred.


2. The specific featureset of the initial interest is as follows. The
current Z networking API is push-based, where the OS pushes data into
an application (via a callback). Whereas BSD Sockets API is pull-
based,
where an application asks OS for new data to process, when an
application feels like. Implementing this pull-style API is the
initial focus of the effort.

3. The work is going to be guided by porting efforts of an actual
application which needs BSD Sockets API (MicroPython). That would
serve
as an initial (out-of-tree) prototype, with relevant code to be
optimized and merged in-tree for wider reuse.


Consequently, questions to the Zephyr community:

1. Are you interested in BSD Sockets like API?
2. Does the plan above sound good? Any important points to take
care of right from the beginning?
3. Or would you do something differently?
4. Any other feedback is appreciated.



Thanks,
Paul

Cheers,
Jukka


Daniel Thompson <daniel.thompson@...>
 

On 27/03/17 08:29, Tomasz Bursztyka wrote:
Hi Hughes,


https://github.com/apache/incubator-mynewt-core/tree/develop/net/ip/mn_socket/include/mn_socket


I’d like to point to Mynewt’s socket APIs as a reference here, a
couple of things we considered:

1- Don’t keep POSIX names unless you are going to be POSIX compliant.
When running the system simulated, it’s often helpful to use system
sockets to communicate and do things, if you use POSIX names you have
conflicts.
1a- This also allows you to have a “true” POSIX mapping layer on
top, for people who have more memory and truly want socket-level
compatibility.

2- FDs just waste memory, add locking and make things harder to debug,
use socket structures.
I don't know Mynewt, but if I want BSD socket API in Zephyr, I would
like to see no prefix in front of types and functions.
Sure, it's not going to be 100% Posix compliant, and it's not the point
here yes.

But I really want to use socket, bind, listen, as I mostly would in any
other OS.
A commonly used pattern for BSD-like interfaces is to have a prefixed API (e.g. net_socket, net_bind) and, introduce a separate mechanism, to alias the name (either linker magic or just #define socket net_socket in a separate header).

The reason for this is to do with the multiple definitions of BSD-like. Making the aliasing *optional* allows an application to provide alternative wrappers if it needs it...


For the FD, I hope we can make any struct pointer to look alike.
Maybe there are limitations however, not sure.
... for example if an application has a big chunk of legacy code scattered liberally with int it might prefer to pay the costs for the fd table (without imposing it on the rest of us).


Daniel



3- Allow for multiple socket providers
(https://github.com/apache/incubator-mynewt-core/blob/develop/net/ip/mn_socket/include/mn_socket/mn_socket_ops.h),
that way it should be easy to “offload” the IP stack for cases (e.g.
WINC1500) where the IP stack is not on-chip, or where somebody wants
to use an existing commercial/industrial IP stack.
Offloading is already handled through net_context, so it will seamlessly
work already.

Br,

Tomasz
_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


Paul Sokolovsky
 

Hello Sterling,

Thanks for the prompt feedback!

On Sun, 26 Mar 2017 09:37:07 -0700
"Sterling Hughes" <sterling@runtime.io> wrote:

+1

https://github.com/apache/incubator-mynewt-core/tree/develop/net/ip/mn_socket/include/mn_socket

I’d like to point to Mynewt’s socket APIs as a reference here, a
couple of things we considered:
Thanks for MyNewt references on the matter, I'll finally have a good
excuse to look into again (the time for which I couldn't find for a
while) ;-).


1- Don’t keep POSIX names unless you are going to be POSIX
compliant. When running the system simulated, it’s often helpful to
use system sockets to communicate and do things, if you use POSIX
names you have conflicts.
Sure, the "BSD Sockets like API" functions would be still
prefixed/namespaced (I actually wanted to bring up a separate
discussion on namespacing of Zephyr APIs). Then if/when it reaches
enough parity that there could be a talk of porting existing apps with
minimal changes, there would be just a header with stuff like:

#define bind net_sock_bind

1a- This also allows you to have a “true” POSIX mapping layer
on top, for people who have more memory and truly want socket-level
compatibility.

2- FDs just waste memory, add locking and make things harder to
debug, use socket structures.
Agree. That was one of the 1st question I got at the minisummit, and my
answer was: "well, we could waste some memory by adding the FD table to
map small integers to underlying structures, but why?". Indeed, by just
casting pointers to integers, we can go a long, long way of using this
API and porting existing apps.

Generally, the work of adding well-known APIs is just a precursor (or
a subset) of implementing much bigger chunk of POSIX functionality and
making Zephyr much more POSIX compliant. So, I leave FDs introduction,
etc. to these later stages of work.

3- Allow for multiple socket providers
(https://github.com/apache/incubator-mynewt-core/blob/develop/net/ip/mn_socket/include/mn_socket/mn_socket_ops.h),
that way it should be easy to “offload” the IP stack for cases (e.g.
WINC1500) where the IP stack is not on-chip, or where somebody wants
to use an existing commercial/industrial IP stack.
Right. Our initial prototyping team here at Linaro consists of Gil
Pitney and me. Gil works on integrating TI CC3200 into Zephyr, and it
has networking offloading at many different levels, and the point is
exactly to consider whether it makes sense to consider offloading on BSD
Sockets API layer, in addition to various layers he already works on.
So, good point, and thanks for the pointer, we'll have a look how
MyNewt does that.

4- If you are interested in unifying this API with Mynewt, we’d be
happy to talk about agreeing on a unified API for sub-embedded
sockets.
I'll definitely have a look and see what commonality can be achieved,
hope Gil will provide feedback too.


Best,

Sterling

On 26 Mar 2017, at 9:06, Paul Sokolovsky wrote:

Hello,

Support for BSD Sockets API in Zephyr is one of the frequently asked
features from different parties. Possibility of such support was a
topic
of discussions at the recent OpenIoT Summit Portland and Zephyr
Mini-summit Budapest. I'm happy to report that I didn't hear a
single "cons" vote, most people I heard or talked with were
positive that they
either interested in it themselves, or it least OK with it if it's
properly layered and doesn't bloat existing networking API.

So, based on that, Linaro Zephyr team would like to proceed with
bootstrapping work on this, collecting initial requirements, and
starting prototyping. I submitted a Jira Epic
https://jira.zephyrproject.org/browse/ZEP-1921 for this feature,
which has a detailed, even if maybe somewhat unstructured
discussion of initial ideas/requirements.

I won't paste it here, instead inviting interested parties to read
it there. Instead, here's a summary of the initial ideas:

1. There's no talk about implementing complete 100% (or 99.9%) POSIX
compliant sockets API. We may get there eventually, but that would
require stakeholders for each expensive or hard to implement
feature. The current approach is that we value lightweight nature
of Zephyr, and
looking towards finding a minimal set of changes (additions) to
provide
BSD Sockets *like* API to Zephyr.

2. The specific featureset of the initial interest is as follows.
The current Z networking API is push-based, where the OS pushes
data into an application (via a callback). Whereas BSD Sockets API
is pull-based,
where an application asks OS for new data to process, when an
application feels like. Implementing this pull-style API is the
initial focus of the effort.

3. The work is going to be guided by porting efforts of an actual
application which needs BSD Sockets API (MicroPython). That would
serve
as an initial (out-of-tree) prototype, with relevant code to be
optimized and merged in-tree for wider reuse.


Consequently, questions to the Zephyr community:

1. Are you interested in BSD Sockets like API?
2. Does the plan above sound good? Any important points to take
care of right from the beginning?
3. Or would you do something differently?
4. Any other feedback is appreciated.



Thanks,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog


Tomasz Bursztyka
 

Hi Daniel,


https://github.com/apache/incubator-mynewt-core/tree/develop/net/ip/mn_socket/include/mn_socket


I’d like to point to Mynewt’s socket APIs as a reference here, a
couple of things we considered:

1- Don’t keep POSIX names unless you are going to be POSIX compliant.
When running the system simulated, it’s often helpful to use system
sockets to communicate and do things, if you use POSIX names you have
conflicts.
1a- This also allows you to have a “true” POSIX mapping layer on
top, for people who have more memory and truly want socket-level
compatibility.

2- FDs just waste memory, add locking and make things harder to debug,
use socket structures.
I don't know Mynewt, but if I want BSD socket API in Zephyr, I would
like to see no prefix in front of types and functions.
Sure, it's not going to be 100% Posix compliant, and it's not the point
here yes.

But I really want to use socket, bind, listen, as I mostly would in any
other OS.
A commonly used pattern for BSD-like interfaces is to have a prefixed API (e.g. net_socket, net_bind) and, introduce a separate mechanism, to alias the name (either linker magic or just #define socket net_socket in a separate header).

The reason for this is to do with the multiple definitions of BSD-like. Making the aliasing *optional* allows an application to provide alternative wrappers if it needs it...
That's the thing: there won't be any other wrappers.
There is Zephyr's net stack with net_context, and there will be BSD socket layer Paul is working on, and that's it.

Tomasz


Daniel Thompson <daniel.thompson@...>
 

On 27/03/17 12:19, Tomasz Bursztyka wrote:
2- FDs just waste memory, add locking and make things harder to debug,
use socket structures.
I don't know Mynewt, but if I want BSD socket API in Zephyr, I would
like to see no prefix in front of types and functions.
Sure, it's not going to be 100% Posix compliant, and it's not the point
here yes.

But I really want to use socket, bind, listen, as I mostly would in any
other OS.
A commonly used pattern for BSD-like interfaces is to have a prefixed
API (e.g. net_socket, net_bind) and, introduce a separate mechanism,
to alias the name (either linker magic or just #define socket
net_socket in a separate header).

The reason for this is to do with the multiple definitions of
BSD-like. Making the aliasing *optional* allows an application to
provide alternative wrappers if it needs it...
That's the thing: there won't be any other wrappers.
There is Zephyr's net stack with net_context, and there will be BSD
socket layer Paul is working on, and that's it.
I was regarding as the mechanism to rename symbols
(net_sock_bind -> bind) to be a form of wrapper around BSD-like sockets, albeit a trivial one.

However providing that renaming is optional it does not preclude the application from providing more advanced wrappers if it needs to. For sure, if the BSD-like sockets follow the advice/prior-art in MyNewt and avoid using FDs, allowing the application to opt-out of that decision probably matters.


Daniel.


Paul Sokolovsky
 

Hello Tomasz,

On Mon, 27 Mar 2017 09:29:31 +0200
Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com> wrote:

Hi Hughes,


https://github.com/apache/incubator-mynewt-core/tree/develop/net/ip/mn_socket/include/mn_socket


I’d like to point to Mynewt’s socket APIs as a reference here, a
couple of things we considered:

1- Don’t keep POSIX names unless you are going to be POSIX
compliant. When running the system simulated, it’s often helpful to
use system sockets to communicate and do things, if you use POSIX
names you have conflicts.
1a- This also allows you to have a “true” POSIX mapping layer
on top, for people who have more memory and truly want socket-level
compatibility.

2- FDs just waste memory, add locking and make things harder to
debug, use socket structures.
I don't know Mynewt, but if I want BSD socket API in Zephyr, I would
like to see no prefix in front of types and functions.
Sure, it's not going to be 100% Posix compliant, and it's not the
point here yes.

But I really want to use socket, bind, listen, as I mostly would in
any other OS.
Please note that these names belong to LIBC namespace, not Zephyr
namespace. So, in our case it's up to Newlib to define them in any it
wants (which is hopefully what we want). I think that good initial
target would be to have a compatibility headers, as I mentioned in
reply to Sterling:

#define socket net_sock_socket
#define listen net_sock_listen
...

(net_sock_ prefix is made and doesn't constitute any specific proposal
- so far).

For the FD, I hope we can make any struct pointer to look alike.
Maybe there are limitations however, not sure.
Right. There may be some limitation, but we should be good to start
with just casting structure pointers to integers and use that as
"socket identifiers". E.g. POSIX (via its publicly available SUSv2
rendition) says:
http://pubs.opengroup.org/onlinepubs/7908799/xns/socket.html "Otherwise
a value of -1 is returned and errno is set to indicate the error.". We
can easily provide that and thus be POSIX compliant.

However, we know that some real-world software likes to test for
negative return value instead:

int fd = socket(...);
if (fd < 0) ...

And indeed, the link above also says "Upon successful completion,
socket() returns a nonnegative integer, the socket file descriptor."
Well, that would be an example of limitation you talk about. But it's
my intuitive impression that majority (well, many) MCUs try to map
RAM/ROM into lower half/quarter of addresspace, so we should be good
here too, subject to actual checking over a long range of various
architectures.


Anyway, I don't want to burden this initial discussion with narrow
issues like above, the point is that there're indeed may be some
limitations to consider, I have some inventory of them in my mind
already, ready to research more, and then will bring up at appropriate
time.




3- Allow for multiple socket providers
(https://github.com/apache/incubator-mynewt-core/blob/develop/net/ip/mn_socket/include/mn_socket/mn_socket_ops.h),
that way it should be easy to “offload” the IP stack for cases
(e.g. WINC1500) where the IP stack is not on-chip, or where
somebody wants to use an existing commercial/industrial IP stack.
Offloading is already handled through net_context, so it will
seamlessly work already.
One of Gil's point was that CC3200 native interface is exactly BSD
Sockets. So, now he needs to convert that API to push-style native
Zephyr API. Then for this work, I will convert Z native push style to
pull style again. That would mean double "impedance conversion" for a
case like CC3200, and an idea to look if it's possible to avoid it. But
leave that topic to Gil, I'm not an offloading specialist.



Br,

Tomasz


--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog


Paul Sokolovsky
 

On Mon, 27 Mar 2017 13:19:16 +0200
Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com> wrote:

[]

That's the thing: there won't be any other wrappers.
There is Zephyr's net stack with net_context, and there will be BSD
socket layer Paul is working on, and that's it.
Nope, then (later, one sweet (or sour) day) there will be a more
general POSIX layer on top of it (and other APIs, threading,
filsystem, console, etc.), and then there will be LIBC on top of it. I'm
not a stakeholder for adding all those layers, but I know that there
are such (and likely will be more as Zephyr gets more adoption), so we
should plan for that.

I think that's positive after all - we don't need to plan how to get
all-the-POSIX compliant sockets right away, at least for now.
Instead, can plan for how to get the most juicy BSD Sockets features
while adding the least code (aka bloat). Then at later time, other folks
can provide their arguments for getting more POSIX compliance (at the
expense of adding more code of course).


Tomasz
--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog


Tomasz Bursztyka
 

Hi Paul,

3- Allow for multiple socket providers
(https://github.com/apache/incubator-mynewt-core/blob/develop/net/ip/mn_socket/include/mn_socket/mn_socket_ops.h),
that way it should be easy to “offload” the IP stack for cases
(e.g. WINC1500) where the IP stack is not on-chip, or where
somebody wants to use an existing commercial/industrial IP stack.
Offloading is already handled through net_context, so it will
seamlessly work already.
One of Gil's point was that CC3200 native interface is exactly BSD
Sockets. So, now he needs to convert that API to push-style native
Zephyr API. Then for this work, I will convert Z native push style to
pull style again. That would mean double "impedance conversion" for a
case like CC3200, and an idea to look if it's possible to avoid it. But
leave that topic to Gil, I'm not an offloading specialist.
The problem with direct offloading - i.e. without net_context etc... - is that it won't integrate
at all properly with systems having multiple network interfaces: ones offering offload, and ones not.
That's why current offload still tightens the "offload device" to a network interface, the core
network stack can handle among other. That will be necessary there will be routing done from
one net-if to another, etc...

The problem of non-linearity of buffer can be solved another way, so I don't see this as a problem here.

In the end, also, it will be way much less work for your layer: it just has to work on top of net_context and
should not care at all about the logic of the network device below.

Br,

Tomasz


Paul Sokolovsky
 

Hello Jukka,

On Mon, 27 Mar 2017 12:37:40 +0300
Jukka Rissanen <jukka.rissanen@linux.intel.com> wrote:

[]
The current approach is that we value lightweight nature of Zephyr,
and
looking towards finding a minimal set of changes (additions) to
provide
BSD Sockets *like* API to Zephyr.
The definition of what is BSD Socket *like* system seems to differ
from person to person.
That's true, and the reason why I informally proposed the "process-wise"
definition above: "minimal set of changes (additions) to provide
BSD Sockets *like* API to Zephyr."

For me the current net_context API in Zephyr
is quite BSD socket like, meaning that the API provides similar
functions that are found in BSD socket API like open, close, bind,
connect, accept etc. So it is quite easy to port the application in
this respect.
That's also true, and I right from the start got a habit to call
net_context "a socket". The API calls you mention are all indeed work
(likely almost) the same. The big difference comes with recv() call -
whereas BSD Sockets API has it conventionally pull-style, in Zephyr
it's push-style, where data gets delivered to an app via a callback.
That's one single feature which makes porting 3rd-party applications
complicated and cumbersome.


The bigger difference between BSD socket API and Zephyr net_context
API is:
* net_context API uses net_buf to pass data. The net_buf does not
provide linear memory but data needs to be partitioned when sending
and read in chunks when receiving. We have helpers defined in nbuf.h
for handling reading/writing data in this case. The issue with linear
memory case is that it uses much more memory as we need to be prepared
to receive at least 1280 byte size chunks of data (IPv6 min data
packet size).
Right. And that part is covered by BSD Sockets' own API - the data is
passed via app-owned buffers, not system-owned buffers. That means that
by definition, BSD Sockets don't support zero-copy operation. While an
obvious drawback, it has its positive sides to, like it offers
possibility for better system vs app separation for security.

* The net_context is asynchronous and caller needs to have callbacks
defined. The BSD socket API is synchronous. The net_context can be
used in synchronous way so this is a smaller issue imho.
As you may already noticed, I prefer to call this distinction
"push-style vs pull-style", because it pinpoints the problem better. The
net_context used "in synchronous way" doesn't provide BSD Sockets
behavior for receives. For that to work, incoming (but unprocessed) data
needs to be queue *per socket*, until an app requests it. And indeed,
that's the one big initial change I would need to make.

Having a BSD socket API on top of net_context will use more memory so
if one is concerned about memory consumption, then using native API
should be preferred.
+100, BSD Sockets like API is not a replacement for native API, only a
helper to port existing applications (mostly libraries in the
real-world cases, I may imagine, but only practice will tell how
people will use it).

[]

--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog


Gil Pitney
 

I think application developers would prefer to see ONE simple,
standard API for networking on which to build all higher level
networking protocols and applications.

However, I understand the Zephyr IP stack needs to target highly
memory constrained systems, and the decision has been made that the
standard POSIX APIs are just not suitable.

So, now the concern is that we are left with two "BSD-like" APIs doing
essentially the same thing, one purporting to use less data space,
though shifting more complexity and code-space up to the application,
and allowing applications to be written one of two (or both?) ways.

So, then, a couple recommendations:

1) Let's start with the standard POSIX APIs:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10

and devolve from there with a list of exceptions with rationale for
why Zephyr cannot or should not comply with the standard.

For example, APIs dealing with file descriptors are reasonable
exceptions, because Zephyr does not support a POSIX file system.

I'd avoid terms like "BSD-like", as the current Zephyr APIs are
arguably "BSD-like" to some degree - just not standard.

2) Enable TCP/IP offload from the socket layer.

The TI CC3220 completely offloads the TCP/IP stack onto a
co-processor, by marshalling BSD socket API calls over SPI to the
network coprocessor.

The current NET_OFFLOAD support in the Zephyr IP stack provides a hook
to call an offload engine. For the TI CC3220, this mapping of
net_context APIs to BSD socket APIs adds some overhead and code
complexity.

However, now that we're talking about adding a BSD socket layer to
Zephyr, offloading directly from the socket layer would be more
natural (something similar to the MyNewt solution, as pointed out by
Sterling).

Otherwise, we have to map BSD sockets -> net_context -> BSD sockets,
with all the required extra overhead of data structures, sync
primitives, and server thread(s) to handle mapping between the two
different networking API usage models.

By doing so, I understand we could potentially bypass the (TBD)
routing table. But that is a use case, AFAIK, not really needed for
the CC3220 typical client IoT node devices.

Then again, the POSIX socket standard specifies that sockets shall
support routing, so maybe we can just make that work somehow?

On 27 March 2017 at 06:27, Paul Sokolovsky <paul.sokolovsky@linaro.org> wrote:
Hello Jukka,

On Mon, 27 Mar 2017 12:37:40 +0300
Jukka Rissanen <jukka.rissanen@linux.intel.com> wrote:

[]
The current approach is that we value lightweight nature of Zephyr,
and
looking towards finding a minimal set of changes (additions) to
provide
BSD Sockets *like* API to Zephyr.
The definition of what is BSD Socket *like* system seems to differ
from person to person.
That's true, and the reason why I informally proposed the "process-wise"
definition above: "minimal set of changes (additions) to provide
BSD Sockets *like* API to Zephyr."

For me the current net_context API in Zephyr
is quite BSD socket like, meaning that the API provides similar
functions that are found in BSD socket API like open, close, bind,
connect, accept etc. So it is quite easy to port the application in
this respect.
That's also true, and I right from the start got a habit to call
net_context "a socket". The API calls you mention are all indeed work
(likely almost) the same. The big difference comes with recv() call -
whereas BSD Sockets API has it conventionally pull-style, in Zephyr
it's push-style, where data gets delivered to an app via a callback.
That's one single feature which makes porting 3rd-party applications
complicated and cumbersome.


The bigger difference between BSD socket API and Zephyr net_context
API is:
* net_context API uses net_buf to pass data. The net_buf does not
provide linear memory but data needs to be partitioned when sending
and read in chunks when receiving. We have helpers defined in nbuf.h
for handling reading/writing data in this case. The issue with linear
memory case is that it uses much more memory as we need to be prepared
to receive at least 1280 byte size chunks of data (IPv6 min data
packet size).
Right. And that part is covered by BSD Sockets' own API - the data is
passed via app-owned buffers, not system-owned buffers. That means that
by definition, BSD Sockets don't support zero-copy operation. While an
obvious drawback, it has its positive sides to, like it offers
possibility for better system vs app separation for security.

* The net_context is asynchronous and caller needs to have callbacks
defined. The BSD socket API is synchronous. The net_context can be
used in synchronous way so this is a smaller issue imho.
As you may already noticed, I prefer to call this distinction
"push-style vs pull-style", because it pinpoints the problem better. The
net_context used "in synchronous way" doesn't provide BSD Sockets
behavior for receives. For that to work, incoming (but unprocessed) data
needs to be queue *per socket*, until an app requests it. And indeed,
that's the one big initial change I would need to make.

Having a BSD socket API on top of net_context will use more memory so
if one is concerned about memory consumption, then using native API
should be preferred.
+100, BSD Sockets like API is not a replacement for native API, only a
helper to port existing applications (mostly libraries in the
real-world cases, I may imagine, but only practice will tell how
people will use it).

[]

--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


Leandro Pereira
 

Paul,

On 03/27/2017 04:16 AM, Paul Sokolovsky wrote:

2- FDs just waste memory, add locking and make things harder to
debug, use socket structures.
Agree. That was one of the 1st question I got at the minisummit, and my
answer was: "well, we could waste some memory by adding the FD table to
map small integers to underlying structures, but why?". Indeed, by just
casting pointers to integers, we can go a long, long way of using this
API and porting existing apps.
Right now we have memory pools, so file descriptors could be really an index to the net_context pool. Since FDs should be treated as opaque identifiers, this should continue to be fine even if things change in the future (such as having multiple mempools for struct net_context).
(We might need a queue per context to properly emulate the read() API, though, so the reference might not be exactly to a net_context pool, but the same idea applies.)

Casting a pointer to an integer as suggested might work, but will most likely be an issue with the common pattern of checking for error by just comparing the sign of the return value, instead of the (technically correct) comparison with -1.

Leandro


Luiz Augusto von Dentz
 

Hi,

On Tue, Mar 28, 2017 at 7:08 PM, Leandro Pereira
<leandro.pereira@intel.com> wrote:
Paul,

On 03/27/2017 04:16 AM, Paul Sokolovsky wrote:

2- FDs just waste memory, add locking and make things harder to
debug, use socket structures.

Agree. That was one of the 1st question I got at the minisummit, and my
answer was: "well, we could waste some memory by adding the FD table to
map small integers to underlying structures, but why?". Indeed, by just
casting pointers to integers, we can go a long, long way of using this
API and porting existing apps.
Right now we have memory pools, so file descriptors could be really an index
to the net_context pool. Since FDs should be treated as opaque identifiers,
this should continue to be fine even if things change in the future (such as
having multiple mempools for struct net_context).
(We might need a queue per context to properly emulate the read() API,
though, so the reference might not be exactly to a net_context pool, but the
same idea applies.)
Indeed we would have to put a queue per net_context which quickly adds
up to the memory footprint, luckily we can use k_poll and avoid having
extra threads for each context. But I guess the fundamental problem is
this is probably intended to interface with real BSD socket/posix
applications/components, which I guess will not be enough for most, so
at the end will it be worth adding this code? What are the things we
want to port on top, can someone present at the mini-summit make a
list?

Casting a pointer to an integer as suggested might work, but will most
likely be an issue with the common pattern of checking for error by just
comparing the sign of the return value, instead of the (technically correct)
comparison with -1.
We can check if there are any paddings on net_context, if there is it
might not be a bad idea to add an id to them, or really make them
compatible with k_poll.

In general Im much more concern about the stack fragmentation, though.
Adding compatibility layers and offloading works against the stack
itself, as they either eat the available memory for enabling features
in the stack or reimplement part of the stack in other layers, not to
mention this takes a lot of time that could be spend in other areas
like security and proper accelerators for heavy tasks like for example
crypto.

--
Luiz Augusto von Dentz


Paul Sokolovsky
 

Hello Gil,

Thanks for the reply, please let me both agree and not agree to some
points.

On Mon, 27 Mar 2017 16:40:30 -0700
Gil Pitney <gil.pitney@linaro.org> wrote:

I think application developers would prefer to see ONE simple,
standard API for networking on which to build all higher level
networking protocols and applications.
I'm on their side, but such application developers will quickly see
that they can't use "ONE simple, standard API" on the smallest of
devices. I guess small footprint and resource usage is a distinctive
trait of Zephyr, and we should not compromise it. So, let there be
layering, and let different application developers choose what they
like/need.

However, I understand the Zephyr IP stack needs to target highly
memory constrained systems, and the decision has been made that the
standard POSIX APIs are just not suitable.
Ack, we in agreement here, per above.

So, now the concern is that we are left with two "BSD-like" APIs doing
essentially the same thing, one purporting to use less data space,
though shifting more complexity and code-space up to the application,
and allowing applications to be written one of two (or both?) ways.
Let me start with the latter - I may imagine a particular application
will want to use either "BSD Socket like API" or "native Zephyr API",
but not both at the same time. If this idea is sound, it may help us
to better structure additions for new API (make them less (least)
intrusive).

Now about terminology - I avoid "BSD-like" term, and specifically call
new API to be developed "BSD Socket like API", that's long but as much
unambiguous as I could come up with. "like" part is to set the
expectations right, so nobody would get an idea that one will be able
to build Chromium against Zephyr 1.8, or something like that ;-). I
wouldn't call current native API "BSD Sockets like API" at all, because
the definitive trait of BSD Sockets is pull-stile API, which native API
lacks.

That all still may be too subtle and too much word-playing, especially
for someone who's not in all of this context, so ideas for better naming
are welcome, but the basic idea is that current API is "native", while
one to be developed is "BSD Sockets like".


So, then, a couple recommendations:

1) Let's start with the standard POSIX APIs:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10

and devolve from there with a list of exceptions with rationale for
why Zephyr cannot or should not comply with the standard.

For example, APIs dealing with file descriptors are reasonable
exceptions, because Zephyr does not support a POSIX file system.
So, you propose top-bottom approach, whereas I explicitly proposed
bottom-top approach: let's start with finding the most divergent
feature from the current native API, let's try to implement it in terms
of existing API, and see what it takes to do that. Then select next
feature, rinse and repeat. IMHO, that's much better for initial
prototyping phase and matches current development model of Zephyr,
where we extensively grow featureset. We certainly will need to pause
at some point and match what we have against spec some time later.

I'd avoid terms like "BSD-like", as the current Zephyr APIs are
arguably "BSD-like" to some degree - just not standard.
Per above, and per my outlook, it's not, and can be as well called
"native". I'll be happy to adopt any other naming, as long as it makes
matters less confusing.


2) Enable TCP/IP offload from the socket layer.

The TI CC3220 completely offloads the TCP/IP stack onto a
co-processor, by marshalling BSD socket API calls over SPI to the
network coprocessor.

The current NET_OFFLOAD support in the Zephyr IP stack provides a hook
to call an offload engine. For the TI CC3220, this mapping of
net_context APIs to BSD socket APIs adds some overhead and code
complexity.

However, now that we're talking about adding a BSD socket layer to
Zephyr, offloading directly from the socket layer would be more
natural (something similar to the MyNewt solution, as pointed out by
Sterling).

Otherwise, we have to map BSD sockets -> net_context -> BSD sockets,
with all the required extra overhead of data structures, sync
primitives, and server thread(s) to handle mapping between the two
different networking API usage models.

By doing so, I understand we could potentially bypass the (TBD)
routing table. But that is a use case, AFAIK, not really needed for
the CC3220 typical client IoT node devices.

Then again, the POSIX socket standard specifies that sockets shall
support routing, so maybe we can just make that work somehow?


On 27 March 2017 at 06:27, Paul Sokolovsky
<paul.sokolovsky@linaro.org> wrote:
Hello Jukka,

On Mon, 27 Mar 2017 12:37:40 +0300
Jukka Rissanen <jukka.rissanen@linux.intel.com> wrote:

[]
The current approach is that we value lightweight nature of
Zephyr, and
looking towards finding a minimal set of changes (additions) to
provide
BSD Sockets *like* API to Zephyr.
The definition of what is BSD Socket *like* system seems to differ
from person to person.
That's true, and the reason why I informally proposed the
"process-wise" definition above: "minimal set of changes
(additions) to provide BSD Sockets *like* API to Zephyr."

For me the current net_context API in Zephyr
is quite BSD socket like, meaning that the API provides similar
functions that are found in BSD socket API like open, close, bind,
connect, accept etc. So it is quite easy to port the application in
this respect.
That's also true, and I right from the start got a habit to call
net_context "a socket". The API calls you mention are all indeed
work (likely almost) the same. The big difference comes with recv()
call - whereas BSD Sockets API has it conventionally pull-style, in
Zephyr it's push-style, where data gets delivered to an app via a
callback. That's one single feature which makes porting 3rd-party
applications complicated and cumbersome.


The bigger difference between BSD socket API and Zephyr net_context
API is:
* net_context API uses net_buf to pass data. The net_buf does not
provide linear memory but data needs to be partitioned when sending
and read in chunks when receiving. We have helpers defined in
nbuf.h for handling reading/writing data in this case. The issue
with linear memory case is that it uses much more memory as we
need to be prepared to receive at least 1280 byte size chunks of
data (IPv6 min data packet size).
Right. And that part is covered by BSD Sockets' own API - the data
is passed via app-owned buffers, not system-owned buffers. That
means that by definition, BSD Sockets don't support zero-copy
operation. While an obvious drawback, it has its positive sides to,
like it offers possibility for better system vs app separation for
security.

* The net_context is asynchronous and caller needs to have
callbacks defined. The BSD socket API is synchronous. The
net_context can be used in synchronous way so this is a smaller
issue imho.
As you may already noticed, I prefer to call this distinction
"push-style vs pull-style", because it pinpoints the problem
better. The net_context used "in synchronous way" doesn't provide
BSD Sockets behavior for receives. For that to work, incoming (but
unprocessed) data needs to be queue *per socket*, until an app
requests it. And indeed, that's the one big initial change I would
need to make.

Having a BSD socket API on top of net_context will use more memory
so if one is concerned about memory consumption, then using native
API should be preferred.
+100, BSD Sockets like API is not a replacement for native API,
only a helper to port existing applications (mostly libraries in the
real-world cases, I may imagine, but only practice will tell how
people will use it).

[]

--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog


Paul Sokolovsky
 

Hello Leandro,

On Tue, 28 Mar 2017 09:08:51 -0700
Leandro Pereira <leandro.pereira@intel.com> wrote:

Paul,

On 03/27/2017 04:16 AM, Paul Sokolovsky wrote:

2- FDs just waste memory, add locking and make things harder to
debug, use socket structures.
Agree. That was one of the 1st question I got at the minisummit,
and my answer was: "well, we could waste some memory by adding the
FD table to map small integers to underlying structures, but why?".
Indeed, by just casting pointers to integers, we can go a long,
long way of using this API and porting existing apps.
Right now we have memory pools, so file descriptors could be really
an index to the net_context pool. Since FDs should be treated as
opaque identifiers, this should continue to be fine even if things
change in the future (such as having multiple mempools for struct
net_context).
Sounds like an interesting idea, thanks for sharing, I'll keep it in
mind and will look into it at later stage.

(We might need a queue per context to properly emulate
the read() API, though, so the reference might not be exactly to a
net_context pool, but the same idea applies.)
Right, that's exactly what I'm working on right now. Well, I keep this
per-context queue external so far, but later we'll need to see whether
we want to introduce a separate "socket" object type to host it (and
any other needed fields), or if we can (configurably!) embed them in
net_context. If we agree with the idea that a particular app will use
either native API, or BSD Sockets API, I think this latter choice is
very viable.

Casting a pointer to an integer as suggested might work, but will
most likely be an issue with the common pattern of checking for error
by just comparing the sign of the return value, instead of the
(technically correct) comparison with -1.
Yep, we'd need to look at the corpus of apps to be ported to Zephyr to
see how grave that issue is. That may take some time (to collect needed
set), though if you have any related data/ideas already, please share
them.


Leandro
[]

--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog


Paul Sokolovsky
 

Hello Luiz Augusto,

On Tue, 28 Mar 2017 23:34:25 +0300
Luiz Augusto von Dentz <luiz.dentz@gmail.com> wrote:

[]

net_context). (We might need a queue per context to properly
emulate the read() API, though, so the reference might not be
exactly to a net_context pool, but the same idea applies.)
Indeed we would have to put a queue per net_context which quickly adds
up to the memory footprint,
As I point in the reply to Leandro, these additions would be #ifdef'ed
on CONFIG_NET_BSD_SOCKETS or something, so application developers will
have a choice whether save memory or have more familiar API. Putting
the queue, etc. into yet different structure (and waste some memory to
link this new structure to net_context) is yet another choice.

luckily we can use k_poll and avoid having
extra threads for each context.
Note that the work I'm going to do is based on the previous experience
with implement BSD-Sockets like pull-style API on top of push-style
native ("raw") lwIP API. There, we were able to achieve that without
any threads (indeed, even with a cooperative RTOS underneath). So, no
planning of having extra threads on the initial stage of work
(push-to-pull "impedance conversion"). And presence of k_poll makes me
positive that we'll be able to implement even poll() or epoll() without
them (but that's 2nd stage of work).

But I guess the fundamental problem is
this is probably intended to interface with real BSD socket/posix
applications/components, which I guess will not be enough for most, so
at the end will it be worth adding this code? What are the things we
want to port on top, can someone present at the mini-summit make a
list?
As my initial RFC message pointed, the initial target is MicroPython
with its "usocket" (micro-socket) module. That's a single app, but it
wraps a large chunk of BSD Sockets API for Python language. And of
course, this RFC is to see what other projects people are interested to
port.

Casting a pointer to an integer as suggested might work, but will
most likely be an issue with the common pattern of checking for
error by just comparing the sign of the return value, instead of
the (technically correct) comparison with -1.
We can check if there are any paddings on net_context, if there is it
might not be a bad idea to add an id to them, or really make them
compatible with k_poll.

In general Im much more concern about the stack fragmentation, though.
Adding compatibility layers and offloading works against the stack
itself,
Well, I don't see how proper, configurable (on/off) layering works
against the stack. I'd say, vice-versa, it allows to leverage it for
more usecases. Offloading, let me skip that part ;-).

as they either eat the available memory for enabling features
in the stack
Let users select whether they want to save memory or development
effort by using a conventional API ;-).

or reimplement part of the stack in other layers,
Well, here I, as present at the mini-summit, can quote what Anas said
on this: he literally said - don't do that. If "higher level wrapper"
needs to reimplement or otherwise put upside-down what native layer
does, let's just rework native layer to be (more) POSIX/etc.
compatible. And well, that's pretty much how we already do it, with
several features for 1.7 done to make it closer to POSIX spirit, and
there're more such things in queue (here's the latest merged:
https://gerrit.zephyrproject.org/r/12455). So, no worries, or rather,
we all are aware of the problem, and ready to do homework on it.

not to mention this takes a lot of time that could be spend in other
areas like security and proper accelerators for heavy tasks like for
example crypto.
As a maintainer in other projects, I can easily relate to that thought
(that I, a maintainer, know better what contributors rather be working
on ;-) ). But here's how it works at Linaro: our aim is to reduce/avoid
fragmentation among ARM vendors (and not at the expense of more
fragmentation with other architectures). So if a member comes to us and
says "we'll use Zephyr if ...", we'd better listen, because there's no
lack of alternatives to Zephyr, and if interested parties select
something else instead, there won't be anything good from that to
Zephyr, or to the industry at all, which will continue in a chaos of
several dozens of adhoc (vs full-featured) RTOSes. And yeah, BSD
Sockets is a frequent request we heard for a while. Other things you
mention are in the plans (ours including) too.


--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog


Daniel Thompson <daniel.thompson@...>
 

On 28/03/17 22:26, Paul Sokolovsky wrote:
So, now the concern is that we are left with two "BSD-like" APIs doing
essentially the same thing, one purporting to use less data space,
though shifting more complexity and code-space up to the application,
and allowing applications to be written one of two (or both?) ways.
Let me start with the latter - I may imagine a particular application
will want to use either "BSD Socket like API" or "native Zephyr API",
but not both at the same time. If this idea is sound, it may help us
to better structure additions for new API (make them less (least)
intrusive).
I'm afraid I *don't* think this idea is sound!

The very reason we want BSD(-like) sockets is because we are importing a big pile of third party library code into our application.

Importing third-party library code must not impose additional requirements on application code (or *other* third party library code) or integration will be a nightmare.


Daniel.


Marcus Shawcroft <marcus.shawcroft@...>
 

On 27 March 2017 at 12:16, Paul Sokolovsky <paul.sokolovsky@linaro.org> wrote:

2- FDs just waste memory, add locking and make things harder to
debug, use socket structures.
Agree. That was one of the 1st question I got at the minisummit, and my
answer was: "well, we could waste some memory by adding the FD table to
map small integers to underlying structures, but why?". Indeed, by just
casting pointers to integers, we can go a long, long way of using this
API and porting existing apps.
Casting pointers to and from integers is legal, but implementation
defined (c99 6.3.2.3 p5 p6). We should avoid implementation defined
behaviour in the language where possible, especially in public APIs.

/Marcus