Re: BSD Sockets in mainline, and how that affects design decisions for the rest of IP stack (e.g. send MTU handling)


Paul Sokolovsky
 

Hello Andrew,

On Wed, 11 Oct 2017 18:53:24 +0000
"Boie, Andrew P" <andrew.p.boie@intel.com> wrote:

There has not been any public talk in mailing list about
userspace/kernel separation and how it affects IP stack etc. so
it is a bit difficult to say anything about this.
That's true, but we can/should think how it may be affected, or
we'll be caught in the cold water with them, and may come up with
random on-spot designs to address such future requirements, instead
of something well thought out.
The userspace work has progressed to the point where we have enough
confidence in the API design to open up the design discussion to a
larger audience; until now enough things have been in flux (or
uncertain) such that we've kept the discussion to the working group
we established for it.

What we are trying to do is get something feature-complete into the
tree for the upcoming 1.10 release, with partial test case coverage
and initial documentation, on an 'experimental' basis; i.e. APIs and
policies are subject to change. Then polish everything up for the
1.11 release, which would be the official debut of this feature.

I have to admit my knowledge of the network stack is quite poor, but
broadly speaking here are a set of slides recently drafted which goes
into some detail about what sort of kernel objects are accessible
from user threads and the sort of restrictions they have. We expect
to expose a subset of existing kernel APIs to user threads, and all
driver APIs which don't involve registration of callbacks. Please
feel free to leave comments in the document, or on this list.

https://docs.google.com/presentation/d/195ciwFwv7s0MX4AvL0KFB1iHm1_gRoXmn54mjS5fki8/edit?usp=sharing
Thanks for sharing these. What caught my attention is "No good way to
assert validity of k_mem_block object passed to k_mem_pool_free() - Just
tell userspace to use heap memory APIs in newlib! No need to re-invent
the C library..."

That's pretty much what I talked about - that with new requirements and
challenges, we may find out that a well-known and proven API like BSD
Sockets is a very good way to address them, instead of continuing to
add complexity to existing adhoc APIs.


I suspect the biggest implication for the network stack is that it
uses registration of callbacks heavily, and it's forbidden to allow
user threads to directly register callbacks that run in supervisor
mode. But you can get around this (for example) by having the
callback do minimal processing of the incoming data and signal a
semaphore to have a user mode worker thread do the rest of the work.
That's half of the work BSD Sockets do - they put network packets as
delivered via a callback into per-socket fifo.

We are also looking into supporting user-mode workqueues. We also
don't (yet) have a clear picture on what support for k_poll APIs we
will have for userspace.

There's also the question of memory buffers, there would need to be
some care taken that any buffers used by the stack that are exposed
to the application contain purely data and no internal data
structures private to the kernel. This constraint is why we don't
provide system call interfaces to k_queue APIs.
net_pkt's and net_buf's as used by native networking API do share this
problem - they have internal kernel data. Not only that, they are also
allocated from the pool, and are small objects, which can't be
protected by MPU or MMU individually. Which means that one application
could have access/corrupt networking data for other apps.

And above you write about protecting kernel from userspace, but is
there a requirement to protect one userspace entity (a thread in our
case, as we don't support processes) from another? I hope there's,
because it doesn't make much sense to go so long way of kernel vs
userspace separation and don't think about task separation. Just
imagine that the could be a thread running OTA, and another thread
running an application level 3rd-party lib. We don't want
vulnerability in the latter to compromise OTA process.

The solution to the problem is well known - don't try to export
kernel-level object (like network buffers) to userspace, just copy
*data* there as needed. That's 2nd part of what BSD Sockets do.

Ideally in the fullness of time, we could migrate some parts of the
network protocol stack to run in user mode, which I think would
enhance the security of the system.

At the moment, current implementation effort is centered around
getting our test cases running in user mode, and getting started on
the formal documentation.

HTH,
Andrew


--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

Join devel@lists.zephyrproject.org to automatically receive all group messages.