BSD Sockets like API: prototyping report #3


Paul Sokolovsky
 

Hello,

This report is quite overdue, and I hoped to have sent it weeks ago, as
the initial implementation of all the basic functions were done long
ago. But I wanted to back it with automated easily reproducible
testing, as well as test against real-world sites or well-known
networking servers with real data sizes (like an odd gigabyte) - and
both aims turned out to be elusive, requiring excursions in unrelated
areas (like UART handling in Zephyr, still ongoing) and many more
investigation and fixes.

Well, now things more or less got together:

1. There's implementation of the following function (with Python
interface, which would still require some refactoring to turn them into
C ones): socket(), bind(), connect(), listen(), accept(), recv(),
send(), close(), getaddrinfo(). Only blocking sockets are supported so
far.

2. These are implemented in "usocket" module of MicroPython, in
"zephyr-socket2" branch:
https://github.com/pfalcon/micropython/blob/zephyr-socket2/zephyr/modusocket.c
To work as expected, this requires master with some additional patches
applied: https://github.com/pfalcon/zephyr/tree/master-micropython .

3. It was tested in following ways:

a) There few (4) automated tests, for basic UDP functionality.
Unfortunately, TCP tests don't work in this manner,
https://jira.zephyrproject.org/browse/ZEP-2001.

b) My "ultimate" test was supposed to be downloading large files (tens
of MBs) from http://archive.ubuntu.com/, checking their hash (I'm using
that site as it readily offers hash values for different algorithms),
then repeating to collect enough traffic (ultimately mentioned
gigabyte). Unfortunately, I didn't have much luck with this "ambitious"
goal. Overall, I got 100% TCP hang on byte counts ranging from
100Kbs to ~10MB. A raw TCP connection hanging is an expected situation
in general (in real-world apps, that's countered by having connection
timeout), but hitting that in 100% of cases would mean that something
not yet fully right with Zephyr stack handling of non-successful
conditions (like retransmits, backoffs, etc.).

c) So, I had to scale down to use my local Apache server. There
everything went much better, and I was able to download ~300MB at the
first try (sorry, keep forgetting to leave it overnight for the
proverbial gigibyte). So, overall, "success" flow is ok both on Zephyr
side and on BSD prototype side, there're no buffer leaks, etc. (Again,
"non-success" cases where there's need for some recovery and retries
is a different story). This concludes (initial) client-side testing.

d) Trying server-side handling, I immediately hit an issue with
(case-dependent) FIN handling:
https://jira.zephyrproject.org/browse/ZEP-2104. Working that around
(proper fix yet needs investigation), I'm happy to report that a simple
HTTP server sample sustains 1000 requests from ApacheBench (ab -n 1000
http://192.0.2.1:8080/).



Further plans:

1. There are some optimizations to do yet on the current code, before
separating socket functions from MicroPython boilerplate.

2. It's clear that in-tree patches won't be ready before 1.8 release
freeze, so instead I plan to concentrate on trying to flush any
smaller changes which may be useful for mainline already, and on
investigating identified issues.




--
Best Regards,
Paul

Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

Join devel@lists.zephyrproject.org to automatically receive all group messages.