Signed-off-by: Jukka Rissanen <jukka.rissanen(a)linux.intel.com>
---
Hi,

current network application APIs in Zephyr are synchronous and a
bit of awkward to use in TCP. Because of these things, I propose
couple of new functions that are asynchronous and easier to use
by the application. These APIs require net_buf fragmentation support
that I sent earlier.

Cheers,
Jukka


include/net/net_socket.h | 141 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)

diff --git a/include/net/net_socket.h b/include/net/net_socket.h
index 88073b1..2451de3 100644
--- a/include/net/net_socket.h
+++ b/include/net/net_socket.h
@@ -143,6 +143,147 @@ int net_reply(struct net_context *context, struct net_buf *buf);
struct simple_udp_connection *
net_context_get_udp_connection(struct net_context *context);

+/**
+ * @brief Callback that is called when there is something received from
+ * the network.
+ *
+ * @details It is callback responsibility to release the net_buf when the
+ * data is no longer needed. The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error
+ * @param iov List of net_buf that contain data
+ * @param user_data User supplied data
+ */
+typedef void (*net_readv_cb_t)(struct net_context *context,
+ int status,
+ struct net_buf *iov,
+ void *user_data);
+
+/**
+ * @brief Register a receiver to get data from network.
+ *
+ * @details Application uses this to get data from network connection.
+ * Caller needs to specify a callback function that is called when data
+ * is received by the socket. When the IP stack has received data, it then
+ * calls user specified callback and it is callback responsibility to
+ * release the net_buf when the data is no longer needed.
+ * This function will return immediately as it only sets a callback to be
+ * called when new data is received. Application can remove the registered
+ * callback by calling the function with same context and setting cb to NULL.
+ *
+ * @param context Network context
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_readv(struct net_context *context,
+ net_readv_cb_t cb,
+ void *user_data);
+
+/**
+ * @brief Callback that is called when user can send
+ * data to the network.
+ *
+ * @details The first call to this function will set *status to 0 and
+ * bytes_sent to 0. Application can then prepare the data to be sent
+ * in net_buf fragment list. The data to be sent is returned to the
+ * caller of the function. If application does not wish to send anything
+ * it can just return NULL. In this case the caller will deallocate the
+ * iov list. Application should set the *status to <0 to indicate more
+ * detailed reason for the error if NULL is returned.
+ *
+ * For successfully sent UDP data, the IP stack will call this
+ * callback again with status set to 0 and bytes_sent telling how many bytes
+ * were sent. If the UDP data was not sent for some reason, then *status
+ * will have value < 0 and bytes_sent is set to 0.
+ *
+ * For TCP, the callback is called when the network connection has been
+ * established. In this case the *status is 0 and bytes_sent is 0.
+ * If there is a connection timeout, the status code will be -ETIMEDOUT
+ * and bytes_sent is set to 0. Other error codes are also possible in
+ * which case the *status < 0 and bytes_sent will tell how many bytes were
+ * successfully sent. If the *status is set to 0 then bytes_sent will tell
+ * how many bytes were sent to the peer.
+ *
+ * The iov fragment list does not contain data that has been successfully
+ * sent. Also the iov->frags->data of the first data fragment will point to
+ * first byte that has not yet been sent. If all the data was sent
+ * successfully, then the first item of iov list will have its frags pointer
+ * set to NULL. Application can send more data if it wishes by returning
+ * a new list of data fragments.
+ *
+ * The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error in stack side
+ * Application can return error code if needed via this pointer.
+ * @param bytes_sent How many bytes were sent.
+ * @param iov List of net_buf that contain data. If this is NULL, then
+ * allocate the buf and fill it with data. If non-NULL, then the protocol
+ * headers are already there and you can append the data.
+ * @param user_data User supplied data
+ *
+ * @return A valid net_buf that needs to be sent,
+ * NULL if user is not able to send anything.
+ */
+typedef struct net_buf *(*net_writev_cb_t)(struct net_context *context,
+ int *status,
+ int bytes_sent,
+ struct net_buf *iov,
+ void *user_data);
+
+/**
+ * @brief Send data to network.
+ *
+ * @details Application uses this to send data to a network connection.
+ * Caller needs to specify a callback function that is called when data
+ * is ready to be sent. The function will return immediately, the timeout
+ * is only used when calling the user callback. For UDP, the timeout is
+ * ignored. For TCP the callback is called after the TCP connection is
+ * established and user is able to send data, or if there is an error
+ * creating a connection or if the connection timeouts.
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This is only
+ * used in TCP.
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_writev(struct net_context *context,
+ int32_t timeout,
+ net_writev_cb_t cb,
+ void *user_data);
+
+/**
+ * @brief Reply data to sender.
+ *
+ * @details This is a helper that will help user to reply data to the
+ * sender. Application can use this function to reply data it received
+ * via net_readv().
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This is only
+ * used in TCP.
+ * @param iov Data to be sent
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_replyv(struct net_context *context,
+ int32_t timeout,
+ struct net_buf *iov,
+ net_writev_cb_t cb,
+ void *user_data);
+
#ifdef __cplusplus
}
#endif
--
2.5.5

Hi Jukka,

On Wed, Apr 27, 2016 at 11:59 AM, Jukka Rissanen
<jukka.rissanen(a)linux.intel.com> wrote:

Signed-off-by: Jukka Rissanen <jukka.rissanen(a)linux.intel.com>
---
Hi,

current network application APIs in Zephyr are synchronous and a
bit of awkward to use in TCP. Because of these things, I propose
couple of new functions that are asynchronous and easier to use
by the application. These APIs require net_buf fragmentation support
that I sent earlier.

Cheers,
Jukka


include/net/net_socket.h | 141 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)

diff --git a/include/net/net_socket.h b/include/net/net_socket.h
index 88073b1..2451de3 100644
--- a/include/net/net_socket.h
+++ b/include/net/net_socket.h

Perhaps it should be name io instead of socket since this is not a BSD
socket API, and I would drop the net_ prefix as that is already
implicit from the parent directory. Or perhaps have the API using
socket version e.g: send*/recv*, but Im afraid a familiar socket API
would require queuing and some way of polling the context so I don't
know if it is doable.

@@ -143,6 +143,147 @@ int net_reply(struct net_context *context, struct net_buf *buf);
struct simple_udp_connection *
net_context_get_udp_connection(struct net_context *context);

+/**
+ * @brief Callback that is called when there is something received from
+ * the network.
+ *
+ * @details It is callback responsibility to release the net_buf when the
+ * data is no longer needed. The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error
+ * @param iov List of net_buf that contain data
+ * @param user_data User supplied data
+ */
+typedef void (*net_readv_cb_t)(struct net_context *context,
+ int status,
+ struct net_buf *iov,
+ void *user_data);
+
+/**
+ * @brief Register a receiver to get data from network.
+ *
+ * @details Application uses this to get data from network connection.
+ * Caller needs to specify a callback function that is called when data
+ * is received by the socket. When the IP stack has received data, it then
+ * calls user specified callback and it is callback responsibility to
+ * release the net_buf when the data is no longer needed.
+ * This function will return immediately as it only sets a callback to be
+ * called when new data is received. Application can remove the registered
+ * callback by calling the function with same context and setting cb to NULL.
+ *
+ * @param context Network context
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_readv(struct net_context *context,
+ net_readv_cb_t cb,
+ void *user_data);

+/**
+ * @brief Callback that is called when user can send
+ * data to the network.
+ *
+ * @details The first call to this function will set *status to 0 and
+ * bytes_sent to 0. Application can then prepare the data to be sent
+ * in net_buf fragment list. The data to be sent is returned to the
+ * caller of the function. If application does not wish to send anything
+ * it can just return NULL. In this case the caller will deallocate the
+ * iov list. Application should set the *status to <0 to indicate more
+ * detailed reason for the error if NULL is returned.
+ *
+ * For successfully sent UDP data, the IP stack will call this
+ * callback again with status set to 0 and bytes_sent telling how many bytes
+ * were sent. If the UDP data was not sent for some reason, then *status
+ * will have value < 0 and bytes_sent is set to 0.
+ *
+ * For TCP, the callback is called when the network connection has been
+ * established. In this case the *status is 0 and bytes_sent is 0.
+ * If there is a connection timeout, the status code will be -ETIMEDOUT
+ * and bytes_sent is set to 0. Other error codes are also possible in
+ * which case the *status < 0 and bytes_sent will tell how many bytes were
+ * successfully sent. If the *status is set to 0 then bytes_sent will tell
+ * how many bytes were sent to the peer.
+ *
+ * The iov fragment list does not contain data that has been successfully
+ * sent. Also the iov->frags->data of the first data fragment will point to
+ * first byte that has not yet been sent. If all the data was sent
+ * successfully, then the first item of iov list will have its frags pointer
+ * set to NULL. Application can send more data if it wishes by returning
+ * a new list of data fragments.
+ *
+ * The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error in stack side
+ * Application can return error code if needed via this pointer.
+ * @param bytes_sent How many bytes were sent.
+ * @param iov List of net_buf that contain data. If this is NULL, then
+ * allocate the buf and fill it with data. If non-NULL, then the protocol
+ * headers are already there and you can append the data.
+ * @param user_data User supplied data
+ *
+ * @return A valid net_buf that needs to be sent,
+ * NULL if user is not able to send anything.
+ */
+typedef struct net_buf *(*net_writev_cb_t)(struct net_context *context,
+ int *status,
+ int bytes_sent,
+ struct net_buf *iov,
+ void *user_data);
+
+/**
+ * @brief Send data to network.
+ *
+ * @details Application uses this to send data to a network connection.
+ * Caller needs to specify a callback function that is called when data
+ * is ready to be sent. The function will return immediately, the timeout
+ * is only used when calling the user callback. For UDP, the timeout is
+ * ignored. For TCP the callback is called after the TCP connection is
+ * established and user is able to send data, or if there is an error
+ * creating a connection or if the connection timeouts.
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This is only
+ * used in TCP.
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_writev(struct net_context *context,
+ int32_t timeout,
+ net_writev_cb_t cb,
+ void *user_data);
+
+/**
+ * @brief Reply data to sender.
+ *
+ * @details This is a helper that will help user to reply data to the
+ * sender. Application can use this function to reply data it received
+ * via net_readv().
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This is only
+ * used in TCP.
+ * @param iov Data to be sent
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_replyv(struct net_context *context,
+ int32_t timeout,
+ struct net_buf *iov,
+ net_writev_cb_t cb,
+ void *user_data);
+
#ifdef __cplusplus
}
#endif
--
2.5.5

--
Luiz Augusto von Dentz

On Wed, 27 Apr 2016 11:59:50 +0300, Jukka Rissanen wrote:

Signed-off-by: Jukka Rissanen <jukka.rissanen(a)linux.intel.com>
---
Hi,

current network application APIs in Zephyr are synchronous and a
bit of awkward to use in TCP. Because of these things, I propose
couple of new functions that are asynchronous and easier to use
by the application. These APIs require net_buf fragmentation support
that I sent earlier.

Cheers,
Jukka


include/net/net_socket.h | 141 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)

diff --git a/include/net/net_socket.h b/include/net/net_socket.h
index 88073b1..2451de3 100644
--- a/include/net/net_socket.h
+++ b/include/net/net_socket.h
@@ -143,6 +143,147 @@ int net_reply(struct net_context *context, struct net_buf *buf);
struct simple_udp_connection *
net_context_get_udp_connection(struct net_context *context);

+/**
+ * @brief Callback that is called when there is something received from
+ * the network.
+ *
+ * @details It is callback responsibility to release the net_buf when the
+ * data is no longer needed. The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error
+ * @param iov List of net_buf that contain data
+ * @param user_data User supplied data
+ */
+typedef void (*net_readv_cb_t)(struct net_context *context,
+ int status,
+ struct net_buf *iov,
+ void *user_data);

I receive a list of net_buf, based on what criteria? A list of packets
or a single packet split in several buffers? If the latter, then you are
relegating to the application developer the job of putting it all
together, which in many cases will be required. Unless we expect that
every single higher level protocol will work with non-continuous chunks
of memory.

+
+/**
+ * @brief Register a receiver to get data from network.
+ *
+ * @details Application uses this to get data from network connection.
+ * Caller needs to specify a callback function that is called when data
+ * is received by the socket. When the IP stack has received data, it then
+ * calls user specified callback and it is callback responsibility to
+ * release the net_buf when the data is no longer needed.
+ * This function will return immediately as it only sets a callback to be
+ * called when new data is received. Application can remove the registered
+ * callback by calling the function with same context and setting cb to NULL.
+ *
+ * @param context Network context
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_readv(struct net_context *context,
+ net_readv_cb_t cb,
+ void *user_data);
+
+/**
+ * @brief Callback that is called when user can send
+ * data to the network.
+ *
+ * @details The first call to this function will set *status to 0 and
+ * bytes_sent to 0. Application can then prepare the data to be sent
+ * in net_buf fragment list. The data to be sent is returned to the
+ * caller of the function. If application does not wish to send anything
+ * it can just return NULL. In this case the caller will deallocate the
+ * iov list. Application should set the *status to <0 to indicate more
+ * detailed reason for the error if NULL is returned.
+ *
+ * For successfully sent UDP data, the IP stack will call this
+ * callback again with status set to 0 and bytes_sent telling how many bytes
+ * were sent. If the UDP data was not sent for some reason, then *status
+ * will have value < 0 and bytes_sent is set to 0.
+ *
+ * For TCP, the callback is called when the network connection has been
+ * established. In this case the *status is 0 and bytes_sent is 0.
+ * If there is a connection timeout, the status code will be -ETIMEDOUT
+ * and bytes_sent is set to 0. Other error codes are also possible in
+ * which case the *status < 0 and bytes_sent will tell how many bytes were
+ * successfully sent. If the *status is set to 0 then bytes_sent will tell
+ * how many bytes were sent to the peer.
+ *
+ * The iov fragment list does not contain data that has been successfully
+ * sent. Also the iov->frags->data of the first data fragment will point to
+ * first byte that has not yet been sent. If all the data was sent
+ * successfully, then the first item of iov list will have its frags pointer
+ * set to NULL. Application can send more data if it wishes by returning
+ * a new list of data fragments.
+ *
+ * The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error in stack side
+ * Application can return error code if needed via this pointer.
+ * @param bytes_sent How many bytes were sent.
+ * @param iov List of net_buf that contain data. If this is NULL, then
+ * allocate the buf and fill it with data. If non-NULL, then the protocol
+ * headers are already there and you can append the data.
+ * @param user_data User supplied data
+ *
+ * @return A valid net_buf that needs to be sent,
+ * NULL if user is not able to send anything.
+ */
+typedef struct net_buf *(*net_writev_cb_t)(struct net_context *context,
+ int *status,
+ int bytes_sent,
+ struct net_buf *iov,
+ void *user_data);

So the way to use this is:
- I have data to send, I call net_writev() and set my callback.
- When data can be sent, callback is called, *status and bytes_sent = 0
- I allocate a net_buf, put my data in it, and return it.
* What's the iov received here? Only for error cases or the callback
can be called multiple times to send the remaining data? In that
case, the idea is that I receive a list of unsent data that I need
to return and that's it?
- When all data was sent, the callback is called again with *status 0
and bytes_sent sent to the total of bytes sent... since the last time
the function was called? Since the first?

The thing with the fragment list, I'm understanding is meant to work as
an iovec, so let's say I have a CoAP server that wants to notify three
different clients of a change in a resource, it would make sense to have
a single buffer with the payload and change the one with the header. If
I understand this correctly, when data was sent successfully, I will not
have access to the net_buf with the payload anymore, so I would need to
recreate it, correct? In this case, there's no gain (and maybe even a
little loss) in doing a list of fragments, when I can just use a single
buffer and copy both header and payload to it.

+
+/**
+ * @brief Send data to network.
+ *
+ * @details Application uses this to send data to a network connection.
+ * Caller needs to specify a callback function that is called when data
+ * is ready to be sent. The function will return immediately, the timeout
+ * is only used when calling the user callback. For UDP, the timeout is
+ * ignored. For TCP the callback is called after the TCP connection is
+ * established and user is able to send data, or if there is an error
+ * creating a connection or if the connection timeouts.
+ *

This means that every time I call net_writev(), the TCP connection is
established? Or is that only the first time? If it's only the first, why
not add a connect() function? And if it's established every time, what's
the point of that?

+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This is only
+ * used in TCP.
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_writev(struct net_context *context,
+ int32_t timeout,
+ net_writev_cb_t cb,
+ void *user_data);
+
+/**
+ * @brief Reply data to sender.
+ *
+ * @details This is a helper that will help user to reply data to the
+ * sender. Application can use this function to reply data it received
+ * via net_readv().
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This is only
+ * used in TCP.
+ * @param iov Data to be sent
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_replyv(struct net_context *context,
+ int32_t timeout,
+ struct net_buf *iov,
+ net_writev_cb_t cb,
+ void *user_data);
+

Is there really no way to get rid of this? Why can't we just have a
'send' function that works for all cases? If the problem is getting the
address to send to, just add a helper function to put this in the
net_buf before sending.

#ifdef __cplusplus
}
#endif
--
2.5.5

Hi Jukka,

Jukka Rissanen <jukka.rissanen(a)linux.intel.com> writes:

Signed-off-by: Jukka Rissanen <jukka.rissanen(a)linux.intel.com>
---
Hi,

current network application APIs in Zephyr are synchronous and a
bit of awkward to use in TCP. Because of these things, I propose
couple of new functions that are asynchronous and easier to use
by the application. These APIs require net_buf fragmentation support
that I sent earlier.

At least from Soletta side, what I feel is lacking from the Zephyr
network stack API-wise is this:

- net_writeto(struct net_context *context, struct net_buf *buf,
struct net_addr *to, uint16_t port)

- net_readfrom(struct net_context *context, struct net_buf *buf,
struct net_addr *from, uint16_t *port);

- net_context_add_addr(struct net_context *context,
struct net_addr *local_addr, uint16_t port);

See what Ivan needed to do here:

https://github.com/solettaproject/soletta/blob/master/src/lib/comms/sol-socket-impl-zephyr.c#L260

Cheers,
Jukka


include/net/net_socket.h | 141 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)

diff --git a/include/net/net_socket.h b/include/net/net_socket.h
index 88073b1..2451de3 100644
--- a/include/net/net_socket.h
+++ b/include/net/net_socket.h
@@ -143,6 +143,147 @@ int net_reply(struct net_context *context, struct net_buf *buf);
struct simple_udp_connection *
net_context_get_udp_connection(struct net_context *context);

+/**
+ * @brief Callback that is called when there is something received from
+ * the network.
+ *
+ * @details It is callback responsibility to release the net_buf when the
+ * data is no longer needed. The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error
+ * @param iov List of net_buf that contain data
+ * @param user_data User supplied data
+ */
+typedef void (*net_readv_cb_t)(struct net_context *context,
+ int status,
+ struct net_buf *iov,
+ void *user_data);
+
+/**
+ * @brief Register a receiver to get data from network.
+ *
+ * @details Application uses this to get data from network connection.
+ * Caller needs to specify a callback function that is called when data
+ * is received by the socket. When the IP stack has received data, it then
+ * calls user specified callback and it is callback responsibility to
+ * release the net_buf when the data is no longer needed.
+ * This function will return immediately as it only sets a callback to be
+ * called when new data is received. Application can remove the registered
+ * callback by calling the function with same context and setting cb to NULL.
+ *
+ * @param context Network context
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_readv(struct net_context *context,
+ net_readv_cb_t cb,
+ void *user_data);
+
+/**
+ * @brief Callback that is called when user can send
+ * data to the network.
+ *
+ * @details The first call to this function will set *status to 0 and
+ * bytes_sent to 0. Application can then prepare the data to be sent
+ * in net_buf fragment list. The data to be sent is returned to the
+ * caller of the function. If application does not wish to send anything
+ * it can just return NULL. In this case the caller will deallocate the
+ * iov list. Application should set the *status to <0 to indicate more
+ * detailed reason for the error if NULL is returned.
+ *
+ * For successfully sent UDP data, the IP stack will call this
+ * callback again with status set to 0 and bytes_sent telling how many bytes
+ * were sent. If the UDP data was not sent for some reason, then *status
+ * will have value < 0 and bytes_sent is set to 0.
+ *
+ * For TCP, the callback is called when the network connection has been
+ * established. In this case the *status is 0 and bytes_sent is 0.
+ * If there is a connection timeout, the status code will be -ETIMEDOUT
+ * and bytes_sent is set to 0. Other error codes are also possible in
+ * which case the *status < 0 and bytes_sent will tell how many bytes were
+ * successfully sent. If the *status is set to 0 then bytes_sent will tell
+ * how many bytes were sent to the peer.
+ *
+ * The iov fragment list does not contain data that has been successfully
+ * sent. Also the iov->frags->data of the first data fragment will point to
+ * first byte that has not yet been sent. If all the data was sent
+ * successfully, then the first item of iov list will have its frags pointer
+ * set to NULL. Application can send more data if it wishes by returning
+ * a new list of data fragments.
+ *
+ * The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error in stack side
+ * Application can return error code if needed via this pointer.
+ * @param bytes_sent How many bytes were sent.
+ * @param iov List of net_buf that contain data. If this is NULL, then
+ * allocate the buf and fill it with data. If non-NULL, then the protocol
+ * headers are already there and you can append the data.
+ * @param user_data User supplied data
+ *
+ * @return A valid net_buf that needs to be sent,
+ * NULL if user is not able to send anything.
+ */
+typedef struct net_buf *(*net_writev_cb_t)(struct net_context *context,
+ int *status,
+ int bytes_sent,
+ struct net_buf *iov,
+ void *user_data);
+
+/**
+ * @brief Send data to network.
+ *
+ * @details Application uses this to send data to a network connection.
+ * Caller needs to specify a callback function that is called when data
+ * is ready to be sent. The function will return immediately, the timeout
+ * is only used when calling the user callback. For UDP, the timeout is
+ * ignored. For TCP the callback is called after the TCP connection is
+ * established and user is able to send data, or if there is an error
+ * creating a connection or if the connection timeouts.
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This is only
+ * used in TCP.
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_writev(struct net_context *context,
+ int32_t timeout,
+ net_writev_cb_t cb,
+ void *user_data);
+
+/**
+ * @brief Reply data to sender.
+ *
+ * @details This is a helper that will help user to reply data to the
+ * sender. Application can use this function to reply data it received
+ * via net_readv().
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This is only
+ * used in TCP.
+ * @param iov Data to be sent
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_replyv(struct net_context *context,
+ int32_t timeout,
+ struct net_buf *iov,
+ net_writev_cb_t cb,
+ void *user_data);
+
#ifdef __cplusplus
}
#endif
--
2.5.5

Cheers,
--
Vinicius

Hi Jukka,

Hi Ivan,

On Wed, 2016-04-27 at 11:59 -0300, Iván Briano wrote:

On Wed, 27 Apr 2016 11:59:50 +0300, Jukka Rissanen wrote:

Signed-off-by: Jukka Rissanen <jukka.rissanen(a)linux.intel.com>
---
Hi,

current network application APIs in Zephyr are synchronous and a
bit of awkward to use in TCP. Because of these things, I propose
couple of new functions that are asynchronous and easier to use
by the application. These APIs require net_buf fragmentation
support
that I sent earlier.

Cheers,
Jukka


include/net/net_socket.h | 141
+++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 141 insertions(+)

diff --git a/include/net/net_socket.h b/include/net/net_socket.h
index 88073b1..2451de3 100644
--- a/include/net/net_socket.h
+++ b/include/net/net_socket.h
@@ -143,6 +143,147 @@ int net_reply(struct net_context *context,
struct net_buf *buf);
 struct simple_udp_connection *
  net_context_get_udp_connection(struct net_context
*context);
 
+/**
+ * @brief Callback that is called when there is something received
from
+ * the network.
+ *
+ * @details It is callback responsibility to release the net_buf
when the
+ * data is no longer needed. The callback is called in fiber
context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error
+ * @param iov List of net_buf that contain data
+ * @param user_data User supplied data
+ */
+typedef void (*net_readv_cb_t)(struct net_context *context,
+        int status,
+        struct net_buf *iov,
+        void *user_data);

I receive a list of net_buf, based on what criteria? A list of
packets
or a single packet split in several buffers? If the latter, then you

The latter, the app would receive list of buffers. This complicates the
application a bit but some API could be used to retrieve the data so
that app would not need to know much about the individual fragments.

are
relegating to the application developer the job of putting it all
together, which in many cases will be required. Unless we expect that
every single higher level protocol will work with non-continuous
chunks
of memory.

+
+/**
+ * @brief Register a receiver to get data from network.
+ *
+ * @details Application uses this to get data from network
connection.
+ * Caller needs to specify a callback function that is called when
data
+ * is received by the socket. When the IP stack has received data,
it then
+ * calls user specified callback and it is callback responsibility
to
+ * release the net_buf when the data is no longer needed.
+ * This function will return immediately as it only sets a
callback to be
+ * called when new data is received. Application can remove the
registered
+ * callback by calling the function with same context and setting
cb to NULL.
+ *
+ * @param context Network context
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_readv(struct net_context *context,
+       net_readv_cb_t cb,
+       void *user_data);
+
+/**
+ * @brief Callback that is called when user can send
+ * data to the network.
+ *
+ * @details The first call to this function will set *status to 0
and
+ * bytes_sent to 0. Application can then prepare the data to be
sent
+ * in net_buf fragment list. The data to be sent is returned to
the
+ * caller of the function. If application does not wish to send
anything
+ * it can just return NULL. In this case the caller will
deallocate the
+ * iov list. Application should set the *status to <0 to indicate
more
+ * detailed reason for the error if NULL is returned.
+ *
+ * For successfully sent UDP data, the IP stack will call this
+ * callback again with status set to 0 and bytes_sent telling how
many bytes
+ * were sent. If the UDP data was not sent for some reason, then
*status
+ * will have value < 0 and bytes_sent is set to 0.
+ *
+ * For TCP, the callback is called when the network connection has
been
+ * established. In this case the *status is 0 and bytes_sent is 0.
+ * If there is a connection timeout, the status code will be
-ETIMEDOUT
+ * and bytes_sent is set to 0. Other error codes are also possible
in
+ * which case the *status < 0 and bytes_sent will tell how many
bytes were
+ * successfully sent. If the *status is set to 0 then bytes_sent
will tell
+ * how many bytes were sent to the peer.
+ *
+ * The iov fragment list does not contain data that has been
successfully
+ * sent. Also the iov->frags->data of the first data fragment will
point to
+ * first byte that has not yet been sent. If all the data was sent
+ * successfully, then the first item of iov list will have its
frags pointer
+ * set to NULL. Application can send more data if it wishes by
returning
+ * a new list of data fragments.
+ *
+ * The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error in stack side
+ * Application can return error code if needed via this pointer.
+ * @param bytes_sent How many bytes were sent.
+ * @param iov List of net_buf that contain data. If this is NULL,
then
+ * allocate the buf and fill it with data. If non-NULL, then the
protocol
+ * headers are already there and you can append the data.
+ * @param user_data User supplied data
+ *
+ * @return A valid net_buf that needs to be sent,
+ * NULL if user is not able to send anything.
+ */
+typedef struct net_buf *(*net_writev_cb_t)(struct net_context
*context,
+    int *status,
+    int bytes_sent,
+    struct net_buf *iov,
+    void *user_data);

So the way to use this is:
 - I have data to send, I call net_writev() and set my callback.
 - When data can be sent, callback is called, *status and bytes_sent
= 0
 - I allocate a net_buf, put my data in it, and return it.
   * What's the iov received here? Only for error cases or the
callback
     can be called multiple times to send the remaining data? In that
     case, the idea is that I receive a list of unsent data that I
need
     to return and that's it?

The latter. So the callback would be called again to send the remaining
data.

 - When all data was sent, the callback is called again with *status
0
   and bytes_sent sent to the total of bytes sent... since the last
time
   the function was called? Since the first?

Since the last time the function was called.

The thing with the fragment list, I'm understanding is meant to work
as
an iovec, so let's say I have a CoAP server that wants to notify
three
different clients of a change in a resource, it would make sense to
have
a single buffer with the payload and change the one with the header.
If
I understand this correctly, when data was sent successfully, I will
not
have access to the net_buf with the payload anymore, so I would need
to
recreate it, correct? In this case, there's no gain (and maybe even a
little loss) in doing a list of fragments, when I can just use a
single
buffer and copy both header and payload to it.

The net_buf's have a refcount so the same buffer can be used to send
same data to multiple clients.

+
+/**
+ * @brief Send data to network.
+ *
+ * @details Application uses this to send data to a network
connection.
+ * Caller needs to specify a callback function that is called when
data
+ * is ready to be sent. The function will return immediately, the
timeout
+ * is only used when calling the user callback. For UDP, the
timeout is
+ * ignored. For TCP the callback is called after the TCP
connection is
+ * established and user is able to send data, or if there is an
error
+ * creating a connection or if the connection timeouts.
+ *

This means that every time I call net_writev(), the TCP connection is
established? Or is that only the first time? If it's only the first,
why
not add a connect() function? And if it's established every time,
what's
the point of that?

The connection would be created only once. Adding the connect() is one
option but it is not really necessary.

+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This
is only
+ * used in TCP.
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_writev(struct net_context *context,
+        int32_t timeout,
+        net_writev_cb_t cb,
+        void *user_data);
+
+/**
+ * @brief Reply data to sender.
+ *
+ * @details This is a helper that will help user to reply data to
the
+ * sender. Application can use this function to reply data it
received
+ * via net_readv().
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This
is only
+ * used in TCP.
+ * @param iov Data to be sent
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_replyv(struct net_context *context,
+        int32_t timeout,
+        struct net_buf *iov,
+        net_writev_cb_t cb,
+        void *user_data);
+

Is there really no way to get rid of this? Why can't we just have a
'send' function that works for all cases? If the problem is getting
the
address to send to, just add a helper function to put this in the
net_buf before sending.

If we are able to put some intelligence to plain send then why not. The
reason for the reply() was that we could have only one context for the
connection (to save some memory). For example in BSD sockets, the
listening socket is accepted by the app in which case when sending one
uses the accepted socket to send data and the addresses in the packet
are set then correctly.
But if the send can figure out itself that it should reverse the
addresses in the sending IP packet, then we can get rid of reply.

 #ifdef __cplusplus
 }
 #endif

Cheers,
Jukka

Hi Johan,

On Wed, 2016-04-27 at 21:56 +0300, Johan Hedberg wrote:

Hi Jukka,

On Wed, Apr 27, 2016, Jukka Rissanen wrote:
 +/**

+ * @brief Callback that is called when there is something received
from
+ * the network.
+ *
+ * @details It is callback responsibility to release the net_buf
when the
+ * data is no longer needed. The callback is called in fiber
context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error
+ * @param iov List of net_buf that contain data
+ * @param user_data User supplied data
+ */
+typedef void (*net_readv_cb_t)(struct net_context *context,
+        int status,
+        struct net_buf *iov,
+        void *user_data);

A couple of things: I think "recv" is more intuitive than "read" when
it
comes to network operations (as opposed to e.g. file access). Same
goes
for "send" vs "write".

We have a net_send() API already that is why I used net_read() and
net_write() here. I will drop the v from the function name in next
version.

I don't think the fact that the net_buf parameter might have extra
fragments after it is enough justification for having it named "iov"
instead of simply "buf". Once fragmentation support is there it
should
be the default assumption that any buffer might have a list of
trailing
fragments. So at most I'd just mention this in the function
documentation.

Sure, the parameter name can be changed.

+/**
+ * @brief Register a receiver to get data from network.
+ *
+ * @details Application uses this to get data from network
connection.
+ * Caller needs to specify a callback function that is called when
data
+ * is received by the socket. When the IP stack has received data,
it then
+ * calls user specified callback and it is callback responsibility
to
+ * release the net_buf when the data is no longer needed.
+ * This function will return immediately as it only sets a
callback to be
+ * called when new data is received. Application can remove the
registered
+ * callback by calling the function with same context and setting
cb to NULL.
+ *
+ * @param context Network context
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_readv(struct net_context *context,
+       net_readv_cb_t cb,
+       void *user_data);

To make the purpose of the function clearer I'd rename this to
net_set_recv_cb() or similar.

Sure, np.

+/**
+ * @brief Callback that is called when user can send
+ * data to the network.
+ *
+ * @details The first call to this function will set *status to 0
and
+ * bytes_sent to 0. Application can then prepare the data to be
sent
+ * in net_buf fragment list. The data to be sent is returned to
the
+ * caller of the function. If application does not wish to send
anything
+ * it can just return NULL. In this case the caller will
deallocate the
+ * iov list. Application should set the *status to <0 to indicate
more
+ * detailed reason for the error if NULL is returned.
+ *
+ * For successfully sent UDP data, the IP stack will call this
+ * callback again with status set to 0 and bytes_sent telling how
many bytes
+ * were sent. If the UDP data was not sent for some reason, then
*status
+ * will have value < 0 and bytes_sent is set to 0.
+ *
+ * For TCP, the callback is called when the network connection has
been
+ * established. In this case the *status is 0 and bytes_sent is 0.
+ * If there is a connection timeout, the status code will be
-ETIMEDOUT
+ * and bytes_sent is set to 0. Other error codes are also possible
in
+ * which case the *status < 0 and bytes_sent will tell how many
bytes were
+ * successfully sent. If the *status is set to 0 then bytes_sent
will tell
+ * how many bytes were sent to the peer.
+ *
+ * The iov fragment list does not contain data that has been
successfully
+ * sent. Also the iov->frags->data of the first data fragment will
point to
+ * first byte that has not yet been sent.

If you want to maintain the same buffer list even after sending all
data
in some of them (something I don't think is a good idea since you
want
to unref and get the bufs back to the pool asap) I'd just set buf-

Hmm, the idea was that already sent buffers would be returned to the
pool immediately and removed from the fragment list. The fragment list
would only contain buffers that need to re-sent. This is needed in TCP,
for UDP we can always send the packets.

len

of all completed buffers to 0, i.e. the start of remaining data would
be
the first buffer in the chain with buf->len > 0. If the idea is to
keep
rotating the return parameter back to the next calls input parameter
I'd
just go forward in the chain and give a pointer to the first buffer
with
data left while having unrefed all the previous ones.

    If all the data was sent
+ * successfully, then the first item of iov list will have its
frags pointer
+ * set to NULL. Application can send more data if it wishes by
returning
+ * a new list of data fragments.
+ *
+ * The callback is called in fiber context.
+ *
+ * @param context Network context
+ * @param status 0 if ok, <0 if there is an error in stack side
+ * Application can return error code if needed via this pointer.
+ * @param bytes_sent How many bytes were sent.
+ * @param iov List of net_buf that contain data. If this is NULL,
then
+ * allocate the buf and fill it with data. If non-NULL, then the
protocol
+ * headers are already there and you can append the data.
+ * @param user_data User supplied data
+ *
+ * @return A valid net_buf that needs to be sent,
+ * NULL if user is not able to send anything.
+ */
+typedef struct net_buf *(*net_writev_cb_t)(struct net_context
*context,
+    int *status,
+    int bytes_sent,
+    struct net_buf *iov,
+    void *user_data);

I'm not really following what exactly the relationship is of the
input
net_buf parameter and the return value of this function. Sounds
unnecessarily complicated to me (the little bit that I did
understand).
Why does this function need an input net_buf parameter to start with?

If the data could not be sent, the stack calls this API again to allow
the app to do something about the situation. This is used in TCP so
that the application can know what data has been transferred
successfully to the peer.

Also, what's the reason this needs to be asynchronous this way? If
the
app is responsible for allocating the net_buf can't it just give it
straight to the stack with net_send() or similar, with an optional
callback to notify completion (or failure) of sending the data over
the
network?

The data to be sent could be given as a parameter when calling
net_write() but there would still be need to call the callback. I did
it like this in first version of API but after some experimenting it
felt more natural that all the data sending would be done from one
place (from cb), and the initial net_write() call would only do
connection establishment when needed.

+ * @brief Reply data to sender.
+ *
+ * @details This is a helper that will help user to reply data to
the
+ * sender. Application can use this function to reply data it
received
+ * via net_readv().
+ *
+ * @param context Network context
+ * @param timeout How long to wait until user can send data. This
is only
+ * used in TCP.
+ * @param iov Data to be sent
+ * @param cb User supplied callback function
+ * @param user_data User supplied data
+ *
+ * @return 0 if cb registration was ok, <0 otherwise
+ */
+int net_replyv(struct net_context *context,
+        int32_t timeout,
+        struct net_buf *iov,
+        net_writev_cb_t cb,
+        void *user_data);

This is basically something like the net_send() I mentioned earlier.

Yes, it is almost the same. As I mentioned to Ivan in another mail,
that in order to avoid to allocate extra context for reply data, the
net_reply() was created. It just swaps the IP addresses of the network
packet. It is not really mandatory but acts as a helper.


Cheers,
Jukka

Hi Vinicius,

On Wed, 2016-04-27 at 12:10 -0300, Vinicius Costa Gomes wrote:

Hi Jukka,

Jukka Rissanen <jukka.rissanen(a)linux.intel.com> writes:

Signed-off-by: Jukka Rissanen <jukka.rissanen(a)linux.intel.com>
---
Hi,

current network application APIs in Zephyr are synchronous and a
bit of awkward to use in TCP. Because of these things, I propose
couple of new functions that are asynchronous and easier to use
by the application. These APIs require net_buf fragmentation
support
that I sent earlier.

At least from Soletta side, what I feel is lacking from the Zephyr
network stack API-wise is this:

- net_writeto(struct net_context *context, struct net_buf *buf,
       struct net_addr *to, uint16_t port)

- net_readfrom(struct net_context *context, struct net_buf *buf,
       struct net_addr *from, uint16_t *port);

- net_context_add_addr(struct net_context *context,
       struct net_addr *local_addr, uint16_t port);

Sure, if the API's make sense then just send the patches.

See what Ivan needed to do here:

https://github.com/solettaproject/soletta/blob/master/src/lib/comms/s
ol-socket-impl-zephyr.c#L260

Cheers,
Jukka