Date   

Re: Using IC manufacturer header files

Maciek Borzecki <maciek.borzecki@...>
 

On Wed, Apr 20, 2016 at 9:28 PM, Cufi, Carles <Carles.Cufi(a)nordicsemi.no> wrote:
Hi Maciek,

-----Original Message-----
From: Maciek Borzecki [mailto:maciek.borzecki(a)gmail.com]
Sent: Wednesday, April 20, 2016 19:24
To: Cufi, Carles <Carles.Cufi(a)nordicsemi.no>
Cc: Nashif, Anas <anas.nashif(a)intel.com>; devel(a)lists.zephyrproject.org
Subject: Re: [devel] Re: Using IC manufacturer header files

On Tue, Apr 19, 2016 at 10:50 PM, Cufi, Carles
<Carles.Cufi(a)nordicsemi.no> wrote:
Hi Maciek,

-----Original Message-----
From: Maciek Borzecki [mailto:maciek.borzecki(a)gmail.com]
Sent: Tuesday, April 19, 2016 17:27
To: Nashif, Anas <anas.nashif(a)intel.com>
Cc: Cufi, Carles <Carles.Cufi(a)nordicsemi.no>;
devel(a)lists.zephyrproject.org
Subject: Re: [devel] Re: Using IC manufacturer header files

On Tue, Apr 19, 2016 at 5:04 PM, Nashif, Anas <anas.nashif(a)intel.com>
wrote:
Hi,
Any files or headers included from an external project that need to
be
kept in sync with upstream can keep their style and formatting to
allow updates in the future. This was the approach taken when we
imported the IP stack from contiki AFAIK.

How about keeping the library/headers external and passing their
location in Kconfig? Similar to what buildroot does with external
kernel/toolchains. Is that acceptable too?

Not sure if you mean pulling the source code from another repository
at build time, since I don't know what buildroot does with
kernel/toolchains.
However the MDK is not currently available in a repository, instead it
is released as a .zip file by Nordic every time it gets updated. We
could of course mirror it in a repo, but that would anyway require us to
update it manually every time a new version is available.

What I meant is that you keep the {S,M}DK externally and just configure
a path to it inside Kconfig. You can add a config entry like
CONFIG_STM32F1_STDLIB_PATH. Then in project/driver makefiles
-I$(CONFIG_STM32F1_STDLIB_PATH) and optionally add some files to the
build as needed. This has one major advantage, that the 3rd party code
is kept outside of Zephyr repository, so you basically don't care about
the naming/conventions/licenses 3rd party libraries would use.
I couldn't find the CONFIG_STM32F1_STDLIB_PATH you mentioned, and I tried to build the "hello_world" sample for STM32 and it worked fine, even though I have no STM SDK installed at all.
Right, but only because the implementation was done in such way.
Frankly, if I used ST's libraries, it would have saved me a lot of
time during porting. Instead, I followed the example set by other,
already supported, targets and implemented STM32 specific pieces from
scratch by looking at the CPU manuals only.

With that I mean that it looks like, so far, archs are self-contained. The problem with the "-I$(NORDIC_MDK_PATH)" approach is that, as you mention later, Nordic might change the folder structure of the .zip file that it provides, rename some files and change it completely, leaving us at the mercy of the vendor.

As for mirroring 3rd party SDK in a repo. No matter the vendor's
distribution method I always pull the source code into a git repository
for two main reasons:
By pulling into the repo I assume you mean actually duplicating the files inside the target repo, instead of "pulling" them from an external repo.

- I don't trust vendors to have a reasonable release management (more
often than not they prove to have none)
Agreed, see point above. Sometimes there's legitimate reasons to change the structure of the SDK provided by a vendor, but the OS build shouldn't be affected by it. Furthermore, relying on a particular version could potentially then introduce dependency hell into the Zephyr build for certain architectures, where some users would start to complain that the OS doesn't build for a particular IC because they have an older (or newer) IC vendor SDK than the one that Zephyr depends on.
I understand that it's nicer from user's perspective that the CPU
support is integrated into the main codebase. I'm just not sure if it
will be possible to maintain this state in the long run. Integrating
code from a 3rd party into the main codebase means that we've
officially made maintaining compatibility our problem. While, this is
ok to do it with IP stack from contiki, it's just one 3rd party
project after all, it will be hard with the multitude of CPUs out
there.

I know it sounds crazy, but OpenEmbedded BSP layers are not that bad
to work with after all. I can imagine that having similar BSP layers
for Zephyr would not be that complicated. These 'layers', maintained
by community or interested parties, could provide a curated hardware
enablement or support libraries.


- you cannot rely on developers putting/extracting SDK's at proper
locations and setting their dev environment
Yes, this is especially troublesome in this case. The MDK is delivered as a .zip file for Linux and there is no installer or standardized path for it.


The multi repository projects can be easily remedied by using `repo`
tool or git submodules.
Although that is true, it would break the current seemingly desirable feature of the Zephyr codebase, and that is it being self-contained. Git submodules or repo add an additional layer of complexity to users, and that is not something I envision introducing at this point.

So far we have a nice, self-contained repo with a subset of (almost) untouched Nordic MDK files in the arch folder. We'll start with that and see how that works in the future.

Thanks,

Carles


--
Maciek Borzecki


Re: remove zephyr-env.sh

Nashif, Anas
 

On 20/04/2016, 15:58, "Perez-Gonzalez, Inaky" <inaky.perez-gonzalez(a)intel.com> wrote:

On Wed, 2016-04-20 at 19:53 +0000, Boie, Andrew P wrote:

From: Perez-Gonzalez, Inaky [mailto:inaky.perez-gonzalez(a)intel.com]
Currently we require the user to source the zephyr-env.sh
environment to
set up the build environment. This is awkward; things build fine
without
it and the user still has to manually setup ZEPHYR_BASE,
ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. zephyr-env.sh has
grown to
I don't believe this is the case.

You do have to set ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. I
put them in my .bashrc:

export ZEPHYR_GCC_VARIANT=zephyr
export ZEPHYR_SDK_INSTALL_DIR=/opt/zephyr-sdk/

However, ZEPHYR_BASE is specific to what Zephyr tree you are working
in. I personally have several that I work in. This is not something I
would want to globally set in my .bashrc. At minimum we would need to
transplant this logic to the Makefiles.
Certainly, ZEPHYR_BASE is tree-specific (so could be the other in
certain cases, but uncommon). I also do set it manually (in fact, as I
said somewhere, I feed it to make's command line)

if you think doing this

# make ZEPHYR_BASE=/path/to/source/tree BOARD=blah blah

hundreds of times per day is more convenient than doing 'source zephyr-env.sh’ once a day per shell then we have a different definition of convenience :-)



From looking at the script it additionally:

1. Appends the ZEPHYR_BASE/scripts directory to the PATH
This is the only one that might have some use--however, it still forces
you to work on the same session; if you fly through windows and tabs,
like I do, it's not as useful.

2. Sources ~/.zephyrrc if it exists (not sure what the use-case for
this is, I don't personally use it).
I haven't heard of anyone ever using this (hopefully they'll come up in
this thread).

I use it and it is also documented in the getting started guide.



Both of these seem optional, but to run stuff like sanitycheck from
the command line you'd have to include a full path.
Can we fold a 'make sanitycheck'-like solution? After all, if we can
figure out access to the Makefile, we can use it for accessing the
scripts too?

then you have to support all the nice options of sanity checks using make, do not know how user-friendly this will be.


Anas


Re: remove zephyr-env.sh

Perez-Gonzalez, Inaky <inaky.perez-gonzalez@...>
 

On Wed, 2016-04-20 at 19:53 +0000, Boie, Andrew P wrote:

From: Perez-Gonzalez, Inaky [mailto:inaky.perez-gonzalez(a)intel.com]
Currently we require the user to source the zephyr-env.sh
environment to
set up the build environment. This is awkward; things build fine
without
it and the user still has to manually setup ZEPHYR_BASE,
ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. zephyr-env.sh has
grown to
I don't believe this is the case.

You do have to set ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. I
put them in my .bashrc:

export ZEPHYR_GCC_VARIANT=zephyr
export ZEPHYR_SDK_INSTALL_DIR=/opt/zephyr-sdk/

However, ZEPHYR_BASE is specific to what Zephyr tree you are working
in. I personally have several that I work in. This is not something I
would want to globally set in my .bashrc. At minimum we would need to
transplant this logic to the Makefiles.
Certainly, ZEPHYR_BASE is tree-specific (so could be the other in
certain cases, but uncommon). I also do set it manually (in fact, as I
said somewhere, I feed it to make's command line)

From looking at the script it additionally:

1. Appends the ZEPHYR_BASE/scripts directory to the PATH
This is the only one that might have some use--however, it still forces
you to work on the same session; if you fly through windows and tabs,
like I do, it's not as useful.

2. Sources ~/.zephyrrc if it exists (not sure what the use-case for
this is, I don't personally use it).
I haven't heard of anyone ever using this (hopefully they'll come up in
this thread).

Both of these seem optional, but to run stuff like sanitycheck from
the command line you'd have to include a full path.
Can we fold a 'make sanitycheck'-like solution? After all, if we can
figure out access to the Makefile, we can use it for accessing the
scripts too?


Re: remove zephyr-env.sh

Benjamin Walsh <benjamin.walsh@...>
 

On Wed, Apr 20, 2016 at 07:53:56PM +0000, Boie, Andrew P wrote:
From: Perez-Gonzalez, Inaky [mailto:inaky.perez-gonzalez(a)intel.com]
Currently we require the user to source the zephyr-env.sh environment to
set up the build environment. This is awkward; things build fine without
it and the user still has to manually setup ZEPHYR_BASE,
ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. zephyr-env.sh has grown to
I don't believe this is the case.

You do have to set ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. I
put them in my .bashrc:

export ZEPHYR_GCC_VARIANT=zephyr
export ZEPHYR_SDK_INSTALL_DIR=/opt/zephyr-sdk/

However, ZEPHYR_BASE is specific to what Zephyr tree you are working
in. I personally have several that I work in. This is not something I
would want to globally set in my .bashrc. At minimum we would need to
transplant this logic to the Makefiles.

From looking at the script it additionally:

1. Appends the ZEPHYR_BASE/scripts directory to the PATH
2. Sources ~/.zephyrrc if it exists (not sure what the use-case for
this is, I don't personally use it).
I use it instead of polluting my .bashrc with zephyr stuff. Basically,
your two export lines above go in there.

I personally like that zephyr-env.sh exists: you just have to source it
and your environment is fully set (if you put the two export lines above
in your ~/.zephyrrc as well).

Both of these seem optional, but to run stuff like sanitycheck from
the command line you'd have to include a full path.


Re: remove zephyr-env.sh

Boie, Andrew P
 

From: Perez-Gonzalez, Inaky [mailto:inaky.perez-gonzalez(a)intel.com]
Currently we require the user to source the zephyr-env.sh environment to
set up the build environment. This is awkward; things build fine without
it and the user still has to manually setup ZEPHYR_BASE,
ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. zephyr-env.sh has grown to
I don't believe this is the case.

You do have to set ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. I put them in my .bashrc:

export ZEPHYR_GCC_VARIANT=zephyr
export ZEPHYR_SDK_INSTALL_DIR=/opt/zephyr-sdk/

However, ZEPHYR_BASE is specific to what Zephyr tree you are working in. I personally have several that I work in. This is not something I would want to globally set in my .bashrc. At minimum we would need to transplant this logic to the Makefiles.

From looking at the script it additionally:

1. Appends the ZEPHYR_BASE/scripts directory to the PATH
2. Sources ~/.zephyrrc if it exists (not sure what the use-case for this is, I don't personally use it).

Both of these seem optional, but to run stuff like sanitycheck from the command line you'd have to include a full path.

Andrew


Re: remove zephyr-env.sh

Perez-Gonzalez, Inaky <inaky.perez-gonzalez@...>
 

On Wed, 2016-04-20 at 19:35 +0000, Cufi, Carles wrote:
Hi Inaky,

From: Perez-Gonzalez, Inaky [mailto:inaky.perez-gonzalez(a)intel.com]
 
https://jira.zephyrproject.org/browse/ZEP-211 has been entered to
track
a proposal to remove zephyr-env.sh.

<<from the ticket>>
Currently we require the user to source the zephyr-env.sh
environment to
set up the build environment. This is awkward; things build fine
without
it and the user still has to manually setup ZEPHYR_BASE,
ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. 
I might be mistaken but I believe ZEPHYR_BASE is indeed set by
zephyr-env.sh, so it does not need to be set manually. In fact I ran
into an issue where I had my own ZEPHYR_BASE set to
"/home/me/src/zephyr" but was working on a separate
"/home/me/src/zephyr-upstream/" copy of the repo and my changes to
Kconfig obviously never showed up, leaving me puzzled. It is somehow
convenient to be able to set the ZEPHYR_BASE to whatever copy of the
repo one is building, but it would be even better if make itself
would do that when building instead, based on the path where one
executes it.
to avoid those issues I usually run make ZEPHYR_BASE=DIR <blah>, which
is kind of more verbose, but gets the trick done.

However, I hear your call. I think it's worthwhile to consider the
posibility of having the Makefile "look" for the source (looking in top
level directories, following a symlink, $ZEPHYR_BASE or fail) -- the
trick is how to do it so it does not require a huge chunk of code in
each app's Makefile (it can't be in a library as we'd have to point to
it)

By the way, and as a side comment, paths containing "~" do not seem
to work in ZEPHYR_BASE and <toolchain>_TOOLCHAIN_PATH. Instead one
has to explicitly set "/home/me".
Mind filing a ticket for it, please?


Re: remove zephyr-env.sh

Carles Cufi
 

Hi Inaky,

-----Original Message-----
From: Perez-Gonzalez, Inaky [mailto:inaky.perez-gonzalez(a)intel.com]
Sent: Wednesday, April 20, 2016 20:45
To: devel(a)lists.zephyrproject.org
Subject: [devel] RFC: remove zephyr-env.sh

Hello All

https://jira.zephyrproject.org/browse/ZEP-211 has been entered to track
a proposal to remove zephyr-env.sh.

<<from the ticket>>
Currently we require the user to source the zephyr-env.sh environment to
set up the build environment. This is awkward; things build fine without
it and the user still has to manually setup ZEPHYR_BASE,
ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR.
I might be mistaken but I believe ZEPHYR_BASE is indeed set by zephyr-env.sh, so it does not need to be set manually. In fact I ran into an issue where I had my own ZEPHYR_BASE set to "/home/me/src/zephyr" but was working on a separate "/home/me/src/zephyr-upstream/" copy of the repo and my changes to Kconfig obviously never showed up, leaving me puzzled. It is somehow convenient to be able to set the ZEPHYR_BASE to whatever copy of the repo one is building, but it would be even better if make itself would do that when building instead, based on the path where one executes it.

By the way, and as a side comment, paths containing "~" do not seem to work in ZEPHYR_BASE and <toolchain>_TOOLCHAIN_PATH. Instead one has to explicitly set "/home/me".

Carles


Re: Using IC manufacturer header files

Carles Cufi
 

Hi Maciek,

-----Original Message-----
From: Maciek Borzecki [mailto:maciek.borzecki(a)gmail.com]
Sent: Wednesday, April 20, 2016 19:24
To: Cufi, Carles <Carles.Cufi(a)nordicsemi.no>
Cc: Nashif, Anas <anas.nashif(a)intel.com>; devel(a)lists.zephyrproject.org
Subject: Re: [devel] Re: Using IC manufacturer header files

On Tue, Apr 19, 2016 at 10:50 PM, Cufi, Carles
<Carles.Cufi(a)nordicsemi.no> wrote:
Hi Maciek,

-----Original Message-----
From: Maciek Borzecki [mailto:maciek.borzecki(a)gmail.com]
Sent: Tuesday, April 19, 2016 17:27
To: Nashif, Anas <anas.nashif(a)intel.com>
Cc: Cufi, Carles <Carles.Cufi(a)nordicsemi.no>;
devel(a)lists.zephyrproject.org
Subject: Re: [devel] Re: Using IC manufacturer header files

On Tue, Apr 19, 2016 at 5:04 PM, Nashif, Anas <anas.nashif(a)intel.com>
wrote:
Hi,
Any files or headers included from an external project that need to
be
kept in sync with upstream can keep their style and formatting to
allow updates in the future. This was the approach taken when we
imported the IP stack from contiki AFAIK.

How about keeping the library/headers external and passing their
location in Kconfig? Similar to what buildroot does with external
kernel/toolchains. Is that acceptable too?

Not sure if you mean pulling the source code from another repository
at build time, since I don't know what buildroot does with
kernel/toolchains.
However the MDK is not currently available in a repository, instead it
is released as a .zip file by Nordic every time it gets updated. We
could of course mirror it in a repo, but that would anyway require us to
update it manually every time a new version is available.

What I meant is that you keep the {S,M}DK externally and just configure
a path to it inside Kconfig. You can add a config entry like
CONFIG_STM32F1_STDLIB_PATH. Then in project/driver makefiles
-I$(CONFIG_STM32F1_STDLIB_PATH) and optionally add some files to the
build as needed. This has one major advantage, that the 3rd party code
is kept outside of Zephyr repository, so you basically don't care about
the naming/conventions/licenses 3rd party libraries would use.
I couldn't find the CONFIG_STM32F1_STDLIB_PATH you mentioned, and I tried to build the "hello_world" sample for STM32 and it worked fine, even though I have no STM SDK installed at all. With that I mean that it looks like, so far, archs are self-contained. The problem with the "-I$(NORDIC_MDK_PATH)" approach is that, as you mention later, Nordic might change the folder structure of the .zip file that it provides, rename some files and change it completely, leaving us at the mercy of the vendor.

As for mirroring 3rd party SDK in a repo. No matter the vendor's
distribution method I always pull the source code into a git repository
for two main reasons:
By pulling into the repo I assume you mean actually duplicating the files inside the target repo, instead of "pulling" them from an external repo.

- I don't trust vendors to have a reasonable release management (more
often than not they prove to have none)
Agreed, see point above. Sometimes there's legitimate reasons to change the structure of the SDK provided by a vendor, but the OS build shouldn't be affected by it. Furthermore, relying on a particular version could potentially then introduce dependency hell into the Zephyr build for certain architectures, where some users would start to complain that the OS doesn't build for a particular IC because they have an older (or newer) IC vendor SDK than the one that Zephyr depends on.

- you cannot rely on developers putting/extracting SDK's at proper
locations and setting their dev environment
Yes, this is especially troublesome in this case. The MDK is delivered as a .zip file for Linux and there is no installer or standardized path for it.


The multi repository projects can be easily remedied by using `repo`
tool or git submodules.
Although that is true, it would break the current seemingly desirable feature of the Zephyr codebase, and that is it being self-contained. Git submodules or repo add an additional layer of complexity to users, and that is not something I envision introducing at this point.

So far we have a nice, self-contained repo with a subset of (almost) untouched Nordic MDK files in the arch folder. We'll start with that and see how that works in the future.

Thanks,

Carles


RFC: remove zephyr-env.sh

Perez-Gonzalez, Inaky <inaky.perez-gonzalez@...>
 

Hello All

https://jira.zephyrproject.org/browse/ZEP-211 has been entered to track
a proposal to remove zephyr-env.sh.

<<from the ticket>>
Currently we require the user to source the zephyr-env.sh environment
to set up the build environment. This is awkward; things build fine
without it and the user still has to manually setup ZEPHYR_BASE,
ZEPHYR_GCC_VARIANT and ZEPHYR_SDK_INSTALL_DIR. zephyr-env.sh has grown
to be huge and most of the code seems to be just verifying the
environment where it is running.

Unless there is a pressing need for it, this is a proposal to remove
it.
<</>>

Does anyone have reasons not to remove it? I'd like to hear.

Thank you!


Re: Using IC manufacturer header files

Maciek Borzecki <maciek.borzecki@...>
 

On Tue, Apr 19, 2016 at 10:50 PM, Cufi, Carles
<Carles.Cufi(a)nordicsemi.no> wrote:
Hi Maciek,

-----Original Message-----
From: Maciek Borzecki [mailto:maciek.borzecki(a)gmail.com]
Sent: Tuesday, April 19, 2016 17:27
To: Nashif, Anas <anas.nashif(a)intel.com>
Cc: Cufi, Carles <Carles.Cufi(a)nordicsemi.no>;
devel(a)lists.zephyrproject.org
Subject: Re: [devel] Re: Using IC manufacturer header files

On Tue, Apr 19, 2016 at 5:04 PM, Nashif, Anas <anas.nashif(a)intel.com>
wrote:
Hi,
Any files or headers included from an external project that need to be
kept in sync with upstream can keep their style and formatting to allow
updates in the future. This was the approach taken when we imported the
IP stack from contiki AFAIK.

How about keeping the library/headers external and passing their
location in Kconfig? Similar to what buildroot does with external
kernel/toolchains. Is that acceptable too?

Not sure if you mean pulling the source code from another repository at build time, since I don't know what buildroot does with kernel/toolchains.
However the MDK is not currently available in a repository, instead it is released as a .zip file by Nordic every time it gets updated. We could of course mirror it in a repo, but that would anyway require us to update it manually every time a new version is available.
What I meant is that you keep the {S,M}DK externally and just
configure a path to it inside Kconfig. You can add a config entry like
CONFIG_STM32F1_STDLIB_PATH. Then in project/driver makefiles
-I$(CONFIG_STM32F1_STDLIB_PATH) and optionally add some files to the
build as needed. This has one major advantage, that the 3rd party code
is kept outside of Zephyr repository, so you basically don't care
about the naming/conventions/licenses 3rd party libraries would use.

As for mirroring 3rd party SDK in a repo. No matter the vendor's
distribution method I always pull the source code into a git
repository for two main reasons:
- I don't trust vendors to have a reasonable release management (more
often than not they prove to have none)
- you cannot rely on developers putting/extracting SDK's at proper
locations and setting their dev environment

The multi repository projects can be easily remedied by using `repo`
tool or git submodules.

Cheers,
--
Maciek Borzecki


Daily JIRA Digest

donotreply@...
 

NEW JIRA items within last 24 hours: 1
[ZEP-208] Zephyr's cc2520 driver (together with Zephyr's IP stack) does not handle fragmentation of packets properly
https://jira.zephyrproject.org/browse/ZEP-208


UPDATED JIRA items within last 24 hours: 0

CLOSED JIRA items within last 24 hours: 0

RESOLVED JIRA items within last 24 hours: 0


Daily Gerrit Digest

donotreply@...
 

NEW within last 24 hours:
- https://gerrit.zephyrproject.org/r/1544 : qmsi: pinmux: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1546 : Bluetooth: Kconfig: Split debug selection into a choice
- https://gerrit.zephyrproject.org/r/1545 : Bluetooth: Add dedicated bt_send() function
- https://gerrit.zephyrproject.org/r/1539 : net: Wakeup TX fiber when packet needs to be sent
- https://gerrit.zephyrproject.org/r/1537 : net: tcp: Disable client role
- https://gerrit.zephyrproject.org/r/1540 : net: ipv4: Print configured IPv4 address to console
- https://gerrit.zephyrproject.org/r/1543 : net: apps: Set IPv4 configuration when doing automatic testing
- https://gerrit.zephyrproject.org/r/1542 : net: apps: Fix IPv4 support in echo-server
- https://gerrit.zephyrproject.org/r/1541 : net: apps: Add TCP support to echo-server application
- https://gerrit.zephyrproject.org/r/1538 : net: 802.15.4: Do not compress TCP packets
- https://gerrit.zephyrproject.org/r/1536 : net: Enable TCP support
- https://gerrit.zephyrproject.org/r/1535 : net: Initial TCP support
- https://gerrit.zephyrproject.org/r/1533 : ci: test: footprint: increase
- https://gerrit.zephyrproject.org/r/1530 : samples: ipm: reset kernel binary name to zephyr

UPDATED within last 24 hours:
- https://gerrit.zephyrproject.org/r/1500 : qmsi: uart: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1506 : qmsi: flash: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1523 : spi: revert changes to API
- https://gerrit.zephyrproject.org/r/1521 : qmsi: use QMSI_LIBRARY instead of QMSI_DRIVERS
- https://gerrit.zephyrproject.org/r/1492 : qmsi: spi: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1353 : samples: Using new GPIO API callbacks
- https://gerrit.zephyrproject.org/r/914 : gpio: Improve the public API to handle multi callbacks
- https://gerrit.zephyrproject.org/r/1354 : cc2520: Using new GPIO API callbacks
- https://gerrit.zephyrproject.org/r/1271 : sensors: Using new GPIO API callbacks
- https://gerrit.zephyrproject.org/r/1445 : gpio: Deprecate API 1.0 callback function
- https://gerrit.zephyrproject.org/r/1508 : microkernel: use _thread_essential_set()
- https://gerrit.zephyrproject.org/r/1493 : qmsi: aio: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1514 : nanokernel: tighten _is_thread_essential()
- https://gerrit.zephyrproject.org/r/1499 : qmsi: adc: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1496 : qmsi: gpio: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1495 : qmsi: aon_counters: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1490 : qmsi: pwm: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1483 : qmsi: watchdog: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1453 : sensor: bmi160: move the printing macro to the header
- https://gerrit.zephyrproject.org/r/1457 : sensor: bmi160: add support for triggers
- https://gerrit.zephyrproject.org/r/1515 : sensor: bmi160: switch to the new logging API
- https://gerrit.zephyrproject.org/r/1458 : sensor: bmi160: add trigger tests to the application
- https://gerrit.zephyrproject.org/r/1452 : sensor: bmi160: add x86 app for Arduino101
- https://gerrit.zephyrproject.org/r/1454 : sensor: bmi160: create two wrappers for bmi160_reg_val_to_range
- https://gerrit.zephyrproject.org/r/1455 : sensor: bmi160: fix bmi160_reg_field_update function
- https://gerrit.zephyrproject.org/r/1451 : sensor: sensor.h: fix typo
- https://gerrit.zephyrproject.org/r/1450 : sensor: bmi160: move sample app to arc subdirectory
- https://gerrit.zephyrproject.org/r/1456 : sensor: bmi160: make some read/write functions global
- https://gerrit.zephyrproject.org/r/1498 : nble: Use string name for GPIO driver
- https://gerrit.zephyrproject.org/r/1522 : doc: add architecture porting guide
- https://gerrit.zephyrproject.org/r/1488 : qmsi: i2c: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1497 : samples: power: use string for driver name
- https://gerrit.zephyrproject.org/r/1482 : qmsi: rtc: use built-in qmsi driver
- https://gerrit.zephyrproject.org/r/1489 : pwm: unify driver names
- https://gerrit.zephyrproject.org/r/1491 : spi: use global init priority
- https://gerrit.zephyrproject.org/r/1494 : aio: rename sample name and make it generic
- https://gerrit.zephyrproject.org/r/1098 : drivers: add qmsi files for Quark MCUs
- https://gerrit.zephyrproject.org/r/1363 : sanitycheck: Add olimexino_stm32 board to sanitycheck
- https://gerrit.zephyrproject.org/r/1361 : boards/olimexino_stm32: add new board
- https://gerrit.zephyrproject.org/r/1325 : power_mgmt: Provide APIs for devices to signal busy to PM policy mgr
- https://gerrit.zephyrproject.org/r/1520 : ipm console: Implement flow control between sender and receiver
- https://gerrit.zephyrproject.org/r/1518 : build: fixes issue in windows Kconfig support
- https://gerrit.zephyrproject.org/r/1519 : doc: add step for windows build configuration
- https://gerrit.zephyrproject.org/r/1516 : x86: make GDT setup optional
- https://gerrit.zephyrproject.org/r/1278 : sensor: fix coding style (bmc150 and lsm9ds0)
- https://gerrit.zephyrproject.org/r/1263 : build: support icx llvm compiler
- https://gerrit.zephyrproject.org/r/1411 : build: add arc support to issm toolchain
- https://gerrit.zephyrproject.org/r/202 : doc: Add Arduino 101 Blinky application tutorial.

MERGED within last 24 hours:
- https://gerrit.zephyrproject.org/r/1534 : include: doc: added zephyr sdk
- https://gerrit.zephyrproject.org/r/1532 : include: doc script: use root makefile rule
- https://gerrit.zephyrproject.org/r/1531 : conf: new version and release vars from KERNELVERSION
- https://gerrit.zephyrproject.org/r/1528 : include: jira script: print a line between items
- https://gerrit.zephyrproject.org/r/1529 : newlib: fix export command
- https://gerrit.zephyrproject.org/r/1525 : sensors: bma280: compute accel scales based on data width
- https://gerrit.zephyrproject.org/r/1526 : sensors: bma280: fix accel data sample calculation
- https://gerrit.zephyrproject.org/r/1527 : sensors: bma280: add support for BMC150 accelerometer


Re: API deprecation / GPIO patch

Tomasz Bursztyka
 

Guys,

Thoughts or answers on that one:

Question: *_are we comfortable with preserving app compatibility, but
breaking driver compatibility?_* I am not so sure about this but would
like some feedback from the larger group.
Tomasz


Re: [RFCv2 1/2] misc: Add timer API

Tomasz Bursztyka
 

Hi Luiz,



+
+struct sys_timer_queue {
+ nano_thread_id_t __thread_id;
+ struct sys_timer *__head;
+ size_t stack_size;
+ char stack[0];
+};
+
+typedef void (*sys_timer_func)(void *user_data);
Giving the pointer on current sys_timer instead you could:

+
+struct sys_timer {
+ struct nano_timer __nano_timer;
+ sys_timer_func func;
+ void *user_data;
remove that pointer, and up to user to use CONTAINER_OF() if he wants to.
(yes I am reusing ideas from Johan ;) )

+ struct sys_timer *__next;
+};
+
+/**
+ * @brief Init timer queue
+ *
+ * @param tq Timer queue object that should be initialized
+ * @param stack_size The stack size in bytes
+ */
+int sys_timer_queue_init(struct sys_timer_queue *tq, size_t stack_size);
Are you ever going to use another queue besides the system's one (the
global one)?
It could simplify a lot the API then. Unless I miss to see some use case
for custom queue

+
+/**
+ * @brief Add timer to a timer queue
+ *
+ * Both timer and timer queue must be initialized.
+ *
+ * @param tq Timer queue object to add the timer
And continuing on that idea, is it then worth asking to which queue you
want to add a timer?

+ * @param t Timer object to be added
+ * @param ticks Amount of time in ticks to wait
+ *
+ * @return Zero on success or (negative) error code otherwise.
+ */
+int sys_timer_queue_add(struct sys_timer_queue *tq, struct sys_timer *t,
+ int ticks);
so you could remove tq parameter.

+
+/**
+ * @brief Cancel timer
+ *
+ * Remove timer from timer queue and stop it so its callback is not called.
+ *
+ * @param tq Timer queue object where the timer was added
+ * @param t Timer object to be canceled
+ *
+ * @return Zero on success or (negative) error code otherwise.
+ */
+int sys_timer_queue_cancel(struct sys_timer_queue *tq, struct sys_timer *t);
here as well.

+
+/**
+ * @brief Init timer
+ *
+ * @param t Timer object to be initialized
+ * @param func Function callback to be called when timer expires
+ * @param user_data Data to be passed in the callback
+ *
+ * @return Zero on success or (negative) error code otherwise.
+ */
+int sys_timer_init(struct sys_timer *t, sys_timer_func func, void *user_data);
+
+/**
+ * @brief Add timer to the global timer queue
+ *
+ * Timer must be initialized.
+ *
+ * @param _t Timer object to be added
+ * @param _ticks Amount of time in ticks to wait
+ *
+ * @return Zero on success or (negative) error code otherwise.
+ */
+#define sys_timer_add(_t, _ticks) sys_timer_queue_add(__global_tq, _t, _ticks)
+
+/**
+ * @brief Cancel global timer
+ *
+ * Remove timer from the global timer queue and stop it so its callback is not
+ * called.
+ *
+ * @param _t Timer object to be canceled
+ *
+ * @return Zero on success or (negative) error code otherwise.
+ */
+#define sys_timer_cancel(_t) sys_timer_queue_cancel(__global_tq, _t)
And these 2 defines would also go away.

And struct sys_timer_queue could also be private.

Tomasz


Re: [RFC] Zephyr IoT Protocols

Joakim Eriksson
 

I fully agree with your plan of implementing the IoT protocols but have a few questions related to the
planned implementation.

What is the minimal SoC required for the implementation of let’s say
DTLS + CoAP + LWM2M + Application (assumed to be small)?

There are a lot of SoC out there in real IoT products already that probably could replace
whatever codebase they currently run (Zigbee, or other 802.15.4, RIOT, Contiki, mbed OS, etc)
so this is very interesting for me at least.

In Contiki we have a stack that can run on STM32W108 which has 16 KB of RAM and 256 KB of
flash that we used for IPSO Interop last year and had the following:
- Full Yanzi Application Layer Stack
- DTLS + CoAP + LWM2M + IPSO Objects

Running in Yanzi LED Lamps and Yanzi Smart Plugs and they was interop tested with Wakaama
and Leshan plus other implementations at the event. This codebase is already in upstream Contiki.

Thread + Nest’s application layer seems to need more than Thread/6LoWPAN + LWM2M and therefore
requires larger devices - I am very interested in seeing which is your target for IoT devices that needs
connectivity and a full IP + Application stack.

Personally i would also vote for calling these IoT protocols rather than M2M protocols. All of them
are Internet protocols and M2M sounds more M-Bus, modbus, and the likes to me. So the title
“Zephyr IoT protocols” i do like, but I suggest having them in something that is IP-network related
rather than in a lib/m2m folder.


Also, there are plenty of reference implementations out there. I think you should at least also have
Leshan (LWM2M server/client library) in the list also, and possibly the Contiki implementations of
MQTT/LWM2M that might not be reference implementations but would match an implementation
that would fit constrained IoT devices (which will be a part of the future as long as RAM costs energy
to retain and money to buy).


Best regards,
— Joakim Eriksson, SICS

On 19 Apr 2016, at 23:22, Santes, Flavio <flavio.santes(a)intel.com> wrote:

[RFC] Zephyr IoT Protocols


Problem definition

Currently, Zephyr does not support any machine-to-machine protocol
(M2M). These protocols were defined for inter-communication between
devices or nodes in a constrained networking environment [1].


Aim and scope

This document proposes the integration of M2M protocols to Zephyr.
Specifically, we are interested in the following protocols:
CoAP [3], LightweightM2M (LWM2M) [2], MQTT [4] and MQTT-SN [5].


State of the art

LWM2M is a device-management protocol frequently used on the top of
CoAP. CoAP requires UDP as a transport protocol. Optionally, CoAP may
use DTLS (on the top of UDP) for securing communications between
devices.

On the other hand, MQTT, a publish-subscribe messaging protocol,
requires TCP as a transport protocol and may use TLS to create a secure
channel.

A lightweight version of MQTT for sensor networks (MQTT-SN) can be
deployed in almost any kind of network and it doesn't require any
specific transport protocol. MQTT-SN was created with short-range
wireless networks in mind. However, MQTT-SN can be used with wired or
wireless networks.

Support for short-range wireless communications in Zephyr is present
via Bluetooth. Furthermore, the Zephyr's TCP/IP stack only offers UDP.
So far, TCP is work in progress. Zephyr offers a networking security
layer by means of DTLS.


Reference implementations

LWM2M + CoAP (wakaama)
A plain C BSD-licensed implementation of LWM2M is available at [6].

MQTT client library
A client-side plain C implementation, licensed under the Eclipse
Public License [7].

MQTT client full implementation
Project Paho provides a full implementation in C++ licensed under
the Eclipse Public License [8].

MQTT-SN
C library licensed under the Eclipse Public License [9].


Integrating M2M protocols with Zephyr

MQTT relies on TCP as a transport protocol, and this could be an
stopper. So far, there is no evidence of any other inconvenience for
integrating LWM2M, CoAP and MQTT-SN.

Proposed roadmap

The integration of the M2M protocols could fit into the lib/m2m
directory (to be created), generating a directory per protocol:

lib/
|-- m2m
|-- coap
|-- lwm2m
|-- mqtt
|-- mqtt_sn

MQTT-SN can be the first protocol added to Zephyr, given the few
dependencies involved in its development and testing. We can continue
with CoAP/LWM2M and finally with MQTT. However, it's not clear if the
Zephyr's licenses are compatible with the Eclipse Public License, so
this could be a stopper (we can always write the code from scratch if
this license is incompatible with Zephyr's).

References

[1] Terminology for Constrained-Node Networks
C. Bormann, et al.
May, 2014
<https://www.rfc-editor.org/rfc/rfc7228.txt>

[2] OMA Lightweight M2M (LWM2M) protocol
Open Mobile Alliance
December, 2013
<http://openmobilealliance.hs-sites.com/
lightweight-m2m-specification-from-oma>

[3] The Constrained Application Protocol (CoAP)
Z. Shelby, et al.
June, 2014
<https://tools.ietf.org/html/rfc7252>

[4] MQTT Version 3.1.1
Andrew Banks and Rahul Gupta
October 2014
<http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.pdf>

[5] MQTT For Sensor Networks Protocol Specification Version 1.2
Andy Stanford-Clark and Hong Linh Truong
November, 2014
<http://mqtt.org/new/wp-content/uploads/2009/06/
MQTT-SN_spec_v1.2.pdf>

[6] LWM2M implementation - Wakaama Project
Eclipse Foundation
<https://github.com/eclipse/wakaama>

[7] MQTT C client - Eclipse Paho
Eclipse Foundation
<https://www.eclipse.org/paho/clients/c/>
GIT repo:
<https://github.com/eclipse/paho.mqtt.c>

[8] MQTT C/C++ Embedded clients - Eclipse Paho
Eclipse Foundation
<https://www.eclipse.org/paho/clients/c/embedded/>
GIT repo:
<http://git.eclipse.org/c/paho/
org.eclipse.paho.mqtt.embedded-c.git>

[9] MQTT-SN - Eclipse Paho
Eclipse Foundation
<https://www.eclipse.org/paho/clients/c/embedded-sn/>
GIT repo:
<http://git.eclipse.org/c/paho/
org.eclipse.paho.mqtt-sn.embedded-c.git/>


[RFC] Zephyr IoT Protocols

Flavio Santes <flavio.santes@...>
 

[RFC] Zephyr IoT Protocols


Problem definition

Currently, Zephyr does not support any machine-to-machine protocol 
(M2M). These protocols were defined for inter-communication between 
devices or nodes in a constrained networking environment [1].


Aim and scope

This document proposes the integration of M2M protocols to Zephyr. 
Specifically, we are interested in the following protocols: 
CoAP [3], LightweightM2M (LWM2M) [2], MQTT [4] and MQTT-SN [5].


State of the art

LWM2M is a device-management protocol frequently used on the top of 
CoAP. CoAP requires UDP as a transport protocol. Optionally, CoAP may 
use DTLS (on the top of UDP) for securing communications between 
devices.

On the other hand, MQTT, a publish-subscribe messaging protocol, 
requires TCP as a transport protocol and may use TLS to create a secure
channel. 

A lightweight version of MQTT for sensor networks (MQTT-SN) can be 
deployed in almost any kind of network and it doesn't require any 
specific transport protocol. MQTT-SN was created with short-range 
wireless networks in mind. However, MQTT-SN can be used with wired or
wireless networks.

Support for short-range wireless communications in Zephyr is present
via Bluetooth. Furthermore, the Zephyr's TCP/IP stack only offers UDP. 
So far, TCP is work in progress. Zephyr offers a networking security
layer by means of DTLS.


Reference implementations 

  LWM2M + CoAP (wakaama)
    A plain C BSD-licensed implementation of LWM2M is available at [6].

  MQTT client library
    A client-side plain C implementation, licensed under the Eclipse
    Public License [7].

  MQTT client full implementation
    Project Paho provides a full implementation in C++ licensed under
    the Eclipse Public License [8].

  MQTT-SN
    C library licensed under the Eclipse Public License [9].


Integrating M2M protocols with Zephyr

MQTT relies on TCP as a transport protocol, and this could be an 
stopper. So far, there is no evidence of any other inconvenience for 
integrating LWM2M, CoAP and MQTT-SN.

Proposed roadmap

The integration of the M2M protocols could fit into the lib/m2m 
directory (to be created), generating a directory per protocol:

lib/
|-- m2m
    |-- coap
    |-- lwm2m
    |-- mqtt
    |-- mqtt_sn

MQTT-SN can be the first protocol added to Zephyr, given the few 
dependencies involved in its development and testing. We can continue
with CoAP/LWM2M and finally with MQTT. However, it's not clear if the 
Zephyr's licenses are compatible with the Eclipse Public License, so 
this could be a stopper (we can always write the code from scratch if
this license is incompatible with Zephyr's).

References

[1] Terminology for Constrained-Node Networks
    C. Bormann, et al.
    May, 2014
    <https://www.rfc-editor.org/rfc/rfc7228.txt>

[2] OMA Lightweight M2M (LWM2M) protocol
    Open Mobile Alliance
    December, 2013
    <http://openmobilealliance.hs-sites.com/
    lightweight-m2m-specification-from-oma>

[3] The Constrained Application Protocol (CoAP)
    Z. Shelby, et al.
    June, 2014
    <https://tools.ietf.org/html/rfc7252>

[4] MQTT Version 3.1.1 
    Andrew Banks and Rahul Gupta 
    October 2014
    <http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.pdf>

[5] MQTT For Sensor Networks Protocol Specification Version 1.2
    Andy Stanford-Clark and Hong Linh Truong    
    November, 2014
    <http://mqtt.org/new/wp-content/uploads/2009/06/
    MQTT-SN_spec_v1.2.pdf>

[6] LWM2M implementation - Wakaama Project
    Eclipse Foundation
    <https://github.com/eclipse/wakaama>

[7] MQTT C client - Eclipse Paho
    Eclipse Foundation
    <https://www.eclipse.org/paho/clients/c/>
    GIT repo:
    <https://github.com/eclipse/paho.mqtt.c>

[8] MQTT C/C++ Embedded clients - Eclipse Paho
    Eclipse Foundation
    <https://www.eclipse.org/paho/clients/c/embedded/>
    GIT repo:
    <http://git.eclipse.org/c/paho/
    org.eclipse.paho.mqtt.embedded-c.git>

[9] MQTT-SN - Eclipse Paho
    Eclipse Foundation
    <https://www.eclipse.org/paho/clients/c/embedded-sn/>
    GIT repo:
    <http://git.eclipse.org/c/paho/
    org.eclipse.paho.mqtt-sn.embedded-c.git/>


Re: Using IC manufacturer header files

Carles Cufi
 

Hi Anas,

-----Original Message-----
From: Nashif, Anas [mailto:anas.nashif(a)intel.com]
Sent: Tuesday, April 19, 2016 17:46
To: Maciek Borzecki <maciek.borzecki(a)gmail.com>
Cc: Cufi, Carles <Carles.Cufi(a)nordicsemi.no>;
devel(a)lists.zephyrproject.org
Subject: Re: [devel] Re: Re: Using IC manufacturer header files

Hi,
Preference is to keep the most essential parts for building a basic
Zephyr application on a specific board self-contained without external
dependencies.
Sounds good. However please bear in mind that those are SoC include headers, not board ones. They define registers and memory maps for the different Nordic System-on-Chip.

Given the current feedback, we'll proceed to use the original header files directly and without modification by simply including them in the Zephyr codebase.

Thanks,

Carles


Re: Using IC manufacturer header files

Carles Cufi
 

Hi Maciek,

-----Original Message-----
From: Maciek Borzecki [mailto:maciek.borzecki(a)gmail.com]
Sent: Tuesday, April 19, 2016 17:27
To: Nashif, Anas <anas.nashif(a)intel.com>
Cc: Cufi, Carles <Carles.Cufi(a)nordicsemi.no>;
devel(a)lists.zephyrproject.org
Subject: Re: [devel] Re: Using IC manufacturer header files

On Tue, Apr 19, 2016 at 5:04 PM, Nashif, Anas <anas.nashif(a)intel.com>
wrote:
Hi,
Any files or headers included from an external project that need to be
kept in sync with upstream can keep their style and formatting to allow
updates in the future. This was the approach taken when we imported the
IP stack from contiki AFAIK.

How about keeping the library/headers external and passing their
location in Kconfig? Similar to what buildroot does with external
kernel/toolchains. Is that acceptable too?

Not sure if you mean pulling the source code from another repository at build time, since I don't know what buildroot does with kernel/toolchains.
However the MDK is not currently available in a repository, instead it is released as a .zip file by Nordic every time it gets updated. We could of course mirror it in a repo, but that would anyway require us to update it manually every time a new version is available.

Thanks,

Carles


Re: Using IC manufacturer header files

Carles Cufi
 

Hi there,

-----Original Message-----
From: Nashif, Anas [mailto:anas.nashif(a)intel.com]
Sent: Tuesday, April 19, 2016 17:04
To: Cufi, Carles <Carles.Cufi(a)nordicsemi.no>;
devel(a)lists.zephyrproject.org
Subject: Re: [devel] Using IC manufacturer header files

Hi,
Any files or headers included from an external project that need to be
kept in sync with upstream can keep their style and formatting to allow
updates in the future. This was the approach taken when we imported the
IP stack from contiki AFAIK.
Sounds like a good idea. Since those files are part of the Nordic MDK, which sees frequent releases, I also think it would not make sense to try and convert them to a different coding style.

Thanks,

Carles


Re: Setting correct ZEPHYR_GCC_VARIANT

Eric Zaluzec <Eric.Zaluzec@...>
 

Thanks Anas. This documentation was what I was missing.

As regards to building with Yocto, it was not necessary use case. The use case was just a quick way to build & rebuild the zephyr kernel with multiple machine types. This is not required.

7561 - 7580 of 8114