Re: Managing/structuring multi image/binary projects for (OTA updatable products etc)

Jan Kloetzke <jan@...>

Hi Marc,

On Mon, Dec 09, 2019 at 07:38:53AM -0800, Marc Reilly wrote:
Hi all,

I've been looking at making product(s) which can do OTA updates, and
how to structure the overall project.
A basic aim is to be able to checkout and build (assuming toolchain
and environment is setup) with minimal commands, and also minimal
manual 'configuration' changes required for a default build.
I'm not really feeling like I'm seeing a nice solution, so was
wondering if anyone else has overcome a similar situation, or anyone
has any ideas to progress.

Following, I've tried to summarize the problem & design as I see it to
make sure I'm on the right track, and then I've got a few example
commands to further illustrate where the problem is for me.

Any and all suggestions welcome.
We faced similar problems when we had to make an IVI system that uses
bare metal + FreeRTOS + Linux in one SoC and requires a consistent build
for all of them. In the end we created our own meta-tool:

Now, Zephyr already came up with the west tool which solves some of the
problems and drives many Zephyr use cases better than Bob could at the
moment. Nonetheless I created some demo recipes a couple of weeks ago to
show how a multi-application-build could look like:

Now, I'm perfectly aware that it creates another layer of complexity.
The recipes effectively replace west to drive the build. But Bob is
meant to solve many other problems too that are constantly coming up
with such more-than-one-image embedded systems: variants management,
reliable incremental building, reproducibility, toolchain setup, Jenkins
integration and automatic artifact caching. Also building and running
unit tests from the same sources on the host in one command was a
requirement for us. The goal is to always execute just one build

$ git clone <project>
$ cd <project>
$ bob build <image>

and start working from there on:

... hack on the source ...
bob build <image>
bob build <unit-tests>
git commit/push/pull
... repeat ...

We're currently expanding the public-visible recipes. There is another
example which shows how FreeRTOS and Linux share some code are built
together as AMP system:

If your product stays in the nRF+mcuboot+zephyr world then sticking to
the nRF Connect SDK may be easier. Otherwise Bob might be worth a look




The product acheives OTA updates, at a general level, by:
- use mcuboot as the bootloader
- app implements SMP service (eg mcumgr/smp_svr example)
- initially program the product with a combination of the mcuboot + app
- further updates can be done OTA, the app handles transmission and copying of the new firmware image to a partition on flash, then resets. the bootloader checks the new firmware image does some partition 'accounting' and then boots the new image.

Typical project elements/dependencies:
- app for the product/device
- board files are defined in the 'app' repo
- common libs (eg for our GATT services which most our devices have)
- zephyr framework
- dependent modules (eg nordic_hal, segger)
- mcuboot
- (references the app board file)

This resembles either the T2 or T3 of west's documented tolopogies. []

- We want 2 primary build artifacts:
- the app (configured as launched from a bootloader)
- bootloader mcuboot
- And then from this with can run a variety of commands to merge and/or sign the build artifacts.
- As a bonus, perhaps another app configuration to make dev/debug easier where the bootloader is not used.


West seems like the ideal tool to use for this, it boils down to a manifest repo (for project & dependencies checkout), and then a series of commands to build the various artifacts and merge & sign them. For example:

west init -m git@...:SomeManifestRepo.git
west build --board=theboard -d build-mcuboot -s mcuboot/boot/zephyr -- -DBOARD_ROOT=/abs/path/to/app
west build --board=theboard -d build-app -s app -- -DBOARD_ROOT=/abs/path/to/app
west sign ...

However, it would be nice to be able to glue this all together with some script etc to simplify the extra elements of the west commands ('theboard' 'BOARD_ROOT') etc. And here is where I become unsure of how this glue script/(c)makefile/etc fits into the topology of the project.. The natural place for the script to run from is the west installation folder itself (ie, the parent folder of all the subprojects) however the natural place for it to 'live' is in the 'manifest-repo'.

- copy 'glue' script into west installation folder ?
- west commands ? in the manifest repo, or maybe another common lib etc.

Join to automatically receive all group messages.