From the ELC-E Zephyr MultiImage BoF hosted by Marti there was some reference to using a tool like bitbake. There was mention of a tool called Bob.

Sending a reference here for all the be able to look at Bob:

https://bobbuildtool.dev/

- k

There were also references to a “partition manager” application/script.
Anybody have a reference to that?

--
Henrik Brix Andersen

You can find the documentation for the tool here - http://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/scripts/partition_manager/partition_manager.html

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded example is the most interesting one to the Zephyr community (even though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple dependent Zephyr images then we can try to setup some Bob recipes that builds the same thing...

/Jan

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded example is the most interesting one to the Zephyr community (even though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple dependent Zephyr images then we can try to setup some Bob recipes that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

- k

To what degree does Bob require Linux tools like bash?

It is not an option to introduce bash as a dependency for Windows users.

Can it be used without bash?

________________________________________
From: devel@lists.zephyrproject.org <devel@lists.zephyrproject.org> on behalf of Kumar Gala via Lists.Zephyrproject.Org <kumar.gala=linaro.org@lists.zephyrproject.org>
Sent: Wednesday, October 30, 2019 8:05 AM
To: jan@kloetzke.net
Cc: devel@lists.zephyrproject.org
Subject: Re: [Zephyr-devel] ELC-E Zephyr Multiimage BoF - Bob Build Tool

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded example is the most interesting one to the Zephyr community (even though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple dependent Zephyr images then we can try to setup some Bob recipes that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

- k

AFAICT it heavily integrates with bash making it a no-go.

________________________________________
From: devel@lists.zephyrproject.org <devel@lists.zephyrproject.org> on behalf of Sebastian Boe via Lists.Zephyrproject.Org <Sebastian.Boe=nordicsemi.no@lists.zephyrproject.org>
Sent: Thursday, October 31, 2019 9:57 AM
To: jan@kloetzke.net; kumar.gala@linaro.org
Cc: devel@lists.zephyrproject.org
Subject: Re: [Zephyr-devel] ELC-E Zephyr Multiimage BoF - Bob Build Tool

To what degree does Bob require Linux tools like bash?

It is not an option to introduce bash as a dependency for Windows users.

Can it be used without bash?

________________________________________
From: devel@lists.zephyrproject.org <devel@lists.zephyrproject.org> on behalf of Kumar Gala via Lists.Zephyrproject.Org <kumar.gala=linaro.org@lists.zephyrproject.org>
Sent: Wednesday, October 30, 2019 8:05 AM
To: jan@kloetzke.net
Cc: devel@lists.zephyrproject.org
Subject: Re: [Zephyr-devel] ELC-E Zephyr Multiimage BoF - Bob Build Tool

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded example is the most interesting one to the Zephyr community (even though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple dependent Zephyr images then we can try to setup some Bob recipes that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

- k

Hi Sebastian,

On Thu, Oct 31, 2019 at 09:17:01AM +0000, Sebastian Boe wrote:

AFAICT it heavily integrates with bash making it a no-go.

IMHO this depends on what Bob should be used for in the context of
Zephyr. At the BoF session most people agreed that the Multiimage
problem goes beyond Zephyr alone. If Bob (in the current version) is a
viable alternative really depends on the use case.

To what degree does Bob require Linux tools like bash?

At the moment bash is still a hard requirement. There are some other
unix tools required but this is going away in the future.

It is not an option to introduce bash as a dependency for Windows users.

You can use Bob on Windows via MSYS2, though. But I agree that this may
not be an option for everybody at the moment.

Can it be used without bash?

This is in the works. Windows support is definitely on the agenda
because there are a bunch of people that have the same requirement. The
(currently stale) PR

https://github.com/BobBuildTool/bob/pull/258

will be updated in the coming weeks. This requires some amount of
refactoring internally so its taking some time. The idea is to make Bob
independent of the acutally used scripting language. The first step is
to support PowerShell on Windows and then maybe Python for
cross-platform scripts.

Regards,
Jan

________________________________________
From: devel@lists.zephyrproject.org <devel@lists.zephyrproject.org> on behalf of Kumar Gala via Lists.Zephyrproject.Org <kumar.gala=linaro.org@lists.zephyrproject.org>
Sent: Wednesday, October 30, 2019 8:05 AM
To: jan@kloetzke.net
Cc: devel@lists.zephyrproject.org
Subject: Re: [Zephyr-devel] ELC-E Zephyr Multiimage BoF - Bob Build Tool

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded example is the most interesting one to the Zephyr community (even though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple dependent Zephyr images then we can try to setup some Bob recipes that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

- k

Hi Kumar,

On Wed, Oct 30, 2019 at 02:05:36AM -0500, Kumar Gala wrote:

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded
example is the most interesting one to the Zephyr community (even
though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple
dependent Zephyr images then we can try to setup some Bob recipes
that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

I'm currently preparing some recipes. It will post them with some
instructions how to build in the next days once I got it working...

Regards,
Jan

"Jan Kloetzke via Lists.Zephyrproject.Org"
<jan=kloetzke.net@lists.zephyrproject.org> writes:

Hi Kumar,

On Wed, Oct 30, 2019 at 02:05:36AM -0500, Kumar Gala wrote:

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded
example is the most interesting one to the Zephyr community (even
though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple
dependent Zephyr images then we can try to setup some Bob recipes
that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

I'm currently preparing some recipes. It will post them with some
instructions how to build in the next days once I got it working...

Sounds great!

Please be aware of this in that case:

https://github.com/zephyrproject-rtos/zephyr/issues/19918

It would be good to see a solution that doesn't have this issue.

Thanks,
Marti

On Fri, Nov 01, 2019 at 09:24:59AM +0000, Bolivar, Marti wrote:

"Jan Kloetzke via Lists.Zephyrproject.Org"
<jan=kloetzke.net@lists.zephyrproject.org> writes:

Hi Kumar,

On Wed, Oct 30, 2019 at 02:05:36AM -0500, Kumar Gala wrote:

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded
example is the most interesting one to the Zephyr community (even
though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple
dependent Zephyr images then we can try to setup some Bob recipes
that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

I'm currently preparing some recipes. It will post them with some
instructions how to build in the next days once I got it working...

Ok, here it is: https://github.com/jkloetzke/zephyr-example-recipes

There is a README.md with a short overview. Basically you just have to
clone the recipes and build it:

$ git clone https://github.com/jkloetzke/zephyr-example-recipes.git \
--recurse-submodules
$ cd zephyr-example-recipes
$ bob dev demo::zephyr -j

Make sure that you have the latest version of Bob. The demo recipes use
the "layers" feature that has not been released yet. It's best to
install Bob via:

$ pip3 install [--user] git+https://github.com/BobBuildTool/bob

Otherwise have a look at

https://bob-build-tool.readthedocs.io/en/latest/installation.html

for further options.

Note that the recipes do not use west but clone the involved
repositories directly and calls cmake itself. This has the advantage
that there is only one tool that fetches/updates sources. Additionally
the OpenAMP sample was put into a dedicated repository because some
refactoring was needed, namely removing the "ExternalProject" link
between the images.

The recipes also show how a toolchain is defined in the recipes. There
is no need for the user to install a toolchain but the right one is
brought in by the recipes automatically. Usually this is done with all
involved tools to maximize reproducibility but this would have just made
the example more complicated.

Sounds great!

Please be aware of this in that case:

https://github.com/zephyrproject-rtos/zephyr/issues/19918

It would be good to see a solution that doesn't have this issue.

This is handled well by Bob. The explicit goal of Bob is to feel like
"make" even though Bob controls the build of whole packages.

Have fun.

/Jan

"Jan Kloetzke via Lists.Zephyrproject.Org"
<jan=kloetzke.net@lists.zephyrproject.org> writes:

On Fri, Nov 01, 2019 at 09:24:59AM +0000, Bolivar, Marti wrote:

"Jan Kloetzke via Lists.Zephyrproject.Org"
<jan=kloetzke.net@lists.zephyrproject.org> writes:

Hi Kumar,

On Wed, Oct 30, 2019 at 02:05:36AM -0500, Kumar Gala wrote:

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded
example is the most interesting one to the Zephyr community (even
though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple
dependent Zephyr images then we can try to setup some Bob recipes
that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

I'm currently preparing some recipes. It will post them with some
instructions how to build in the next days once I got it working...

Ok, here it is: https://github.com/jkloetzke/zephyr-example-recipes

There is a README.md with a short overview. Basically you just have to
clone the recipes and build it:

$ git clone https://github.com/jkloetzke/zephyr-example-recipes.git \
--recurse-submodules
$ cd zephyr-example-recipes
$ bob dev demo::zephyr -j

Make sure that you have the latest version of Bob. The demo recipes use
the "layers" feature that has not been released yet. It's best to
install Bob via:

$ pip3 install [--user] git+https://github.com/BobBuildTool/bob

Otherwise have a look at

https://bob-build-tool.readthedocs.io/en/latest/installation.html

for further options.

Note that the recipes do not use west but clone the involved
repositories directly and calls cmake itself. This has the advantage
that there is only one tool that fetches/updates sources.

Seems like zephyr.yaml is the file controlling this in a map called
checkoutSCM. (I haven't read the bob docs, but I'm sure that's covered
there.)

I made a quick hack west extension to convert the upstream zephyr
manifest into the format used there, in case it's useful:

https://github.com/mbolivar/tree/bob-the-builder

Example:

$ west gen-bob-checkout
checkoutSCM:
- commit: 07380358424edecdefeb28ed7cb43e27b530014d
dir: zephyr
scm: git
url: https://github.com/zephyrproject-rtos/zephyr
- commit: 04ff67a0826a51041e51034faf8fc4d3eeacd846
dir: modules/hal/atmel
scm: git
url: https://github.com/zephyrproject-rtos/hal_atmel
- commit: e01f3bce2a94847253369efb9a081f5c0e9ec882
dir: tools/ci-tools
scm: git
url: https://github.com/zephyrproject-rtos/ci-tools
- commit: 99129c5efc907ea613c4b73ccff07581feb58a7a
dir: modules/lib/civetweb
scm: git
url: https://github.com/zephyrproject-rtos/civetweb
[...]

Just in case it's useful to anyone who wants all the projects but
doesn't want to do the paperwork themselves.

Additionally
the OpenAMP sample was put into a dedicated repository because some
refactoring was needed, namely removing the "ExternalProject" link
between the images.

The recipes also show how a toolchain is defined in the recipes. There
is no need for the user to install a toolchain but the right one is
brought in by the recipes automatically. Usually this is done with all
involved tools to maximize reproducibility but this would have just made
the example more complicated.

Sounds great!

Please be aware of this in that case:

https://github.com/zephyrproject-rtos/zephyr/issues/19918

It would be good to see a solution that doesn't have this issue.

This is handled well by Bob. The explicit goal of Bob is to feel like
"make" even though Bob controls the build of whole packages.

Have fun.

/Jan

On Mon, Nov 04, 2019 at 10:34:37PM +0000, Bolivar, Marti wrote:

"Jan Kloetzke via Lists.Zephyrproject.Org"
<jan=kloetzke.net@lists.zephyrproject.org> writes:

On Fri, Nov 01, 2019 at 09:24:59AM +0000, Bolivar, Marti wrote:

"Jan Kloetzke via Lists.Zephyrproject.Org"
<jan=kloetzke.net@lists.zephyrproject.org> writes:

Hi Kumar,

On Wed, Oct 30, 2019 at 02:05:36AM -0500, Kumar Gala wrote:

On Oct 29, 2019, at 5:08 AM, jan@kloetzke.net wrote:

Hi,

I think the https://github.com/BobBuildTool/bob-example-embedded
example is the most interesting one to the Zephyr community (even
though it's using FreeRTOS+Linux at the moment).

If you guys have a particular example that already builds multiple
dependent Zephyr images then we can try to setup some Bob recipes
that builds the same thing...

/Jan

You can take a look at the open-amp sample, it creates 2 zephyr images:

https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/ipc/openamp

I'm currently preparing some recipes. It will post them with some
instructions how to build in the next days once I got it working...

Ok, here it is: https://github.com/jkloetzke/zephyr-example-recipes

There is a README.md with a short overview. Basically you just have to
clone the recipes and build it:

$ git clone https://github.com/jkloetzke/zephyr-example-recipes.git \
--recurse-submodules
$ cd zephyr-example-recipes
$ bob dev demo::zephyr -j

Make sure that you have the latest version of Bob. The demo recipes use
the "layers" feature that has not been released yet. It's best to
install Bob via:

$ pip3 install [--user] git+https://github.com/BobBuildTool/bob

Otherwise have a look at

https://bob-build-tool.readthedocs.io/en/latest/installation.html

for further options.

Note that the recipes do not use west but clone the involved
repositories directly and calls cmake itself. This has the advantage
that there is only one tool that fetches/updates sources.

Seems like zephyr.yaml is the file controlling this in a map called
checkoutSCM. (I haven't read the bob docs, but I'm sure that's covered
there.)

Yes, that's correct. It's covered here:

https://bob-build-tool.readthedocs.io/en/latest/manual/configuration.html#checkoutscm

I made a quick hack west extension to convert the upstream zephyr
manifest into the format used there, in case it's useful:

https://github.com/mbolivar/tree/bob-the-builder

Example:

$ west gen-bob-checkout
checkoutSCM:
- commit: 07380358424edecdefeb28ed7cb43e27b530014d
dir: zephyr
scm: git
url: https://github.com/zephyrproject-rtos/zephyr
- commit: 04ff67a0826a51041e51034faf8fc4d3eeacd846
dir: modules/hal/atmel
scm: git
url: https://github.com/zephyrproject-rtos/hal_atmel
- commit: e01f3bce2a94847253369efb9a081f5c0e9ec882
dir: tools/ci-tools
scm: git
url: https://github.com/zephyrproject-rtos/ci-tools
- commit: 99129c5efc907ea613c4b73ccff07581feb58a7a
dir: modules/lib/civetweb
scm: git
url: https://github.com/zephyrproject-rtos/civetweb
[...]

Just in case it's useful to anyone who wants all the projects but
doesn't want to do the paperwork themselves.

That looks indeed useful.

I was wondering if there is a way to select only the needed
repositories? West seems to always clone all known projects up-front.
OTOH Bob's approach is normally to checkout the sources individually for
each project and then build these individually. AFAICT this won't work
for Zephyr because all repositories must be present to be gathered in
one large CMake project. So the structure ends up with a single large
source package. Are there any plans to make this more modular in the
future?

Additionally
the OpenAMP sample was put into a dedicated repository because some
refactoring was needed, namely removing the "ExternalProject" link
between the images.

The recipes also show how a toolchain is defined in the recipes. There
is no need for the user to install a toolchain but the right one is
brought in by the recipes automatically. Usually this is done with all
involved tools to maximize reproducibility but this would have just made
the example more complicated.

Sounds great!

Please be aware of this in that case:

https://github.com/zephyrproject-rtos/zephyr/issues/19918

It would be good to see a solution that doesn't have this issue.

This is handled well by Bob. The explicit goal of Bob is to feel like
"make" even though Bob controls the build of whole packages.

Have fun.

/Jan

Hi,

"Jan Kloetzke via Lists.Zephyrproject.Org"
<jan=kloetzke.net@lists.zephyrproject.org> writes:

On Mon, Nov 04, 2019 at 10:34:37PM +0000, Bolivar, Marti wrote:

I made a quick hack west extension to convert the upstream zephyr
manifest into the format used there, in case it's useful:

https://github.com/mbolivar/tree/bob-the-builder

Example:

$ west gen-bob-checkout
checkoutSCM:
- commit: 07380358424edecdefeb28ed7cb43e27b530014d
dir: zephyr
scm: git
url: https://github.com/zephyrproject-rtos/zephyr
- commit: 04ff67a0826a51041e51034faf8fc4d3eeacd846
dir: modules/hal/atmel
scm: git
url: https://github.com/zephyrproject-rtos/hal_atmel
- commit: e01f3bce2a94847253369efb9a081f5c0e9ec882
dir: tools/ci-tools
scm: git
url: https://github.com/zephyrproject-rtos/ci-tools
- commit: 99129c5efc907ea613c4b73ccff07581feb58a7a
dir: modules/lib/civetweb
scm: git
url: https://github.com/zephyrproject-rtos/civetweb
[...]

Just in case it's useful to anyone who wants all the projects but
doesn't want to do the paperwork themselves.

That looks indeed useful.

I was wondering if there is a way to select only the needed
repositories? West seems to always clone all known projects up-front.

Plain "west update" updates all the repositories. You can say "west
update project-1 project-2 ..." to just work on a subset of them.

OTOH Bob's approach is normally to checkout the sources individually for
each project and then build these individually. AFAICT this won't work
for Zephyr because all repositories must be present to be gathered in
one large CMake project. So the structure ends up with a single large
source package. Are there any plans to make this more modular in the
future?

If you have cloned all the modules your application needs to build,
you're fine. You don't need to download all of them.

Indeed, you don't need west at all if you manually set ZEPHYR_MODULES
before building:

https://docs.zephyrproject.org/latest/guides/west/without-west.html#specifying-modules

But west doesn't manage dependencies between repositories. That would
take it into package manager territory. We are staying away from that
as a design decision.

It's up to the user to either manage their application's dependencies
explicitly, or just fetch the kitchen sink. Either way, Kconfig
ultimately controls what gets built into the final firmware image.

Marti