Hi there,

-----Original Message-----
From: users@... <users@...> On
Behalf Of Jørn Villesen Christensen via Lists.Zephyrproject.Org
Sent: 11 October 2019 21:20
To: users@...
Cc: users@...
Subject: [Zephyr-users] Recursive checkout with West

Hi Zephyr community,

I am a little bit uncertain whether this is the correct list to ask for
help; if not I would appreciate if you could guide me to the correct
place :-)

TLDR: (Current) main question: Is it possible to make West work with
recursive West-based projects? I.e. have a top level project (managed by
West) that has another dependency (Zephyr) that also have a west.yml
file?

A little more background:

I am new to Zephyr and to West, but I am working on a project where we
are considering to use both :-) We do have other external dependencies
and we also have a variety of products that a) uses the same
dependencies (drivers, controllers, etc.) and b) may or may not use
Zephyr, or different configurations of Zephyr.

So in a learning step, we tried to make a top level project that would
depend on Zephyr and another project. The intention was just to see how
West and the build system would behave. What happens is that West is
able to check out the project and to download the dependencies in the
top level west.yml. However, it does not download the dependencies of
Zephyr.

Is this possible? Or is this really unintended use of West (and Zephyr)?

Not currently, but work to support this is ongoing right now, since this has been requested multiple times already and we realize it is a critical feature to have.

Feel free to track progress and add your requirements here:
https://github.com/zephyrproject-rtos/west/issues/221

Thank you for your response and link to the issue you are working on. But it seems to me that what you are working on / discussing is a slightly different issue - or at least different solution (than I had imagined) to the challenges I face. Let me sketch it out.

My vision / idea (a.k.a. recursive parsing):
I envisioned that we had a top level project with a west.yml. Upon running west update, west would check out project dependencies (as today), but then it would examine each of these projects to see if they would contain a west.yml. If so, it would run a west update on those as well, ignoring that it is already in a (parent) west project.

What I understand you are creating:
You are creating functionality to merge multiple west.yml files into one. If I understand correctly, the idea is to write a top-level west template file (west-top.yml) and then merge this with all west.yml files from the project dependencies (that are west projects). This creates a merged west.yml file that the top level project need to maintain.

Have you (as the Zephyr community) considered the recursive parsing approach?

Below are my thoughts on (implementing) this. If you find it interesting, I would love some feedback. If you already considered this (and decided against it) feel free to skip reading the last part of my mail. But I would like to hear your thoughts on the subject :-) (Perhaps you can link me to a discussion elsewhere :) ).

=== Longer story: ===

The main benefits of recursive parsing (as I see it):
The top level project would only have a single west.yml file to maintain - and it would be short. It would not need to keep track of all dependency dependencies. If a project dependency is to be updated, you would not need to run the merge command again (and resolve possible conflicts from intermittent changes).

Challenges in build procedure:
I am not that familiar with West's build system - except that it uses cmake and ninja in the background - but I do realise that special care need to be taken when building project dependencies. As I see it, there are two situations:

1. If the project is *not* a west-managed project: How are these compiled and linked today? [1]   I would expect the procedures to be transferable to the recursive parsing build procedure - with minor tweaks.
2. If the project *is* a west-managed project: Normally this project would be the top project and need to be linked to a final executable. But in the recursive parsing scheme, these projects need to be compiled only, and then only linked in the top level project. I guess such logic would be able to be build into the West build system.

[1]: Not really a question. I don't know it, but it is on my todo list to read up upon. So ignore this question unless it has significance for this discussion.

Then there is an issue about duplicated (sub) dependencies:
Say we have a top level project that depends on suba and subb. Each of these two sub projects depends on subc. Here are two situations:

1. suba and subb both depend on the same version of subc.
2. suba and subb depend on different versions of subc.

I think the discussion on how to handle this should be deferred to another place than this (already too long) mail - but I guess this is an issue that would also have to be considered in the merge-yml-files approach that you are working on atm.

If you got this far: Thank you for taking your time to read my thoughts :-)

BR
Jørn