Re: Recursive checkout with West


Jørn Villesen Christensen <extjvc@...>
 

Hi Marti,

Thank you too - and for the links :)

On 16/10/2019 22.43, Bolivar, Marti wrote:
Hi Jørn,

Thank you for taking the time to write down your thoughts.

Jørn Villesen Christensen via Lists.Zephyrproject.Org

My vision / idea (a.k.a. recursive parsing):
I envisioned that we had a top level project with a west.yml. Upon running west update, west would check out project dependencies (as today), but then it would examine each of these projects to see if they would contain a west.yml. If so, it would run a west update on those as well, ignoring that it is already in a (parent) west project.

Doing this automatically would be undesirable to at those users who do
not want to automatically include all upstream modules in their
downstream (e.g. if they want to exclude modules that are only relevant
to arches, socs, etc. that they are targeting, for bandwidth
optimization or due to License/IP allergies).

Hm... I just assumed that if you were including a project you would always a) need all dependencies of that project, and b) be ok with their licenses. But ok... point accepted :-)


=== Longer story: ===

The main benefits of recursive parsing (as I see it):
The top level project would only have a single west.yml file to maintain - and it would be short. It would not need to keep track of all dependency dependencies. If a project dependency is to be updated, you would not need to run the merge command again (and resolve possible conflicts from intermittent changes).

As far as I can tell, the existing proposal described on GitHub
satisfies these requirements:

1. one west.yml
2. it's short
3. you don't have to manually track changes in other west.yml files

I was not aware of this work that you are doing. I only saw the #221 issue (West manifest imports) which (afaIcs) is more about merging several manifests files rather than the import feature you are documenting.

I read your documentation (most of it at least :) ) and it indeed sounds like the thing I was looking for. So thank you. Would you have a road map for when this feature would be implemented?

(I tested with my west 0.6.3 and it does not seem to recognize the import keyword.)

If that doesn't work for you, can you explain more about why your
specific proposed mechanism of automatically and recursively pulling in
all west.yml files in all subprojects is better?

I think your proposal is indeed what I am looking for. Yes, there is a slight functional difference whether it is recursive or not. I just assumed recursive would be beneficial / logical, but you made a point that it might not be what everybody wants. I think for our project the one-level import may be sufficient. At least for now.

(Note: I originally thought that these explicit imports would not be
recursive to start out, but I'm going to try to make that work.)

When I read your proposal I was thinking that the recursive-feature could be optional:

  • If import is boolean (today), this could be changed to enumeration with following keywords:
    • false, off, none: No import is done - same as false today.
    • true, top: Same as true today - yielding an error if any sub projects use import keyword
    • top-ignore: Same as true today but without yielding an error is sub projects use import keyword. Just ignore it.
    • recursive: Recursive import.
  • If import is mapping, one could add a key recursive => {false / false-ignore / true}
I am not sure whether there is need for this detailed control; I just wanted to share the idea.
Challenges in build procedure:
I think it's not really a build system problem as much as it is a west
init / west update problem.

The build sytem just uses 'west list' to get the default ZEPHYR_MODULES:

https://docs.zephyrproject.org/latest/guides/modules.html

Since 'west list' will still work the same way after this feature is
implemented, I expect no impact on zephyr's build system.

The meat of the feature is making west init and west update work
properly when there multiple files, some of which may also need to be
cloned from separate repositories.

Thank you for the info (west list and build system). I'll take your word for it that the challenge lies with init and update. :-) 

BR
Jørn


Join users@lists.zephyrproject.org to automatically receive all group messages.