Quick update.

1. [*] "yay!"... except it's not clear why tests run by Shippable are not deterministic: https://github.com/zephyrproject-rtos/zephyr/pull/13803 randconfig maybe? I digress.

Non-determinism found and explained: https://github.com/zephyrproject-rtos/zephyr/pull/13973

Thanks to the reviewers for the very quick reviews.

2. As the opposite experience, I also got for another work in progress / hack / draft PR:
Documentation build with 'make htmldocs' failed.
... and that's it; no logs or error message whatsoever, just this line. I have some guesses of what fails in CI while everything works for me (it is a documentation _hack_) but zero information on what exactly or how.

Still unclear.

3. In the same draft PR as 2. I "forgot" a license header in a temporary file. While it does not provide the command and log, the zephyrbot told me in very nice terms what was wrong. What's still confusing however is why the zephyr-ci-tools/scripts/check_compliance.py [-m Licence] script which I did run before pushing keeps telling me everything is fine and didn't save me the round trip time (and embarrassment). A typo would also cost (another) round-trip. This script really looks like the check zephyrbot runs but... it's apparently not or not quite?

Mystery solved: https://github.com/zephyrproject-rtos/ci-tools/pull/25


Another mystery being fixed by Ulf: CODEOWNERS failing locally but not in CI
https://github.com/zephyrproject-rtos/ci-tools/pull/23
https://github.com/zephyrproject-rtos/zephyr/pull/13978

Any layer of indirection in CI creates distance between developers and CI and makes developers care less about failures that don't affect them directly. Developers are not "consumers" and the vast majority seems to prefer having too much information than too little and solve problems by themselves but if the only option they have is not to care about some failures then they don't.

There may be reasons for not all CI details to be available and/or easy to reproduce but they don't apply to stuff like building docs or license checks or other simple sanitycheck.


1. https://app.shippable.com/github/zephyrproject-rtos/zephyr/runs/35577
2. & 3. https://github.com/zephyrproject-rtos/zephyr/pull/13805