Re: Seeing and running what CI does (was: CODEOWNERS check in CI)
A couple things I find confusing:
- There's only the ("beautified") output but not the commands that produced them as is typical in CI, so no obvious indication of what to do to test the next git push and make sure it will actually fix the issues before pushing again.
- The comments with the output are typically some distance away from the red/green "traffic lights" with unrelated stuff in between
- The "Details" buttons on the other hand *are* in the traffic lights and seem really designed to point at the output, I think most people expect them to but they point at the documentation instead.
> you do not have to look at the actual CI log which might be verbose and misleading
Well (most of) the actual CI log is or should be what one gets anyway when running the sanitycheck / compliance.py / etc. scripts locally so if that output is confusing then users will just submit patches to make it better (as I just did) and keep everything consistent.
The less difference and distance between users and CI, the better.
> The actual output is always available in a comment posted by zephyrbot in the same PR. The comment is updated for every run
It's interesting that this comment preserves some review history unlike the aggressive way the "traffic lights" in particular and github in general rewrite and bury review history. So this is Good but unfortunately inconsistent with (Bad) github so a bit confusing again. For instance it's difficult to relate the history of this comment to the corresponding git pushes and commits.
> We are planning at some point to use the Checks API from GH which would make things more clear.
Do you know some other (unrelated) project(s) to look at that already use this API?
"Nashif, Anas" <anas.nashif@...>
Status is actually available as a comment in the PR. We are planning at some point to use the Checks API from GH which would make things more clear.
The scripts now provide the output so you do not have to look at the actual CI log which might be verbose and misleading. If any information is missing in the comments posted by CI, please let us know.
<devel@...> on behalf of Marc Herbert <marc.herbert@...>
I asked this question on Slack some time back and Anas answered it's not possible (yet?) for various reasons, one of them related to github accounts and security.
Fortunately something even more useful is already possible: running the checks yourself. Just run this compliance_script.py script:
I know run this script *before* uploading anything to github as it obviously saves round trips and time.
You also need the "sanitycheck" script in the other, main repo.
I think sanitycheck is mentioned in the online documentation and the check_compliance about to be.
On a related CI topic, here's why the code coverage report is very often bogus:
https://github.com/codecov/codecov-bash/issues/83 thumbs up to "vote" for it
<devel@...> on behalf of Stenberg <bjorn@...>
Can we please make the "Details" links for these checks show the actual output of the check command, rather than the current generic information about why the check exists?