You mean test that runs in an emulator or using the unit test type in ztest, which builds as a host binary?
From: Jeremy Bettis <jbettis@...>
Date: Thursday, February 3, 2022 at 3:37 PM
To: Mai, Enjia <enjia.mai@...>
Cc: Nashif, Anas <anas.nashif@...>, Hake Huang <hake.huang@...>, maciej.perkowski@... <maciej.perkowski@...>, testing-wg@... <testing-wg@...>, Wang, Guojun <guojun.wang@...>
Subject: Re: [testing-wg] Test coverage and its measurements
I've noticed there is only coverage for posix and qemu_x86 on https://app.codecov.io/gh/zephyrproject-rtos/zephyr.
Do you have any guides for making a unittest for non posix code?
For example, if I wanted to create a simple unit test for the code in /arch/arm/core where would I start? I'm fine with having to fake out most of the dependencies.
Thanks for bringing and clarifying that question.
And excuse me, Hake and all, I cannot attend this meeting tonight due to family activities of CNY.
Coverage reports and having test coverage is a test activity, has nothing to do with Safety. Maybe it was not discussed here in this group in the past, but it’s something that belongs here IMO.
Coverage analysis is not conducted in the testing working group instead there is another team who is testing the kernel API coverage. IIRC it is safety team, correct me if I am wrong @Mai.Enjia@...
I was recently asked by my colleague about quality standards related to Zephyr’s contributions, in particular to testing. A relevant section of our docs states:
“When major new functionality is added, tests for the new functionality MUST be added to the automated test suite. All new APIs MUST be documented and tested and tests MUST cover at least 80% of the added functionality
using the code coverage tool and reporting provided by the project.”
Is there some more detailed explanation of this part? In particular: “tests MUST cover at least 80% of the added functionality using the code coverage tool and reporting provided by the project.” How this code coverage is evaluated? Is this enforced during
a PR process?
Another question is related to the Bluetooth stack. How the overall test coverage of Bluetooth stack in Zephyr is measured? Are BubbleSim tests included in such calculations?
Maybe we can have a short discussion related to test coverage measurements during our today’s wg meeting?
| Software Test Developer
M +48 728 395 111 | Kraków, Poland
Jeremy Bettis | ChromeOS
FAFT lead | jbettis@... | 303-257-2486