Re: Test coverage and its measurements


Nashif, Anas
 

Hi,

Coverage reports and having test coverage is a test activity, has nothing to do with Safety. Maybe it was not discussed here in this group in the past, but it’s something that belongs here IMO.

 

Anas

 

From: testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Date: Thursday, February 3, 2022 at 9:14 AM
To: maciej.perkowski@... <maciej.perkowski@...>, testing-wg@... <testing-wg@...>
Cc: Wang, Guojun <guojun.wang@...>
Subject: Re: [testing-wg] Test coverage and its measurements

Coverage analysis is not conducted in the testing working group instead there is another team who is testing the kernel API coverage. IIRC it is safety team, correct me if I am wrong @Mai.Enjia@...

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of Perkowski, Maciej via lists.zephyrproject.org
Sent: 2022
23 17:14
To: testing-wg@...
Cc: Wang, Guojun <guojun.wang@...>
Subject: [EXT] [testing-wg] Test coverage and its measurements

 

Caution: EXT Email

Dear Testers,

I was recently asked by my colleague about quality standards related to Zephyr’s contributions, in particular to testing. A relevant section of our docs states:

“When major new functionality is added, tests for the new functionality MUST be added to the automated test suite. All new APIs MUST be documented and tested and tests MUST cover at least 80% of the added functionality using the code coverage tool and reporting provided by the project.”

https://docs.zephyrproject.org/latest/contribute/index.html#other-commit-expectations
Is there some more detailed explanation of this part? In particular: “tests MUST cover at least 80% of the added functionality using the code coverage tool and reporting provided by the project.”  How this code coverage is evaluated? Is this enforced during a PR process?
Another question is related to the Bluetooth stack. How the overall test coverage of Bluetooth stack in Zephyr is measured? Are BubbleSim tests included in such calculations?
Maybe we can have a short discussion related to test coverage measurements during our today’s wg meeting?
Cheers,

Maciej Perkowski | Software Test Developer
M +48 728 395 111 | Kraków, Poland
nordicsemi.com | devzone.nordicsemi.com

Nordic_logo_signature

 

Join testing-wg@lists.zephyrproject.org to automatically receive all group messages.