Perez-Gonzalez, Inaky <inaky.perez-gonzalez@...>
From: Hannikainen, Jaakko
Sent: Friday, August 5, 2016 5:51 AM
To: Perez-Gonzalez, Inaky <inaky.perez-gonzalez(a)intel.com>; devel(a)lists.zephyrproject.org
Subject: RE: About the testing framework
Guess our targest are partially aligned then (thanks, 'open-source' development). At any case, Zephyr needs a centralized testing framework. I quickly wrote one up for testing purposes, but it's meant to run with precisely the module being tested linked in, and nothing else of Zephyr, instead being linked with eg. GNU's libc. I haven't done any work for actual integration with the existing testing scripts. The framework itself can be ported pretty quickly to a more Zephyr-compatible way though (a couple lines of code + some ifdef), so that we could have one framework for both integration and unit testing - unifying the test architecture. How well does your framework work without the Zephyr core, and is it easily portable to be run with your own computer without virtualization?
The infrastructure does two parts; one [the server] allows you to access and share (remote) target hardware; the client side builds and deploys to targets, running the code in the target and evaluating if tests pass. If it is running in the local machine as part of what you are conceiving (what it calls an static check), then it can do it without the server side (this I use for example to run checkpatch verifications, etc).
That said, nothing is tied to Zephyr itself-every part is configurable to support other OSes or whichever.
ZEP-578 could help testing a lot. If we move from ifdef-compiled code to if(defined()), the code can be tested a lot easier in a unit testing context. Before including the .c file, just change the macro contents simple numbers to variables, and set the variables at each unit test.
ACK-I'm all to minimize #ifdef blocks; I think we can do a reasonable job of making multiple things static without introducing complexity in the code itself.
Drivers can also be pretty easily unit-tested, just mock out the hardware apis and check the driver's code itself, and ensure it uses the mocked device correctly. This applies even if ZEP-578 doesn't get any wind, then the test suite code has to be split into multiple binaries, one for each test configuration.
Agreed-this helps exercising the extra paths and complements running the driver code in the hardware to make sure it works in the iron.