users@lists.zephyrproject.org | Using sanitycheck to run my project tests

Home Messages Hashtags Subgroups

Date Date 1 - 8 of 8

Using sanitycheck to run my project tests

Rodrigo Peixoto <rodrigopex@...> #978 Hi all. I was creating my unit, integration, and regression test for a project I am involved, I was wondering if it is possible to use the sanitycheck script to run my own tests that are not related to zephyr kernel. Folder hierarchy: my_project_folder/ include/ src/ CMakeLists.txt ... tests/ test_at/ testcase.yaml src/ ... I have run in my own tests folder: $ sanitycheck -p native_posix -T . --list-tests - build.action_data - build.action_data_offset - build.action_enqueue - build.action_send_sync_thread - build.base_action - build.send_at_vsx_off - build.send_at_vsx_on - build.send_at_vsx_on_status_off 8 total. It shows 8 tests in total. When I run: $ sanitycheck -p native_posix -T . --outdir ./santy-out Building testcase defconfigs... 1 tests selected, 107 tests discarded due to filters total complete: 1/ 1 100% failed: 0 1 of 1 tests passed with 0 warnings in 2 seconds It shows 1 test had run ok, but I already changed the test to make it fails but the result is always the same. Another weird thing is the number of tests of 107 discard and 1 selected... How can I run that properly? I would like to do that without copying the tests to the kernel folder. Is it possible? Thank you. Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex
More All Messages By This Member
Nashif, Anas #979 --list-tests give you the list of sub-tests in a test project, sanitycheck results report on the test project level, not the sub-tests. Anas toggle quoted message Show quoted text From: users@... [mailto:users@...] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 9:22 AM To: users@... Subject: [Zephyr-users] Using sanitycheck to run my project tests Hi all. I was creating my unit, integration, and regression test for a project I am involved, I was wondering if it is possible to use the sanitycheck script to run my own tests that are not related to zephyr kernel. Folder hierarchy: my_project_folder/ include/ src/ CMakeLists.txt ... tests/ test_at/ testcase.yaml src/ ... I have run in my own tests folder: $ sanitycheck -p native_posix -T . --list-tests - build.action_data - build.action_data_offset - build.action_enqueue - build.action_send_sync_thread - build.base_action - build.send_at_vsx_off - build.send_at_vsx_on - build.send_at_vsx_on_status_off 8 total. It shows 8 tests in total. When I run: $ sanitycheck -p native_posix -T . --outdir ./santy-out Building testcase defconfigs... 1 tests selected, 107 tests discarded due to filters total complete: 1/ 1 100% failed: 0 1 of 1 tests passed with 0 warnings in 2 seconds It shows 1 test had run ok, but I already changed the test to make it fails but the result is always the same. Another weird thing is the number of tests of 107 discard and 1 selected... How can I run that properly? I would like to do that without copying the tests to the kernel folder. Is it possible? Thank you. Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex
More All Messages By This Member
Rodrigo Peixoto <rodrigopex@...> #980 But when I force an error it doesn't affect the result. Any clue? Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:46 GMT-03:00 Nashif, Anas <anas.nashif@...>: toggle quoted message Show quoted text --list-tests give you the list of sub-tests in a test project, sanitycheck results report on the test project level, not the sub-tests. Anas From: users@... [mailto:users@lists.zephyrproject.org] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 9:22 AM To: users@... Subject: [Zephyr-users] Using sanitycheck to run my project tests Hi all. I was creating my unit, integration, and regression test for a project I am involved, I was wondering if it is possible to use the sanitycheck script to run my own tests that are not related to zephyr kernel. Folder hierarchy: my_project_folder/ include/ src/ CMakeLists.txt ... tests/ test_at/ testcase.yaml src/ ... I have run in my own tests folder: $ sanitycheck -p native_posix -T . --list-tests - build.action_data - build.action_data_offset - build.action_enqueue - build.action_send_sync_thread - build.base_action - build.send_at_vsx_off - build.send_at_vsx_on - build.send_at_vsx_on_status_off 8 total. It shows 8 tests in total. When I run: $ sanitycheck -p native_posix -T . --outdir ./santy-out Building testcase defconfigs... 1 tests selected, 107 tests discarded due to filters total complete: 1/ 1 100% failed: 0 1 of 1 tests passed with 0 warnings in 2 seconds It shows 1 test had run ok, but I already changed the test to make it fails but the result is always the same. Another weird thing is the number of tests of 107 discard and 1 selected... How can I run that properly? I would like to do that without copying the tests to the kernel folder. Is it possible? Thank you. Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex
More All Messages By This Member
Rodrigo Peixoto <rodrigopex@...> #981 When I run the command: $ sanitycheck -p native_posix -T . --outdir ./sanity-out --detailed-report ./sanity-out/tests.report The content of the report is: <testsuites><testsuite errors="0" failures="0" name="Sanitycheck" skipped="0" tests="0" time="0" /></testsuites> Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:53 GMT-03:00 Rodrigo Peixoto <rodrigopex@...>: toggle quoted message Show quoted text But when I force an error it doesn't affect the result. Any clue? Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:46 GMT-03:00 Nashif, Anas <anas.nashif@...>: --list-tests give you the list of sub-tests in a test project, sanitycheck results report on the test project level, not the sub-tests. Anas From: users@... [mailto:users@...ect.org] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 9:22 AM To: users@... Subject: [Zephyr-users] Using sanitycheck to run my project tests Hi all. I was creating my unit, integration, and regression test for a project I am involved, I was wondering if it is possible to use the sanitycheck script to run my own tests that are not related to zephyr kernel. Folder hierarchy: my_project_folder/ include/ src/ CMakeLists.txt ... tests/ test_at/ testcase.yaml src/ ... I have run in my own tests folder: $ sanitycheck -p native_posix -T . --list-tests - build.action_data - build.action_data_offset - build.action_enqueue - build.action_send_sync_thread - build.base_action - build.send_at_vsx_off - build.send_at_vsx_on - build.send_at_vsx_on_status_off 8 total. It shows 8 tests in total. When I run: $ sanitycheck -p native_posix -T . --outdir ./santy-out Building testcase defconfigs... 1 tests selected, 107 tests discarded due to filters total complete: 1/ 1 100% failed: 0 1 of 1 tests passed with 0 warnings in 2 seconds It shows 1 test had run ok, but I already changed the test to make it fails but the result is always the same. Another weird thing is the number of tests of 107 discard and 1 selected... How can I run that properly? I would like to do that without copying the tests to the kernel folder. Is it possible? Thank you. Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex
More All Messages By This Member
Nashif, Anas #982 What kind of error did you force? What do you have in testcase.yaml? Can you run with -vv? Please open an issue on GH and attach output there. Anas From: <users@...> on behalf of Rodrigo Peixoto <rodrigopex@...> Date: Thursday, 19 July 2018 at 09:53 To: Anas Nashif <anas.nashif@...> Cc: "users@..." <users@...> Subject: Re: [Zephyr-users] Using sanitycheck to run my project tests But when I force an error it doesn't affect the result. Any clue? Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:46 GMT-03:00 Nashif, Anas <anas.nashif@...>: toggle quoted message Show quoted text --list-tests give you the list of sub-tests in a test project, sanitycheck results report on the test project level, not the sub-tests. Anas From: users@... [mailto:users@...] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 9:22 AM To: users@... Subject: [Zephyr-users] Using sanitycheck to run my project tests Hi all. I was creating my unit, integration, and regression test for a project I am involved, I was wondering if it is possible to use the sanitycheck script to run my own tests that are not related to zephyr kernel. Folder hierarchy: my_project_folder/ include/ src/ CMakeLists.txt ... tests/ test_at/ testcase.yaml src/ ... I have run in my own tests folder: $ sanitycheck -p native_posix -T . --list-tests - build.action_data - build.action_data_offset - build.action_enqueue - build.action_send_sync_thread - build.base_action - build.send_at_vsx_off - build.send_at_vsx_on - build.send_at_vsx_on_status_off 8 total. It shows 8 tests in total. When I run: $ sanitycheck -p native_posix -T . --outdir ./santy-out Building testcase defconfigs... 1 tests selected, 107 tests discarded due to filters total complete: 1/ 1 100% failed: 0 1 of 1 tests passed with 0 warnings in 2 seconds It shows 1 test had run ok, but I already changed the test to make it fails but the result is always the same. Another weird thing is the number of tests of 107 discard and 1 selected... How can I run that properly? I would like to do that without copying the tests to the kernel folder. Is it possible? Thank you. Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex
More All Messages By This Member
Rodrigo Peixoto <rodrigopex@...> #983 Maybe it is not an issue, but a miss use of my part. Should I open the issue? My testcase.yaml is: tests: build: build_only: true The error I am forcing is a zassert to be false in one of the tests that must run. When I run this manually, the error appears. Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:59 GMT-03:00 Nashif, Anas <anas.nashif@...>: toggle quoted message Show quoted text What kind of error did you force? What do you have in testcase.yaml? Can you run with -vv? Please open an issue on GH and attach output there. Anas From: <users@...> on behalf of Rodrigo Peixoto <rodrigopex@...> Date: Thursday, 19 July 2018 at 09:53 To: Anas Nashif <anas.nashif@...> Cc: "users@..." <users@...> Subject: Re: [Zephyr-users] Using sanitycheck to run my project tests But when I force an error it doesn't affect the result. Any clue? Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:46 GMT-03:00 Nashif, Anas <anas.nashif@...>: --list-tests give you the list of sub-tests in a test project, sanitycheck results report on the test project level, not the sub-tests. Anas From: users@... [mailto:users@lists.zephyrproject.org] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 9:22 AM To: users@... Subject: [Zephyr-users] Using sanitycheck to run my project tests Hi all. I was creating my unit, integration, and regression test for a project I am involved, I was wondering if it is possible to use the sanitycheck script to run my own tests that are not related to zephyr kernel. Folder hierarchy: my_project_folder/ include/ src/ CMakeLists.txt ... tests/ test_at/ testcase.yaml src/ ... I have run in my own tests folder: $ sanitycheck -p native_posix -T . --list-tests - build.action_data - build.action_data_offset - build.action_enqueue - build.action_send_sync_thread - build.base_action - build.send_at_vsx_off - build.send_at_vsx_on - build.send_at_vsx_on_status_off 8 total. It shows 8 tests in total. When I run: $ sanitycheck -p native_posix -T . --outdir ./santy-out Building testcase defconfigs... 1 tests selected, 107 tests discarded due to filters total complete: 1/ 1 100% failed: 0 1 of 1 tests passed with 0 warnings in 2 seconds It shows 1 test had run ok, but I already changed the test to make it fails but the result is always the same. Another weird thing is the number of tests of 107 discard and 1 selected... How can I run that properly? I would like to do that without copying the tests to the kernel folder. Is it possible? Thank you. Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex
More All Messages By This Member
Nashif, Anas #984 You should get rid of the build_only toggle quoted message Show quoted text From: users@... [mailto:users@...] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 10:06 AM To: Nashif, Anas <anas.nashif@...> Cc: users@... Subject: Re: [Zephyr-users] Using sanitycheck to run my project tests Maybe it is not an issue, but a miss use of my part. Should I open the issue? My testcase.yaml is: tests: build: build_only: true The error I am forcing is a zassert to be false in one of the tests that must run. When I run this manually, the error appears. Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:59 GMT-03:00 Nashif, Anas <anas.nashif@...>: What kind of error did you force? What do you have in testcase.yaml? Can you run with -vv? Please open an issue on GH and attach output there. Anas From: <users@...> on behalf of Rodrigo Peixoto <rodrigopex@...> Date: Thursday, 19 July 2018 at 09:53 To: Anas Nashif <anas.nashif@...> Cc: "users@..." <users@...> Subject: Re: [Zephyr-users] Using sanitycheck to run my project tests But when I force an error it doesn't affect the result. Any clue? Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:46 GMT-03:00 Nashif, Anas <anas.nashif@...>: --list-tests give you the list of sub-tests in a test project, sanitycheck results report on the test project level, not the sub-tests. Anas From: users@... [mailto:users@...] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 9:22 AM To: users@... Subject: [Zephyr-users] Using sanitycheck to run my project tests Hi all. I was creating my unit, integration, and regression test for a project I am involved, I was wondering if it is possible to use the sanitycheck script to run my own tests that are not related to zephyr kernel. Folder hierarchy: my_project_folder/ include/ src/ CMakeLists.txt ... tests/ test_at/ testcase.yaml src/ ... I have run in my own tests folder: $ sanitycheck -p native_posix -T . --list-tests - build.action_data - build.action_data_offset - build.action_enqueue - build.action_send_sync_thread - build.base_action - build.send_at_vsx_off - build.send_at_vsx_on - build.send_at_vsx_on_status_off 8 total. It shows 8 tests in total. When I run: $ sanitycheck -p native_posix -T . --outdir ./santy-out Building testcase defconfigs... 1 tests selected, 107 tests discarded due to filters total complete: 1/ 1 100% failed: 0 1 of 1 tests passed with 0 warnings in 2 seconds It shows 1 test had run ok, but I already changed the test to make it fails but the result is always the same. Another weird thing is the number of tests of 107 discard and 1 selected... How can I run that properly? I would like to do that without copying the tests to the kernel folder. Is it possible? Thank you. Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex
More All Messages By This Member
Rodrigo Peixoto <rodrigopex@...> #986 Anas, The command worked after removing the build_only. I think the documentation is outdated. For example: "Build only: If true, don't try to run the test under QEMU even if the selected platform supports it.". In this part of the documentation and other several points, it seems to be related to QEMU, but it is not right I guess. Please confirm that I would make a pull request with the adjustments. Now, I am trying to generate the coverage data but it is not working yet. I have added the -C flag at the command and found the gcda and gcdo files at build/CMakefiles/app.dir/src. I have pointed the outdir to there but the coverage is still not running properly. Any clue on that? Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 12:03 GMT-03:00 Nashif, Anas <anas.nashif@...>: toggle quoted message Show quoted text You should get rid of the build_only From: users@... [mailto:users@lists.zephyrproject.org] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 10:06 AM To: Nashif, Anas <anas.nashif@...> Cc: users@... Subject: Re: [Zephyr-users] Using sanitycheck to run my project tests Maybe it is not an issue, but a miss use of my part. Should I open the issue? My testcase.yaml is: tests: build: build_only: true The error I am forcing is a zassert to be false in one of the tests that must run. When I run this manually, the error appears. Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:59 GMT-03:00 Nashif, Anas <anas.nashif@...>: What kind of error did you force? What do you have in testcase.yaml? Can you run with -vv? Please open an issue on GH and attach output there. Anas From: <users@...> on behalf of Rodrigo Peixoto <rodrigopex@...> Date: Thursday, 19 July 2018 at 09:53 To: Anas Nashif <anas.nashif@...> Cc: "users@..." <users@...> Subject: Re: [Zephyr-users] Using sanitycheck to run my project tests But when I force an error it doesn't affect the result. Any clue? Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex 2018-07-19 10:46 GMT-03:00 Nashif, Anas <anas.nashif@...>: --list-tests give you the list of sub-tests in a test project, sanitycheck results report on the test project level, not the sub-tests. Anas From: users@... [mailto:users@lists.zephyrproject.org] On Behalf Of Rodrigo Peixoto Sent: Thursday, July 19, 2018 9:22 AM To: users@... Subject: [Zephyr-users] Using sanitycheck to run my project tests Hi all. I was creating my unit, integration, and regression test for a project I am involved, I was wondering if it is possible to use the sanitycheck script to run my own tests that are not related to zephyr kernel. Folder hierarchy: my_project_folder/ include/ src/ CMakeLists.txt ... tests/ test_at/ testcase.yaml src/ ... I have run in my own tests folder: $ sanitycheck -p native_posix -T . --list-tests - build.action_data - build.action_data_offset - build.action_enqueue - build.action_send_sync_thread - build.base_action - build.send_at_vsx_off - build.send_at_vsx_on - build.send_at_vsx_on_status_off 8 total. It shows 8 tests in total. When I run: $ sanitycheck -p native_posix -T . --outdir ./santy-out Building testcase defconfigs... 1 tests selected, 107 tests discarded due to filters total complete: 1/ 1 100% failed: 0 1 of 1 tests passed with 0 warnings in 2 seconds It shows 1 test had run ok, but I already changed the test to make it fails but the result is always the same. Another weird thing is the number of tests of 107 discard and 1 selected... How can I run that properly? I would like to do that without copying the tests to the kernel folder. Is it possible? Thank you. Best regards, Rodrigo Peixoto Co-founder and Technical advisor +55 (82) 98144-8585 http://ayna.tech \| Skype: rodrigopex
More All Messages By This Member

1 - 8 of 8

1

Previous Topic Next Topic

Home Hashtags Subgroups Terms