Zephyr: Testing WG


Hake Huang
 

Hi All,

This weekly meeting agenda:


1. Tire 1 Board testing list discussion.

Are those boards listed in this link still valid? https://github.com/zephyrproject-rtos/zephyr/wiki/Board-Testing-Introduction



1. release board testing requirements definition?

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Fissues%2F38566%23issuecomment-1204232175&;data=05%7C01%7Chake.huang%40nxp.com%7C45edd7f65d4945b3f28d08da7b49505e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637957852057490514%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mmmJgxYR47EA5KZfbJ5eY%2F732or2vbeUEhgsZutFWfM%3D&reserved=0

A. Select a TMS system.(TWG need propose one)

* Allure system (https://qameta.io/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fqameta.io%2F&data=05%7C01%7Chake.huang%40nxp.com%7Cbf18863ac69240665bde08da7bab23ab%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637958272220298306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=910oA%2BwDRYGadxRSHCizP2QoEPSW%2FufNaw%2BqfRDEmy0%3D&reserved=0>)
* Testomat https://testomat.io/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftestomat.io%2F&data=05%7C01%7Chake.huang%40nxp.com%7Cbf18863ac69240665bde08da7bab23ab%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637958272220298306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=a1Refd511adSDDEOjPGUVhb0Wj0wvGOAOyUp2YDrtHQ%3D&reserved=0>
* Zephyr Enterprise Version
https://smartbear.com/test-management/zephyr-enterprise/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmartbear.com%2Ftest-management%2Fzephyr-enterprise%2F&data=05%7C01%7Chake.huang%40nxp.com%7Cbf18863ac69240665bde08da7bab23ab%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637958272220298306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2MCUM6cg%2B0WtjdhqoRs6S1GfQKS%2FhzFeX91IH4V6qos%3D&reserved=0>
https://www.youtube.com/watch?v=sfXfW9eM4dA<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DsfXfW9eM4dA&data=05%7C01%7Chake.huang%40nxp.com%7Cbf18863ac69240665bde08da7bab23ab%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637958272220298306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0PNFwHaLiKRNym85mvQgsKGCeSIrw9TWyn14SNupnQQ%3D&reserved=0>

* Zephyr Plugin for Jira
(https://support.smartbear.com/zephyr-squad-cloud/docs/about.html<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsupport.smartbear.com%2Fzephyr-squad-cloud%2Fdocs%2Fabout.html&data=05%7C01%7Chake.huang%40nxp.com%7Cbf18863ac69240665bde08da7bab23ab%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637958272220298306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=98OUDGwlp1NoBrs2Gi1sHsTOuXE8YAQ6Vuw1lc3%2BIv0%3D&reserved=0> )

* Testrail
https://blog.gurock.com/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gurock.com%2F&data=05%7C01%7Chake.huang%40nxp.com%7Cbf18863ac69240665bde08da7bab23ab%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637958272220298306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jXO%2F21y6G7HpykF06zTxUpF9fyqYWYKy0ObVIo4C6fA%3D&reserved=0>

Criteria:
- coop with GitHub
- coop with Jira (issue tracker system)
- traceability
- dashboard
- customizable reporting





1. New ztest API migration status and issues discussion.

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Fissues%2F47002&;data=05%7C01%7Chake.huang%40nxp.com%7C45edd7f65d4945b3f28d08da7b49505e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637957852057490514%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=q6lO2aEwRjzgbSjebnBku%2B1mLLoXxBgwqYXpspfL5wQ%3D&amp;reserved=0





1. round table



Regards,

Hake


Regards,
Hake

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021年6月14日 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022年8月18日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:


Caution: EXT Email
Meeting Agenda: https://docs.google.com/document/d/1dXrQJBAxNPbJp78rb-v5m8R6pa-Nx5_QHELZRefLiFE/edit#heading=h.b7v1k1kxoko<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1dXrQJBAxNPbJp78rb-v5m8R6pa-Nx5_QHELZRefLiFE%2Fedit%23heading%3Dh.b7v1k1kxoko&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=58NClTdq6jxckGRsMnhXiGxpUu%2F2ucbyjWCm11EKZRY%3D&reserved=0>



________________________________________________________________________________
Join Microsoft Teams Meeting<https://eur01.safelinks.protection.outlook.com/ap/t-59584e83/?url=https%3A%2F%2Fteams.microsoft.com%2Fl%2Fmeetup-join%2F19%253ameeting_YTcwYjc1M2ItYWFmMy00MTM1LTgxNDctM2ZmZGRlMTM5NjNk%2540thread.v2%2F0%3Fcontext%3D%257b%2522Tid%2522%253a%2522af0096d9-700c-411a-b795-b3dd7122bad2%2522%252c%2522Oid%2522%253a%2522841a7c92-7816-4faf-9887-5e334e88f6d8%2522%257d&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=6FsHJ7xti4o5sSfm5%2B%2F4s0bdPB%2F6IKWcxkxQ3DiV7no%3D&reserved=0>
+1 321-558-6518<tel:+1%20321-558-6518,,831716531#> United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdialin.teams.microsoft.com%2F325d775d-c910-441e-90d0-353ebaa56cdd%3Fid%3D831716531&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=2KLSY7C556%2BNiClnR66qVJ2aRK3luFhRTuhAfCcccnc%3D&reserved=0> | Reset PIN<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmysettings.lync.com%2Fpstnconferencing&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=%2FN5l8KybBHmPv2xZWgO9GhiQJArSvxVpMBC%2BlVxc24o%3D&reserved=0> | Learn more about Teams<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FJoinTeamsMeeting&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=48MgCObcpx%2Bv%2FmJJX8gvb%2FqWiJ9jb8Z07Y9tkTsvg1g%3D&reserved=0> | Meeting options<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fteams.microsoft.com%2FmeetingOptions%2F%3ForganizerId%3D841a7c92-7816-4faf-9887-5e334e88f6d8%26tenantId%3Daf0096d9-700c-411a-b795-b3dd7122bad2%26threadId%3D19_meeting_YTcwYjc1M2ItYWFmMy00MTM1LTgxNDctM2ZmZGRlMTM5NjNk%40thread.v2%26messageId%3D0%26language%3Den-US&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916902415%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=MhLvYQlvufjhw1v0cpClxUQaDm6Wx2IBTRwXd4ogep8%3D&reserved=0>


________________________________________________________________________________


Hake Huang
 

Hi All,

This weekly meeting agenda


1. Tire 1 Board testing list discussion follow up.

Are those boards listed in this link still valid? https://github.com/zephyrproject-rtos/zephyr/wiki/Board-Testing-Introduction<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Fwiki%2FBoard-Testing-Introduction&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039470131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qbiZr6UCTZphKVPlIyOraOiVsOnq4t6KJvePLNj3KgY%3D&reserved=0>



1. release board testing requirements definition?

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Fissues%2F38566%23issuecomment-1204232175&;data=05%7C01%7Chake.huang%40nxp.com%7C45edd7f65d4945b3f28d08da7b49505e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637957852057490514%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=mmmJgxYR47EA5KZfbJ5eY%2F732or2vbeUEhgsZutFWfM%3D&amp;reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Fissues%2F38566%23issuecomment-1204232175&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039470131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=spHR3OKfyjqKKnoui9zhsEe%2FRcUkTGrX1BAS16iJswI%3D&reserved=0>

A. Select a TMS system.(TWG need propose one)

* Allure system (https://qameta.io/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fqameta.io%2F&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039470131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4oaNRGr4q8JDzJAGQkLFd4LCpDQb3K7FfyGEq2Fl%2FGY%3D&reserved=0>)
* Testomat https://testomat.io/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftestomat.io%2F&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039470131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eFxjZ9px3iWjHeA6d7sAYUtZvoJaDgUAhEwNMdeSD9U%3D&reserved=0>
* Zephyr Enterprise Version
https://smartbear.com/test-management/zephyr-enterprise/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmartbear.com%2Ftest-management%2Fzephyr-enterprise%2F&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039470131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2FqHZSme3h8ebTxEuE3Z%2Fl59zM5yNZrXcX1nEYTDZRa0%3D&reserved=0>
https://www.youtube.com/watch?v=sfXfW9eM4dA<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DsfXfW9eM4dA&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039470131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=cRbcpH%2BSjiYItbyINQyZTOegdKwWQXMRmQo0o9YyFCI%3D&reserved=0>

* Zephyr Plugin for Jira
(https://support.smartbear.com/zephyr-squad-cloud/docs/about.html<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsupport.smartbear.com%2Fzephyr-squad-cloud%2Fdocs%2Fabout.html&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039470131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=i%2B16uIc1hn%2BUJAema1sUwqHWv3TbGifRsgbqMYyrrJU%3D&reserved=0> )

* Testrail
https://blog.gurock.com/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gurock.com%2F&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039626357%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=uq45RNJbMmB6noQjo7r1TmUIfFauKMwYFxBeQHYIM1k%3D&reserved=0>

* oepnsearch

Criteria:
- coop with GitHub
- coop with Jira (issue tracker system)
- traceability
- dashboard
- customizable reporting





1. New ztest API migration status and issues discussion.

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Fissues%2F47002&;data=05%7C01%7Chake.huang%40nxp.com%7C45edd7f65d4945b3f28d08da7b49505e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637957852057490514%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=q6lO2aEwRjzgbSjebnBku%2B1mLLoXxBgwqYXpspfL5wQ%3D&amp;reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Fissues%2F47002&data=05%7C01%7Chake.huang%40nxp.com%7C40ddc6a67c4d4ff116be08da80cb196d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637963907039626357%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LEWY9PtPlr8IeCIHFqpgxiZ98nhsWYHUzTiNwP3iB5o%3D&reserved=0>





1. round table

1.


Regards,
Hake

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021年6月14日 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022年8月25日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:


Caution: EXT Email
Meeting Agenda: https://docs.google.com/document/d/1dXrQJBAxNPbJp78rb-v5m8R6pa-Nx5_QHELZRefLiFE/edit#heading=h.b7v1k1kxoko<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1dXrQJBAxNPbJp78rb-v5m8R6pa-Nx5_QHELZRefLiFE%2Fedit%23heading%3Dh.b7v1k1kxoko&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=58NClTdq6jxckGRsMnhXiGxpUu%2F2ucbyjWCm11EKZRY%3D&reserved=0>



________________________________________________________________________________
Join Microsoft Teams Meeting<https://eur01.safelinks.protection.outlook.com/ap/t-59584e83/?url=https%3A%2F%2Fteams.microsoft.com%2Fl%2Fmeetup-join%2F19%253ameeting_YTcwYjc1M2ItYWFmMy00MTM1LTgxNDctM2ZmZGRlMTM5NjNk%2540thread.v2%2F0%3Fcontext%3D%257b%2522Tid%2522%253a%2522af0096d9-700c-411a-b795-b3dd7122bad2%2522%252c%2522Oid%2522%253a%2522841a7c92-7816-4faf-9887-5e334e88f6d8%2522%257d&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=6FsHJ7xti4o5sSfm5%2B%2F4s0bdPB%2F6IKWcxkxQ3DiV7no%3D&reserved=0>
+1 321-558-6518<tel:+1%20321-558-6518,,831716531#> United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdialin.teams.microsoft.com%2F325d775d-c910-441e-90d0-353ebaa56cdd%3Fid%3D831716531&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=2KLSY7C556%2BNiClnR66qVJ2aRK3luFhRTuhAfCcccnc%3D&reserved=0> | Reset PIN<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmysettings.lync.com%2Fpstnconferencing&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=%2FN5l8KybBHmPv2xZWgO9GhiQJArSvxVpMBC%2BlVxc24o%3D&reserved=0> | Learn more about Teams<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FJoinTeamsMeeting&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=48MgCObcpx%2Bv%2FmJJX8gvb%2FqWiJ9jb8Z07Y9tkTsvg1g%3D&reserved=0> | Meeting options<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fteams.microsoft.com%2FmeetingOptions%2F%3ForganizerId%3D841a7c92-7816-4faf-9887-5e334e88f6d8%26tenantId%3Daf0096d9-700c-411a-b795-b3dd7122bad2%26threadId%3D19_meeting_YTcwYjc1M2ItYWFmMy00MTM1LTgxNDctM2ZmZGRlMTM5NjNk%40thread.v2%26messageId%3D0%26language%3Den-US&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916902415%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=MhLvYQlvufjhw1v0cpClxUQaDm6Wx2IBTRwXd4ogep8%3D&reserved=0>


________________________________________________________________________________


Hake Huang
 

Hi Anas,

In today’s meeting, we are impressed by your introduce the new test report structure with many thanks, and we notice that you stress the new TSM system as well.

To break down targets into well accepted steps, I’d like to ask your help to list your intended criteria. There are too many items in your introduction, and I guess most of us can’t get full track.
For T1 boards:

1. The T1 boards has test commitment

2. The T1 boards need to be broadly available

3. The T1 boards need to be debug able

What is the criteria to select test suite for T1 boards?

As the TSM system, when can we use the newer one? Or still pending for community feedback?

Thanks again.

Regards,
Hake

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021年6月14日 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022年8月25日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:


Caution: EXT Email
Meeting Agenda: https://docs.google.com/document/d/1dXrQJBAxNPbJp78rb-v5m8R6pa-Nx5_QHELZRefLiFE/edit#heading=h.b7v1k1kxoko<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1dXrQJBAxNPbJp78rb-v5m8R6pa-Nx5_QHELZRefLiFE%2Fedit%23heading%3Dh.b7v1k1kxoko&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=58NClTdq6jxckGRsMnhXiGxpUu%2F2ucbyjWCm11EKZRY%3D&reserved=0>



________________________________________________________________________________
Join Microsoft Teams Meeting<https://eur01.safelinks.protection.outlook.com/ap/t-59584e83/?url=https%3A%2F%2Fteams.microsoft.com%2Fl%2Fmeetup-join%2F19%253ameeting_YTcwYjc1M2ItYWFmMy00MTM1LTgxNDctM2ZmZGRlMTM5NjNk%2540thread.v2%2F0%3Fcontext%3D%257b%2522Tid%2522%253a%2522af0096d9-700c-411a-b795-b3dd7122bad2%2522%252c%2522Oid%2522%253a%2522841a7c92-7816-4faf-9887-5e334e88f6d8%2522%257d&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=6FsHJ7xti4o5sSfm5%2B%2F4s0bdPB%2F6IKWcxkxQ3DiV7no%3D&reserved=0>
+1 321-558-6518<tel:+1%20321-558-6518,,831716531#> United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdialin.teams.microsoft.com%2F325d775d-c910-441e-90d0-353ebaa56cdd%3Fid%3D831716531&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=2KLSY7C556%2BNiClnR66qVJ2aRK3luFhRTuhAfCcccnc%3D&reserved=0> | Reset PIN<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmysettings.lync.com%2Fpstnconferencing&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=%2FN5l8KybBHmPv2xZWgO9GhiQJArSvxVpMBC%2BlVxc24o%3D&reserved=0> | Learn more about Teams<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FJoinTeamsMeeting&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=48MgCObcpx%2Bv%2FmJJX8gvb%2FqWiJ9jb8Z07Y9tkTsvg1g%3D&reserved=0> | Meeting options<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fteams.microsoft.com%2FmeetingOptions%2F%3ForganizerId%3D841a7c92-7816-4faf-9887-5e334e88f6d8%26tenantId%3Daf0096d9-700c-411a-b795-b3dd7122bad2%26threadId%3D19_meeting_YTcwYjc1M2ItYWFmMy00MTM1LTgxNDctM2ZmZGRlMTM5NjNk%40thread.v2%26messageId%3D0%26language%3Den-US&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916902415%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=MhLvYQlvufjhw1v0cpClxUQaDm6Wx2IBTRwXd4ogep8%3D&reserved=0>


________________________________________________________________________________


Hake Huang
 

Hi All,

This weekly meeting agenda.


1. Tire 1 Board testing list discussion follow up.
For T1 boards:

1. The T1 boards has test commitment

2. The T1 boards need to be broadly available

3. The T1 boards need to be debug able

What is the criteria to select test suite for T1 boards? Do we need wait for the twist test report ready?


2. TMS system discussion

when can we use the newer one? Or still pending for community feedback?



3. Query for code coverage from community.
Install the code coverage report in html format for

· https://github.com/zephyrproject-rtos/zephyr/tree/main/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Ftree%2Fmain%2F&data=05%7C01%7Chake.huang%40nxp.com%7C8c5edc3052b54548d58008da8b2bef37%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637975318082359658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=VZt2eOrSsRHp7TmDcufRr%2FujnwXD6%2FQFequUpSA2ctc%3D&reserved=0>

· https://github.com/zephyrproject-rtos/zephyr/tree/v2.7-auditable-branch/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzephyrproject-rtos%2Fzephyr%2Ftree%2Fv2.7-auditable-branch%2F&data=05%7C01%7Chake.huang%40nxp.com%7C8c5edc3052b54548d58008da8b2bef37%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637975318082359658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ufvlf8KQUxtViIY3PzdYsxODyODVh2ly8V59hdI6hFc%3D&reserved=0>
on a publicly available web server accessible to the community.


4. Ztest_new migration status


5. Round table

Regards,
Hake


-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021年6月14日 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022年9月1日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:


Caution: EXT Email
Meeting Agenda: https://docs.google.com/document/d/1dXrQJBAxNPbJp78rb-v5m8R6pa-Nx5_QHELZRefLiFE/edit#heading=h.b7v1k1kxoko<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1dXrQJBAxNPbJp78rb-v5m8R6pa-Nx5_QHELZRefLiFE%2Fedit%23heading%3Dh.b7v1k1kxoko&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=58NClTdq6jxckGRsMnhXiGxpUu%2F2ucbyjWCm11EKZRY%3D&reserved=0>



________________________________________________________________________________
Join Microsoft Teams Meeting<https://eur01.safelinks.protection.outlook.com/ap/t-59584e83/?url=https%3A%2F%2Fteams.microsoft.com%2Fl%2Fmeetup-join%2F19%253ameeting_YTcwYjc1M2ItYWFmMy00MTM1LTgxNDctM2ZmZGRlMTM5NjNk%2540thread.v2%2F0%3Fcontext%3D%257b%2522Tid%2522%253a%2522af0096d9-700c-411a-b795-b3dd7122bad2%2522%252c%2522Oid%2522%253a%2522841a7c92-7816-4faf-9887-5e334e88f6d8%2522%257d&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=6FsHJ7xti4o5sSfm5%2B%2F4s0bdPB%2F6IKWcxkxQ3DiV7no%3D&reserved=0>
+1 321-558-6518<tel:+1%20321-558-6518,,831716531#> United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdialin.teams.microsoft.com%2F325d775d-c910-441e-90d0-353ebaa56cdd%3Fid%3D831716531&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=2KLSY7C556%2BNiClnR66qVJ2aRK3luFhRTuhAfCcccnc%3D&reserved=0> | Reset PIN<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmysettings.lync.com%2Fpstnconferencing&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=%2FN5l8KybBHmPv2xZWgO9GhiQJArSvxVpMBC%2BlVxc24o%3D&reserved=0> | Learn more about Teams<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FJoinTeamsMeeting&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916746181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=48MgCObcpx%2Bv%2FmJJX8gvb%2FqWiJ9jb8Z07Y9tkTsvg1g%3D&reserved=0> | Meeting options<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fteams.microsoft.com%2FmeetingOptions%2F%3ForganizerId%3D841a7c92-7816-4faf-9887-5e334e88f6d8%26tenantId%3Daf0096d9-700c-411a-b795-b3dd7122bad2%26threadId%3D19_meeting_YTcwYjc1M2ItYWFmMy00MTM1LTgxNDctM2ZmZGRlMTM5NjNk%40thread.v2%26messageId%3D0%26language%3Den-US&data=05%7C01%7Chake.huang%40nxp.com%7C8a3f2c866a9349d9741208da50146d09%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C637910345916902415%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=MhLvYQlvufjhw1v0cpClxUQaDm6Wx2IBTRwXd4ogep8%3D&reserved=0>


________________________________________________________________________________


Nashif, Anas
 

Hi Hake,

Regarding reference boards and tiers, I think we need to expand on this and start drafting a proposal on what all of this means and how these fits in the release criteria alongside other items we want to have. Take the process meeting discussion, the discussion we had in the testing WG and create a more detailed proposal of what the goal is, the expectations and how this will translate into execution.

 

Regarding test scope, a thorough review of our tests and test structure and a way for us to label/tag tests in a structured way are needed. This should also come as a detailed proposal. Maybe a review of all our tests, their types, classes, priorities and how important it is to run certain tests on hardware is going to be necessary to proceed with this.

 

As for the test management system and where test reports would go, we discussed having a comparison table with the proposed solutions and some criteria for selection, do we have this already? What are the current from runners?

 

Anas

 

From: testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Date: Thursday, August 25, 2022 at 11:17 AM
To: Nashif, Anas <anas.nashif@...>
Cc: testing-wg@... Group <testing-wg@...>
Subject: Re: [testing-wg] Zephyr: Testing WG

Hi Anas,

 

In todays meeting, we are impressed by your introduce the new test report structure with many thanks, and we notice that you stress the new TSM system as well.

 

To break down targets into well accepted steps, Id like to ask your help to list your intended criteria.  There are too many items in your introduction, and I guess most of us cant get full track.

For T1 boards:

1.       The T1 boards has test commitment

2.       The T1 boards need to be broadly available

3.       The T1 boards need to be debug able

 

What is the criteria to select test suite for T1 boards?

 

As the TSM system, when can we use the newer one? Or still pending for community feedback?

 

Thanks again.

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
825日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi Anas,

 

Thanks a lot for your reply.

 

We have discussed the boards criteria see below and test scope(Piotr Gołyźniak(Nordic) will send out his summary table for all testcases).

 

1.   At least one from each architecture(arm , arm64, x86, upsqure, intel-adsp)

2.   At least one for each sub-platform (e.g. NXP RT series, LPC series, imx8 series)

RT cortex-m7/cortex-mx33, LPC cortex-m4/m0, imx8 arm64/amp(cortex m / a)

3.      At least one for each function group as leading platform(networking, api)

 

As to the TMS comparison table please see attachment.

 

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of Nashif, Anas via lists.zephyrproject.org
Sent: 2022
91 21:45
To: Hake Huang <hake.huang@...>
Cc: testing-wg@... Group <testing-wg@...>
Subject: [EXT] Re: [testing-wg] Zephyr: Testing WG

 

Caution: EXT Email

Hi Hake,

Regarding reference boards and tiers, I think we need to expand on this and start drafting a proposal on what all of this means and how these fits in the release criteria alongside other items we want to have. Take the process meeting discussion, the discussion we had in the testing WG and create a more detailed proposal of what the goal is, the expectations and how this will translate into execution.

 

Regarding test scope, a thorough review of our tests and test structure and a way for us to label/tag tests in a structured way are needed. This should also come as a detailed proposal. Maybe a review of all our tests, their types, classes, priorities and how important it is to run certain tests on hardware is going to be necessary to proceed with this.

 

As for the test management system and where test reports would go, we discussed having a comparison table with the proposed solutions and some criteria for selection, do we have this already? What are the current from runners?

 

Anas

 

From: testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Date: Thursday, August 25, 2022 at 11:17 AM
To: Nashif, Anas <anas.nashif@...>
Cc: testing-wg@... Group <testing-wg@...>
Subject: Re: [testing-wg] Zephyr: Testing WG

Hi Anas,

 

In todays meeting, we are impressed by your introduce the new test report structure with many thanks, and we notice that you stress the new TSM system as well.

 

To break down targets into well accepted steps, Id like to ask your help to list your intended criteria.  There are too many items in your introduction, and I guess most of us cant get full track.

For T1 boards:

1.       The T1 boards has test commitment

2.       The T1 boards need to be broadly available

3.       The T1 boards need to be debug able

 

What is the criteria to select test suite for T1 boards?

 

As the TSM system, when can we use the newer one? Or still pending for community feedback?

 

Thanks again.

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
825日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

Agenda for this weekly meeting

 

1.     Tire 1 Board testing  list discussion follow up.

https://github.com/zephyrproject-rtos/zephyr/issues/38566#issuecomment-1204232175 

 

  • T1 boards Proposal discussion:

At least one from each architecture(arm , arm64, x86, upsqure, intel-adsp)

At least one for each sub-platform (e.g. NXP RT series, LPC series, imx8 series)

·         RT cortex-m7/cortex-mx33, LPC cortex-m4/m0, imx8 arm64/amp(cortex m / a)

At least one for each function group as leading platform(networking, api)

  • Test cases review structure draft: see attachment

2.    TMS system discussion. See attachment

 

 

3.    Round table

 

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
98日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

This weekly meeting agenda.

 

1.     Tire 1 Board testing  list discussion follow up.

https://github.com/zephyrproject-rtos/zephyr/issues/38566#issuecomment-1204232175 

 

·         T1 boards Proposal discussion:

·         Test cases review structure draft:

Based on current test cases -> review them (remove bluetooth, networking) review link is below:

https://docs.google.com/spreadsheets/d/1NAC2SscIW4Wmz_9MScDeZ7VnXi0vO8dHkrKnUCT7Bfk/edit?usp=sharing 

process working group background information:

https://docs.google.com/presentation/d/10Q-B0Upv1bBaoQRNM5n9yUEMeXO6NINnFNDkA86RCsM/edit?usp=sharing

2.    TMS system discussion. 

 https://github.com/marketplace/testquality/plan/MDIyOk1hcmtldHBsYWNlTGlzdGluZ1BsYW44MTM=#pricing-and-setup  

 

3.    Round table

 

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
915日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

This is this weekly meeting minutes.

Highlights:

We will be continue discussing the T1 boards test suites, you can use the template below to fill your proposed cases.

https://docs.google.com/spreadsheets/d/1NAC2SscIW4Wmz_9MScDeZ7VnXi0vO8dHkrKnUCT7Bfk/edit

 

Thursday 9/15 2022

Attendee: 



  • Aaron Massey(Google)
  • AI Semjonvos(Google)
  • Alexey Brodkin(Synopsys)
  • Artern Panfilov(Synopsys)
  • Danny (Oticon)
  • David Leach(NXP)
  • Enjia Mai(intel)
  • Eugeniy Paltsev(Synopsys)
  • Hake Huang (NXP)
  • Hongbo(Intel)
  • Hein Simon(Baumer Electric AG)
  • Lixin Guo(Intel)
  • Jocelyn Li (Intel)
  • Jereme Bettis(Google)
  • Jinru wang(Synopsys)
  • Kosycarz Piotr (Nordic)
  • Maciej Perkowski (Nordic)
  • Meng xianglin(Intel)
  • Matthew Holenko (Antmicro)
  • Nashif Anas(Intel)
  • Paul Fagerburg(google)
  • Peng Cheng(Intel)
  • Piotr Zierhoffer (Antmicro)
  • Piotr Gołyźniak(Nordic)
  • Qi Chervl(T-Mobile)
  • Rob Woolley(Wind River)
  • Ruffer Dennis(T-Mobile) 
  • Shao ming(Intel)
  • Steven Wang Li(Intel)
  • Stephanos loannidis(Nodic)
  • Rattanakone Raam(T Mobile)
  • Yerabolu Spoorthy Priya(Intel) 
  • Yves Vandervennet(NXP)

 

    Tire 1 Board testing  list discussion follow up.

https://github.com/zephyrproject-rtos/zephyr/issues/38566#issuecomment-1204232175 

 

·         T1 boards Proposal discussion:

·         Test cases review structure draft:

Based on current test cases -> review them (remove bluetooth, networking) review link is below:

https://docs.google.com/spreadsheets/d/1NAC2SscIW4Wmz_9MScDeZ7VnXi0vO8dHkrKnUCT7Bfk/edit?usp=sharing 

 

<maceij> split the sheet into submdules and ask owner to review them?

1.    General guide line.

a.      How to group the test

b.      Case shall be running on as many boards as possible

c.      Case shall be run without special fixtures/add-on boards

d.      Each module shall has at leat one test(may not true for some peripheral)

e.      Exclude samples

                     Or start from 0 cases and select cases in to T1 bottom up way

We can use the template here(https://docs.google.com/spreadsheets/d/1NAC2SscIW4Wmz_9MScDeZ7VnXi0vO8dHkrKnUCT7Bfk/edit)

process working group background information:

https://docs.google.com/presentation/d/10Q-B0Upv1bBaoQRNM5n9yUEMeXO6NINnFNDkA86RCsM/edit?usp=sharing

 

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
915日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

This weekly meeting agenda.

 

l  We have basic list of testcases, please review and feedback, review link is below:

https://docs.google.com/spreadsheets/d/1NAC2SscIW4Wmz_9MScDeZ7VnXi0vO8dHkrKnUCT7Bfk/edit?usp=sharing 

 

l  round table

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
929日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 



HI All,

 

This weekly meeting minutes. We have finalize the proposed minimal testing suites as below, and we expect to more members to confirm next week.

Thursday 9/29 2022

Attendee: 



  • Aaron Massey(Google)
  • AI Semjonvos(Google)
  • Alexey Brodkin(Synopsys)
  • Artern Panfilov(Synopsys)
  • Danny (Oticon)
  • David Leach(NXP)
  • Enjia Mai(intel)
  • Eugeniy Paltsev(Synopsys)
  • Hake Huang (NXP)
  • Hongbo(Intel)
  • Hein Simon(Baumer Electric AG)
  • Lixin Guo(Intel)
  • Jocelyn Li (Intel)
  • Jereme Bettis(Google)
  • Jinru wang(Synopsys)
  • Kosycarz Piotr (Nordic)
  • Maciej Perkowski (Nordic)
  • Meng xianglin(Intel)
  • Matthew Holenko (Antmicro)
  • Nashif Anas(Intel)
  • Paul Fagerburg(google)
  • Peng Cheng(Intel)
  • Piotr Zierhoffer (Antmicro)
  • Piotr Gołyźniak(Nordic)
  • Qi Chervl(T-Mobile)
  • Rob Woolley(Wind River)
  • Ruffer Dennis(T-Mobile) 
  • Shao ming(Intel)
  • Steven Wang Li(Intel)
  • Stephanos loannidis(Nodic)
  • Rattanakone Raam(T Mobile)
  • Yerabolu Spoorthy Priya(Intel) 
  • Yves Vandervennet(NXP)




  • We have basic list of testcases, please review and feedback, review link is below:

https://docs.google.com/spreadsheets/d/1NAC2SscIW4Wmz_9MScDeZ7VnXi0vO8dHkrKnUCT7Bfk/edit?usp=sharing 

 

Below are the proposed minimal test cases for T1 boards

 

test scenario name

test application path

subsystem

comments on why the case is necessary

arch.*.*

tests/arch

arch

architecture compliance testing

kernel.*

tests/kernel

kernel

kernel compatible testing

drivers.adc

tests/drivers/adc/adc_api

ADC

adc driver testing

drivers.counter.basic_api

tests/drivers/counter/Counter_baseic_api

COUNTER

counter driver testing

drivers.dac.api

tests/drivers/dac/dac_api

DAC

dac api testing

drivers.dma.chan_blen_transfer

tests/drivers/dma/chan_blen_transfer

DMA

dma api testing

drivers.gpio.2pin

tests/drivers/gpio/gpio_basic_api

GPIO

gpio api testing

drivers.spi.loopback

tests/drivers/spi/spi_loopback

SPI

SPI api testing

pm

tests/subsys/pm/

Power mgmt

All the power managent function should be tested if supported.

 

 

  • l  round table

 

https://github.com/zephyrproject-rtos/zephyr/issues/50354  Semjonovs will follow up

 

Regards

Hake


Hake Huang
 

Hi All,

 

This weekly meeting agenda.

 

l  We have basic list of testcases, please review and feedback, review link is below:

 

test scenario name

test application path

subsystem

comments on why the case is necessary

arch.*.*

tests/arch

arch

architecture compliance testing

kernel.*

tests/kernel

kernel

kernel compatible testing

drivers.adc

tests/drivers/adc/adc_api

ADC

adc driver testing

drivers.counter.basic_api

tests/drivers/counter/Counter_baseic_api

COUNTER

counter driver testing

drivers.dac.api

tests/drivers/dac/dac_api

DAC

dac api testing

drivers.dma.chan_blen_transfer

tests/drivers/dma/chan_blen_transfer

DMA

dma api testing

drivers.gpio.2pin

tests/drivers/gpio/gpio_basic_api

GPIO

gpio api testing

drivers.spi.loopback

tests/drivers/spi/spi_loopback

SPI

SPI api testing

pm

tests/subsys/pm/

Power mgmt

All the power managent function should be tested if supported.

 

 

l  round table

 

Regards,

Hake

 

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
106日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

This weekly meeting agenda

 

1.     Discussion on T1 test suites

 

Proposal: add tag to every cases scenario, 

Rules are:

1.       Only some filters can be allowed.

CONFIG_PICOLIBC_SUPPORTED, will create a github issue and Stephan will help to solve (https://github.com/zephyrproject-rtos/zephyr/issues/51043)

2.         No arch dependency

 

Now the test list for T1 boards are

 

test scenario name

test application path

subsystem

comments on why the case is necessary

kernel.*

tests/kernel

kernel

kernel compatible testing

drivers.counter.basic_api

tests/drivers/counter/Counter_baseic_api

COUNTER

counter driver testing

drivers.dma.chan_blen_transfer

tests/drivers/dma/chan_blen_transfer

DMA

dma api testing

drivers.gpio.2pin

tests/drivers/gpio/gpio_basic_api

GPIO

gpio api testing

Pm

tests/subsys/pm/

Power mgmt

All the power managent function should be tested if supported.

 

source

https://docs.google.com/spreadsheets/d/1NAC2SscIW4Wmz_9MScDeZ7VnXi0vO8dHkrKnUCT7Bfk/edit?usp=sharing 

https://docs.google.com/document/d/1Z_9zhmM6EK09N1lQj-rvaScoVPDG_abJtV85JE7c3n4/edit?usp=sharing 

 

 

 

Regards,

Hake

 

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
1013日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

This weekly meeting agenda

 

1.     T1 board test cases follow up

https://github.com/zephyrproject-rtos/zephyr/pull/51287

 

https://github.com/zephyrproject-rtos/zephyr/pull/51288

 

and the overall features request link is here

https://github.com/zephyrproject-rtos/zephyr/issues/51286

 

2.     Round table

 

Regards,

Hake

 

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
1020日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

Agenda for this weekly meeting.

 

1.     Follow up for T1 board test cases.

 

2.     Round table.

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
1027日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

We do not have specific topic for this weekly meeting. But we can quick sync on the status with

 

1.     Tag for test cases refactor

https://github.com/zephyrproject-rtos/zephyr/issues/51718

 

2.     Opensearch status if Anash can present.

 

Just reminder for all the weekly meeting is moved 1 hour later due to Day time saving in winter.  

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
1110日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Perkowski, Maciej
 

Hi Hake,
Could you please specify if the original time stays or was it changed: I.e. is the meeting at
7:00-8:00 (UTC-08:00) Pacific Time or at some other time?

Cheers

 

Maciej Perkowski | Software Test Developer
M +48 728 395 111 | Kraków, Poland
nordicsemi.com | devzone.nordicsemi.com

Nordic_logo_signature

 

From: testing-wg@... <testing-wg@...> On Behalf Of Hake Huang via lists.zephyrproject.org
Sent: Thursday, November 10, 2022 5:50 AM
To: testing-wg@...
Subject: Re: [testing-wg] Zephyr: Testing WG

 

Caution:This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

 

Hi All,

 

We do not have specific topic for this weekly meeting. But we can quick sync on the status with

 

1.       Tag for test cases refactor

https://github.com/zephyrproject-rtos/zephyr/issues/51718

 

2.       Opensearch status if Anash can present.

 

Just reminder for all the weekly meeting is moved 1 hour later due to Day time saving in winter.  

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 20221110日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

This weekly meeting agenda:

 

1.         Follow up: Issue on how to split build and run in twister, by Piotr(Nordic).

2.         Around table discussion.

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
1117日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

This weekly meeting agenda:

 

1.     Round table discussion.

 

 

2.     Follow up issues

https://github.com/zephyrproject-rtos/zephyr/issues/51718

 

add  a default meta named ‘level’ as below

 

level:

-      unit/integration/acceptance/system/general

-      simulation/hardware/host

 

yellow  is default one

 

Regards,

Hake

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
1124日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Hake Huang
 

Hi All,

 

This weekly meeting agenda:

 

Today I did not get any special topics from any member, so we can have a short sync, and round table discussion/

 

  1. Round table discussion.

 

 

Regards,

Hake

 

 

-----Original Appointment-----
From: Brett Preston <bpreston@...>
Sent: 2021
614 9:58
To: Brett Preston
Cc: bpreston@...; kstewart@...; Hake Huang; anas.nashif@...; Mai, Enjia
Subject: [EXT] Zephyr: Testing WG
When: 2022
121日星期四 7:00-8:00 (UTC-08:00) Pacific Time (US & Canada).
Where:

 

Caution: EXT Email



 

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________