Hi Brett,

I lost write access to the meeting agenda file, can you help to grant me write access? Thanks in advance.

Regards,

Hake

Caution: EXT Email

Hi All,

This weekly meeting agenda:

1.     ZDS recap.

a)      Twister V2

2.     MAX Pro test supporting.

a)      NXP setup maxpro testing environment, and if you like, we can support your platform monthly testing (MAX Pro) as well,

b)      Background info is here

c)      Current issues found is reported here

3.     PR talk

a)      Twister support distribute run

https://github.com/zephyrproject-rtos/zephyr/pull/46078

b)      Fix the dead loop in checking device status

https://github.com/zephyrproject-rtos/zephyr/pull/45263

Regards,

Hake

Caution: EXT Email

Hi All,

This weekly meeting agenda

1.     PR talk

a)      False negative issue

https://github.com/zephyrproject-rtos/zephyr/pull/45943

b)      Code scan path

       https://github.com/zephyrproject-rtos/zephyr/pull/45759

c)      cmsis_dsp parse issue

       https://github.com/zephyrproject-rtos/zephyr/pull/45832

d)      discard handling

https://github.com/zephyrproject-rtos/zephyr/pull/45684

2.     Round table

Regards,

Hake

Caution: EXT Email

Hi All,

This weekly meeting agenda

1. Follow up “with what tested and put the summary to the public”

a.       The test suite can get consistent results, and we test on important coverage(modules, drivers, platform). How to get coverage improved.

1. Current situation:

l  The test suites are based on API coverage, with additional user scenario + API parameter enhancement.

l  We only have line/branch coverage analysis with x86 platforms.

b.        Reporting(Grafana) and how we do reporting. Test plan and test cases manager. Find a way to take all the results, align on TMS.

1. Current situation:

l  We use test_result repo and allow any one from community to upload their results. Question: Do we need encourage PR owner to provide a test result link in their board testing result for given PR?

l  We provide a Grafana web site with statistic report on test result. (maybe we need feedback on what need add)

l  Zephyr does not have TMS system so far(which is mainly for historic result comparison), to utilize a TMS system we need below:

1)        Test cases id and name shall be mapped consistently, change test name will confuse the test result report over history.

2)        Test case mapped with test suite and test plan shall be provide API to update, as well as test result.

3)        Any one can generate test report without limitation.

c.       Need to create a process to upload test results(json format) for a certain PR on actual HW. (mainly ensure the test is consistent w/wo this PR), select the proper test suites. Also need to isolate real board testing with qemu/cmock way.

1.       Current situation:

We have setup a flow based on test_result, which anyone can upload their test result for given commit. Question: do we need provide a PR result, which does not check the version? Also report github ticket based on upload test result? The scripts is ready.

Maintainer.yml help to get more precise cases in PR-CI

Do we need to create more cases on cmock way? which means we need urge API owner to provide test with mocked.

2.     PR talk

https://github.com/zephyrproject-rtos/zephyr/pull/45684/files

Regards,

Hake

Caution: EXT Email

Hi Hake,

               Could we add one topic about this PR https://github.com/zephyrproject-rtos/zephyr/pull/44976 ? I want to push this further to fix the passrate impact in our daily test report, thanks.

Thanks,

Peng

Hi All,

This weekly meeting agenda:

1. Follow up “with what tested and put the summary to the public”

a.     The test suite can get consistent results, and we test on important coverage(modules, drivers, platform). How to get coverage improved.

1. Current situation:

l  The test suites are based on API coverage, with additional user scenario + API parameter enhancement.

l  We only have line/branch coverage analysis with x86 platforms.

b.      Reporting(Grafana) and how we do reporting. Test plan and test cases manager. Find a way to take all the results, align on TMS.

1. Current situation:

l  We use test_result repo and allow any one from community to upload their results. Question: Do we need encourage PR owner to provide a test result link in their board testing result for given PR?

l  We provide a Grafana web site with statistic report on test result. (maybe we need feedback on what need add)

l  Zephyr does not have TMS system so far(which is mainly for historic result comparison), to utilize a TMS system we need below:

1)       Test cases id and name shall be mapped consistently, change test name will confuse the test result report over history.

2)       Test case mapped with test suite and test plan shall be provide API to update, as well as test result.

3)       Any one can generate test report without limitation.

c.      Need to create a process to upload test results(json format) for a certain PR on actual HW. (mainly ensure the test is consistent w/wo this PR), select the proper test suites. Also need to isolate real board testing with qemu/cmock way.

1.     Current situation:

We have setup a flow based on test_result, which anyone can upload their test result for given commit. Question: do we need provide a PR result, which does not check the version? Also report github ticket based on upload test result? The scripts is ready.

Maintainer.yml help to get more precise cases in PR-CI

Do we need to create more cases on cmock way? which means we need urge API owner to provide test with mocked.

Regards,

Hake

Caution: EXT Email

Updated.

This weekly meeting agenda:

1. PR talk https://github.com/zephyrproject-rtos/zephyr/pull/44976

2. Follow up “with what tested and put the summary to the public”

a.    The test suite can get consistent results, and we test on important coverage(modules, drivers, platform). How to get coverage improved.

1. Current situation:

l  The test suites are based on API coverage, with additional user scenario + API parameter enhancement.

l  We only have line/branch coverage analysis with x86 platforms.

b.     Reporting(Grafana) and how we do reporting. Test plan and test cases manager. Find a way to take all the results, align on TMS.

1. Current situation:

l  We use test_result repo and allow any one from community to upload their results. Question: Do we need encourage PR owner to provide a test result link in their board testing result for given PR?

l  We provide a Grafana web site with statistic report on test result. (maybe we need feedback on what need add)

l  Zephyr does not have TMS system so far(which is mainly for historic result comparison), to utilize a TMS system we need below:

1)     Test cases id and name shall be mapped consistently, change test name will confuse the test result report over history.

2)     Test case mapped with test suite and test plan shall be provide API to update, as well as test result.

3)     Any one can generate test report without limitation.

c.     Need to create a process to upload test results(json format) for a certain PR on actual HW. (mainly ensure the test is consistent w/wo this PR), select the proper test suites. Also need to isolate real board testing with qemu/cmock way.

1.    Current situation:

We have setup a flow based on test_result, which anyone can upload their test result for given commit. Question: do we need provide a PR result, which does not check the version? Also report github ticket based on upload test result? The scripts is ready.

Maintainer.yml help to get more precise cases in PR-CI

Do we need to create more cases on cmock way? which means we need urge API owner to provide test with mocked.

Caution: EXT Email

Hi All,

This weekly meeting agenda:

1. Follow up “with what tested and put the summary to the public”

1. The test suite can get consistent results, and we test on important coverage(modules, drivers, platform). How to get coverage improved.
1. Current situation:

l  The test suites are based on API coverage, with additional user scenario + API parameter enhancement.

l  We only have line/branch coverage analysis with x86 platforms.

2.  Reporting(Grafana) and how we do reporting. Test plan and test cases manager. Find a way to take all the results, align on TMS.
1. Current situation:

l  We use test_result repo and allow any one from community to upload their results. Question: Do we need encourage PR owner to provide a test result link in their board testing result for given PR?

l  We provide a Grafana web site with statistic report on test result. (maybe we need feedback on what need add)

l  Zephyr does not have TMS system so far(which is mainly for historic result comparison), to utilize a TMS system we need below:

1)     Test cases id and name shall be mapped consistently, change test name will confuse the test result report over history.

2)     Test case mapped with test suite and test plan shall be provide API to update, as well as test result.

3)     Any one can generate test report without limitation.

3. Need to create a process to upload test results(json format) for a certain PR on actual HW. (mainly ensure the test is consistent w/wo this PR), select the proper test suites. Also need to isolate real board testing with qemu/cmock way.

1.    Current situation:

We have setup a flow based on test_result, which anyone can upload their test result for given commit. Question: do we need provide a PR result, which does not check the version? Also report github ticket based on upload test result? The scripts is ready.

Maintainer.yml help to get more precise cases in PR-CI

Do we need to create more cases on cmock way? which means we need urge API owner to provide test with mocked.

Regards,

Hake

Caution: EXT Email

Hi All,

This weekly meeting agenda:

1.     Round table discussion.

1.       Monthly testing report summary.

1)      To export more information to community, do we need to have a monthly testing summary?

2)      If yes, the contents will be include

l  Board test report link

l  Issues that reported

l  new features with testing scripts

Regards,

Hake

Caution: EXT Email

Hi All,

This weekly meeting agenda:

1.     Round table discussion

2.     Monthly testing report summary.

1)     To export more information to community, do we need to have a monthly testing summary?

2)     If yes, the contents will be include

l  Board test report link

l  Issues that reported

l  new features with testing scripts

Regards,

Hake

Caution: EXT Email

Hi All,

This weekly meeting agenda: 

1.     Round table check

2.     V2.7.2-RC1 status updates.

3.     Code coverage process information gathering.

a)         Background: I get some query from Safety working group on code coverage process and howtos, want to sync current information. 

Regards,

Hake

Caution: EXT Email

Hi Hake,

Apologies, I won’t be able to attend this week.

Thank You,

Spoorthy Priya Yerabolu.

Hi All,

No specific topic for this weekly meeting. We can go round table and have a quick sync. Thanks.

Regards,

Hake

Caution: EXT Email

Hi All,

No specific topic for this weekly meeting. We can go round table and have a quick sync. Thanks.

Regards,

Hake

Caution: EXT Email

Hi All,

This weekly meeting agenda.

1.     Feedback for https://github.com/PerMac/TwisterV2

There is one PR from my side

https://github.com/PerMac/TwisterV2/pull/1

2.     Twister with suite name issue

https://github.com/zephyrproject-rtos/zephyr/issues/44539#issuecomment-1090106223

https://github.com/zephyrproject-rtos/zephyr/issues/44438

3.     Round table

Regards,

Hake

Caution: EXT Email

Hi All,

Agenda of this weekly meeting:

1.     Pytest for twister v2 demo.

2.     Round table discussion

Regards,

Hake

Caution: EXT Email

Dear Hake and all,

I am sorry, I cannot join today’s meeting. Have a good day.

Enjia

Hi All,

Agenda for this weekly meeting:

1.       Round table discussion.

2.       Demo of pytest usage in twister V2

3.       Follow up on actions of test scenario naming convention.

Regards,

Hake

Caution: EXT Email

Hi All,

Agenda for this weekly meeting:

1.     Round table discussion.

2.     Demo of pytest usage in twister V2

3.     Follow up on actions of test scenario naming convention.

Regards,

Hake

Caution: EXT Email

Hi All,

Agenda for this weekly meeting:

1.     Round table discussion

2.     Follow up on actions of test scenario naming convention issues.

3.     Discord testing channel alignment

1)     I create #failure case discussion, #ztest features, #twister related discussion

Regards,

Hake

Caution: EXT Email

Add one topic

Caution: EXT Email