Date   

Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

Hake Huang
 

Hi Maciej

 

It is 2:00pm to 3:00pm, (GMT+00:00) UTC

 

Regards,

Hake

 

From: Perkowski, Maciej <Maciej.Perkowski@...>
Sent: 2020
1110 16:21
To: Hake Huang <hake.huang@...>; testing-wg@...
Subject: RE: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

 

Caution: EXT Email

Hi Hake,
Could you please state the time in UTC? Is it as in call for RSVP: 14-15 UTC?

Maciej Perkowski | Software Test Developer
M +48 728 395 111 | Kraków, Poland
nordicsemi.com | devzone.nordicsemi.com

Nordic_logo_signature

 

From: testing-wg@... <testing-wg@...> On Behalf Of Hake Huang via lists.zephyrproject.org
Sent: Tuesday, November 10, 2020 4:47 AM
To: testing-wg@...
Subject: Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

 

Hi All,

 

This is a notice to that weekly call will be 1 hour later for regions that do not use day time saving shift(e.g. China), if you have problem with this time, please propose your favourite time.

 

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
119 21:50
To: testing-wg@...
Subject: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

 

Caution: EXT Email

Reminder: Zephyr: Testing WG weekly call

When: Monday, 9 November 2020, 2:00pm to 3:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

Perkowski, Maciej
 

Hi Hake,
Could you please state the time in UTC? Is it as in call for RSVP: 14-15 UTC?

Maciej Perkowski | Software Test Developer
M +48 728 395 111 | Kraków, Poland
nordicsemi.com | devzone.nordicsemi.com

Nordic_logo_signature

 

From: testing-wg@... <testing-wg@...> On Behalf Of Hake Huang via lists.zephyrproject.org
Sent: Tuesday, November 10, 2020 4:47 AM
To: testing-wg@...
Subject: Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

 

Hi All,

 

This is a notice to that weekly call will be 1 hour later for regions that do not use day time saving shift(e.g. China), if you have problem with this time, please propose your favourite time.

 

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
119 21:50
To: testing-wg@...
Subject: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

 

Caution: EXT Email

Reminder: Zephyr: Testing WG weekly call

When: Monday, 9 November 2020, 2:00pm to 3:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

Hake Huang
 

Hi All,

 

This is a notice to that weekly call will be 1 hour later for regions that do not use day time saving shift(e.g. China), if you have problem with this time, please propose your favourite time.

 

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
119 21:50
To: testing-wg@...
Subject: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

 

Caution: EXT Email

Reminder: Zephyr: Testing WG weekly call

When: Monday, 9 November 2020, 2:00pm to 3:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder: Zephyr: Testing WG weekly call

When: Monday, 9 November 2020, 2:00pm to 3:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
 
 
________________________________________________________________________________


Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/02/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

Hake Huang
 

Hi All,

 

This is this weekly meeting agenda.

 

1.     versions.json with weekly usage case intro. (Anas, 10 minuts)

https://github.com/zephyrproject-rtos/zephyr/pull/29583/files/2b962288f2058828282d100c18540928adc539bd..418ab8529567743a30a77ee6e5efdb4f66f7be54

 

2.     Feedback on Release metrics document. (Anas, Meciej, 15 mintues)

https://docs.google.com/document/d/1y7mtAhmvl5z1ohazNgXFEXdFg6RO-J38QsFA7twPPW8/edit?pli=1#

 

3.     Test manager sytem survey and typical process discussion. (Maksim, 15 minutes)

 

4.     Round table disscussion. (all, 20 mintues)

 

Regards,

Hake

 

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
119 21:50
To: testing-wg@...
Subject: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 11/09/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

 

Caution: EXT Email

Reminder: Zephyr: Testing WG weekly call

When: Monday, 9 November 2020, 2:00pm to 3:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Zephyr: Testing WG weekly call - Mon, 11/02/2020 2:00pm-3:00pm, Please RSVP #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder: Zephyr: Testing WG weekly call

When: Monday, 2 November 2020, 2:00pm to 3:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
 
 
________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Nashif, Anas
 

Hi,

Same here, won’t be able to attend today. Sorry.

 

Anas

 

From: <testing-wg@...> on behalf of "Perkowski, Maciej" <maciej.perkowski@...>
Date: Monday, 2 November 2020 at 02:57
To: "hake.huang@..." <hake.huang@...>, "testing-wg@..." <testing-wg@...>
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Hi,

I will have to skip today’s meeting as well.

Regards,

Maciej

Maciej Perkowski | Software Test Developer
M +48 728 395 111 | Kraków, Poland
nordicsemi.com | devzone.nordicsemi.com

Nordic_logo_signature

 

From: testing-wg@... <testing-wg@...> On Behalf Of Hake Huang via lists.zephyrproject.org
Sent: Monday, November 2, 2020 4:35 AM
To: testing-wg@...
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Hi All,

 

This is this weekly meeting agenda.

 

1.  Updates on versions.json update PR. (Anas, 10 minuts)

https://github.com/zephyrproject-rtos/zephyr/pull/29583/files/2b962288f2058828282d100c18540928adc539bd..418ab8529567743a30a77ee6e5efdb4f66f7be54

 

2.  Feedback on Release metrics document. (Anas, Meciej, 15 mintues)

https://docs.google.com/document/d/1y7mtAhmvl5z1ohazNgXFEXdFg6RO-J38QsFA7twPPW8/edit?pli=1#

 

3.  Seek for help on gPTP testing setup. (Hake, 15 minutes)

Does any one have the test envrion setting on Windows?

https://github.com/zephyrproject-rtos/zephyr/issues/29599

 

4.  Round table disscussion. (all, 20 mintues)

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Perkowski, Maciej
 

Hi,

I will have to skip today’s meeting as well.

Regards,

Maciej

Maciej Perkowski | Software Test Developer
M +48 728 395 111 | Kraków, Poland
nordicsemi.com | devzone.nordicsemi.com

Nordic_logo_signature

 

From: testing-wg@... <testing-wg@...> On Behalf Of Hake Huang via lists.zephyrproject.org
Sent: Monday, November 2, 2020 4:35 AM
To: testing-wg@...
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Hi All,

 

This is this weekly meeting agenda.

 

1.     Updates on versions.json update PR. (Anas, 10 minuts)

https://github.com/zephyrproject-rtos/zephyr/pull/29583/files/2b962288f2058828282d100c18540928adc539bd..418ab8529567743a30a77ee6e5efdb4f66f7be54

 

2.     Feedback on Release metrics document. (Anas, Meciej, 15 mintues)

https://docs.google.com/document/d/1y7mtAhmvl5z1ohazNgXFEXdFg6RO-J38QsFA7twPPW8/edit?pli=1#

 

3.     Seek for help on gPTP testing setup. (Hake, 15 minutes)

Does any one have the test envrion setting on Windows?

https://github.com/zephyrproject-rtos/zephyr/issues/29599

 

4.     Round table disscussion. (all, 20 mintues)

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Hake Huang
 

Hi All,

 

This is this weekly meeting agenda.

 

1.     Updates on versions.json update PR. (Anas, 10 minuts)

https://github.com/zephyrproject-rtos/zephyr/pull/29583/files/2b962288f2058828282d100c18540928adc539bd..418ab8529567743a30a77ee6e5efdb4f66f7be54

 

2.     Feedback on Release metrics document. (Anas, Meciej, 15 mintues)

https://docs.google.com/document/d/1y7mtAhmvl5z1ohazNgXFEXdFg6RO-J38QsFA7twPPW8/edit?pli=1#

 

3.     Seek for help on gPTP testing setup. (Hake, 15 minutes)

Does any one have the test envrion setting on Windows?

https://github.com/zephyrproject-rtos/zephyr/issues/29599

4.     Round table disscussion. (all, 20 mintues)

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Hake Huang
 

 

Thanks Anas.

 

My reason to create a new weekly json file.

 

  1. I find it is difficult to create a compatible json file. and I want to keep the versions.json consistent so that the existing parser will not change.
  2. I want to make it easy to select a weekly test once you know the week of year. So I propose a dict format:

{<Week of year> : <version tag>}

 

So it will be very easy to retrieve any given weekly’s hash tag, by

Data = json.load(…)

if week in data:

   version  = data[week]

 

I see your PR at https://github.com/zephyrproject-rtos/zephyr/pull/29583/files/2b962288f2058828282d100c18540928adc539bd..418ab8529567743a30a77ee6e5efdb4f66f7be54 , and I leave my comments there. Thanks.

 

Regards,
Hake

 

From: Nashif, Anas <anas.nashif@...>
Sent: 2020
1027 19:55
To: Hake Huang <hake.huang@...>; testing-wg@...
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

I did not suggest using another file, we should use exactly the same file but expand the data we store there:

  • Add date
  • Add a flag for weekly

 

I do not think creating a new file is needed..

 

Anas

 

From: <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Date: Tuesday, 27 October 2020 at 02:04
To: "testing-wg@..." <testing-wg@...>
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Hi All,

 

In this weekly meeting, Anas proposes that we shall create a regular candency to upload our test result. If we can not afford daily result upload, then we shall do a weekly upload in a given version, this PR is to create a new https://testing.zephyrproject.org/daily_tests/weeklys.json which contains the week of year and version tag in dict, so that we can use this json file to sync our efforts. In such way, we can get a full report with as many boards as possible in a given tag. Please kindly give you comments.

 

https://github.com/zephyrproject-rtos/test_results/pull/141

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Hake Huang
 

Hi All,

 

Anash update this feature in https://github.com/zephyrproject-rtos/zephyr/pull/29583/files/2b962288f2058828282d100c18540928adc539bd..418ab8529567743a30a77ee6e5efdb4f66f7be54

 

Please check this PR, I have close my former PR.

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of Hake Huang via lists.zephyrproject.org
Sent: 2020
1027 17:00
To: Erwan Gouriou <erwan.gouriou@...>
Cc: testing-wg@...
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Hi Erwan,

 

Thanks for your comments. Please check my inline comments below(lead by Hake)

 

Regards,

Hake

 

From: Erwan Gouriou <erwan.gouriou@...>
Sent: 2020
1027 16:42
To: Hake Huang <hake.huang@...>
Cc: testing-wg@...
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Hi Hake,

 

I have few questions indeed:

1. Does it mean that https://testing.zephyrproject.org/daily_tests/versions.json would not be used anymore ?

    I suggest we keep it as weekly frequency will not be sufficient in release periods.

     [Hake] the versions is kept also.

2. Is there a gating protocol in order to avoid publishing a "bad" commit with build issues that would prevent

    having meaningful results? If there's only one test result upload per week, we need to ensure a minimum quality

    is guaranteed

     [Hake] we have some discussion this week, but we need define a scope on what to check, let me creat an issue here.

3. The duration of an automated test campaign on 14 boards is less than 3 hours (when there are no major issues).

    This of course depends on the machine used, but even with twice this duration, this can be done in a night.

    I'm curious about the obstacles to perform such campaigns on a daily basis.

       [Hake] According to my experiences, there are some efforts required, e.g. I meet download issue regularly and some time an application blocks the board due to many things. So it is no a zero cost thing, (qemu board testing is totally different from real board testing). Also the machine performance is an issue. Not everyone has a powerful build machine, for me the configurations are 2 4-core desktop PCs, which take me a whole day to finish 6 boards if there is not issues found, If something stack, I need restart another round of test cycle from beginning.

 

Cheers

Erwan

 

 

 

 

On Tue, 27 Oct 2020 at 07:04, Hake Huang <hake.huang@...> wrote:

Hi All,

 

In this weekly meeting, Anas proposes that we shall create a regular candency to upload our test result. If we can not afford daily result upload, then we shall do a weekly upload in a given version, this PR is to create a new https://testing.zephyrproject.org/daily_tests/weeklys.json which contains the week of year and version tag in dict, so that we can use this json file to sync our efforts. In such way, we can get a full report with as many boards as possible in a given tag. Please kindly give you comments.

 

https://github.com/zephyrproject-rtos/test_results/pull/141

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Nashif, Anas
 

I did not suggest using another file, we should use exactly the same file but expand the data we store there:

  • Add date
  • Add a flag for weekly

 

I do not think creating a new file is needed..

 

Anas

 

From: <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Date: Tuesday, 27 October 2020 at 02:04
To: "testing-wg@..." <testing-wg@...>
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Hi All,

 

In this weekly meeting, Anas proposes that we shall create a regular candency to upload our test result. If we can not afford daily result upload, then we shall do a weekly upload in a given version, this PR is to create a new https://testing.zephyrproject.org/daily_tests/weeklys.json which contains the week of year and version tag in dict, so that we can use this json file to sync our efforts. In such way, we can get a full report with as many boards as possible in a given tag. Please kindly give you comments.

 

https://github.com/zephyrproject-rtos/test_results/pull/141

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Erwan Gouriou
 

Thanks Hake for the feedback, it seems I underestimated the impact of test machine performance.


On Tue, 27 Oct 2020 at 09:59, Hake Huang <hake.huang@...> wrote:

Hi Erwan,

 

Thanks for your comments. Please check my inline comments below(lead by Hake)

 

Regards,

Hake

 

From: Erwan Gouriou <erwan.gouriou@...>
Sent: 2020
1027 16:42
To: Hake Huang <hake.huang@...>
Cc: testing-wg@...
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Hi Hake,

 

I have few questions indeed:

1. Does it mean that https://testing.zephyrproject.org/daily_tests/versions.json would not be used anymore ?

    I suggest we keep it as weekly frequency will not be sufficient in release periods.

     [Hake] the versions is kept also.

2. Is there a gating protocol in order to avoid publishing a "bad" commit with build issues that would prevent

    having meaningful results? If there's only one test result upload per week, we need to ensure a minimum quality

    is guaranteed

     [Hake] we have some discussion this week, but we need define a scope on what to check, let me creat an issue here.

3. The duration of an automated test campaign on 14 boards is less than 3 hours (when there are no major issues).

    This of course depends on the machine used, but even with twice this duration, this can be done in a night.

    I'm curious about the obstacles to perform such campaigns on a daily basis.

       [Hake] According to my experiences, there are some efforts required, e.g. I meet download issue regularly and some time an application blocks the board due to many things. So it is no a zero cost thing, (qemu board testing is totally different from real board testing). Also the machine performance is an issue. Not everyone has a powerful build machine, for me the configurations are 2 4-core desktop PCs, which take me a whole day to finish 6 boards if there is not issues found, If something stack, I need restart another round of test cycle from beginning.

 

Cheers

Erwan

 

 

 

 

On Tue, 27 Oct 2020 at 07:04, Hake Huang <hake.huang@...> wrote:

Hi All,

 

In this weekly meeting, Anas proposes that we shall create a regular candency to upload our test result. If we can not afford daily result upload, then we shall do a weekly upload in a given version, this PR is to create a new https://testing.zephyrproject.org/daily_tests/weeklys.json which contains the week of year and version tag in dict, so that we can use this json file to sync our efforts. In such way, we can get a full report with as many boards as possible in a given tag. Please kindly give you comments.

 

https://github.com/zephyrproject-rtos/test_results/pull/141

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Hake Huang
 

Hi Erwan,

 

Thanks for your comments. Please check my inline comments below(lead by Hake)

 

Regards,

Hake

 

From: Erwan Gouriou <erwan.gouriou@...>
Sent: 2020
1027 16:42
To: Hake Huang <hake.huang@...>
Cc: testing-wg@...
Subject: Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Hi Hake,

 

I have few questions indeed:

1. Does it mean that https://testing.zephyrproject.org/daily_tests/versions.json would not be used anymore ?

    I suggest we keep it as weekly frequency will not be sufficient in release periods.

     [Hake] the versions is kept also.

2. Is there a gating protocol in order to avoid publishing a "bad" commit with build issues that would prevent

    having meaningful results? If there's only one test result upload per week, we need to ensure a minimum quality

    is guaranteed

     [Hake] we have some discussion this week, but we need define a scope on what to check, let me creat an issue here.

3. The duration of an automated test campaign on 14 boards is less than 3 hours (when there are no major issues).

    This of course depends on the machine used, but even with twice this duration, this can be done in a night.

    I'm curious about the obstacles to perform such campaigns on a daily basis.

       [Hake] According to my experiences, there are some efforts required, e.g. I meet download issue regularly and some time an application blocks the board due to many things. So it is no a zero cost thing, (qemu board testing is totally different from real board testing). Also the machine performance is an issue. Not everyone has a powerful build machine, for me the configurations are 2 4-core desktop PCs, which take me a whole day to finish 6 boards if there is not issues found, If something stack, I need restart another round of test cycle from beginning.

 

Cheers

Erwan

 

 

 

 

On Tue, 27 Oct 2020 at 07:04, Hake Huang <hake.huang@...> wrote:

Hi All,

 

In this weekly meeting, Anas proposes that we shall create a regular candency to upload our test result. If we can not afford daily result upload, then we shall do a weekly upload in a given version, this PR is to create a new https://testing.zephyrproject.org/daily_tests/weeklys.json which contains the week of year and version tag in dict, so that we can use this json file to sync our efforts. In such way, we can get a full report with as many boards as possible in a given tag. Please kindly give you comments.

 

https://github.com/zephyrproject-rtos/test_results/pull/141

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Erwan Gouriou
 

Hi Hake,

I have few questions indeed:
1. Does it mean that https://testing.zephyrproject.org/daily_tests/versions.json would not be used anymore ?
    I suggest we keep it as weekly frequency will not be sufficient in release periods.
2. Is there a gating protocol in order to avoid publishing a "bad" commit with build issues that would prevent
    having meaningful results? If there's only one test result upload per week, we need to ensure a minimum quality
    is guaranteed
3. The duration of an automated test campaign on 14 boards is less than 3 hours (when there are no major issues).
    This of course depends on the machine used, but even with twice this duration, this can be done in a night.
    I'm curious about the obstacles to perform such campaigns on a daily basis.

Cheers
Erwan




On Tue, 27 Oct 2020 at 07:04, Hake Huang <hake.huang@...> wrote:

Hi All,

 

In this weekly meeting, Anas proposes that we shall create a regular candency to upload our test result. If we can not afford daily result upload, then we shall do a weekly upload in a given version, this PR is to create a new https://testing.zephyrproject.org/daily_tests/weeklys.json which contains the week of year and version tag in dict, so that we can use this json file to sync our efforts. In such way, we can get a full report with as many boards as possible in a given tag. Please kindly give you comments.

 

https://github.com/zephyrproject-rtos/test_results/pull/141

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

Hake Huang
 

Hi All,

 

In this weekly meeting, Anas proposes that we shall create a regular candency to upload our test result. If we can not afford daily result upload, then we shall do a weekly upload in a given version, this PR is to create a new https://testing.zephyrproject.org/daily_tests/weeklys.json which contains the week of year and version tag in dict, so that we can use this json file to sync our efforts. In such way, we can get a full report with as many boards as possible in a given tag. Please kindly give you comments.

 

https://github.com/zephyrproject-rtos/test_results/pull/141

 

Regards,

Hake

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 22:20
To: testing-wg@...
Subject: [EXT] [testing-wg] Updated Event: Zephyr: Testing WG weekly call #cal-invite

 

Caution: EXT Email

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Updated Event: Zephyr: Testing WG weekly call #cal-invite

testing-wg@lists.zephyrproject.org Calendar <noreply@...>
 

Zephyr: Testing WG weekly call

When:
Monday, 2 November 2020
2:00pm to 3:00pm
(UTC+00:00) UTC
Repeats: Weekly on Monday

Where:
Microsoft Teams Meeting

Organizer: testing-wg@...

An RSVP is requested. Click here to RSVP

Description:

________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
 
 
________________________________________________________________________________


Zephyr: Testing WG weekly call - Mon, 10/26/2020 1:00pm-2:00pm, Please RSVP #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder: Zephyr: Testing WG weekly call

When: Monday, 26 October 2020, 1:00pm to 2:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
 
 
________________________________________________________________________________


Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 10/26/2020 1:00pm-2:00pm, Please RSVP #cal-reminder

Perkowski, Maciej
 

Hi,

I am a bit sick and I won’t join today’s meeting,

Regards

 

Maciej Perkowski | Software Test Developer
M +48 728 395 111 | Kraków, Poland
nordicsemi.com | devzone.nordicsemi.com

Nordic_logo_signature

 

From: testing-wg@... <testing-wg@...> On Behalf Of Hake Huang via lists.zephyrproject.org
Sent: Monday, October 26, 2020 9:18 AM
To: testing-wg@...
Subject: Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 10/26/2020 1:00pm-2:00pm, Please RSVP #cal-reminder

 

Hi All,

 

This weekly meeting agenda:

 

1.  Teting wiki link announcement. (10 minutes, Hake)

(https://github.com/zephyrproject-rtos/zephyr/wiki/Continuous-Integration)

 

2.  Discussion on Quality release Metrics on release v2.4.0. (15 minutes, Maciej)

https://docs.google.com/document/d/1y7mtAhmvl5z1ohazNgXFEXdFg6RO-J38QsFA7twPPW8/edit?pli=1#

 

3.     Round table discussion. (30 minutes)

a)       Test plan draft continues.

b)      Function team engagement.

 

Regards,

Hake

 

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 20:50
To: testing-wg@...
Subject: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 10/26/2020 1:00pm-2:00pm, Please RSVP #cal-reminder

 

Caution: EXT Email

Reminder: Zephyr: Testing WG weekly call

When: Monday, 26 October 2020, 1:00pm to 2:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________


Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 10/26/2020 1:00pm-2:00pm, Please RSVP #cal-reminder

Hake Huang
 

Hi All,

 

This weekly meeting agenda:

 

1.  Teting wiki link announcement. (10 minutes, Hake)

(https://github.com/zephyrproject-rtos/zephyr/wiki/Continuous-Integration)

 

2.  Discussion on Quality release Metrics on release v2.4.0. (15 minutes, Maciej)

https://docs.google.com/document/d/1y7mtAhmvl5z1ohazNgXFEXdFg6RO-J38QsFA7twPPW8/edit?pli=1#

 

3.     Round table discussion. (30 minutes)

a)      Test plan draft continues.

b)      Function team engagement.

 

Regards,

Hake

 

 

From: testing-wg@... <testing-wg@...> On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Sent: 2020
1026 20:50
To: testing-wg@...
Subject: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 10/26/2020 1:00pm-2:00pm, Please RSVP #cal-reminder

 

Caution: EXT Email

Reminder: Zephyr: Testing WG weekly call

When: Monday, 26 October 2020, 1:00pm to 2:00pm, (GMT+00:00) UTC

Where:Microsoft Teams Meeting

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description:

________________________________________________________________________________

+1 321-558-6518 United States, Orlando (Toll)

Conference ID: 831 716 531#

Local numbers | Reset PIN | Learn more about Teams | Meeting options

 

 

________________________________________________________________________________

61 - 80 of 329