|
Re: [EXT] [testing-wg] Event: Zephyr: Testing WG weekly call - 06/28/2021
#cal-reminder
Hake Huang
Hi All,
This weekly meeting agenda:
1. Issues discussion on PR https://github.com/zephyrproject-rtos/test_results/pull/384 and how to create public test-report repo that can be used to accommodate test issues, and transfer to zephyr main repo if confirmed manually.
2. Round table discussion.
Regards, Hake
From: testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Event: Zephyr: Testing WG weekly call - 06/21/2021
#cal-reminder
testing-wg@lists.zephyrproject.org Calendar <noreply@...>
Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Event: Zephyr: Testing WG weekly call - 06/21/2021
#cal-reminder
Hake Huang
Hi All,
This weekly meeting agenda.
1. Scripts to check failure case in CI report and create github issue. (10 minutes) https://github.com/zephyrproject-rtos/test_results/pull/384 2. Round table
Regards, Hake
From: testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Event: Zephyr: Testing WG weekly call - 06/14/2021
#cal-reminder
testing-wg@lists.zephyrproject.org Calendar <noreply@...>
Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Event: Zephyr: Testing WG weekly call - 06/14/2021
#cal-reminder
Hake Huang
Hi All,
This weekly meeting agenda:
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
l Round table
Regards, Hake
From: testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Event: Zephyr: Testing WG weekly call - 06/07/2021
#cal-reminder
testing-wg@lists.zephyrproject.org Calendar <noreply@...>
Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Event: Zephyr: Testing WG weekly call - 06/07/2021
#cal-reminder
Hake Huang
This is zephyr testing mini-conference meeting calendar on 6/8
Regards, Hake
From: Hake Huang
Hi All,
This weekly meeting agenda:
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
Regards, Hake
From:
testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Event: Zephyr: Testing WG weekly call - 06/07/2021
#cal-reminder
Hake Huang
Hi All,
This weekly meeting agenda:
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
Regards, Hake
From: testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Testing related miniconf during Zephyr Summit
Perkowski, Maciej
Hi, Regards,
Maciej Perkowski
| Software Test Developer
|
|
|
|
Event: Zephyr: Testing WG weekly call - 05/31/2021
#cal-reminder
testing-wg@lists.zephyrproject.org Calendar <noreply@...>
Reminder: Zephyr: Testing WG weekly call When: Where: Organizer: testing-wg@... An RSVP is requested. Click here to RSVP Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/31/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
Hake Huang
Hi All,
This weekly meeting agenda:
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit o Maciej/Anas twister usage o Maciej will send out discussion from github discussion. o [TBD]Cloud testing process, board testing report publishing o [TBD]Benchmarks/footprint/performances o [TBD/Rob] memory benchmark introduction.
Last week items follow up:
1. Nfs issue? a) Is it related to zephyr tool-chain or all gcc tool-chain? 2. How to support multiply SDK releases?
Regards, Hake
From: testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 31 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
Hake Huang
Hi Alex,
By last night’s meeting, below is the guide on how to use 3rd party tool-chain.
https://docs.zephyrproject.org/latest/getting_started/toolchain_3rd_party_x_compilers.html
Regards, Hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email I'm not an expert in NFS by any means, but looking at its description here https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/What_is_the_NFSv3_mount_%22noac%22_option%3F I'm not sure if it's going to make a difference.
What worries me is all the "computations" happen locally on the same machine, it's only tools that are being used are located on NFS share. So since there's no distributed (between multiple clients) execution of say "twister" script, why top-level VFS capabilities are not enough here?
-Alexey From: Hake Huang <hake.huang@...>
Do you try noac option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake,
That's what we have on the client machine: ----------------------->8------------------- rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=5,sec=sys,mountaddr=11.22.33.44,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=11.22.33.44 ----------------------->8-------------------
-Alexey From: Hake Huang <hake.huang@...>
Hi Alex,
How do you mount the nfs? I mean what is the option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake, all,
Our scenario for build problem reproduction is quite simple:
If we move Zephyr SDK right to the build machine the issue goes away. Use of simulated platforms is important because their executions happen in parallel, while execution on a real target boards is gated by very limited amount of available boards (we may have a few, but not 10s or even 100s).
Another interesting observation is with use of the same one technical user for execution of all "twister" instances (and subsequently all children processes) we quite early hit a limit of processes existing simultaneously for 1 user. On our servers we have "max user processes" ("ulimit -u") set to 4096 by default. And so the first thing we learned it's necessary to set it to something much higher, which we don now to get NFS-related problem reproduced.
Now, NFS-related problem seem to be related to some issues with synchronization as at some point we get a lot of python processes waiting for something with the following stack-trace: ---------------------->8---------------------- $ cat /proc/30876/stack [<ffffffff9a912076>] futex_wait_queue_me+0xc6/0x130 [<ffffffff9a912e1b>] futex_wait+0x17b/0x280 [<ffffffff9a914b66>] do_futex+0x106/0x5a0 [<ffffffff9a915080>] SyS_futex+0x80/0x190 [<ffffffff9af8dede>] system_call_fastpath+0x25/0x2a [<ffffffffffffffff>] 0xffffffffffffffff ---------------------->8----------------------
We're still in progress of debugging that. Once we have more data, we'll happily share it!
-Alexey From:
testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Hi All,
Today’s weekly meeting agenda
Follow up last weekly meeting
1. Mcuboot can be ROM? 2. Multi-branch checking in parallel. Alex, do you send out your scenario summary? 3. https://github.com/zephyrproject-rtos/zephyr/issues/34571 according to my experiences, it is because your board uart output some exception character which looks like a EOF to console handler.
General topics:
1. V.2.6.0 board testing discussion. a) Any common pending issues found in this release?
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
3. Round table discussion
Regards, Hake
From:
testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
I'm not an expert in NFS by any means, but looking at its description here https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/What_is_the_NFSv3_mount_%22noac%22_option%3F I'm
not sure if it's going to make a difference.
What worries me is all the "computations" happen locally on the same machine, it's only tools that are being used are located on NFS share. So since there's no distributed (between multiple clients) execution of say "twister" script, why top-level VFS capabilities
are not enough here?
-Alexey
From: Hake Huang <hake.huang@...>
Sent: Monday, May 24, 2021 3:01 PM To: Alexey Brodkin <abrodkin@...> Cc: testing-wg@... <testing-wg@...>; Eugeniy Paltsev <paltsev@...> Subject: RE: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP #cal-reminder Do you try noac option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake,
That's what we have on the client machine: ----------------------->8------------------- rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=5,sec=sys,mountaddr=11.22.33.44,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=11.22.33.44 ----------------------->8-------------------
-Alexey From: Hake Huang <hake.huang@...>
Hi Alex,
How do you mount the nfs? I mean what is the option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake, all,
Our scenario for build problem reproduction is quite simple:
If we move Zephyr SDK right to the build machine the issue goes away. Use of simulated platforms is important because their executions happen in parallel, while execution on a real target boards is gated by very limited amount of available boards (we may have a few, but not 10s or even 100s).
Another interesting observation is with use of the same one technical user for execution of all "twister" instances (and subsequently all children processes) we quite early hit a limit of processes existing simultaneously for 1 user. On our servers we have "max user processes" ("ulimit -u") set to 4096 by default. And so the first thing we learned it's necessary to set it to something much higher, which we don now to get NFS-related problem reproduced.
Now, NFS-related problem seem to be related to some issues with synchronization as at some point we get a lot of python processes waiting for something with the following stack-trace: ---------------------->8---------------------- $ cat /proc/30876/stack [<ffffffff9a912076>] futex_wait_queue_me+0xc6/0x130 [<ffffffff9a912e1b>] futex_wait+0x17b/0x280 [<ffffffff9a914b66>] do_futex+0x106/0x5a0 [<ffffffff9a915080>] SyS_futex+0x80/0x190 [<ffffffff9af8dede>] system_call_fastpath+0x25/0x2a [<ffffffffffffffff>] 0xffffffffffffffff ---------------------->8----------------------
We're still in progress of debugging that. Once we have more data, we'll happily share it!
-Alexey From:
testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Hi All,
Today’s weekly meeting agenda
Follow up last weekly meeting
1. Mcuboot can be ROM? 2. Multi-branch checking in parallel. Alex, do you send out your scenario summary? 3. https://github.com/zephyrproject-rtos/zephyr/issues/34571 according to my experiences, it is because your board uart output some exception character which looks like a EOF to console handler.
General topics:
1. V.2.6.0 board testing discussion. a) Any common pending issues found in this release?
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
3. Round table discussion
Regards, Hake
From:
testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
Hi Hake,
That's what we have on the client machine:
----------------------->8-------------------
rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=5,sec=sys,mountaddr=11.22.33.44,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=11.22.33.44
----------------------->8-------------------
-Alexey
From: Hake Huang <hake.huang@...>
Sent: Monday, May 24, 2021 2:47 PM To: Alexey Brodkin <abrodkin@...> Cc: testing-wg@... <testing-wg@...>; Eugeniy Paltsev <paltsev@...> Subject: RE: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP #cal-reminder Hi Alex,
How do you mount the nfs? I mean what is the option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake, all,
Our scenario for build problem reproduction is quite simple:
If we move Zephyr SDK right to the build machine the issue goes away. Use of simulated platforms is important because their executions happen in parallel, while execution on a real target boards is gated by very limited amount of available boards (we may have a few, but not 10s or even 100s).
Another interesting observation is with use of the same one technical user for execution of all "twister" instances (and subsequently all children processes) we quite early hit a limit of processes existing simultaneously for 1 user. On our servers we have "max user processes" ("ulimit -u") set to 4096 by default. And so the first thing we learned it's necessary to set it to something much higher, which we don now to get NFS-related problem reproduced.
Now, NFS-related problem seem to be related to some issues with synchronization as at some point we get a lot of python processes waiting for something with the following stack-trace: ---------------------->8---------------------- $ cat /proc/30876/stack [<ffffffff9a912076>] futex_wait_queue_me+0xc6/0x130 [<ffffffff9a912e1b>] futex_wait+0x17b/0x280 [<ffffffff9a914b66>] do_futex+0x106/0x5a0 [<ffffffff9a915080>] SyS_futex+0x80/0x190 [<ffffffff9af8dede>] system_call_fastpath+0x25/0x2a [<ffffffffffffffff>] 0xffffffffffffffff ---------------------->8----------------------
We're still in progress of debugging that. Once we have more data, we'll happily share it!
-Alexey From:
testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Hi All,
Today’s weekly meeting agenda
Follow up last weekly meeting
1. Mcuboot can be ROM? 2. Multi-branch checking in parallel. Alex, do you send out your scenario summary? 3. https://github.com/zephyrproject-rtos/zephyr/issues/34571 according to my experiences, it is because your board uart output some exception character which looks like a EOF to console handler.
General topics:
1. V.2.6.0 board testing discussion. a) Any common pending issues found in this release?
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
3. Round table discussion
Regards, Hake
From:
testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
Hi Hake, all,
Our scenario for build problem reproduction is quite simple:
If we move Zephyr SDK right to the build machine the issue goes away.
Use of simulated platforms is important because their executions happen in parallel, while execution on a real target boards is gated by very limited amount of available boards (we may have a few, but not 10s or even 100s).
Another interesting observation is with use of the same one technical user for execution of all "twister" instances (and subsequently all children processes) we quite early hit a limit of processes existing simultaneously for 1 user. On our servers we
have "max user processes" ("ulimit -u") set to 4096 by default. And so the first thing we learned it's necessary to set it to something much higher, which we don now to get NFS-related problem reproduced.
Now, NFS-related problem seem to be related to some issues with synchronization as at some point we get a lot of python processes waiting for something with the following stack-trace:
---------------------->8----------------------
$ cat /proc/30876/stack
[<ffffffff9a912076>] futex_wait_queue_me+0xc6/0x130
[<ffffffff9a912e1b>] futex_wait+0x17b/0x280
[<ffffffff9a914b66>] do_futex+0x106/0x5a0
[<ffffffff9a915080>] SyS_futex+0x80/0x190
[<ffffffff9af8dede>] system_call_fastpath+0x25/0x2a
[<ffffffffffffffff>] 0xffffffffffffffff---------------------->8----------------------
We're still in progress of debugging that. Once we have more data, we'll happily share it!
-Alexey
From: testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Sent: Monday, May 24, 2021 7:50 AM To: testing-wg@... <testing-wg@...> Subject: Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP #cal-reminder Hi All,
Today’s weekly meeting agenda
Follow up last weekly meeting
1. Mcuboot can be ROM? 2. Multi-branch checking in parallel. Alex, do you send out your scenario summary? 3. https://github.com/zephyrproject-rtos/zephyr/issues/34571 according to my experiences, it is because your board uart output some exception character which looks like a EOF to console handler.
General topics:
1. V.2.6.0 board testing discussion. a) Any common pending issues found in this release?
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
3. Round table discussion
Regards, Hake
From: testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
________________________________________________________________________________
+1 321-558-6518 United States, Orlando (Toll)
Conference ID: 831 716 531#
Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
Hake Huang
I just remember we have file sync issue before, so try to add noac will disable the cache in nfs, which will downgrade the performance, but sync the files always.
Regards, Hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email I'm not an expert in NFS by any means, but looking at its description here https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/What_is_the_NFSv3_mount_%22noac%22_option%3F I'm not sure if it's going to make a difference.
What worries me is all the "computations" happen locally on the same machine, it's only tools that are being used are located on NFS share. So since there's no distributed (between multiple clients) execution of say "twister" script, why top-level VFS capabilities are not enough here?
-Alexey From: Hake Huang <hake.huang@...>
Do you try noac option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake,
That's what we have on the client machine: ----------------------->8------------------- rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=5,sec=sys,mountaddr=11.22.33.44,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=11.22.33.44 ----------------------->8-------------------
-Alexey From: Hake Huang <hake.huang@...>
Hi Alex,
How do you mount the nfs? I mean what is the option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake, all,
Our scenario for build problem reproduction is quite simple:
If we move Zephyr SDK right to the build machine the issue goes away. Use of simulated platforms is important because their executions happen in parallel, while execution on a real target boards is gated by very limited amount of available boards (we may have a few, but not 10s or even 100s).
Another interesting observation is with use of the same one technical user for execution of all "twister" instances (and subsequently all children processes) we quite early hit a limit of processes existing simultaneously for 1 user. On our servers we have "max user processes" ("ulimit -u") set to 4096 by default. And so the first thing we learned it's necessary to set it to something much higher, which we don now to get NFS-related problem reproduced.
Now, NFS-related problem seem to be related to some issues with synchronization as at some point we get a lot of python processes waiting for something with the following stack-trace: ---------------------->8---------------------- $ cat /proc/30876/stack [<ffffffff9a912076>] futex_wait_queue_me+0xc6/0x130 [<ffffffff9a912e1b>] futex_wait+0x17b/0x280 [<ffffffff9a914b66>] do_futex+0x106/0x5a0 [<ffffffff9a915080>] SyS_futex+0x80/0x190 [<ffffffff9af8dede>] system_call_fastpath+0x25/0x2a [<ffffffffffffffff>] 0xffffffffffffffff ---------------------->8----------------------
We're still in progress of debugging that. Once we have more data, we'll happily share it!
-Alexey From:
testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Hi All,
Today’s weekly meeting agenda
Follow up last weekly meeting
1. Mcuboot can be ROM? 2. Multi-branch checking in parallel. Alex, do you send out your scenario summary? 3. https://github.com/zephyrproject-rtos/zephyr/issues/34571 according to my experiences, it is because your board uart output some exception character which looks like a EOF to console handler.
General topics:
1. V.2.6.0 board testing discussion. a) Any common pending issues found in this release?
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
3. Round table discussion
Regards, Hake
From:
testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
Hake Huang
Do you try noac option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake,
That's what we have on the client machine: ----------------------->8------------------- rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=5,sec=sys,mountaddr=11.22.33.44,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=11.22.33.44 ----------------------->8-------------------
-Alexey From: Hake Huang <hake.huang@...>
Hi Alex,
How do you mount the nfs? I mean what is the option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake, all,
Our scenario for build problem reproduction is quite simple:
If we move Zephyr SDK right to the build machine the issue goes away. Use of simulated platforms is important because their executions happen in parallel, while execution on a real target boards is gated by very limited amount of available boards (we may have a few, but not 10s or even 100s).
Another interesting observation is with use of the same one technical user for execution of all "twister" instances (and subsequently all children processes) we quite early hit a limit of processes existing simultaneously for 1 user. On our servers we have "max user processes" ("ulimit -u") set to 4096 by default. And so the first thing we learned it's necessary to set it to something much higher, which we don now to get NFS-related problem reproduced.
Now, NFS-related problem seem to be related to some issues with synchronization as at some point we get a lot of python processes waiting for something with the following stack-trace: ---------------------->8---------------------- $ cat /proc/30876/stack [<ffffffff9a912076>] futex_wait_queue_me+0xc6/0x130 [<ffffffff9a912e1b>] futex_wait+0x17b/0x280 [<ffffffff9a914b66>] do_futex+0x106/0x5a0 [<ffffffff9a915080>] SyS_futex+0x80/0x190 [<ffffffff9af8dede>] system_call_fastpath+0x25/0x2a [<ffffffffffffffff>] 0xffffffffffffffff ---------------------->8----------------------
We're still in progress of debugging that. Once we have more data, we'll happily share it!
-Alexey From:
testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Hi All,
Today’s weekly meeting agenda
Follow up last weekly meeting
1. Mcuboot can be ROM? 2. Multi-branch checking in parallel. Alex, do you send out your scenario summary? 3. https://github.com/zephyrproject-rtos/zephyr/issues/34571 according to my experiences, it is because your board uart output some exception character which looks like a EOF to console handler.
General topics:
1. V.2.6.0 board testing discussion. a) Any common pending issues found in this release?
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
3. Round table discussion
Regards, Hake
From:
testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
Hake Huang
Hi Alex,
How do you mount the nfs? I mean what is the option?
Regards, hake
From: Alexey Brodkin <Alexey.Brodkin@...>
Caution: EXT Email Hi Hake, all,
Our scenario for build problem reproduction is quite simple:
If we move Zephyr SDK right to the build machine the issue goes away. Use of simulated platforms is important because their executions happen in parallel, while execution on a real target boards is gated by very limited amount of available boards (we may have a few, but not 10s or even 100s).
Another interesting observation is with use of the same one technical user for execution of all "twister" instances (and subsequently all children processes) we quite early hit a limit of processes existing simultaneously for 1 user. On our servers we have "max user processes" ("ulimit -u") set to 4096 by default. And so the first thing we learned it's necessary to set it to something much higher, which we don now to get NFS-related problem reproduced.
Now, NFS-related problem seem to be related to some issues with synchronization as at some point we get a lot of python processes waiting for something with the following stack-trace: ---------------------->8---------------------- $ cat /proc/30876/stack [<ffffffff9a912076>] futex_wait_queue_me+0xc6/0x130 [<ffffffff9a912e1b>] futex_wait+0x17b/0x280 [<ffffffff9a914b66>] do_futex+0x106/0x5a0 [<ffffffff9a915080>] SyS_futex+0x80/0x190 [<ffffffff9af8dede>] system_call_fastpath+0x25/0x2a [<ffffffffffffffff>] 0xffffffffffffffff ---------------------->8----------------------
We're still in progress of debugging that. Once we have more data, we'll happily share it!
-Alexey From:
testing-wg@... <testing-wg@...> on behalf of Hake Huang <hake.huang@...>
Hi All,
Today’s weekly meeting agenda
Follow up last weekly meeting
1. Mcuboot can be ROM? 2. Multi-branch checking in parallel. Alex, do you send out your scenario summary? 3. https://github.com/zephyrproject-rtos/zephyr/issues/34571 according to my experiences, it is because your board uart output some exception character which looks like a EOF to console handler.
General topics:
1. V.2.6.0 board testing discussion. a) Any common pending issues found in this release?
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
3. Round table discussion
Regards, Hake
From:
testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
|
Re: [EXT] [testing-wg] Zephyr: Testing WG weekly call - Mon, 05/24/2021 1:00pm-2:00pm, Please RSVP
#cal-reminder
Hake Huang
Hi All,
Today’s weekly meeting agenda
Follow up last weekly meeting
1. Mcuboot can be ROM? 2. Multi-branch checking in parallel. Alex, do you send out your scenario summary? 3. https://github.com/zephyrproject-rtos/zephyr/issues/34571 according to my experiences, it is because your board uart output some exception character which looks like a EOF to console handler.
General topics:
1. V.2.6.0 board testing discussion. a) Any common pending issues found in this release?
https://github.com/zephyrproject-rtos/zephyr/wiki/2021-Zephyr-Developer-Summit
3. Round table discussion
Regards, Hake
From: testing-wg@... <testing-wg@...>
On Behalf Of testing-wg@... Calendar via lists.zephyrproject.org
Caution: EXT Email Reminder: Zephyr: Testing WG weekly call When: Monday, 24 May 2021, 1:00pm to 2:00pm, (GMT+00:00) UTC Where:Microsoft Teams Meeting An RSVP is requested. Click here to RSVP Organizer: testing-wg@... Description: Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing ________________________________________________________________________________ +1 321-558-6518 United States, Orlando (Toll) Conference ID: 831 716 531# Local numbers | Reset PIN | Learn more about Teams | Meeting options
________________________________________________________________________________
|
|
|
