Date   

Zephyr: Testing WG weekly call - Mon, 03/18/2019 8:00am-9:00am #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder:
Zephyr: Testing WG weekly call

When:
Monday, 18 March 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH

An RSVP is requested. Click here to RSVP


Zephyr: Testing WG weekly call - Mon, 03/25/2019 8:00am-9:00am #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder:
Zephyr: Testing WG weekly call

When:
Monday, 25 March 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH

An RSVP is requested. Click here to RSVP


some idea on providing test services to community

Hake Huang
 

Hi All,

 

Pre last night’s discussion, here is the briefing of my idea.

 

1.     We can announce that we accept pull request testing.(binary test can be later support)

a)      A pull request ID shall be provide.

b)      A configure file shall include in the pull request, which defines:

1.      platform to test, default all.

2.      Test suites to test, default all.

 

2.     once a pull request are created, a trigger script shall be created to pull the pull request to each test volunteer’s testing branch. This can be done via shippable.

 

3.     The selected volunteers can start testing the pull request by his own condition.

 

4.     Volunteers shall have a mechanism to upload their test report to testrails, with a given report name.

A comments will be added automatically to the pull request, once the report is uploaded.

 

I demo my internal system to do #3 automatically, which is based on Jenkins blueocean pipeline files.

https://jenkins.io/doc/book/blueocean/

and NXP board pipeline files are defined here

https://github.com/hakehuang/zephyr_app_scm

 

I am working on the #4 for a unified junit report based on console log only.

 

Regards,

Hake


Re: some idea on providing test services to community

Nashif, Anas
 

Hake,

 

I looked at the code you referenced below. I am wondering why you do most of this instead of relying on the filter capabilities already available in sanitycheck. Right now you are able to run:

 

sanitycheck  --device-testing --device-serial /dev/ttyACM0 -p frdm_k64f

 

and get all tests (and samples) run on the device and produce a junit file with the test results without having to maintain your own whitelist/blacklist or any device specific tests. This also will work on tests that are not part of Zephyr. It is available for all boards and works nicely. The results can be then uploaded to testrail for specific PRs.

 

This can of course all be done from Jenkins, but most developers or testers will be able to do that on their own and directly.

 

To make sure wall are using the same environment, we can use a docker image available already publicly with the SDK pre-installed.

 

Anas

 

From: testing-wg@... [mailto:testing-wg@...] On Behalf Of Hake Huang
Sent: Monday, March 25, 2019 11:13 PM
To: testing-wg@...
Subject: [testing-wg] some idea on providing test services to community

 

Hi All,

 

Pre last night’s discussion, here is the briefing of my idea.

 

1.      We can announce that we accept pull request testing.(binary test can be later support)

a)       A pull request ID shall be provide.

b)       A configure file shall include in the pull request, which defines:

1.       platform to test, default all.

2.       Test suites to test, default all.

 

2.      once a pull request are created, a trigger script shall be created to pull the pull request to each test volunteer’s testing branch. This can be done via shippable.

 

3.      The selected volunteers can start testing the pull request by his own condition.

 

4.      Volunteers shall have a mechanism to upload their test report to testrails, with a given report name.

A comments will be added automatically to the pull request, once the report is uploaded.

 

I demo my internal system to do #3 automatically, which is based on Jenkins blueocean pipeline files.

https://jenkins.io/doc/book/blueocean/

and NXP board pipeline files are defined here

https://github.com/hakehuang/zephyr_app_scm

 

I am working on the #4 for a unified junit report based on console log only.

 

Regards,

Hake


Re: some idea on providing test services to community

Hake Huang
 

Hi Anas,

 

Thanks a lot for the reviewing.

 

I create our own method is just because our board farm limitation. The board farm run machine can’t support Linux host. And Yes, you are right using the sanity check is a good option for most community users. So my Jenkins solution is not a must(#3). What I propose is the whole process, not the way to do automation.

 

Besides, I want to learn how the filter list is thus created. And so I start to analyze them one by one. To me the list is just a tracking list for features, as in NXP we have not define a clear supporting strategy for all boards in general.

 

Btw, I am using the zephyr docker which is at https://github.com/zephyrproject-rtos/ci-dockerfiles.git this CI works fine for me.

 

Regards,

Hake

 

From: Nashif, Anas [mailto:anas.nashif@...]
Sent: Thursday, March 28, 2019 3:19 AM
To: Hake Huang <hake.huang@...>; testing-wg@...
Subject: RE: some idea on providing test services to community

 

Hake,

 

I looked at the code you referenced below. I am wondering why you do most of this instead of relying on the filter capabilities already available in sanitycheck. Right now you are able to run:

 

sanitycheck  --device-testing --device-serial /dev/ttyACM0 -p frdm_k64f

 

and get all tests (and samples) run on the device and produce a junit file with the test results without having to maintain your own whitelist/blacklist or any device specific tests. This also will work on tests that are not part of Zephyr. It is available for all boards and works nicely. The results can be then uploaded to testrail for specific PRs.

 

This can of course all be done from Jenkins, but most developers or testers will be able to do that on their own and directly.

 

To make sure wall are using the same environment, we can use a docker image available already publicly with the SDK pre-installed.

 

Anas

 

From: testing-wg@... [mailto:testing-wg@...] On Behalf Of Hake Huang
Sent: Monday, March 25, 2019 11:13 PM
To: testing-wg@...
Subject: [testing-wg] some idea on providing test services to community

 

Hi All,

 

Pre last nights discussion, here is the briefing of my idea.

 

1.     We can announce that we accept pull request testing.(binary test can be later support)

a)      A pull request ID shall be provide.

b)      A configure file shall include in the pull request, which defines:

1.      platform to test, default all.

2.      Test suites to test, default all.

 

2.     once a pull request are created, a trigger script shall be created to pull the pull request to each test volunteers testing branch. This can be done via shippable.

 

3.     The selected volunteers can start testing the pull request by his own condition.

 

4.     Volunteers shall have a mechanism to upload their test report to testrails, with a given report name.

A comments will be added automatically to the pull request, once the report is uploaded.

 

I demo my internal system to do #3 automatically, which is based on Jenkins blueocean pipeline files.

https://jenkins.io/doc/book/blueocean/

and NXP board pipeline files are defined here

https://github.com/hakehuang/zephyr_app_scm

 

I am working on the #4 for a unified junit report based on console log only.

 

Regards,

Hake


Zephyr: Testing WG weekly call - Mon, 04/01/2019 8:00am-9:00am #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder:
Zephyr: Testing WG weekly call

When:
Monday, 1 April 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH

An RSVP is requested. Click here to RSVP


Zephyr: Testing WG weekly call - Mon, 04/08/2019 8:00am-9:00am #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder:
Zephyr: Testing WG weekly call

When:
Monday, 8 April 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH

An RSVP is requested. Click here to RSVP


Re: some idea on providing test services to community

Hake Huang
 

Hi Anas,

 

In yesterday’s meeting, you mentioned that the developer will provide justify on the whitelist selection. Do those all listed in the individual pull request? Is there a filter to retrieve them?

 

Regards,

Hake


Re: some idea on providing test services to community

Nashif, Anas
 

Hi,

No filter to find those, the whitelisting in the tests is happening for various reasons, sometimes just to limit level of execution in CI to a limited set of boards and architectures. It definitely needs to be optimized. Any whitelisting that happens now is required to have a justification the commit message and a comment in the yaml file.

 

Anas

 

From: Hake Huang [mailto:hake.huang@...]
Sent: Monday, April 8, 2019 11:12 PM
To: Nashif, Anas <anas.nashif@...>; testing-wg@...
Cc: Hake Huang <hake.huang@...>
Subject: RE: [testing-wg] some idea on providing test services to community

 

Hi Anas,

 

In yesterday’s meeting, you mentioned that the developer will provide justify on the whitelist selection. Do those all listed in the individual pull request? Is there a filter to retrieve them?

 

Regards,

Hake


Cancelled Event: Zephyr: Testing WG weekly call - Monday, 15 April 2019 #cal-cancelled

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Cancelled: Zephyr: Testing WG weekly call

This event has been cancelled.

When:
Monday, 15 April 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Cancelled Event: Zephyr: Testing WG weekly call - Monday, 22 April 2019 #cal-cancelled

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Cancelled: Zephyr: Testing WG weekly call

This event has been cancelled.

When:
Monday, 22 April 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Zephyr: Testing WG weekly call - Mon, 04/29/2019 8:00am-9:00am #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder:
Zephyr: Testing WG weekly call

When:
Monday, 29 April 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH

An RSVP is requested. Click here to RSVP


Cancelled Event: Zephyr: Testing WG weekly call - Monday, 13 May 2019 #cal-cancelled

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Cancelled: Zephyr: Testing WG weekly call

This event has been cancelled.

When:
Monday, 13 May 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Upcoming Event: Zephyr: Testing WG weekly call - Mon, 05/06/2019 8:00am-9:00am, Please RSVP #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder: Zephyr: Testing WG weekly call

When: Monday, 6 May 2019, 8:00am to 9:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/679527144

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description: Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Cancelled Event: Zephyr: Testing WG weekly call - Monday, 20 May 2019 #cal-cancelled

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Cancelled: Zephyr: Testing WG weekly call

This event has been cancelled.

When:
Monday, 20 May 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Upcoming Event: Zephyr: Testing WG weekly call - Mon, 06/03/2019 8:00am-9:00am, Please RSVP #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder: Zephyr: Testing WG weekly call

When: Monday, 3 June 2019, 8:00am to 9:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/679527144

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description: Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Cancelled Event: Zephyr: Testing WG weekly call - Monday, 10 June 2019 #cal-cancelled

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Cancelled: Zephyr: Testing WG weekly call

This event has been cancelled.

When:
Monday, 10 June 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Upcoming Event: Zephyr: Testing WG weekly call - Mon, 06/17/2019 8:00am-9:00am, Please RSVP #cal-reminder

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Reminder: Zephyr: Testing WG weekly call

When: Monday, 17 June 2019, 8:00am to 9:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/679527144

An RSVP is requested. Click here to RSVP

Organizer: testing-wg@...

Description: Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Cancelled Event: Zephyr: Testing WG weekly call - Monday, 24 June 2019 #cal-cancelled

testing-wg@lists.zephyrproject.org Calendar <testing-wg@...>
 

Cancelled: Zephyr: Testing WG weekly call

This event has been cancelled.

When:
Monday, 24 June 2019
8:00am to 9:00am
(GMT-07:00) America/Los Angeles

Where:
https://zoom.us/j/679527144

Organizer:
testing-wg@...

Description:
Zephyr Testing WG Meeting
Meeting Agenda: https://docs.google.com/document/d/1Qti_6mFPkctk9v2vnbz-IMe0ZZO2FJEpX72FaNnEfpE/edit?usp=sharing
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/679527144

Or iPhone one-tap :
    US: +16465588656,,679527144# or +16699006833,,679527144# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 679 527 144
    International numbers available: https://zoom.us/u/ed7Ng0QxH


Testing Zephyr PTP in Renode, TestRail integration

pzierhoffer@...
 

Hi

As I mentioned a few meetings back, we are doing gPTP tests in Renode - an open source simulation framework for multinode systems we made (www.renode.io).

We were asked by Intel to work on general PTP support a few months ago, but after we were done there were some changes to Zephyr that broke it. Then we were asked to prepare a testing suite using Renode.

Renode typically uses the Python-based Robot Framework (https://robotframework.org/) to describe tests.
The main idea there is to write tests in text, using so called "keywords". A keyword can compare two values, run something on a host system, or run other keywords or Python code. The way it is structured it makes it relatively easy for a non-programmer to read/write tests. Additionally, Robot gives very comprehensive reports.

Our PTP tests are based on two Microchip SAM E70 devices, connected via Ethernet, both running similar Zephyr code.

It is possible to test it manually, following our tutorial: https://renode.readthedocs.io/en/latest/tutorials/zephyr-ptp-testing.html

If you have Renode built, testing our pregenerated Zephyr binaries requires you to simply run "./test.sh tests/platforms/SAME70.robot" in the Renode directory. You can also provide your own binaries using command line switches.

Right now we have six test cases:

- Single Node Should Send A PDelay Request Packet (verifying if the protocol starts at all)
- Slave Should Call The Phase Dis Callback (happens when a slave selects a master node)
- Master Should Send Announce Packets With The Expected Parameters (here we analyze the content of packets)
- Master Should Send Sync And Fup Packets With The Expected Parameters (verifying proper timestamping)
- Master Should Send Syncs In Valid Intervals
- Slave Should Sync Its Clock To Master (starting from very desynchronised clocks, we check if the clock on the Ethernet controller is roughly synced after some time)

Adding more test cases is relatively simple and kind of depends on your imagination (everyone's welcome to contribute :) ).
We are still working on adding gPTP support to our Linux-based model. Then we'll be able to verify Zephyr against Linux clients as well.


As I said before, you can run these tests manually if you'd like, but our main approach is to run them online.
We have a cloud-based builder and test runner, for which I attach some screenshots. The main reason to have our own runner system is to be able to exploit Renode's capabilities and give users more information than they'd normally get.
For example, right now we provide save files you can load into your local Renode installation to observe the last state of the tested environment - and continue running and debugging it. The possibilities are endless and we will add new options over time.

I mentioned Robot's reports - I attach a sample report from a PTP test run on today's Zephyr. They are both in HTML and XML form.

The next step for us, as requested by Intel, is to add our results to TestRail, and I would like to ask for guidance on how to proceed.

I will be happy to answer all the questions you might have regarding the framework, Robot tests or this suite in particular.

Best regards

Piotr Zierhoffer

21 - 40 of 396