Topics

[RFC] SMP support for RISC-V privilege architecture


Katsuhiro Suzuki
 

Hello All,

I'm working on SMP support for RISC-V privilege architecture.
https://github.com/katsuster/zephyr/tree/riscv_virt_pc_smp

It seems worked well on QEMU RV32 virt pc (*) porting.
(RV64 not tested yet)

But I don't understand Zephyr's SMP mechanism perfectly and
don't know required conditions (Ex. regression tests) if I
want to change these area.

I'm welcome comments or review of my patch.

(*)QEMU RV32 virt pc:
https://github.com/zephyrproject-rtos/zephyr/pull/28749

Diffs of config from default:
CONFIG_RISCV_USE_SWITCH=y (added by me)
CONFIG_SMP=y
CONFIG_MP_NUM_CPUS=4

Command line:
qemu-system-riscv32 -nographic -machine virt -net none \
-chardev stdio,id=con,mux=on -serial chardev:con \
-mon chardev=con,mode=readline -kernel zephyr/zephyr.elf \
-cpu rv32 -smp cpus=4 -bios none

Best Regards,
Katsuhiro Suzuki


Katsuhiro Suzuki
 

Hello All,

I'm working on SMP support for RISC-V privilege architecture.
https://github.com/katsuster/zephyr/tree/riscv_virt_pc_smp

It seems worked well on QEMU RV32 virt pc (*) porting.
(RV64 not tested yet)

But I don't understand Zephyr's SMP mechanism perfectly and
don't know required conditions (Ex. regression tests) if I
want to change these area.

I'm welcome comments or review of my patch.

(*)QEMU RV32 virt pc:
https://github.com/zephyrproject-rtos/zephyr/pull/28749

Diffs of config from default:
CONFIG_RISCV_USE_SWITCH=y (added by me)
CONFIG_SMP=y
CONFIG_MP_NUM_CPUS=4

Command line:
qemu-system-riscv32 -nographic -machine virt -net none \
-chardev stdio,id=con,mux=on -serial chardev:con \
-mon chardev=con,mode=readline -kernel zephyr/zephyr.elf \
-cpu rv32 -smp cpus=4 -bios none

Best Regards,
Katsuhiro Suzuki


Kumar Gala
 

On Oct 6, 2020, at 4:04 AM, Katsuhiro Suzuki <katsuhiro@...> wrote:

Hello All,

I'm working on SMP support for RISC-V privilege architecture.
https://github.com/katsuster/zephyr/tree/riscv_virt_pc_smp

It seems worked well on QEMU RV32 virt pc (*) porting.
(RV64 not tested yet)

But I don't understand Zephyr's SMP mechanism perfectly and
don't know required conditions (Ex. regression tests) if I
want to change these area.

I'm welcome comments or review of my patch.

(*)QEMU RV32 virt pc:
https://github.com/zephyrproject-rtos/zephyr/pull/28749

Diffs of config from default:
CONFIG_RISCV_USE_SWITCH=y (added by me)
CONFIG_SMP=y
CONFIG_MP_NUM_CPUS=4

Command line:
qemu-system-riscv32 -nographic -machine virt -net none \
-chardev stdio,id=con,mux=on -serial chardev:con \
-mon chardev=con,mode=readline -kernel zephyr/zephyr.elf \
-cpu rv32 -smp cpus=4 -bios none

Best Regards,
Katsuhiro Suzuki
Would probably be good to add a second SMP config for qemu_riscv32 for SMP=y. You can look at the boards/x86/qemu_x86 to see examples of having different configs.

As for testing, I’m guessing the main test to make sure runs and passes is tests/kernel/smp.

Andrew, can probably mention other tests or aspects to be aware of.

- k


Katsuhiro Suzuki
 

Hello Kumar,

Sorry for multi posting. Zephyr ML has very long delay to deliver my mail...??

Anyway, thanks a lot for your comment.

On 2020/10/06 23:57, Kumar Gala wrote:

On Oct 6, 2020, at 4:04 AM, Katsuhiro Suzuki <katsuhiro@...> wrote:

Hello All,

I'm working on SMP support for RISC-V privilege architecture.
https://github.com/katsuster/zephyr/tree/riscv_virt_pc_smp

It seems worked well on QEMU RV32 virt pc (*) porting.
(RV64 not tested yet)

But I don't understand Zephyr's SMP mechanism perfectly and
don't know required conditions (Ex. regression tests) if I
want to change these area.

I'm welcome comments or review of my patch.

(*)QEMU RV32 virt pc:
https://github.com/zephyrproject-rtos/zephyr/pull/28749

Diffs of config from default:
CONFIG_RISCV_USE_SWITCH=y (added by me)
CONFIG_SMP=y
CONFIG_MP_NUM_CPUS=4

Command line:
qemu-system-riscv32 -nographic -machine virt -net none \
-chardev stdio,id=con,mux=on -serial chardev:con \
-mon chardev=con,mode=readline -kernel zephyr/zephyr.elf \
-cpu rv32 -smp cpus=4 -bios none

Best Regards,
Katsuhiro Suzuki
Would probably be good to add a second SMP config for qemu_riscv32 for SMP=y. You can look at the boards/x86/qemu_x86 to see examples of having different configs.
I'll add the config.

BTW, QEMU can choose SMP mode or non-SMP mode by command option.
qemu_x86 Zephyr users never want to use non-SMP mode?

I remember a help of CONFIG_USE_SWTCH said that:
"In uniprocess situations where the
architecture provides both, _arch_switch incurs more somewhat
overhead and may be slower.".

I think RISC-V may cover high-end from low-end, so I try to keep
old style context switch mechanism. On the other hand, it seems
x86_64 port don't implement old style.
Should we keep old one or it depends on each architectures?


As for testing, I’m guessing the main test to make sure runs and passes is tests/kernel/smp.
Andrew, can probably mention other tests or aspects to be aware of.
Thanks! I'll try it and report results.


- k
Best Regards,
Katsuhiro Suzuki


Boie, Andrew P
 

(*)QEMU RV32 virt pc:
https://github.com/zephyrproject-rtos/zephyr/pull/28749
That PR looks fine. But you definitely will have some work to do to get SMP working on Risc-V, starting with changing the arch to select CONFIG_SWITCH, and then implement arch_start_cpu(), arch_curr_cpu(), etc.

But I don't understand Zephyr's SMP mechanism perfectly and don't know
required conditions (Ex. regression tests) if I want to change these area.
There are an assortment of tests with 'smp' in their tags, defined by testcase.yaml.

So you can do:

sanitycheck -p <your target name> -t smp

To quickly run these.

HTH
Andrew

-----Original Message-----
From: devel@... <devel@...> On
Behalf Of Katsuhiro Suzuki
Sent: Tuesday, October 6, 2020 1:54 AM
To: zephyr-devel <zephyr-devel@...>
Subject: [Zephyr-devel] [RFC] SMP support for RISC-V privilege architecture

Hello All,

I'm working on SMP support for RISC-V privilege architecture.
https://github.com/katsuster/zephyr/tree/riscv_virt_pc_smp

It seems worked well on QEMU RV32 virt pc (*) porting.
(RV64 not tested yet)

But I don't understand Zephyr's SMP mechanism perfectly and don't know
required conditions (Ex. regression tests) if I want to change these area.

I'm welcome comments or review of my patch.



Command line:
qemu-system-riscv32 -nographic -machine virt -net none \
-chardev stdio,id=con,mux=on -serial chardev:con \
-mon chardev=con,mode=readline -kernel zephyr/zephyr.elf \
-cpu rv32 -smp cpus=4 -bios none

Best Regards,
Katsuhiro Suzuki




Katsuhiro Suzuki
 

Hello Andrew, Kumar,

Thank you for pointing about regression tests.


On 2020/10/07 9:34, Boie, Andrew P wrote:
(*)QEMU RV32 virt pc:
https://github.com/zephyrproject-rtos/zephyr/pull/28749
That PR looks fine. But you definitely will have some work to do to get SMP working on Risc-V, starting with changing the arch to select CONFIG_SWITCH, and then implement arch_start_cpu(), arch_curr_cpu(), etc.
Yes, right.
For sanitycheck, I added another patch to change configs of qemu_rv32_virt
board. It selected CONFIG_USE_SWITCH and related context switch configs.
And set "y" to CONFIG_SMP in board_defconfig.

These SMP patch and additional configs patch are here:
https://github.com/katsuster/zephyr/tree/riscv_virt_pc_smp_v2

And I created these patches can be added on PR's patch (not conflict):
https://github.com/zephyrproject-rtos/zephyr/pull/28749


But I don't understand Zephyr's SMP mechanism perfectly and don't know
required conditions (Ex. regression tests) if I want to change these area.
There are an assortment of tests with 'smp' in their tags, defined by testcase.yaml.
So you can do:
sanitycheck -p <your target name> -t smp
To quickly run these.
I have tested "sanitycheck -p qemu_rv32_virt -t smp" and fixed some bugs.
Finally, my implementation passed all 3 SMP test sets.

I'll prepare pull request for these patches. Thank you very much!


HTH
Andrew
Best Regards,
Katsuhiro Suzuki


-----Original Message-----
From: devel@... <devel@...> On
Behalf Of Katsuhiro Suzuki
Sent: Tuesday, October 6, 2020 1:54 AM
To: zephyr-devel <zephyr-devel@...>
Subject: [Zephyr-devel] [RFC] SMP support for RISC-V privilege architecture

Hello All,

I'm working on SMP support for RISC-V privilege architecture.
https://github.com/katsuster/zephyr/tree/riscv_virt_pc_smp

It seems worked well on QEMU RV32 virt pc (*) porting.
(RV64 not tested yet)

But I don't understand Zephyr's SMP mechanism perfectly and don't know
required conditions (Ex. regression tests) if I want to change these area.

I'm welcome comments or review of my patch.



Command line:
qemu-system-riscv32 -nographic -machine virt -net none \
-chardev stdio,id=con,mux=on -serial chardev:con \
-mon chardev=con,mode=readline -kernel zephyr/zephyr.elf \
-cpu rv32 -smp cpus=4 -bios none

Best Regards,
Katsuhiro Suzuki