On 21 Jul 2022, at 17:36, Halder, Ayan Kumar <ayan.kumar.halder@...> wrote:

Hi all,

I am trying to run aarch32 armv8r binary on Xen as domU guest and I am getting a data abort.

I have changed the memory address in dts node as follows (based on my fvp address) :-
- dram0: memory@0 {
+ dram0: memory@30000000 {
compatible = "mmio-dram";
- reg = <0x0 DT_SIZE_M(128)>;
+ reg = <0x30000000 DT_SIZE_M(128)>;
};

I need some pointers to debug the data abort.

The disassembly of our interest is :-

static int pl011_init(const struct device *dev)
{
30001ec8: e92d41f0 push {r4, r5, r6, r7, r8, lr}
const struct pl011_config *config = dev->config;
struct pl011_data *data = dev->data;
30001ecc: e5906010 ldr r6, [r0, #16]
{
30001ed0: e1a04000 mov r4, r0
/*
* If working in SBSA mode, we assume that UART is already configured,
* or does not require configuration at all (if UART is emulated by
* virtualization software).
*/
if (!data->sbsa) {
30001ed4: e5d67004 ldrb r7, [r6, #4]
30001ed8: e3570000 cmp r7, #0
30001edc: 1a000025 bne 30001f78 <pl011_init+0xb0>
const struct pl011_config *config = dev->config;
30001ee0: e5902004 ldr r2, [r0, #4]
return (volatile struct pl011_regs *const)DEVICE_MMIO_GET(dev);
30001ee4: e5925000 ldr r5, [r2] <<<<<<<<<<<----------------------------- The value of r5 is 0 and r2 is 0x300005d14
uint64_t bauddiv = (((uint64_t)clk) << PL011_FBRD_WIDTH)
30001ee8: e5921004 ldr r1, [r2, #4]
get_uart(dev)->cr &= ~PL011_CR_UARTEN;
30001eec: e5953030 ldr r3, [r5, #48] ; 0x30 <<<<<<<<<<<---------------------------- Data abort occurs here as r5 is 0

I can see that “DEVICE_MMIO_MAP(dev, K_MEM_CACHE_NONE);” becomes a nop as DEVICE_MMIO_IS_IN_RAM is undefined.

I do not understand how it is supposed to find the device address. The dtb is loaded at 0x38000000. This does not fall in the range of r2’s value.
Please provide some pointers.

Hi Ayan,

Seems to me that the data abort is around the pl011 initialisation, the values for it comes from the DT you
used to build Zephyr, could you share it?

Are you using a PL011 using passthrough or are you using the emulated PL011 from Xen (SBSA)?

Cheers,
Luca

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

On 21/07/2022 18:01, Luca Fancellu wrote:

On 21 Jul 2022, at 17:36, Halder, Ayan Kumar <ayan.kumar.halder@...> wrote:

Hi all,

I am trying to run aarch32 armv8r binary on Xen as domU guest and I am getting a data abort.

I have changed the memory address in dts node as follows (based on my fvp address) :-
- dram0: memory@0 {
+ dram0: memory@30000000 {
compatible = "mmio-dram";
- reg = <0x0 DT_SIZE_M(128)>;
+ reg = <0x30000000 DT_SIZE_M(128)>;
};

I need some pointers to debug the data abort.

The disassembly of our interest is :-

static int pl011_init(const struct device *dev)
{
30001ec8: e92d41f0 push {r4, r5, r6, r7, r8, lr}
const struct pl011_config *config = dev->config;
struct pl011_data *data = dev->data;
30001ecc: e5906010 ldr r6, [r0, #16]
{
30001ed0: e1a04000 mov r4, r0
/*
* If working in SBSA mode, we assume that UART is already configured,
* or does not require configuration at all (if UART is emulated by
* virtualization software).
*/
if (!data->sbsa) {
30001ed4: e5d67004 ldrb r7, [r6, #4]
30001ed8: e3570000 cmp r7, #0
30001edc: 1a000025 bne 30001f78 <pl011_init+0xb0>
const struct pl011_config *config = dev->config;
30001ee0: e5902004 ldr r2, [r0, #4]
return (volatile struct pl011_regs *const)DEVICE_MMIO_GET(dev);
30001ee4: e5925000 ldr r5, [r2] <<<<<<<<<<<----------------------------- The value of r5 is 0 and r2 is 0x300005d14
uint64_t bauddiv = (((uint64_t)clk) << PL011_FBRD_WIDTH)
30001ee8: e5921004 ldr r1, [r2, #4]
get_uart(dev)->cr &= ~PL011_CR_UARTEN;
30001eec: e5953030 ldr r3, [r5, #48] ; 0x30 <<<<<<<<<<<---------------------------- Data abort occurs here as r5 is 0

I can see that “DEVICE_MMIO_MAP(dev, K_MEM_CACHE_NONE);” becomes a nop as DEVICE_MMIO_IS_IN_RAM is undefined.

I do not understand how it is supposed to find the device address. The dtb is loaded at 0x38000000. This does not fall in the range of r2’s value.
Please provide some pointers.

Hi Ayan,

Hi Luca,

Seems to me that the data abort is around the pl011 initialisation, the values for it comes from the DT you
used to build Zephyr, could you share it?

Yes, the error comes around pl011 initialisation. PFA the dt used to build Zephyr (zephyr.dts)

Also find attached r52_zephyr_pass.dts (zephyr passthrough dt) and r52_fvp.dts (xen dt).

Are you using a PL011 using passthrough or are you using the emulated PL011 from Xen (SBSA)?

I am using PL011 via passthrough.

uart2 ie uart@9c0b0000 is assigned to zephyr.

Can you let me know what am I missing ?

- Ayan

Cheers,
Luca

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

On 22 Jul 2022, at 19:44, Ayan Kumar Halder <ayankuma@...> wrote:


On 21/07/2022 18:01, Luca Fancellu wrote:

On 21 Jul 2022, at 17:36, Halder, Ayan Kumar <ayan.kumar.halder@...> wrote:

Hi all,

I am trying to run aarch32 armv8r binary on Xen as domU guest and I am getting a data abort.

I have changed the memory address in dts node as follows (based on my fvp address) :-
- dram0: memory@0 {
+ dram0: memory@30000000 {
compatible = "mmio-dram";
- reg = <0x0 DT_SIZE_M(128)>;
+ reg = <0x30000000 DT_SIZE_M(128)>;
};

I need some pointers to debug the data abort.

The disassembly of our interest is :-

static int pl011_init(const struct device *dev)
{
30001ec8: e92d41f0 push {r4, r5, r6, r7, r8, lr}
const struct pl011_config *config = dev->config;
struct pl011_data *data = dev->data;
30001ecc: e5906010 ldr r6, [r0, #16]
{
30001ed0: e1a04000 mov r4, r0
/*
* If working in SBSA mode, we assume that UART is already configured,
* or does not require configuration at all (if UART is emulated by
* virtualization software).
*/
if (!data->sbsa) {
30001ed4: e5d67004 ldrb r7, [r6, #4]
30001ed8: e3570000 cmp r7, #0
30001edc: 1a000025 bne 30001f78 <pl011_init+0xb0>
const struct pl011_config *config = dev->config;
30001ee0: e5902004 ldr r2, [r0, #4]
return (volatile struct pl011_regs *const)DEVICE_MMIO_GET(dev);
30001ee4: e5925000 ldr r5, [r2] <<<<<<<<<<<----------------------------- The value of r5 is 0 and r2 is 0x300005d14
uint64_t bauddiv = (((uint64_t)clk) << PL011_FBRD_WIDTH)
30001ee8: e5921004 ldr r1, [r2, #4]
get_uart(dev)->cr &= ~PL011_CR_UARTEN;
30001eec: e5953030 ldr r3, [r5, #48] ; 0x30 <<<<<<<<<<<---------------------------- Data abort occurs here as r5 is 0

I can see that “DEVICE_MMIO_MAP(dev, K_MEM_CACHE_NONE);” becomes a nop as DEVICE_MMIO_IS_IN_RAM is undefined.

I do not understand how it is supposed to find the device address. The dtb is loaded at 0x38000000. This does not fall in the range of r2’s value.
Please provide some pointers.

Hi Ayan,

Hi Luca,

Seems to me that the data abort is around the pl011 initialisation, the values for it comes from the DT you
used to build Zephyr, could you share it?

Yes, the error comes around pl011 initialisation. PFA the dt used to build Zephyr (zephyr.dts)

Also find attached r52_zephyr_pass.dts (zephyr passthrough dt) and r52_fvp.dts (xen dt).

Are you using a PL011 using passthrough or are you using the emulated PL011 from Xen (SBSA)?

I am using PL011 via passthrough.

uart2 ie uart@9c0b0000 is assigned to zephyr.

Can you let me know what am I missing ?

Hi Ayan,

Can you try to leave only one serial device in the DT? Like:

uart0: uart@9c0b0000 {
compatible = "arm,pl011";
reg = < 0x9c0b0000 0x1000 >;
status = "okay";
interrupts = < 0x0 0x7 0x2 0xa0 >;
interrupt-names = "irq_7";
label = "UART_0";
clocks = < &uartclk >;
current-speed = < 0x1c200 >;
};

And adjust:

zephyr,console = &uart0;
zephyr,shell-uart = &uart0;

Afterwards be sure you have these in the .config:

CONFIG_UART_PL011=y
CONFIG_UART_PL011_PORT0=y

Is it a DomU right? Not Dom0?

In this case please check if the interrupt controller addresses are pointing to the vGIC for R platform, I know
only the addresses of the Cortex-A port (the one upstreamed).

Cheers,
Luca

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.