Thanks Erik for the pointer. There was a race issue in initializing the pin before the driver was ready.

I changed from

SYS_INIT(board_init, POST_KERNEL, CONFIG_KERNEL_INIT_PRIORITY_DEFAULT);

to

SYS_INIT(board_init, APPLICATION, CONFIG_KERNEL_INIT_PRIORITY_DEFAULT);

and the pins worked as expected.