native_posix on Windows?


Amir Gonnen
 

Hi.
I noticed that WSL is not supported on native_posix board, due to its lack of 32-bit support.
Is there any way of running native_posix on windows, using some other toolchain that provides posix compatible runtime environment? (such as cygwin, mingw, or even Visual Studio?)
If not, is running native code on Window on the roadmap?

Thanks,
Amir


Carles Cufi
 

Hi Amir,

 

As you well point out, the native_posix board and the posix arch don’t work on WSL on Windows. I don’t think anyone has actually tried to use MSYS2 to build for native posix, I guess it should theoretically be possible since our MSYS2 build is still supported (albeit being deprecated by the native Windows one).

 

What I think needs to be done in order to run on Windows natively is to add a “win” architecture and then a native_win board for it, which doesn’t rely on POSIX or Linux system calls at all. To my knowledge this is not yet on the roadmap, but patches would certainly be welcome.

 

I’ve copied Alberto who is the original author of the native_posix board and the posix arch.

 

Carles

 

From: users@... <users@...> On Behalf Of Amir Gonnen
Sent: 29 July 2018 17:55
To: users@...
Subject: [Zephyr-users] native_posix on Windows?

 

Hi.
I noticed that WSL is not supported on native_posix board, due to its lack of 32-bit support.
Is there any way of running native_posix on windows, using some other toolchain that provides posix compatible runtime environment? (such as cygwin, mingw, or even Visual Studio?)
If not, is running native code on Window on the roadmap?

Thanks,
Amir


Andy Ross
 

In principle it certainly could work with cygwin or msys, have you tried?  All it wants is a POSIX C runtime and a command line toolchain (which I'm pretty sure it gets with just "gcc" off of PATH, right?).  If not, that would be a bug reasonably work fixing.


Andy


On 7/29/2018 8:54 AM, Amir Gonnen wrote:

Hi.
I noticed that WSL is not supported on native_posix board, due to its lack of 32-bit support.
Is there any way of running native_posix on windows, using some other toolchain that provides posix compatible runtime environment? (such as cygwin, mingw, or even Visual Studio?)
If not, is running native code on Window on the roadmap?

Thanks,
Amir


Leandro Pereira
 

On 07/30/2018 10:16 AM, Andy Ross wrote:
In principle it certainly could work with cygwin or msys, have you tried?  All it wants is a POSIX C runtime and a command line toolchain (which I'm pretty sure it gets with just "gcc" off of PATH, right?).  If not, that would be a bug reasonably work fixing.
I haven't tried building the posix_native with it yet, but I've heard good things about Midipix: https://midipix.org/


Leandro


Amir Gonnen
 

Giving it a shot!
Tried to configure native_posix with msys2 (32bit):

cmake -GNinja -DBOARD=native_posix -DZEPHYR_TOOLCHAIN_VARIANT=cross-compile -DCROSS_COMPILE=C:\msys32\usr\bin CMAKE_C_COMPILER=C:\msys32\usr\bin\gcc.exe  CMAKE_CXX_COMPILER=C:\msys32\usr\bin\gcc.exe ..\

check_host_is_ok fails:
-- Found PythonInterp: C:/Python37/python.exe (found suitable version "3.7", minimum required is "3.4")
c:/Projects/zephyr//scripts/check_host_is_ok.py:5: DeprecationWarning: 'U' mode is deprecated
  f = open('{}/Kconfig'.format(os.environ["ZEPHYR_BASE"]), 'U')
Traceback (most recent call last):
  File "c:/Projects/zephyr//scripts/check_host_is_ok.py", line 16, in 
    main()
  File "c:/Projects/zephyr//scripts/check_host_is_ok.py", line 13, in main
    crash_if_zephyr_was_cloned_with_wrong_line_endings()
  File "c:/Projects/zephyr//scripts/check_host_is_ok.py", line 10, in crash_if_zephyr_was_cloned_with_wrong_line_endings
    assert f.newlines == '\n', error_msg
AssertionError: Re-clone with autocrlf false. $ git config --global core.autocrlf false
CMake Error at C:/Projects/zephyr/cmake/app/boilerplate.cmake:121 (message):
  command failed with return code: 1
Call Stack (most recent call first):
  CMakeLists.txt:2 (include)

This happens even when re-cloning with core.autocrlf set to false

So I tried overriding it by commenting the assert on check_host_is_ok.
After doing that CMake passes, but ninja fails:

[6/67] Building C object zephyr/CMakeFiles/offsets.dir/arch/posix/core/offsets/offsets.c.obj
FAILED: zephyr/CMakeFiles/offsets.dir/arch/posix/core/offsets/offsets.c.obj
C:\msys32\usr\bin\gcc.exe -DBUILD_VERSION=v1.12.0-1099-g1b6e5f614 -DKERNEL -D_FORTIFY_SOURCE=2 -D_POSIX_C_SOURCE=200809 -D__ZEPHYR__=1 -I../../../kernel/include -I../../../arch/posix/include -I../../../arch/posix/soc/inf_clock -I../../../arch/posix/soc/inf_clock/include -I../../../arch/posix/soc/include -I../../../boards/posix/native_posix -I../../../include -I../../../include/drivers -Izephyr/include/generated -m32   -Os -g -Wall -Wformat -Wformat-security -Wno-format-zero-length -imacros C:/Projects/zephyr/samp
r/arch/posix/core/offsets/offsets.c.obj -MF zephyr\CMakeFiles\offsets.dir\arch\posix\core\offsets\offsets.c.obj.d -o zephyr/CMakeFiles/offsets.dir/arch/posix/core/offsets/offsets.c.obj   -c C:/Projects/zephyr/arch/posix/core/offsets/offsets.c
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s: Assembler messages:
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:23: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:23: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:29: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:29: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:35: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:35: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:41: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:41: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:47: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:47: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:53: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:53: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:59: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:59: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:65: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:65: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:71: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:71: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:77: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:77: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:83: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:83: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:89: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:89: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:95: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:95: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:101: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:101: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:107: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:107: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:113: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:113: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:119: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:119: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:125: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:125: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:131: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:131: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:137: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:137: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:143: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:143: Error: junk at end of line, first unrecognized character is `K'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:149: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:149: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:155: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:155: Error: junk at end of line, first unrecognized character is `_'
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:161: Warning: .type pseudo-op used outside of .def/.endef: ignored.
/c/Users/agonnen/AppData/Local/Temp/ccCteZX2.s:161: Error: junk at end of line, first unrecognized character is `_'
ninja: build stopped: subcommand failed.

Any idea?


Amir Gonnen
 

I did some progress but the issue is not resolved.
The issues I've seen were:
  • Windows newline format 
  • In Linux gcc creates ELF image, while on windows gcc creates COFF image. Zephyr explicitly assumes ELF format in several places (in gen_offset_header.py for example).
  • On COFF inline assembly looks a little different, for example the usage of .scl, .type etc. (inline asm in gcc.h)
  • COFF format automatically adds underscore to symbols. Some symbols are defined in ldscript and referred to in the sources. Either an underscore needs to be removed from the sources or added to the ldscript.
  • __in_section_unique passes __FILE__ to the section declaration which makes the assembly code invalid (it doesn't like a name built from quoted path)
After working around all the above issues I got to the point where build completes successfully and creates an executable image (Yay!).
Unfortunately it does not run, Windows claims this is not a valid executable. Something probably still needs to be changed on the ldscript to make it a valid PE image, I'm not sure what.

Unless anyone has some inputs, I'm going to stop this effort for now.
Just for the reference I'm attaching a patch with my workarounds.

Amir