msys mingw64 trouble

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Sven
Posts: 3830
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

msys mingw64 trouble

Post by Sven » Mon Aug 05, 2019 10:15 pm

It is annoying that Jumbo needs some CygWin DLLs at runtime. Therefore I am trying to switch to MinGW64 under MSYS2 64bit, expecting a way to build a standalone binary that does not require additional DLLs apart from Windows system DLLs.

First step was to use MSYS gcc. Build worked fine, no problem at runtime. But now the exe depended on three MSYS DLLs (msys-2.0.dll, msys-gcc_s-seh-1.dll, msys-stdc++-6.dll) instead of three CygWin DLLs.

I tried to get rid of these DLLs by linking statically, so I added "-static -static-libstdc++" to the compiler options. Build worked fine but the exe crashed immediately, no idea why, no stacktrace in gdb, maybe stackframes corrupted?

Also adding "-static -static-libstdc++" did not help to get rid of the msys-2.0.dll dependency. So I reverted the compiler option changes and went to the next step: actually using MINGW64. I did a "pacman -S mingw-w64-x86_64-gcc" and then restarted the shell window via "MSYS2 MINGW 64bit" start menu item.

Result: build worked, unwanted DLL dependencies were gone, but the exe corrupted the board.

Has someone else had similar trouble, and how did you solve it?
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)

Daniel Shawul
Posts: 3762
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: msys mingw64 trouble

Post by Daniel Shawul » Mon Aug 05, 2019 10:36 pm

I cross compile from linux for windows using mingw32-g++. You would need to build with the "-static" option.
And that to your makefile and you never have to worry about compiling for windows.

jdart
Posts: 3835
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: msys mingw64 trouble

Post by jdart » Tue Aug 06, 2019 1:27 pm

MSVC Community Edition is free: https://visualstudio.microsoft.com/downloads/

Dann Corbit
Posts: 10186
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: msys mingw64 trouble

Post by Dann Corbit » Tue Aug 06, 2019 4:59 pm

I use msys2 instead of msys.
I sometimes have problems with clang builds, but never with gcc.
Is your project open source? If so I might try some experiments building it after I get back from vacation
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Sven
Posts: 3830
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

Re: msys mingw64 trouble

Post by Sven » Tue Aug 06, 2019 7:56 pm

jdart wrote:
Tue Aug 06, 2019 1:27 pm
MSVC Community Edition is free: https://visualstudio.microsoft.com/downloads/
Thanks Jon. I'm using it already for interactive development and debugging but both gcc and mingw produce way faster code for Jumbo than MSVC.
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)

Sven
Posts: 3830
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

Re: msys mingw64 trouble

Post by Sven » Tue Aug 06, 2019 8:00 pm

Dann Corbit wrote:
Tue Aug 06, 2019 4:59 pm
I use msys2 instead of msys.
I sometimes have problems with clang builds, but never with gcc.
Is your project open source? If so I might try some experiments building it after I get back from vacation
I use msys2, too, my whole description above was referring to msys2, I only missed the "2" in the subject line by accident ...

And no, Jumbo is not open source. But I could lend you a private copy if you like :wink:
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)

Dann Corbit
Posts: 10186
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: msys mingw64 trouble

Post by Dann Corbit » Wed Aug 07, 2019 6:36 am

Sven wrote:
Tue Aug 06, 2019 8:00 pm
Dann Corbit wrote:
Tue Aug 06, 2019 4:59 pm
I use msys2 instead of msys.
I sometimes have problems with clang builds, but never with gcc.
Is your project open source? If so I might try some experiments building it after I get back from vacation
I use msys2, too, my whole description above was referring to msys2, I only missed the "2" in the subject line by accident ...

And no, Jumbo is not open source. But I could lend you a private copy if you like :wink:
I will be in the Idaho panhandle until Monday,
I guess by then you will already have ironed out your problem.
If there is still some issue with Msys2, I would be interested to take a look
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

RubiChess
Posts: 118
Joined: Fri Mar 30, 2018 5:20 am

Re: msys mingw64 trouble

Post by RubiChess » Wed Aug 07, 2019 12:15 pm

Sven wrote:
Mon Aug 05, 2019 10:15 pm
It is annoying that Jumbo needs some CygWin DLLs at runtime. Therefore I am trying to switch to MinGW64 under MSYS2 64bit, expecting a way to build a standalone binary that does not require additional DLLs apart from Windows system DLLs.

First step was to use MSYS gcc. Build worked fine, no problem at runtime. But now the exe depended on three MSYS DLLs (msys-2.0.dll, msys-gcc_s-seh-1.dll, msys-stdc++-6.dll) instead of three CygWin DLLs.

I tried to get rid of these DLLs by linking statically, so I added "-static -static-libstdc++" to the compiler options. Build worked fine but the exe crashed immediately, no idea why, no stacktrace in gdb, maybe stackframes corrupted?

Also adding "-static -static-libstdc++" did not help to get rid of the msys-2.0.dll dependency. So I reverted the compiler option changes and went to the next step: actually using MINGW64. I did a "pacman -S mingw-w64-x86_64-gcc" and then restarted the shell window via "MSYS2 MINGW 64bit" start menu item.

Result: build worked, unwanted DLL dependencies were gone, but the exe corrupted the board.

Has someone else had similar trouble, and how did you solve it?
Do you use link-time-optimization / -flto switch?
What you describe sounds like the problem I described in viewtopic.php?f=7&t=71429

Maybe you can also track it down to getline() call.
- Andreas

Sven
Posts: 3830
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

Re: msys mingw64 trouble

Post by Sven » Wed Aug 07, 2019 2:30 pm

RubiChess wrote:
Wed Aug 07, 2019 12:15 pm
Sven wrote:
Mon Aug 05, 2019 10:15 pm
It is annoying that Jumbo needs some CygWin DLLs at runtime. Therefore I am trying to switch to MinGW64 under MSYS2 64bit, expecting a way to build a standalone binary that does not require additional DLLs apart from Windows system DLLs.

First step was to use MSYS gcc. Build worked fine, no problem at runtime. But now the exe depended on three MSYS DLLs (msys-2.0.dll, msys-gcc_s-seh-1.dll, msys-stdc++-6.dll) instead of three CygWin DLLs.

I tried to get rid of these DLLs by linking statically, so I added "-static -static-libstdc++" to the compiler options. Build worked fine but the exe crashed immediately, no idea why, no stacktrace in gdb, maybe stackframes corrupted?

Also adding "-static -static-libstdc++" did not help to get rid of the msys-2.0.dll dependency. So I reverted the compiler option changes and went to the next step: actually using MINGW64. I did a "pacman -S mingw-w64-x86_64-gcc" and then restarted the shell window via "MSYS2 MINGW 64bit" start menu item.

Result: build worked, unwanted DLL dependencies were gone, but the exe corrupted the board.

Has someone else had similar trouble, and how did you solve it?
Do you use link-time-optimization / -flto switch?
What you describe sounds like the problem I described in viewtopic.php?f=7&t=71429

Maybe you can also track it down to getline() call.
- Andreas
Thanks for your remarks.

I use -flto for the release version but not for debug builds. When using MSYS2 gcc/g++ (not MinGW) and linking statically, I get a working engine without -flto (otherwise it crashes). I do not use getline() but fgets(). I was unable to get rid of the msys-2.0.dll reference, though, so the main goal would be missed that way.

When using MinGW (in order to remove the unwanted DLL deps) I always get a binary that corrupts the board, even without -flto, so even in the debug build. It will be quite hard to track this down but of course I will try (tomorrow).
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)

Sven
Posts: 3830
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

Solved! (Re: msys mingw64 trouble)

Post by Sven » Thu Aug 15, 2019 11:46 am

Sven wrote:
Wed Aug 07, 2019 2:30 pm
RubiChess wrote:
Wed Aug 07, 2019 12:15 pm
Sven wrote:
Mon Aug 05, 2019 10:15 pm
It is annoying that Jumbo needs some CygWin DLLs at runtime. Therefore I am trying to switch to MinGW64 under MSYS2 64bit, expecting a way to build a standalone binary that does not require additional DLLs apart from Windows system DLLs.

First step was to use MSYS gcc. Build worked fine, no problem at runtime. But now the exe depended on three MSYS DLLs (msys-2.0.dll, msys-gcc_s-seh-1.dll, msys-stdc++-6.dll) instead of three CygWin DLLs.

I tried to get rid of these DLLs by linking statically, so I added "-static -static-libstdc++" to the compiler options. Build worked fine but the exe crashed immediately, no idea why, no stacktrace in gdb, maybe stackframes corrupted?

Also adding "-static -static-libstdc++" did not help to get rid of the msys-2.0.dll dependency. So I reverted the compiler option changes and went to the next step: actually using MINGW64. I did a "pacman -S mingw-w64-x86_64-gcc" and then restarted the shell window via "MSYS2 MINGW 64bit" start menu item.

Result: build worked, unwanted DLL dependencies were gone, but the exe corrupted the board.

Has someone else had similar trouble, and how did you solve it?
Do you use link-time-optimization / -flto switch?
What you describe sounds like the problem I described in viewtopic.php?f=7&t=71429

Maybe you can also track it down to getline() call.
- Andreas
Thanks for your remarks.

I use -flto for the release version but not for debug builds. When using MSYS2 gcc/g++ (not MinGW) and linking statically, I get a working engine without -flto (otherwise it crashes). I do not use getline() but fgets(). I was unable to get rid of the msys-2.0.dll reference, though, so the main goal would be missed that way.

When using MinGW (in order to remove the unwanted DLL deps) I always get a binary that corrupts the board, even without -flto, so even in the debug build. It will be quite hard to track this down but of course I will try (tomorrow).
I finally managed to identify the root cause for MinGW not working for Jumbo. As already noticed, the "-flto" switch had to be disabled for both MinGW and MSYS2 gcc when linking statically (which I now do and which finally helped me to get rid of additional DLLs with MinGW :-) ).

As to the "board corrupted" problem, I had to replace the GCC code of my popcount and bitScanForward routines:

Code: Select all

#ifdef __LP64__
     return __builtin_popcountll(x);
 #else
     return __builtin_popcount(x);
#endif /* __LP64__ */
...
#ifdef __LP64__
    result = __builtin_ctzll(x);
#else
    if (LOW32_OF_64(x) != 0) {
        result = __builtin_ctz(LOW32_OF_64(x));
    } else {
        result = __builtin_ctz(HIGH32_OF_64(x)) + 32;
    }
#endif /* __LP64__ */
by this version:

Code: Select all

#if defined(__LP64__) || defined(__MINGW64__)
     return __builtin_popcountll(x);
 #else
     return __builtin_popcount(x);
#endif /* defined(__LP64__) || defined(__MINGW64__) */
...
#if defined(__LP64__) || defined(__MINGW64__)
    result = __builtin_ctzll(x);
#else
    if (LOW32_OF_64(x) != 0) {
        result = __builtin_ctz(LOW32_OF_64(x));
    } else {
        result = __builtin_ctz(HIGH32_OF_64(x)) + 32;
    }
#endif /* defined(__LP64__) || defined(__MINGW64__) */
and everything started to work perfectly! The reason is that MinGW somehow does not (or at least, not reliably) set the __LP64__ preprocessor macro, in contrast to standard GCC. I found that by adding one more ASSERT statement that did a board validation already for the initial position and showed me that the number of black pieces in the initial position were "zero" but was ok for white pieces, telling me that something had to be wrong with popcount and 64 bit :-)

I will now start with some testing and hope to make a new MinGW-based release during the next days.
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)

Post Reply