Current state of compiler differences STOCKFISH

[dangi12012]

So I took a few hours to compare how well different compilers do for Stockfish in terms of nps:
We are talking about the latest master branch in SF as of 06.12.2021 -> https://github.com/official-stockfish/Stockfish

Also the latest compilers were installed clang-12 and gcc-11 made the default by
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-12 100


Anyways here is the comparison:

Code: Select all

./stockfish bench 128 1 20 default depth

clang: 57488149 - 2562659nps (run in ubuntu)
gcc:   57488149 - 2451102nps (run in ubuntu)
wingw: 57488149 - 2494495nps (run in windows)
msvc:  57488149 - 1999935nps (run in windows)

To compile on MSVC the makfile cannot be used. I created a sln for that I can share for anyone interested. It also has the appropriate macros defined so that BMI and AVX2 are used.
Run to run variance was pretty low for example my second run in windows was 1999949nps. It seems that MSVC cannot find some optimisations that clang and gcc found.

So there we have it. Current Stockfish with current compilers and the difference between gcc and clang is low but clear. Clang comes out ahead far outside of error of margin (at least on ryzen). Oh yeah and disabling AVX2 made performance 10% of what it is with it enabled.

I could not get native clang to work on windows (link time errors) so does anyone have experience? Download and build this: https://github.com/llvm/llvm-project and then build like you would without the makefile in windows.

[Sopel]

How many benches did you run for each build? Did you do build or profile-build?

[Deberger]

Are you using profile-build?

Code: Select all

make profile-build ARCH=x86-64-bmi2

[Ras]

I could not get native clang to work on windows (link time errors) so does anyone have experience?

This one works (at least for my engine): https://github.com/mstorsjo/llvm-mingw

[hgm]

[Moderation] Warning to dangi: please refrain from personal attacks on other posters. If you think they posted material that violates the TalkChess starter, use the 'report' button ('!') on those postings to have moderation deal with them.

Note that 'trolling' is a subjective opinion, and not a charter violation to begin with, and that reporting a posting for invalid reasons won't lead to its deletion. Off-topic posting in general will only be acted against when it takes on a magnitude that distracts too much from the original discussion, in which case the discussion will be split to separate the topics.

[dangi12012]

I could not get native clang to work on windows (link time errors) so does anyone have experience?

This one works (at least for my engine): https://github.com/mstorsjo/llvm-mingw

Tried that. Didnt work. I can compile with clang. But i get infinite linker errors afterwards.
I will leave it at that. I suspect native windows clang will be just as fast as linux clang because the code is 99% identical.

So clang is best in beginning of 2022 for chessprogramming.

[Sopel]

I still don't see how you come to that conclusion. Without knowing the standard deviation of the performances, and the ways the binaries were produced, we cannot tell which compiler is better.

[Ras]

I can compile with clang. But i get infinite linker errors afterwards.

Here the relevant options I'm using for that Clang:
-target x86_64-pc-windows-gnu -O2 -flto -mthreads -mconsole -static -fuse-ld=lld --rtlib=compiler-rt -s -Wl,-s

32 bit Windows: -target x86_64-pc-windows-gnu
64 bit ARM Windows: -target aarch64-pc-windows-gnu
32 bit ARM Windows: -target armv7-pc-windows-gnu

The download zip I tried successfully: llvm-mingw-20210423-msvcrt-x86_64.zip
One caveat is that my engine is in C, not C++.

[OfekShochat]

regarding sopel's questions, you cant really draw any conclusions without running multiple benches and specifying the standard deviation/variance...

[dangi12012]

regarding sopel's questions, you cant really draw any conclusions without running multiple benches and specifying the standard deviation/variance...

It was remarkably consistent. Tests ran 1min and scores deviated by 1-2% max.