GCC 4.8 made me chose Clang

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Sven
Posts: 3783
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

Re: GCC 4.8 made me chose Clang

Post by Sven » Sun Oct 27, 2013 11:05 am

Maybe a signedness issue in the expression where you calculate the maximum gain of the move? What is the type of vEP, vOP, Material[...].eg? Just guessing of course ...

Sven

User avatar
lucasart
Posts: 3035
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: GCC 4.8 made me chose Clang

Post by lucasart » Sun Oct 27, 2013 11:17 am

trojanfoe wrote:
lucasart wrote: * clang does not accept GNU extensions (std=c++11 but not std=gnu++11). That means I need to throw away variable length arrays. They were so nice for triangular PV array on the stack, without them, I will now have to rely on an ugle global structure.
That isn't true; I've just compiled this, without complaint, using Apple clang 5.0 (I used -std=c++11):

Code: Select all

#include <cstdio>
#include <cstring>

FILE * concat_fopen &#40;char *s1, char *s2, char *mode&#41;
&#123;
   char str&#91;strlen &#40;s1&#41; + strlen &#40;s2&#41; + 1&#93;;
   strcpy &#40;str, s1&#41;;
   strcat &#40;str, s2&#41;;
   return fopen &#40;str, mode&#41;;
&#125;
Perhaps the Apple version has VLA added, but not the standard one. My exact version of clang is: "Debian clang version 3.2-7ubuntu1 (tags/RELEASE_32/final) (based on LLVM 3.2)"

Code: Select all

$ clang++ ./src/*.cc -o ./dc_clang  -std=c++11 -Wall -Wshadow -DNDEBUG -O3 -msse4.2 -fno-rtti -s

./src/search.cc&#58;165&#58;25&#58; error&#58; variable length array of non-POD element type 'move&#58;&#58;move_t'
        move&#58;&#58;move_t subtree_pv&#91;MAX_PLY - ss->ply&#93;;
                               ^
./src/search.cc&#58;284&#58;25&#58; error&#58; variable length array of non-POD element type 'move&#58;&#58;move_t'
        move&#58;&#58;move_t subtree_pv&#91;MAX_PLY - ss->ply&#93;;
                               ^
2 errors generated.
lucasart wrote:
* clang does not accept flto (link time optimization). This was a great source of performance in GCC 4.7.3 when compiling my code, as I hate to have to inline everything: I prefer to rely on flto to do it for me.
Apple clang 5.0 does support LTO and as that is based on LLVM 3.3 SVN, I'm surprised it's not in the non-Apple version.
The problem seems to be trickier than that. I don't know what is wrong exactly and how to fix it, but it's either Debian or Ubuntu's fault, not Clang's. The whole thing is not packaged correctly. Here is a perfect description of the problem I am facing with flto:
http://stackoverflow.com/questions/1629 ... ng-o4-with
lucasart wrote:
* the Clang compile is 10% slower than GCC 4.7 !
No idea about that; I've never compared.
Perhaps I could get back some of the performance if I can get flto to work.
You mention the use of variable length array, which is a feature I would never consider using if I wanted to create portable code (std::vector is the portable variable-length array).
The performance of VLA and std::vector is not even remotely comparable. When you "allocate" a VLA on the stack, all you do is shift the base pointer (a 1 cycle operation). VLA is basically an elegant way of representing the triangular PV in a nice recursive manner. The only other solution I can think of, which preserves the performance, is to use a global array, which is not as pretty.
You might be better off getting it to work with gcc 4.8 and thus having a better code base, rather than assuming these faults are with the compiler rather than your code.
Yes, you're probably right. I will get to the bottom of this, and everytime I commit something I will compile with both Clang and GCC to make sure I'm not starting to introduce code that is non portable and interpreted by both compilers differently.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

syzygy
Posts: 4425
Joined: Tue Feb 28, 2012 10:56 pm

Re: GCC 4.8 made me chose Clang

Post by syzygy » Sun Oct 27, 2013 11:22 am

lucasart wrote:* GCC 4.8.1, with of without flto: both compiles are wrong. Neither have the correct node count, and both versions don't even have the same node counts (ie. they are both broken in different ways).
Before you blame a compiler, blame your code.

User avatar
trojanfoe
Posts: 65
Joined: Sun Jul 31, 2011 9:57 am
Location: Waterlooville, Hampshire, UK

Re: GCC 4.8 made me chose Clang

Post by trojanfoe » Sun Oct 27, 2013 11:22 am

lucasart wrote: Perhaps the Apple version has VLA added, but not the standard one. My exact version of clang is: "Debian clang version 3.2-7ubuntu1 (tags/RELEASE_32/final) (based on LLVM 3.2)"
Perhaps try the packages provided by the LLVM project themselves? That will be llvm-3.4 (at time of writing).

User avatar
rvida
Posts: 481
Joined: Thu Apr 16, 2009 10:00 am
Location: Slovakia, EU

Re: GCC 4.8 made me chose Clang

Post by rvida » Sun Oct 27, 2013 11:39 am

lucasart wrote:I made a very stupid mistake today: updated my Ubuntu from 13.04 to 13.10.
lucasart wrote: * clang does not accept flto (link time optimization).
It does. You just need to enable the gold linker (ubutu 13.10 defaults to vanilla gnu ld):

Code: Select all

ln -s -f /usr/bin/ld.gold /usr/bin/ld
... and correct some screwed up paths in the llvm-3.4 package:

Code: Select all

ln -s /usr/lib/llvm-3.4/lib/libLTO.so /usr/lib/libLTO.so
ln -s /usr/lib/llvm-3.4/lib/LLVMgold.so /usr/lib/LLVMgold.so

User avatar
lucasart
Posts: 3035
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: GCC 4.8 made me chose Clang

Post by lucasart » Sun Oct 27, 2013 12:08 pm

Sven Schüle wrote:Maybe a signedness issue in the expression where you calculate the maximum gain of the move? What is the type of vEP, vOP, Material[...].eg? Just guessing of course ...

Sven
I found the bug. It was all my fault indeed! It's basically an access outside the bounds of an array...

The line that triggers it is:

Code: Select all

+ Material&#91;B.get_piece_on&#40;ss->m.tsq&#40;))&#93;.eg
What happens is that the array 'Material' is defined and initialized for values 0=PAWN up to 5=KING. But in the case '6=NO_PIECE', it turns out that GCC 4.6, 4.7, and Clang have inserted BY CHANCE some extra padding with zero value just after (which is what we want: Material[NO_PIECE] = {0,0}). But GCC 4.8 did not insert that padding (or maybe it did and didn't put a zero value there).
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

User avatar
lucasart
Posts: 3035
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: GCC 4.8 made me chose Clang

Post by lucasart » Sun Oct 27, 2013 12:16 pm

So problem solved, but still: the GCC 4.8.1 compile is 5% slower than the 4.7.3 compile. This is a really big performance regression: same code, same compile flags. 5% slower :shock:
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

Michel
Posts: 2008
Joined: Sun Sep 28, 2008 11:50 pm

Re: GCC 4.8 made me chose Clang

Post by Michel » Sun Oct 27, 2013 12:29 pm

Probably it will be 5% faster on other code.... 5%=5elo. That is just noise.

User avatar
lucasart
Posts: 3035
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: GCC 4.8 made me chose Clang

Post by lucasart » Sun Oct 27, 2013 12:31 pm

This is quite an amazing demonstration of the power of unit testing and git:
  • without unit testing, I would not even know that the GCC 4.8 compile was broken, so I would just live with bugs endlessly, so long as these bugs remained silent enough not to be seen. (I'm talking about the node count unit test, like stockfish bench).
  • thanks to git, finding the offending commit was a O(log N) dichotomic operation.
Without all that, I would have to resort to the worst solution of all, which is to dump the search tree and compare. I hope I never have to do that... ever!
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

Rein Halbersma
Posts: 685
Joined: Tue May 22, 2007 9:13 am

Re: GCC 4.8 made me chose Clang

Post by Rein Halbersma » Sun Oct 27, 2013 12:47 pm

lucasart wrote:This is quite an amazing demonstration of the power of unit testing and git:
  • without unit testing, I would not even know that the GCC 4.8 compile was broken, so I would just live with bugs endlessly, so long as these bugs remained silent enough not to be seen. (I'm talking about the node count unit test, like stockfish bench).
  • thanks to git, finding the offending commit was a O(log N) dichotomic operation.
Without all that, I would have to resort to the worst solution of all, which is to dump the search tree and compare. I hope I never have to do that... ever!
Couldn't agree with you more on the value of unit testing. Note that your tests are technically known as "integration tests". A unit test suite would test all the various functions (move gen, eval) in isolation. The benchmark node count is a much higher level kind of test.

BTW, I got an error with Clang SVN on your code even with std=gnu++11 because Clang will only allow VLA on POD (Plain Old Data) types and your type move_t is not a POD. The rules for POD classes changed in C++11: they need to be both trivial (essentially no user-defined overrides of compiler generated default, copy and move constructors, assignment operators and destructors) and standard-layout (no virtuals, only one access level). To fix it, you need to change the default constructor to

Code: Select all

move_t&#40;) = default;
Anything else (zero initialization, or even providing move_t() {} yourself, makes move_t a non-trivial class). Fortunately, you are already in the good habit of initalization variables at the point of first use, rather than at the top of its scope (Stockfish authors, I am talking to you!) so it shouldn't change the meaning of your code.

BTW, it is good practice to run your code on at least 2 current compilers (latest stable release of g++ and Clang). On Linux, you could even add Intel 14.0 to the mix which is both free and since June almost fully C++11 compliant.

Post Reply