Page 5 of 59

Re: 7-men Syzygy attempt

Posted: Thu Mar 15, 2018 4:59 am
by noobpwnftw
I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.

It does not seem to affect the correctness(confirm?).

Re: 7-men Syzygy attempt

Posted: Thu Mar 15, 2018 6:49 am
by Dann Corbit
noobpwnftw wrote:I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.
. Other
It does not seem to affect the correctness(confirm?).
Expand the array. Otherwise undefined behavior

Re: 7-men Syzygy attempt

Posted: Thu Mar 15, 2018 7:06 am
by syzygy
noobpwnftw wrote:I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.

It does not seem to affect the correctness(confirm?).
Yes this is part of the code that has to be adapted to deal with 16-bit values. The problem starts when (dtz-101)/2 exceeds 255 (actually a bit earlier). This should not affect the generation process and statistics are probably collected correctly.

Re: 7-men Syzygy attempt

Posted: Thu Mar 15, 2018 1:08 pm
by koedem
I realized only now, you generated a complete material balance in 30 seconds only? I have to imagine the ones with pawns take much more RAM because you need to cache the possible promotions so those might be more tricky. But nontheless I am surprised that the pawnless ones go so smooth.

Which hardware configuration did you use? Also, in case one needs a lot of RAM already for the pawnless ones, do you think you could upload some of them when they're finished? (maybe via some torrent or so)

Re: 7-men Syzygy attempt

Posted: Thu Mar 15, 2018 5:24 pm
by syzygy
No, the 33 seconds is only for the "resetting cursed losses" pass.

Re: 7-men Syzygy attempt

Posted: Thu Mar 15, 2018 7:12 pm
by duncan
noobpwnftw wrote: With your new 16-bit compression code available, I'm pretty confident that full set of 7-men Syzygy will be available soon.

I already have the necessary hardware(both steps in physical RAM without --disk) and will provide hosting for all the resulting files.
great achievement if you do this. Thanks.

Re: 7-men Syzygy attempt

Posted: Thu Mar 15, 2018 9:11 pm
by Nordlandia
Bojun Guo: have you checked if probing 7-man inflict larger speed penalty than usual. Probing 6-men cause slight engine speed reduction, not sure how much force 7-man add on speed.

Maybe too early to say.

Re: 7-men Syzygy attempt

Posted: Thu Mar 15, 2018 10:36 pm
by noobpwnftw
Currently all work is about making sure the generator can produce correct results and the file format can hold them, since I don't want to find something broken in the middle of the building campaign...

A quick example is this, the second part overflowed when building KQQQQPvK, luckily it caused crash because MSB before expansion was 1 so the index became negative, otherwise it would just remain silent with a wrong index.

If anyone interested, please also help with checking(take my fork as reference) for such overflows when piece count, index shift range and pawn mask has increased in the 7-men case.

Re: 7-men Syzygy attempt

Posted: Fri Mar 16, 2018 3:36 am
by Dann Corbit
noobpwnftw wrote:Currently all work is about making sure the generator can produce correct results and the file format can hold them, since I don't want to find something broken in the middle of the building campaign...

A quick example is this, the second part overflowed when building KQQQQPvK, luckily it caused crash because MSB before expansion was 1 so the index became negative, otherwise it would just remain silent with a wrong index.

If anyone interested, please also help with checking(take my fork as reference) for such overflows when piece count, index shift range and pawn mask has increased in the 7-men case.
Odd errors trying to use clang.
file defs.h has this:
typedef unsigned long long bitboard;

And yet I get these warnings/errors:

Code: Select all

dcorbit@dcorbit MINGW64 /f/project/dcorbit/tb-7-men/src
$ make
make[1]: Entering directory 'F:/project/dcorbit/tb-7-men/src'
clang -pthread  -O3 -march=native -pipe -D_GNU_SOURCE -Wall -std=c99 -fsanitize=undefined -Wno-array-bounds -flto -fprofile-use -lgcov -c tbgen.c -DHYPER -DUSE_POPCNT -DREGULAR -o objsr/tbgen.o
clang.exe: warning: -lgcov: 'linker' input unused [-Wunused-command-line-argument]
error: Could not read profile default.profdata: No such file or directory
In file included from tbgen.c:16:
In file included from ./board.h:33:
./hyper.h:47:7: warning: implicit declaration of function '_mm_cvtsi64x_si128' is invalid in C99 [-Wimplicit-function-declaration]
  o = _mm_cvtsi64x_si128(occ);
      ^
./hyper.h:47:5: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
  o = _mm_cvtsi64x_si128(occ);
    ^ ~~~~~~~~~~~~~~~~~~~~~~~
In file included from tbgen.c:18:
In file included from ./board.c:9:
./hyper.c:139:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
    xmm1 = _mm_cvtsi64x_si128(bb);
         ^ ~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:140:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
    xmm2 = _mm_cvtsi64x_si128(bb2);
         ^ ~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:142:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
    xmm1 = _mm_cvtsi64x_si128(bit[sq]);
         ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:163:8: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
  xmm1 = _mm_cvtsi64x_si128(0x0001020304050607);
       ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:164:8: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
  xmm2 = _mm_cvtsi64x_si128(0x08090a0b0c0d0e0f);
       ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from tbgen.c:89:
./rtbgen.c:128:5: error: 'asm goto' constructs are not supported yet
    jump_bit_set(bb, sq, lab);
    ^
./generics.c:218:11: note: expanded from macro 'jump_bit_set'
  __asm__ goto ("bt %1, %0; jc %l[lab]" : : "r" (x), "r" ((long64)(y)) : : lab);
          ^
1 warning and 8 errors generated.
make[1]: *** [Makefile.general:30: tbgen.o] Error 1
make[1]: Leaving directory 'F:/project/dcorbit/tb-7-men/src'
make: *** [Makefile:34: rtbgen] Error 2

Re: 7-men Syzygy attempt

Posted: Sun Mar 18, 2018 9:09 am
by noobpwnftw
koedem wrote: Which hardware configuration did you use? Also, in case one needs a lot of RAM already for the pawnless ones, do you think you could upload some of them when they're finished? (maybe via some torrent or so)
I used a 4-way Xeon with some physical RAM more than 1TB to build them. Actually the pawnful ones require less memory because they are built per file.
I have hosted all 3-6 files in my FTP and there is a folder for some test files of 7 piece TBs. Currently the my further building is on hold because there will be a DTZ format change in order to hold all cases in 7 piece TBs, it may or may not require a rebuild of these files.

Don't take anything for granted, these files may not be correct at all.

P.S. on hosting, you can use whatever download tools you want, there is no limit on bandwidth usage nor will I null route anyone.
I had a very bad experience even downloading single threaded somewhere else and I can't use torrent.