7-men Syzygy attempt

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
noobpwnftw
Posts: 258
Joined: Sun Nov 08, 2015 10:10 pm

Re: 7-men Syzygy attempt

Post by noobpwnftw » Thu Mar 15, 2018 4:59 am

I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.

It does not seem to affect the correctness(confirm?).

Dann Corbit
Posts: 9051
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 7-men Syzygy attempt

Post by Dann Corbit » Thu Mar 15, 2018 6:49 am

noobpwnftw wrote:I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.
. Other
It does not seem to affect the correctness(confirm?).
Expand the array. Otherwise undefined behavior
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

syzygy
Posts: 4343
Joined: Tue Feb 28, 2012 10:56 pm

Re: 7-men Syzygy attempt

Post by syzygy » Thu Mar 15, 2018 7:06 am

noobpwnftw wrote:I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.

It does not seem to affect the correctness(confirm?).
Yes this is part of the code that has to be adapted to deal with 16-bit values. The problem starts when (dtz-101)/2 exceeds 255 (actually a bit earlier). This should not affect the generation process and statistics are probably collected correctly.

koedem
Posts: 44
Joined: Fri Mar 18, 2016 9:45 pm

Re: 7-men Syzygy attempt

Post by koedem » Thu Mar 15, 2018 1:08 pm

I realized only now, you generated a complete material balance in 30 seconds only? I have to imagine the ones with pawns take much more RAM because you need to cache the possible promotions so those might be more tricky. But nontheless I am surprised that the pawnless ones go so smooth.

Which hardware configuration did you use? Also, in case one needs a lot of RAM already for the pawnless ones, do you think you could upload some of them when they're finished? (maybe via some torrent or so)

syzygy
Posts: 4343
Joined: Tue Feb 28, 2012 10:56 pm

Re: 7-men Syzygy attempt

Post by syzygy » Thu Mar 15, 2018 5:24 pm

No, the 33 seconds is only for the "resetting cursed losses" pass.

duncan
Posts: 8093
Joined: Mon Jul 07, 2008 8:50 pm

Re: 7-men Syzygy attempt

Post by duncan » Thu Mar 15, 2018 7:12 pm

noobpwnftw wrote: With your new 16-bit compression code available, I'm pretty confident that full set of 7-men Syzygy will be available soon.

I already have the necessary hardware(both steps in physical RAM without --disk) and will provide hosting for all the resulting files.
great achievement if you do this. Thanks.

User avatar
Nordlandia
Posts: 2082
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: 7-men Syzygy attempt

Post by Nordlandia » Thu Mar 15, 2018 9:11 pm

Bojun Guo: have you checked if probing 7-man inflict larger speed penalty than usual. Probing 6-men cause slight engine speed reduction, not sure how much force 7-man add on speed.

Maybe too early to say.

noobpwnftw
Posts: 258
Joined: Sun Nov 08, 2015 10:10 pm

Re: 7-men Syzygy attempt

Post by noobpwnftw » Thu Mar 15, 2018 10:36 pm

Currently all work is about making sure the generator can produce correct results and the file format can hold them, since I don't want to find something broken in the middle of the building campaign...

A quick example is this, the second part overflowed when building KQQQQPvK, luckily it caused crash because MSB before expansion was 1 so the index became negative, otherwise it would just remain silent with a wrong index.

If anyone interested, please also help with checking(take my fork as reference) for such overflows when piece count, index shift range and pawn mask has increased in the 7-men case.

Dann Corbit
Posts: 9051
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 7-men Syzygy attempt

Post by Dann Corbit » Fri Mar 16, 2018 3:36 am

noobpwnftw wrote:Currently all work is about making sure the generator can produce correct results and the file format can hold them, since I don't want to find something broken in the middle of the building campaign...

A quick example is this, the second part overflowed when building KQQQQPvK, luckily it caused crash because MSB before expansion was 1 so the index became negative, otherwise it would just remain silent with a wrong index.

If anyone interested, please also help with checking(take my fork as reference) for such overflows when piece count, index shift range and pawn mask has increased in the 7-men case.
Odd errors trying to use clang.
file defs.h has this:
typedef unsigned long long bitboard;

And yet I get these warnings/errors:

Code: Select all

dcorbit@dcorbit MINGW64 /f/project/dcorbit/tb-7-men/src
$ make
make[1]: Entering directory 'F:/project/dcorbit/tb-7-men/src'
clang -pthread  -O3 -march=native -pipe -D_GNU_SOURCE -Wall -std=c99 -fsanitize=undefined -Wno-array-bounds -flto -fprofile-use -lgcov -c tbgen.c -DHYPER -DUSE_POPCNT -DREGULAR -o objsr/tbgen.o
clang.exe: warning: -lgcov: 'linker' input unused [-Wunused-command-line-argument]
error: Could not read profile default.profdata: No such file or directory
In file included from tbgen.c:16:
In file included from ./board.h:33:
./hyper.h:47:7: warning: implicit declaration of function '_mm_cvtsi64x_si128' is invalid in C99 [-Wimplicit-function-declaration]
  o = _mm_cvtsi64x_si128(occ);
      ^
./hyper.h:47:5: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
  o = _mm_cvtsi64x_si128(occ);
    ^ ~~~~~~~~~~~~~~~~~~~~~~~
In file included from tbgen.c:18:
In file included from ./board.c:9:
./hyper.c:139:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
    xmm1 = _mm_cvtsi64x_si128(bb);
         ^ ~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:140:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
    xmm2 = _mm_cvtsi64x_si128(bb2);
         ^ ~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:142:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
    xmm1 = _mm_cvtsi64x_si128(bit[sq]);
         ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:163:8: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
  xmm1 = _mm_cvtsi64x_si128(0x0001020304050607);
       ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:164:8: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
  xmm2 = _mm_cvtsi64x_si128(0x08090a0b0c0d0e0f);
       ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from tbgen.c:89:
./rtbgen.c:128:5: error: 'asm goto' constructs are not supported yet
    jump_bit_set(bb, sq, lab);
    ^
./generics.c:218:11: note: expanded from macro 'jump_bit_set'
  __asm__ goto ("bt %1, %0; jc %l[lab]" : : "r" (x), "r" ((long64)(y)) : : lab);
          ^
1 warning and 8 errors generated.
make[1]: *** [Makefile.general:30: tbgen.o] Error 1
make[1]: Leaving directory 'F:/project/dcorbit/tb-7-men/src'
make: *** [Makefile:34: rtbgen] Error 2
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

noobpwnftw
Posts: 258
Joined: Sun Nov 08, 2015 10:10 pm

Re: 7-men Syzygy attempt

Post by noobpwnftw » Sun Mar 18, 2018 9:09 am

koedem wrote: Which hardware configuration did you use? Also, in case one needs a lot of RAM already for the pawnless ones, do you think you could upload some of them when they're finished? (maybe via some torrent or so)
I used a 4-way Xeon with some physical RAM more than 1TB to build them. Actually the pawnful ones require less memory because they are built per file.
I have hosted all 3-6 files in my FTP and there is a folder for some test files of 7 piece TBs. Currently the my further building is on hold because there will be a DTZ format change in order to hold all cases in 7 piece TBs, it may or may not require a rebuild of these files.

Don't take anything for granted, these files may not be correct at all.

P.S. on hosting, you can use whatever download tools you want, there is no limit on bandwidth usage nor will I null route anyone.
I had a very bad experience even downloading single threaded somewhere else and I can't use torrent.

Post Reply