I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.
It does not seem to affect the correctness(confirm?).
7-men Syzygy attempt
Moderators: hgm, Rebel, chrisw
-
- Posts: 684
- Joined: Sun Nov 08, 2015 11:10 pm
- Full name: Bojun Guo
-
- Posts: 12662
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: 7-men Syzygy attempt
Expand the array. Otherwise undefined behaviornoobpwnftw wrote:I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.
. Other
It does not seem to affect the correctness(confirm?).
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 5647
- Joined: Tue Feb 28, 2012 11:56 pm
Re: 7-men Syzygy attempt
Yes this is part of the code that has to be adapted to deal with 16-bit values. The problem starts when (dtz-101)/2 exceeds 255 (actually a bit earlier). This should not affect the generation process and statistics are probably collected correctly.noobpwnftw wrote:I found a problem(bug?) after raising the stats limit:
Although later we need to raise the map to 16-bits for large DTZ compression, in the current pawn code the MAX_PLY is 768, so this array here written with [(MAX_PLY - 1 - DRAW_RULE) / 2] is overflowed.
It does not seem to affect the correctness(confirm?).
-
- Posts: 105
- Joined: Fri Mar 18, 2016 10:45 pm
Re: 7-men Syzygy attempt
I realized only now, you generated a complete material balance in 30 seconds only? I have to imagine the ones with pawns take much more RAM because you need to cache the possible promotions so those might be more tricky. But nontheless I am surprised that the pawnless ones go so smooth.
Which hardware configuration did you use? Also, in case one needs a lot of RAM already for the pawnless ones, do you think you could upload some of them when they're finished? (maybe via some torrent or so)
Which hardware configuration did you use? Also, in case one needs a lot of RAM already for the pawnless ones, do you think you could upload some of them when they're finished? (maybe via some torrent or so)
-
- Posts: 5647
- Joined: Tue Feb 28, 2012 11:56 pm
Re: 7-men Syzygy attempt
No, the 33 seconds is only for the "resetting cursed losses" pass.
-
- Posts: 12038
- Joined: Mon Jul 07, 2008 10:50 pm
Re: 7-men Syzygy attempt
great achievement if you do this. Thanks.noobpwnftw wrote: With your new 16-bit compression code available, I'm pretty confident that full set of 7-men Syzygy will be available soon.
I already have the necessary hardware(both steps in physical RAM without --disk) and will provide hosting for all the resulting files.
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: 7-men Syzygy attempt
Bojun Guo: have you checked if probing 7-man inflict larger speed penalty than usual. Probing 6-men cause slight engine speed reduction, not sure how much force 7-man add on speed.
Maybe too early to say.
Maybe too early to say.
-
- Posts: 684
- Joined: Sun Nov 08, 2015 11:10 pm
- Full name: Bojun Guo
Re: 7-men Syzygy attempt
Currently all work is about making sure the generator can produce correct results and the file format can hold them, since I don't want to find something broken in the middle of the building campaign...
A quick example is this, the second part overflowed when building KQQQQPvK, luckily it caused crash because MSB before expansion was 1 so the index became negative, otherwise it would just remain silent with a wrong index.
If anyone interested, please also help with checking(take my fork as reference) for such overflows when piece count, index shift range and pawn mask has increased in the 7-men case.
A quick example is this, the second part overflowed when building KQQQQPvK, luckily it caused crash because MSB before expansion was 1 so the index became negative, otherwise it would just remain silent with a wrong index.
If anyone interested, please also help with checking(take my fork as reference) for such overflows when piece count, index shift range and pawn mask has increased in the 7-men case.
-
- Posts: 12662
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: 7-men Syzygy attempt
Odd errors trying to use clang.noobpwnftw wrote:Currently all work is about making sure the generator can produce correct results and the file format can hold them, since I don't want to find something broken in the middle of the building campaign...
A quick example is this, the second part overflowed when building KQQQQPvK, luckily it caused crash because MSB before expansion was 1 so the index became negative, otherwise it would just remain silent with a wrong index.
If anyone interested, please also help with checking(take my fork as reference) for such overflows when piece count, index shift range and pawn mask has increased in the 7-men case.
file defs.h has this:
typedef unsigned long long bitboard;
And yet I get these warnings/errors:
Code: Select all
dcorbit@dcorbit MINGW64 /f/project/dcorbit/tb-7-men/src
$ make
make[1]: Entering directory 'F:/project/dcorbit/tb-7-men/src'
clang -pthread -O3 -march=native -pipe -D_GNU_SOURCE -Wall -std=c99 -fsanitize=undefined -Wno-array-bounds -flto -fprofile-use -lgcov -c tbgen.c -DHYPER -DUSE_POPCNT -DREGULAR -o objsr/tbgen.o
clang.exe: warning: -lgcov: 'linker' input unused [-Wunused-command-line-argument]
error: Could not read profile default.profdata: No such file or directory
In file included from tbgen.c:16:
In file included from ./board.h:33:
./hyper.h:47:7: warning: implicit declaration of function '_mm_cvtsi64x_si128' is invalid in C99 [-Wimplicit-function-declaration]
o = _mm_cvtsi64x_si128(occ);
^
./hyper.h:47:5: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
o = _mm_cvtsi64x_si128(occ);
^ ~~~~~~~~~~~~~~~~~~~~~~~
In file included from tbgen.c:18:
In file included from ./board.c:9:
./hyper.c:139:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
xmm1 = _mm_cvtsi64x_si128(bb);
^ ~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:140:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
xmm2 = _mm_cvtsi64x_si128(bb2);
^ ~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:142:10: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
xmm1 = _mm_cvtsi64x_si128(bit[sq]);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:163:8: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
xmm1 = _mm_cvtsi64x_si128(0x0001020304050607);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./hyper.c:164:8: error: assigning to '__m128i' (vector of 2 'long long' values) from incompatible type 'int'
xmm2 = _mm_cvtsi64x_si128(0x08090a0b0c0d0e0f);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from tbgen.c:89:
./rtbgen.c:128:5: error: 'asm goto' constructs are not supported yet
jump_bit_set(bb, sq, lab);
^
./generics.c:218:11: note: expanded from macro 'jump_bit_set'
__asm__ goto ("bt %1, %0; jc %l[lab]" : : "r" (x), "r" ((long64)(y)) : : lab);
^
1 warning and 8 errors generated.
make[1]: *** [Makefile.general:30: tbgen.o] Error 1
make[1]: Leaving directory 'F:/project/dcorbit/tb-7-men/src'
make: *** [Makefile:34: rtbgen] Error 2
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 684
- Joined: Sun Nov 08, 2015 11:10 pm
- Full name: Bojun Guo
Re: 7-men Syzygy attempt
I used a 4-way Xeon with some physical RAM more than 1TB to build them. Actually the pawnful ones require less memory because they are built per file.koedem wrote: Which hardware configuration did you use? Also, in case one needs a lot of RAM already for the pawnless ones, do you think you could upload some of them when they're finished? (maybe via some torrent or so)
I have hosted all 3-6 files in my FTP and there is a folder for some test files of 7 piece TBs. Currently the my further building is on hold because there will be a DTZ format change in order to hold all cases in 7 piece TBs, it may or may not require a rebuild of these files.
Don't take anything for granted, these files may not be correct at all.
P.S. on hosting, you can use whatever download tools you want, there is no limit on bandwidth usage nor will I null route anyone.
I had a very bad experience even downloading single threaded somewhere else and I can't use torrent.