Unique position data files

[sje]

Starting with the initial array and running for depth from zero to five, here are the resulting unique position data files:

Code: Select all

 Records      Size Name
       1        59 unique0
      20      1214 unique1
     400     24960 unique2
    5362    340266 unique3
   72078   4648186 unique4
  822518  53811652 unique5

https://dl.dropboxusercontent.com/u/31633927/Unique/unique0
https://dl.dropboxusercontent.com/u/31633927/Unique/unique1
https://dl.dropboxusercontent.com/u/31633927/Unique/unique2
https://dl.dropboxusercontent.com/u/31633927/Unique/unique3
https://dl.dropboxusercontent.com/u/31633927/Unique/unique4
https://dl.dropboxusercontent.com/u/31633927/Unique/unique5

Each record for each file consists of seven fields. The first six are the FEN for the position. The seventh field is the number of times that position appears at the given depth. Note that the half move counter does not affect position identity in this context.

[sje]

The unique6 file is too big (9,417,681 lines, 624,844,375 bytes) to post.

[sje]

How long does it take to make the unique position files?

Single thread, 2.66 GHz Xeon:

Code: Select all

UniqueSt(0): 1   Proc: 0.000   Wall: 0.000   Frequency: 15384.6   Period: 6.5e-05
UniqueSt(1): 20   Proc: 0.001   Wall: 0.001   Frequency: 17331   Period: 5.77e-05
UniqueSt(2): 400   Proc: 0.023   Wall: 0.023   Frequency: 17182.1   Period: 5.82e-05
UniqueSt(3): 5362   Proc: 0.321   Wall: 0.322   Frequency: 16636.7   Period: 6.0108e-05
UniqueSt(4): 72078   Proc: 4.515   Wall: 4.528   Frequency: 15917.8   Period: 6.28227e-05
UniqueSt(5): 822518   Proc: 58.637   Wall: 58.801   Frequency: 13988.1   Period: 7.14893e-05
UniqueSt(6): 9417681   Proc: 15:28.338   Wall: 15:35.260   Frequency: 10069.6   Period: 9.9309e-05

The routine does not use any transposition table assistance, although such would certainly speed up the calculation.

The above times include generating the FEN for each position and for writing all the records to an output file.

[sje]

Four calculation threads, quad core 2.66 GHz Xeon (2006 Mac Pro):

Code: Select all

UniqueMt(0): 1   Proc: 0.000   Wall: 0.000   Frequency: inf   Period: 0
UniqueMt(1): 20   Proc: 0.030   Wall: 0.090   Frequency: 221.107   Period: 0.0045227
UniqueMt(2): 400   Proc: 0.092   Wall: 0.271   Frequency: 1474.84   Period: 0.000678037
UniqueMt(3): 5362   Proc: 0.130   Wall: 0.371   Frequency: 14434.7   Period: 6.92777e-05
UniqueMt(4): 72078   Proc: 0.307   Wall: 0.399   Frequency: 180635   Period: 5.53603e-06
UniqueMt(5): 822518   Proc: 8.542   Wall: 2.827   Frequency: 290907   Period: 3.43753e-06
UniqueMt(6): 9417681   Proc: 5:54.852   Wall: 1:41.527   Frequency: 92759.7   Period: 1.07805e-05
UniqueMt(7): 96400068   Proc: 4:00:16.531   Wall: 1:10:58.097   Frequency: 22639.2   Period: 4.41711e-05

The above does not include FEN string formatting and file output.

[sje]

And on a 3.4 GHz i7-2600 quad code with hyperthreading:

Code: Select all

UniqueMt(0): 1   Proc: 0.000   Wall: 0.000   Frequency: inf   Period: 0
UniqueMt(1): 20   Proc: 0.008   Wall: 0.013   Frequency: 1498.69   Period: 0.00066725
UniqueMt(2): 400   Proc: 0.040   Wall: 0.026   Frequency: 14876   Period: 6.72225e-05
UniqueMt(3): 5362   Proc: 0.080   Wall: 0.026   Frequency: 202600   Period: 4.93584e-06
UniqueMt(4): 72078   Proc: 0.192   Wall: 0.045   Frequency: 1.58808e+06   Period: 6.29693e-07
UniqueMt(5): 822518   Proc: 4.448   Wall: 0.645   Frequency: 1.27391e+06   Period: 7.84983e-07
UniqueMt(6): 9417681   Proc: 2:57.867   Wall: 24.782   Frequency: 380020   Period: 2.63144e-06
UniqueMt(7): 96400068   Proc: 2:07:44.046   Wall: 18:18.916   Frequency: 87722.8   Period: 1.13995e-05

At this point, the program can't get any further because it runs out of memory, or at least physical memory. A change would be needed to use disk storage, and that would not be a trivial change.

[Ajedrecista]

Hello Steven:

And on a 3.4 GHz i7-2600 quad code with hyperthreading:

Code: Select all

UniqueMt(0): 1   Proc: 0.000   Wall: 0.000   Frequency: inf   Period: 0
UniqueMt(1): 20   Proc: 0.008   Wall: 0.013   Frequency: 1498.69   Period: 0.00066725
UniqueMt(2): 400   Proc: 0.040   Wall: 0.026   Frequency: 14876   Period: 6.72225e-05
UniqueMt(3): 5362   Proc: 0.080   Wall: 0.026   Frequency: 202600   Period: 4.93584e-06
UniqueMt(4): 72078   Proc: 0.192   Wall: 0.045   Frequency: 1.58808e+06   Period: 6.29693e-07
UniqueMt(5): 822518   Proc: 4.448   Wall: 0.645   Frequency: 1.27391e+06   Period: 7.84983e-07
UniqueMt(6): 9417681   Proc: 2:57.867   Wall: 24.782   Frequency: 380020   Period: 2.63144e-06
UniqueMt(7): 96400068   Proc: 2:07:44.046   Wall: 18:18.916   Frequency: 87722.8   Period: 1.13995e-05

At this point, the program can't get any further because it runs out of memory, or at least physical memory. A change would be needed to use disk storage, and that would not be a trivial change.

Benchmark of unique positions in an Intel Pentium D930 of year 2006 (3 GHz and 2 GB of RAM) using JetChess 1.0.0.0:

[d]rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1

Code: Select all

Settings: hash = 80 MB; steps = 1.
20 (positions after 1 half move).
Time: 0 ms  (0:00:00.000).


Settings: hash = 80 MB; steps = 1.
400 (positions after 2 half moves).
Time: 0 ms  (0:00:00.000).

Settings: hash = 80 MB; steps = 1.
5,362 (positions after 3 half moves).
Time: 1 ms  (0:00:00.001).


Settings: hash = 80 MB; steps = 1.
72,078 (positions after 4 half moves).
Time: 64 ms  (0:00:00.064).


Settings: hash = 80 MB; steps = 1.
822,518 (positions after 5 half moves).
Time: 656 ms (0:00:00.656).


Settings: hash = 320 MB; steps = 1.
9,417,681 (positions after 6 half moves).
Time: 9.059 s (0:00:09.059).


Settings: hash = 1.25 GB; steps = 2.
96,400,068 (positions after 7 half moves).
Time: 197.433 s (0:03:17.433).


Settings: hash = 1.25 GB; steps = 32.
988,187,354 (positions after 8 half moves).
Time: 14211.285 s (3:56:51.285).

Positions(8) was splitted in 32 steps:

Code: Select all

 i  step(i)

 0  30880243
 1  30881695
 2  30882081
 3  30875564
 4  30878233
 5  30879257
 6  30884928
 7  30881729
 8  30883135
 9  30880814
10  30883178
11  30876341
12  30883674
13  30877213
14  30883155
15  30884022
16  30875665
17  30887529
18  30879728
19  30876364
20  30879448
21  30885697
22  30877070
23  30884906
24  30885634
25  30880013
26  30879312
27  30883307
28  30880058
29  30882182
30  30878479
31  30876700

Doing a lineal extrapolation estimate(i) = [32/(i+1)]*SUM{[step(i)],i=0,i} and computing the relative error rel_err(i) = estimate(i)/(988,187,354) - 1:

Code: Select all

 i  estimate(i)   rel_err(i)

 0   988167776   -0.001981 %
 1   988191008    0.000370 %
 2   988202869    0.001570 %
 3   988156664   -0.003106 %
 4   988146022   -0.004183 %
 5   988144389   -0.004348 %
 6   988169147   -0.001842 %
 7   988174920   -0.001258 %
 8   988184409   -0.000298 %
 9   988184573   -0.000281 %
10   988191584    0.000428 %
11   988179195   -0.000826 %
12   988186762   -0.000060 %
13   988178480   -0.000898 %
14   988183979   -0.000342 %
15   988190524    0.000321 %
16   988180568   -0.000687 %
17   988192811    0.000552 %
18   988190626    0.000331 %
19   988183277   -0.000413 %
20   988181327   -0.000610 %
21   988188644    0.000131 %
22   988183322   -0.000408 %
23   988188892    0.000156 %
24   988194948    0.000768 %
25   988193620    0.000634 %
26   988191559    0.000426 %
27   988194211    0.000694 %
28   988193096    0.000581 %
29   988194320    0.000705 %
30   988191643    0.000434 %
31   988187354    0.000000 %

Positions(9) is impossible of calculate with JetChess (hash overflows). I remember that JetChess is single core, 32-bit software.

Regards from Spain.

Ajedrecista.