Page 1 of 4

AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Sun Jan 07, 2018 1:19 pm
by fastgm
Conditions:
Dual AMD Opteron 6376, Cutechess-Cli, 100 games, fixed 60 seconds per move, opening book: Noomen_2-move_Testsuite.pgn, no adjudication rules, no tablebases

Stockfish 8, 31 threads, 16384 MB Hash
vs
Stockfish 8, 31 threads, 512 MB Hash

Code: Select all

1   Stockfish 8 T31 16384    +7  +3/=96/-1 51.00%   51.0/100
2   Stockfish 8 T31 512      -7  +1/=96/-3 49.00%   49.0/100
 
 Engine                 Depth       Time   Games     Moves  Average
--------------------------------------------------------------------
 Stockfish 8 T31 16384  42.11  114:02:17     100      7078    58.00
 Stockfish 8 T31 512    42.54  113:58:48     100      7076    57.99

    Program                  Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------------
  1 Stockfish 8 T31 16384  : 3003   16  12   100    51.0 %   2997   96.0 %
  2 Stockfish 8 T31 512    : 2997   12  16   100    49.0 %   3003   96.0 %
 
Wins   = 3
Draws  = 96
Losses = 1
Av.Op. Elo = 3000
 
Result     : 51.0/100 (+3,=96,-1)
Perf.      : 51.0 %
Margins    :
 68 %      : (+  1.1,-  0.9 %) -> [ 50.1, 52.1 %]
 95 %      : (+  2.3,-  1.7 %) -> [ 49.3, 53.3 %]
 99.7 %    : (+  3.8,-  2.4 %) -> [ 48.6, 54.8 %]
 
Elo        : 3007
Margins    :
 68 %      : (+  8,-  6) -> [3001,3015]
 95 %      : (+ 16,- 12) -> [2995,3023]
 99.7 %    : (+ 26,- 16) -> [2990,3033]
 
Games        :    100 (finished)
 
White Wins   :      4 ( 4.0 %)
Black Wins   :      0 ( 0.0 %)
Draws        :     96 (96.0 %)
 
White Perf.  : 52.0 %
Black Perf.  : 48.0 %
 
Individual statistics:
 
1 Stockfish 8 T31 16384  : 3003  100 (+  3,= 96,-  1), 51.0 %
2 Stockfish 8 T31 512    : 2997  100 (+  1,= 96,-  3), 49.0 %
All games:
http://www.fastgm.de/schach/SF8-T31-163 ... ec-100.pgn
.

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Sun Jan 07, 2018 2:08 pm
by Modern Times
I've never been a fan of these huge hash sizes. Perhaps they make sense for infinite analysis where in conjunction with large pages they give a boost. But I think maybe it also varies depending on CPU, motherboard and RAM architecture as well.

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Sun Jan 07, 2018 2:29 pm
by elcabesa
Can you try to see what happens with 21 core vs 16 core?

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Sun Jan 07, 2018 2:51 pm
by syzygy
So in a 100-game match played at 1 minute/move and with many cores, handicapping SF8 by giving it a small hash table does not "cripple" it.

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Mon Jan 08, 2018 7:55 am
by Ozymandias
The only one single condition that "crippled" it, was the absence of a book.

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Mon Jan 08, 2018 2:13 pm
by Jouni
Really surprising! 128 GB SF limit is definitely no problem. TCEC should go to 1 GB for optimal play?

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Mon Jan 08, 2018 4:18 pm
by CMCanavessi
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Mon Jan 08, 2018 4:44 pm
by Ozymandias
CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Mon Jan 08, 2018 5:48 pm
by CMCanavessi
Ozymandias wrote:
CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.
No, it's not crippling but you don't just have 1 cause, you have to add them all up.

1GB hash + no TBs + no book + 1 year old version + weird time control = cripple.

If you separate them, the effect is not that noticeable.

Re: AlphaZero Chess test - SF8 16 GB vs. 512 MB

Posted: Mon Jan 08, 2018 7:49 pm
by Ozymandias
CMCanavessi wrote:
Ozymandias wrote:
CMCanavessi wrote:
Ozymandias wrote:The only one single condition that "crippled" it, was the absence of a book.
And TBs.
TBs have a bigger impact at ultra fast TCs. IIRC, this was 1min/move on good HW, so not exactly crippling.
No, it's not crippling but you don't just have 1 cause, you have to add them all up.

1GB hash + no TBs + no book + 1 year old version + weird time control = cripple.

If you separate them, the effect is not that noticeable.
The book alone is noticeable.