Page 1 of 2
Komodo 8 - 5-men Syzygy tablebases
Posted: Sat Jan 10, 2015 7:26 pm
by fastgm
What are the benefits of using
5-men Syzygy tablebases?
Here a test with
Komodo 8 on one core with 128 MB Hash.
The tablebases were on a solid-state drive - Samsung 840 EVO.
Four different time controls were tested, each 3000 games or 1520 at highest time control.
15 sec + 0.15 sec /
60 sec + 0.60 sec /
240 sec + 2.40 sec /
960 sec + 9.60 sec
Code: Select all
TC = 15"+0.15"
TB5 = 5-men Syzygy tablebases
T1 = 1 thread
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3005 8 8 3000 51.4 % 2995 60.8 %
2 Komodo 8 T1 : 2995 8 8 3000 48.6 % 3005 60.8 %
Result : 1542.5/3000 (+630,=1825,-545)
Elo +10
TC = 60"+0.6"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3009 7 7 3000 52.5 % 2991 68.2 %
2 Komodo 8 T1 : 2991 7 7 3000 47.5 % 3009 68.2 %
Result : 1576.0/3000 (+553,=2046,-401)
Elo +18
TC = 4'+2.4"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3006 6 6 3000 51.7 % 2994 75.4 %
2 Komodo 8 T1 : 2994 6 6 3000 48.3 % 3006 75.4 %
Result : 1551.5/3000 (+420,=2263,-317)
Elo +12
TC = 16'+9.6"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3006 8 8 1520 51.7 % 2994 79.8 %
2 Komodo 8 T1 : 2994 8 8 1520 48.3 % 3006 79.8 %
Result : 785.5/1520 (+179,=1213,-128)
Elo +12
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Sat Jan 10, 2015 9:10 pm
by fenchel
fastgm wrote:What are the benefits of using
5-men Syzygy tablebases?
Here a test with
Komodo 8 on one core with 128 MB Hash.
The tablebases were on a solid-state drive - Samsung 840 EVO.
Four different time controls were tested, each 3000 games or 1520 at highest time control.
15 sec + 0.15 sec /
60 sec + 0.60 sec /
240 sec + 2.40 sec /
960 sec + 9.60 sec
Code: Select all
TC = 15"+0.15"
TB5 = 5-men Syzygy tablebases
T1 = 1 thread
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3005 8 8 3000 51.4 % 2995 60.8 %
2 Komodo 8 T1 : 2995 8 8 3000 48.6 % 3005 60.8 %
Result : 1542.5/3000 (+630,=1825,-545)
Elo +10
TC = 60"+0.6"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3009 7 7 3000 52.5 % 2991 68.2 %
2 Komodo 8 T1 : 2991 7 7 3000 47.5 % 3009 68.2 %
Result : 1576.0/3000 (+553,=2046,-401)
Elo +18
TC = 4'+2.4"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3006 6 6 3000 51.7 % 2994 75.4 %
2 Komodo 8 T1 : 2994 6 6 3000 48.3 % 3006 75.4 %
Result : 1551.5/3000 (+420,=2263,-317)
Elo +12
TC = 16'+9.6"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3006 8 8 1520 51.7 % 2994 79.8 %
2 Komodo 8 T1 : 2994 8 8 1520 48.3 % 3006 79.8 %
Result : 785.5/1520 (+179,=1213,-128)
Elo +12
Thank you so much for performing this test. I think it is especially valuable at this present time (e.g., with all the endgame knowledge debates going on with SF, the recent TCEC, etc).
Also, thanks very much for trying different TCs, though of course for now they have overlapping error bars.
I think it would be great to see more of these kinds of tests =) (6-man, SF, small error bars, threads, etc)
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Sat Jan 10, 2015 11:36 pm
by Leto
Thanks for performing this test. A similar test with 6 men tablebases would be even more interesting, I'm certain they will add more than 20 elo at the 960 seconds + 9.60 seconds time control, maybe 50 elo.
Game in 15 minutes + 9.60 sec is rapid, would a slower time control such as game in 2 hours show more elo gains from 6 men tablebases. I'd say yes, but I don't know how much.
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Sun Jan 11, 2015 10:52 am
by cc2150dx
I mainly use 5 men syzygy bases to determine the wins/loss/draws quicker.
And, off-topic I use the bases on a regular HDD with Houdini, SF, K with no slowdowns.
Jason
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Wed Jan 21, 2015 8:20 pm
by fastgm
Here another test, this time
Komodo 8 against Stockfish 5, Komodo once without and then with tablebases, 1 thread, 128 MB Hash.
Three different time controls (TC) were tested, each 3000 or 1550 games at highest time control.
60 sec + 0.60 sec / 240 sec + 2.40 sec / 960 sec + 9.60 sec
Code: Select all
TC = 60"+0.60"
TB5 = 5-men Syzygy tablebases
T1 = 1 Thread
Program Elo + - Games Score Av.Op. Draws
--------------------------------------------------------------------
1 Stockfish 5 T1 : 3005 9 9 3000 51.4 % 2995 46.1 %
2 Komodo 8 T1 : 2995 9 9 3000 48.6 % 3005 46.1 %
Result : 1540.5/3000 (+849,=1383,-768)
Elo +9
Program Elo + - Games Score Av.Op. Draws
--------------------------------------------------------------------
1 Stockfish 5 T1 : 3002 9 9 3000 50.5 % 2998 48.2 %
2 Komodo 8 T1 TB5 : 2998 9 9 3000 49.5 % 3002 48.2 %
Result : 1516.5/3000 (+793,=1447,-760)
Elo +4 TB benefit: 5 Elo
--------------------------------------------------------------------
TC = 4'+2.4"
Program Elo + - Games Score Av.Op. Draws
--------------------------------------------------------------------
1 Stockfish 5 T1 : 3004 9 9 3000 51.2 % 2996 52.2 %
2 Komodo 8 T1 : 2996 9 9 3000 48.8 % 3004 52.2 %
Result : 1536.5/3000 (+754,=1565,-681)
Elo +8
Program Elo + - Games Score Av.Op. Draws
--------------------------------------------------------------------
1 Stockfish 5 T1 : 3002 9 9 3000 50.5 % 2998 52.1 %
2 Komodo 8 T1 TB5 : 2998 9 9 3000 49.5 % 3002 52.1 %
Result : 1516.5/3000 (+735,=1563,-702)
Elo +4 TB benefit: 4 Elo
--------------------------------------------------------------------
TC = 16'+9.6"
Program Elo + - Games Score Av.Op. Draws
--------------------------------------------------------------------
1 Komodo 8 T1 : 3004 11 11 1550 51.3 % 2996 60.4 %
2 Stockfish 5 T1 : 2996 11 11 1550 48.7 % 3004 60.4 %
Result : 795.0/1550 (+327,=936,-287)
Elo +9
Program Elo + - Games Score Av.Op. Draws
--------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3006 11 11 1550 51.7 % 2994 59.2 %
2 Stockfish 5 T1 : 2994 11 11 1550 48.3 % 3006 59.2 %
Result : 802.0/1550 (+343,=918,-289)
Elo +12 TB benefit: 3 Elo
The use of Syzygy tablebases seems only marginally increasing the playing strength of Komodo 8 against Stockfish 5, in average 4 Elo. The gain decreases with increasing time limit.
But due to the error bars, even with this large number of games, this statement is not necessarily cogent, even if the trend suggests this.
Once again Komodo shows a increase in strenght with increasing time limit.
Next is a test with the upcoming Stockfish 6, then the test Komodo 8 against Stockfish 6.
Andreas
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Thu Jan 22, 2015 4:33 pm
by shrapnel
cc2150dx wrote:I mainly use 5 men syzygy bases to determine the wins/loss/draws quicker.
And, off-topic I use the bases on a regular HDD with Houdini, SF, K with no slowdowns.
Jason
You may not notice it, but there will DEFINITELY be slowdowns if you don't use an SSD.
SSDs are so much faster than HDDs....almost unbelievable the difference it makes.
And off-topic, I recently wanted to add a large-capacity HDD to my System, but the thought of installing a large, clunky HDD so put me off that I decided to go with a large capacity SSD instead, even though it cost me more.
No regrets.
Believe me, all your programs ( not just chess) will run so much faster with an SSD that it will make you wonder how you ever put up with HDDs !
On second thoughts, DON'T upgrade to SSD !
Once you do, there is no going back to HDDs !
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Fri Jan 23, 2015 2:22 pm
by Norm Pollock
In between no TBs and SSD, are usb 2.0 and usb 3.0 flash drives. Does anyone know how they rate compared to an SSD?
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Sat Jan 24, 2015 1:39 am
by syzygy
cc2150dx wrote:And, off-topic I use the bases on a regular HDD with Houdini, SF, K with no slowdowns.
Since the usual ignoramus felt a need to contest what you wrote here, I'll just say that 5-piece TBs on a regular HDD indeed won't cause a slowdown on a modern PC as the data will be cached in RAM (ignoring the first seconds of TB access after a reboot of the PC).
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Sat Jan 24, 2015 5:37 am
by lucasart
fastgm wrote:What are the benefits of using
5-men Syzygy tablebases?
Here a test with
Komodo 8 on one core with 128 MB Hash.
The tablebases were on a solid-state drive - Samsung 840 EVO.
Four different time controls were tested, each 3000 games or 1520 at highest time control.
15 sec + 0.15 sec /
60 sec + 0.60 sec /
240 sec + 2.40 sec /
960 sec + 9.60 sec
Code: Select all
TC = 15"+0.15"
TB5 = 5-men Syzygy tablebases
T1 = 1 thread
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3005 8 8 3000 51.4 % 2995 60.8 %
2 Komodo 8 T1 : 2995 8 8 3000 48.6 % 3005 60.8 %
Result : 1542.5/3000 (+630,=1825,-545)
Elo +10
TC = 60"+0.6"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3009 7 7 3000 52.5 % 2991 68.2 %
2 Komodo 8 T1 : 2991 7 7 3000 47.5 % 3009 68.2 %
Result : 1576.0/3000 (+553,=2046,-401)
Elo +18
TC = 4'+2.4"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3006 6 6 3000 51.7 % 2994 75.4 %
2 Komodo 8 T1 : 2994 6 6 3000 48.3 % 3006 75.4 %
Result : 1551.5/3000 (+420,=2263,-317)
Elo +12
TC = 16'+9.6"
Program Elo + - Games Score Av.Op. Draws
---------------------------------------------------------------------
1 Komodo 8 T1 TB5 : 3006 8 8 1520 51.7 % 2994 79.8 %
2 Komodo 8 T1 : 2994 8 8 1520 48.3 % 3006 79.8 %
Result : 785.5/1520 (+179,=1213,-128)
Elo +12
This is very much in line with the result I measured myself a while ago, which was +10 elo in 15"+0.15". However, I never had resources and patience to do longer tests.
It is interesting to note that the elo gain does not shrink at longer tc, even at the very long 16'+9.6". Normally any elo gain tends to shrink at longer tc, as draw rate increases, so anything that does not shrink at this very long tc is remarkable.
I wish Marco Costalba could read this, and stop claiming that syzygy TB have zero elo impact.
Re: Komodo 8 - 5-men Syzygy tablebases
Posted: Sat Jan 24, 2015 10:00 am
by Jouni
So we get +4 ELO from 5 piece set. IPON test gave +6 for Stockfish. Isn't these very modest gain from 960 MB of data?? There has been +5 gain from 10 parameters change in Stockfish testing.