I re-run test (3*3000) games with different book and slightly modified settings. Syzygy files on HD. Now I got this TB gain:
SF8 +9,2
Houdini5 +6,5
Komodo10 +0,7(!?)
I You look at average depth You see, that Komodo with TBs has lower than without, but SF has bigger depth with TBs. For Komodo You obviously need some 10k games to see any statistical gain.
The value of endgame tablebases
Moderators: hgm, Rebel, chrisw
-
- Posts: 1056
- Joined: Fri Mar 10, 2006 6:07 am
- Location: Basque Country (Spain)
Re: The value of endgame tablebases
I did a test and used gaviota tablebases and scorpio bitbases on a HD with 40 moves games in 1 minute.mjlef wrote:That is a very useful test. If you do the same for 6 piece, I would love to see the results.Jouni wrote:In 3000 games bullet selftest I got (5 piece syzygy):
SF 8 +11 ELO
Houdini 5 +14 ELO
Komodo 10 +7 ELO
Equal to 2-3 great patches .
Were you results with a fast SSD or regular HD? Note for 5 piece it does not matter so much since the whole WLD files set is only about 500 megs, which most machines cache in memory anyway. It should matter more for 6 piece.
1) danasah with 4 men gaviota tablebases + 5 men scorpio bitbases vs danasah gives between 7 and 14 Elo points, it seems that matched what I should be expected.
2) danasah with 5 men gaviota tablebases + 6 men scorpio bitbases vs danasah loses 30 Elo points.
I have not played many games and I have to do more tests,
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: The value of endgame tablebases
Is RAID configuration a cheaper alternative to be able to store 6-men Nalimov, for engine matches and Let's Check Analyse [Fritz 15].
Current prices for 1.2 TB SSDs is too high at the present moment.
http://www.ebay.com/itm/like/2321071100 ... rmvSB=true
For example
2x 600 GB SSDs in raid configuration should let me be able to to store whole 6-men set in depth to mate format.
Current prices for 1.2 TB SSDs is too high at the present moment.
http://www.ebay.com/itm/like/2321071100 ... rmvSB=true
For example
2x 600 GB SSDs in raid configuration should let me be able to to store whole 6-men set in depth to mate format.
-
- Posts: 1494
- Joined: Thu Mar 30, 2006 2:08 pm
Re: The value of endgame tablebases
You actually need a lot of games to reduce error margins enough. 3000 game is roughly +/- 7.7 elo. 10000 games is roughly +/- 4.4 elo. In the past we showed something like +10 elo for 5 piece Syzygy and more fr 6 piece. But we have changed some endgame eval rules in Komodo since running this. So perhaps Syzygy help less now. But I cannot say due to the larger error margins. Most of our tests have been using SSD which is faster, although it should not matter too much for 5 piece games since they quickly get cached to RAM by most operating systems (the working win-loss-draw set is only around 500 MB).Jouni wrote:I re-run test (3*3000) games with different book and slightly modified settings. Syzygy files on HD. Now I got this TB gain:
SF8 +9,2
Houdini5 +6,5
Komodo10 +0,7(!?)
I You look at average depth You see, that Komodo with TBs has lower than without, but SF has bigger depth with TBs. For Komodo You obviously need some 10k games to see any statistical gain.
BTW Komodo has a "Smart Syzygy" function for people with HD instead of SSD. When turned on it always looks up 5 piece positions, but only looks up 6 piece positions at the depth the user sets. 6 piece are very slow to access on a HD, so this feature gives a reasonable mix of accuracy versus speed. If you ever try a 6 piece run, I would like to hear how this feature performs on your HD system.
Thanks for running these.
-
- Posts: 3293
- Joined: Wed Mar 08, 2006 8:15 pm
Re: The value of endgame tablebases
Is there a command to read all tablebases at once to RAM (or cache) for faster access or do I remember wrong!?
Jouni
-
- Posts: 303
- Joined: Fri Jun 21, 2013 5:18 am
- Location: Orion Spiral Arm
Re: The value of endgame tablebases
from your Syzygy directory, try this:Jouni wrote:Is there a command to read all tablebases at once to RAM (or cache) for faster access or do I remember wrong!?
type *.rtbw >nul
i7-5930K @4.5GHz, H100i Hydro Cooler, 64GB DDR4 Corsair Dominator Platinum @3000MHz, ASUS X99 Deluxe mboard, 1TB EVO 850 SSD
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: The value of endgame tablebases
Possible 8-Man Cook
[d]1b4r1/2k5/8/8/1B6/6p1/2R3B1/6K1 b - - 0 6
[pgn][Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "New game"]
[Black "?"]
[Result "*"]
[SetUp "1"]
[FEN "1b4r1/2k5/2N5/6q1/8/6pB/1RR3K1/4B3 b - - 0 1"]
[PlyCount "10"]
1... Qd5+ 2. Kg1 Qxc6 3. Ba5+ Kd6 4. Bb4+ Kc7 5. Bg2 Qxc2 6. Rxc2+ *
[/pgn]
[d]1b4r1/2k5/8/8/1B6/6p1/2R3B1/6K1 b - - 0 6
[pgn][Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "New game"]
[Black "?"]
[Result "*"]
[SetUp "1"]
[FEN "1b4r1/2k5/2N5/6q1/8/6pB/1RR3K1/4B3 b - - 0 1"]
[PlyCount "10"]
1... Qd5+ 2. Kg1 Qxc6 3. Ba5+ Kd6 4. Bb4+ Kc7 5. Bg2 Qxc2 6. Rxc2+ *
[/pgn]
-
- Posts: 3293
- Joined: Wed Mar 08, 2006 8:15 pm
Re: The value of endgame tablebases
I ran additional 12000 games with bases cached in RAM (of course rtbw and rtbz files both). Should be optimal conditions? But surprisingly I get lower 5 piece gain now. Gain was from 0 (zero) to +5,7 ELO. Not much from 1 GB of data.
Jouni
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: The value of endgame tablebases
Any major difference between M.2 and regular SSD in terms of tablebase speed?
-
- Posts: 686
- Joined: Thu Mar 03, 2011 4:57 pm
- Location: Germany
Re: The value of endgame tablebases
It depends on the engine probably and how it handles the endgame.
iCE has only 3 men TB, so I'd say this counts as no TB support and it has no problem in this position
All the 5 drawing moves make it top of the list. Ra8 is losing.
If Ra2 is forced iCE orders the 3 winning moves also top of the list
iCE is able to access Gaviota TB but I did not really see a real gain by using it. Tests were almost neutral and so I decided to not include the access functionality in release candidates.
iCE has only 3 men TB, so I'd say this counts as no TB support and it has no problem in this position
Code: Select all
39/46 00:09 -0,66 72. Ke5 (20.660 kn)
39/46 00:09 -0,66 72. Ke6 (20.660 kn)
39/46 00:09 -0,66 72. Ke7 g5 73. Kf6 (20.660 kn)
39/46 00:09 -0,66 72. Ra3 (20.660 kn)
39/46 00:09 -0,75 72. Ra7 g5 73. Ke6 Rb2 74.Rb5+ (20.660 kn)
39/46 00:09 -1,01 72. Ra8 g5 73. Ke5 Rf4 74. Rg8 (20.660 kn)
If Ra2 is forced iCE orders the 3 winning moves also top of the list
Code: Select all
39/48 00:12 +2,40 72... g5 73. Ke6 Rf4 74. g4 (27.459 kn)
39/48 00:12 +2,40 72... Rf4 73. Ra3 Kg4 74. Ra5 (27.459 kn)
39/48 00:12 +2,05 72... Re4 73. Kd5 Rf4 74. Ra3 (27.459 kn)
39/48 00:12 +0,66 72... Rc4 73. Ke5 g5 74. Kf6 (27.459 kn)