Komodo 8 - 5-men Syzygy tablebases

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
fastgm
Posts: 414
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Komodo 8 - 5-men Syzygy tablebases

Post by fastgm » Sat Jan 10, 2015 6:26 pm

What are the benefits of using 5-men Syzygy tablebases?

Here a test with Komodo 8 on one core with 128 MB Hash.
The tablebases were on a solid-state drive - Samsung 840 EVO.
Four different time controls were tested, each 3000 games or 1520 at highest time control.
15 sec + 0.15 sec / 60 sec + 0.60 sec / 240 sec + 2.40 sec / 960 sec + 9.60 sec

Code: Select all

TC  = 15"+0.15"
TB5 = 5-men Syzygy tablebases
T1  = 1 thread
 
    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3005   8   8   3000    51.4 %   2995   60.8 %
  2 Komodo 8 T1      : 2995   8   8   3000    48.6 %   3005   60.8 %

Result : 1542.5/3000 (+630,=1825,-545)
 
Elo  +10 
 
TC  = 60"+0.6"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3009   7   7   3000    52.5 %   2991   68.2 %
  2 Komodo 8 T1      : 2991   7   7   3000    47.5 %   3009   68.2 %

Result : 1576.0/3000 (+553,=2046,-401)
 
Elo  +18 
 
TC  = 4'+2.4"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3006   6   6   3000    51.7 %   2994   75.4 %
  2 Komodo 8 T1      : 2994   6   6   3000    48.3 %   3006   75.4 %

Result : 1551.5/3000 (+420,=2263,-317)
 
Elo  +12
  
TC  = 16'+9.6"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3006   8   8   1520    51.7 %   2994   79.8 %
  2 Komodo 8 T1      : 2994   8   8   1520    48.3 %   3006   79.8 %

Result : 785.5/1520 (+179,=1213,-128)
 
Elo  +12

fenchel
Posts: 36
Joined: Thu Dec 04, 2014 5:01 am

Re: Komodo 8 - 5-men Syzygy tablebases

Post by fenchel » Sat Jan 10, 2015 8:10 pm

fastgm wrote:What are the benefits of using 5-men Syzygy tablebases?

Here a test with Komodo 8 on one core with 128 MB Hash.
The tablebases were on a solid-state drive - Samsung 840 EVO.
Four different time controls were tested, each 3000 games or 1520 at highest time control.
15 sec + 0.15 sec / 60 sec + 0.60 sec / 240 sec + 2.40 sec / 960 sec + 9.60 sec

Code: Select all

TC  = 15"+0.15"
TB5 = 5-men Syzygy tablebases
T1  = 1 thread
 
    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3005   8   8   3000    51.4 %   2995   60.8 %
  2 Komodo 8 T1      : 2995   8   8   3000    48.6 %   3005   60.8 %

Result : 1542.5/3000 (+630,=1825,-545)
 
Elo  +10 
 
TC  = 60"+0.6"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3009   7   7   3000    52.5 %   2991   68.2 %
  2 Komodo 8 T1      : 2991   7   7   3000    47.5 %   3009   68.2 %

Result : 1576.0/3000 (+553,=2046,-401)
 
Elo  +18 
 
TC  = 4'+2.4"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3006   6   6   3000    51.7 %   2994   75.4 %
  2 Komodo 8 T1      : 2994   6   6   3000    48.3 %   3006   75.4 %

Result : 1551.5/3000 (+420,=2263,-317)
 
Elo  +12
  
TC  = 16'+9.6"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3006   8   8   1520    51.7 %   2994   79.8 %
  2 Komodo 8 T1      : 2994   8   8   1520    48.3 %   3006   79.8 %

Result : 785.5/1520 (+179,=1213,-128)
 
Elo  +12
Thank you so much for performing this test. I think it is especially valuable at this present time (e.g., with all the endgame knowledge debates going on with SF, the recent TCEC, etc).

Also, thanks very much for trying different TCs, though of course for now they have overlapping error bars.

I think it would be great to see more of these kinds of tests =) (6-man, SF, small error bars, threads, etc)

User avatar
Leto
Posts: 2028
Joined: Thu May 04, 2006 1:40 am
Location: Dune

Re: Komodo 8 - 5-men Syzygy tablebases

Post by Leto » Sat Jan 10, 2015 10:36 pm

Thanks for performing this test. A similar test with 6 men tablebases would be even more interesting, I'm certain they will add more than 20 elo at the 960 seconds + 9.60 seconds time control, maybe 50 elo.

Game in 15 minutes + 9.60 sec is rapid, would a slower time control such as game in 2 hours show more elo gains from 6 men tablebases. I'd say yes, but I don't know how much.

cc2150dx
Posts: 176
Joined: Sat Nov 30, 2013 8:51 am
Location: Canada

Re: Komodo 8 - 5-men Syzygy tablebases

Post by cc2150dx » Sun Jan 11, 2015 9:52 am

I mainly use 5 men syzygy bases to determine the wins/loss/draws quicker.

And, off-topic I use the bases on a regular HDD with Houdini, SF, K with no slowdowns.


Jason
Komodo tester

fastgm
Posts: 414
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Re: Komodo 8 - 5-men Syzygy tablebases

Post by fastgm » Wed Jan 21, 2015 7:20 pm

Here another test, this time Komodo 8 against Stockfish 5, Komodo once without and then with tablebases, 1 thread, 128 MB Hash.
Three different time controls (TC) were tested, each 3000 or 1550 games at highest time control.
60 sec + 0.60 sec / 240 sec + 2.40 sec / 960 sec + 9.60 sec

Code: Select all

TC  = 60"+0.60"
TB5 = 5-men Syzygy tablebases
T1  = 1 Thread


     Program            Elo    +   -   Games   Score   Av.Op.  Draws
  --------------------------------------------------------------------
   1 Stockfish 5 T1   : 3005   9   9   3000    51.4 %   2995   46.1 %
   2 Komodo 8 T1      : 2995   9   9   3000    48.6 %   3005   46.1 %
 
Result : 1540.5/3000 (+849,=1383,-768)

Elo  +9


     Program            Elo    +   -   Games   Score   Av.Op.  Draws
  --------------------------------------------------------------------
   1 Stockfish 5 T1   : 3002   9   9   3000    50.5 %   2998   48.2 %
   2 Komodo 8 T1 TB5  : 2998   9   9   3000    49.5 %   3002   48.2 %

Result : 1516.5/3000 (+793,=1447,-760)

Elo  +4                   TB benefit: 5 Elo

--------------------------------------------------------------------

 
TC  = 4'+2.4"

     Program            Elo    +   -   Games   Score   Av.Op.  Draws
  --------------------------------------------------------------------
   1 Stockfish 5 T1   : 3004   9   9   3000    51.2 %   2996   52.2 %
   2 Komodo 8 T1      : 2996   9   9   3000    48.8 %   3004   52.2 %

Result : 1536.5/3000 (+754,=1565,-681)
 
Elo  +8


     Program            Elo    +   -   Games   Score   Av.Op.  Draws
  --------------------------------------------------------------------
   1 Stockfish 5 T1   : 3002   9   9   3000    50.5 %   2998   52.1 %
   2 Komodo 8 T1 TB5  : 2998   9   9   3000    49.5 %   3002   52.1 %

Result : 1516.5/3000 (+735,=1563,-702)

Elo  +4                   TB benefit: 4 Elo

-------------------------------------------------------------------- 

  
TC  = 16'+9.6"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------
  1 Komodo 8 T1      : 3004   11  11  1550    51.3 %   2996   60.4 %
  2 Stockfish 5 T1   : 2996   11  11  1550    48.7 %   3004   60.4 %

Result : 795.0/1550 (+327,=936,-287)

Elo  +9


    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 --------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3006   11  11  1550    51.7 %   2994   59.2 %
  2 Stockfish 5 T1   : 2994   11  11  1550    48.3 %   3006   59.2 %

Result : 802.0/1550 (+343,=918,-289)

Elo  +12                   TB benefit: 3 Elo

The use of Syzygy tablebases seems only marginally increasing the playing strength of Komodo 8 against Stockfish 5, in average 4 Elo. The gain decreases with increasing time limit.
But due to the error bars, even with this large number of games, this statement is not necessarily cogent, even if the trend suggests this.

Once again Komodo shows a increase in strenght with increasing time limit.

Next is a test with the upcoming Stockfish 6, then the test Komodo 8 against Stockfish 6.

Andreas

shrapnel
Posts: 1248
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Komodo 8 - 5-men Syzygy tablebases

Post by shrapnel » Thu Jan 22, 2015 3:33 pm

cc2150dx wrote:I mainly use 5 men syzygy bases to determine the wins/loss/draws quicker.

And, off-topic I use the bases on a regular HDD with Houdini, SF, K with no slowdowns.


Jason
You may not notice it, but there will DEFINITELY be slowdowns if you don't use an SSD.
SSDs are so much faster than HDDs....almost unbelievable the difference it makes.
And off-topic, I recently wanted to add a large-capacity HDD to my System, but the thought of installing a large, clunky HDD so put me off that I decided to go with a large capacity SSD instead, even though it cost me more.
No regrets.
Believe me, all your programs ( not just chess) will run so much faster with an SSD that it will make you wonder how you ever put up with HDDs !
On second thoughts, DON'T upgrade to SSD ! :lol: Once you do, there is no going back to HDDs !
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Norm Pollock
Posts: 1017
Joined: Thu Mar 09, 2006 3:15 pm
Location: Long Island, NY, USA
Contact:

Re: Komodo 8 - 5-men Syzygy tablebases

Post by Norm Pollock » Fri Jan 23, 2015 1:22 pm

In between no TBs and SSD, are usb 2.0 and usb 3.0 flash drives. Does anyone know how they rate compared to an SSD?

syzygy
Posts: 4458
Joined: Tue Feb 28, 2012 10:56 pm

Re: Komodo 8 - 5-men Syzygy tablebases

Post by syzygy » Sat Jan 24, 2015 12:39 am

cc2150dx wrote:And, off-topic I use the bases on a regular HDD with Houdini, SF, K with no slowdowns.
Since the usual ignoramus felt a need to contest what you wrote here, I'll just say that 5-piece TBs on a regular HDD indeed won't cause a slowdown on a modern PC as the data will be cached in RAM (ignoring the first seconds of TB access after a reboot of the PC).

User avatar
lucasart
Posts: 3046
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: Komodo 8 - 5-men Syzygy tablebases

Post by lucasart » Sat Jan 24, 2015 4:37 am

fastgm wrote:What are the benefits of using 5-men Syzygy tablebases?

Here a test with Komodo 8 on one core with 128 MB Hash.
The tablebases were on a solid-state drive - Samsung 840 EVO.
Four different time controls were tested, each 3000 games or 1520 at highest time control.
15 sec + 0.15 sec / 60 sec + 0.60 sec / 240 sec + 2.40 sec / 960 sec + 9.60 sec

Code: Select all

TC  = 15"+0.15"
TB5 = 5-men Syzygy tablebases
T1  = 1 thread
 
    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3005   8   8   3000    51.4 %   2995   60.8 %
  2 Komodo 8 T1      : 2995   8   8   3000    48.6 %   3005   60.8 %

Result : 1542.5/3000 (+630,=1825,-545)
 
Elo  +10 
 
TC  = 60"+0.6"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3009   7   7   3000    52.5 %   2991   68.2 %
  2 Komodo 8 T1      : 2991   7   7   3000    47.5 %   3009   68.2 %

Result : 1576.0/3000 (+553,=2046,-401)
 
Elo  +18 
 
TC  = 4'+2.4"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3006   6   6   3000    51.7 %   2994   75.4 %
  2 Komodo 8 T1      : 2994   6   6   3000    48.3 %   3006   75.4 %

Result : 1551.5/3000 (+420,=2263,-317)
 
Elo  +12
  
TC  = 16'+9.6"

    Program            Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------
  1 Komodo 8 T1 TB5  : 3006   8   8   1520    51.7 %   2994   79.8 %
  2 Komodo 8 T1      : 2994   8   8   1520    48.3 %   3006   79.8 %

Result : 785.5/1520 (+179,=1213,-128)
 
Elo  +12
This is very much in line with the result I measured myself a while ago, which was +10 elo in 15"+0.15". However, I never had resources and patience to do longer tests.

It is interesting to note that the elo gain does not shrink at longer tc, even at the very long 16'+9.6". Normally any elo gain tends to shrink at longer tc, as draw rate increases, so anything that does not shrink at this very long tc is remarkable.

I wish Marco Costalba could read this, and stop claiming that syzygy TB have zero elo impact.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

Jouni
Posts: 2029
Joined: Wed Mar 08, 2006 7:15 pm

Re: Komodo 8 - 5-men Syzygy tablebases

Post by Jouni » Sat Jan 24, 2015 9:00 am

So we get +4 ELO from 5 piece set. IPON test gave +6 for Stockfish. Isn't these very modest gain from 960 MB of data?? There has been +5 gain from 10 parameters change in Stockfish testing.
Jouni

Post Reply