Self-play at different time controls

[Adam Hair]

Code: Select all

   # PLAYER                       : RATING    POINTS  PLAYED    (%)
   1 SF 141130 1'+1"              :   28.5     332.0     600   55.3%
   2 SF 141130 2'+0               :    6.7     307.5     600   51.2%
   3 SF 141130 40/1'              :   -8.0     291.0     600   48.5%
   4 SF 141130 2" per move        :  -27.1     269.5     600   44.9%

Each time control played 200 games (reverse colors) against the other time controls. Not many games, but enough to determine which produces the higher quality of play.

For reference, the computer used is 25% faster than the CCRL reference computer.

Next, Komodo 8.

[Ferdy]

Nice TC tests. Do you have a record of average depth?

[Adam Hair]

I will see if I can extract that information from the pgn. If you feel like doing it, here is the pgn:
https://www.mediafire.com/?fhiodqcv360pstp

[Ferdy]

Using Protools by Ed.

Code: Select all

Engine                 Depth       Time   Games     Moves  Average Forfeit
SF 141130 1+1          22.88   19:57:32     600     35336    2.03     0
SF 141130 2sec_move    24.18   19:48:40     600     35734    2.00     0
SF 141130 40/1         23.67   17:16:46     600     35346    1.76     0
SF 141130 2+0          20.92   16:29:08     600     35201    1.69     0


Time control comparison between engines

Depth     : Average search depth
Time      : Total time engine used
Moves     : Total moves engine played
Average   : Average time per move in centi-seconds
Forfeit   : Games engine lost due to time forfeit

List is sorted on Average Time indicating the engine that uses the most time tops.

[Laskos]

Using Protools by Ed.

Code: Select all

Engine                 Depth       Time   Games     Moves  Average Forfeit
SF 141130 1+1          22.88   19:57:32     600     35336    2.03     0
SF 141130 2sec_move    24.18   19:48:40     600     35734    2.00     0
SF 141130 40/1         23.67   17:16:46     600     35346    1.76     0
SF 141130 2+0          20.92   16:29:08     600     35201    1.69     0


Time control comparison between engines

Depth     : Average search depth
Time      : Total time engine used
Moves     : Total moves engine played
Average   : Average time per move in centi-seconds
Forfeit   : Games engine lost due to time forfeit

List is sorted on Average Time indicating the engine that uses the most time tops.

Thank you and Adam for this info. So, the best use of time for SF seems to be a close match between 1' + 1'' and 2' + 0''. The worse clearly 2''/move.

The average depth is a bit tricky, in the endgame depth and depth differences are enhanced, while not improving much the quality of games and averaging will give profit to 2''/move over 2' + 0'' and 1' + 1''.

Strength and Time seem to be the most relevant here.

[Ferdy]

Using Protools by Ed.

Code: Select all

Engine                 Depth       Time   Games     Moves  Average Forfeit
SF 141130 1+1          22.88   19:57:32     600     35336    2.03     0
SF 141130 2sec_move    24.18   19:48:40     600     35734    2.00     0
SF 141130 40/1         23.67   17:16:46     600     35346    1.76     0
SF 141130 2+0          20.92   16:29:08     600     35201    1.69     0


Time control comparison between engines

Depth     : Average search depth
Time      : Total time engine used
Moves     : Total moves engine played
Average   : Average time per move in centi-seconds
Forfeit   : Games engine lost due to time forfeit

List is sorted on Average Time indicating the engine that uses the most time tops.

Thank you and Adam for this info. So, the best use of time for SF seems to be a close match between 1' + 1'' and 2' + 0''. The worse clearly 2''/move.

The average depth is a bit tricky, in the endgame depth and depth differences are enhanced, while not improving much the quality of games and averaging will give profit to 2''/move over 2' + 0'' and 1' + 1''.

Strength and Time seem to be the most relevant here.

I agree smarter use of time will win here, the 2' could be strong in middle game but will use less time on extended game. The 40/1 could not compete with 1'+1" because of that increment and probably most of the games are already decided close to 40 moves, so the 2nd period of 40/1 is no longer a serious threat. The TC with increment can afford to extend its time a little bit on complicated positions. It can withstand at long games too because of that increment.

[Adam Hair]

Komodo 8

Code: Select all

   # PLAYER                : RATING    POINTS  PLAYED    (%)
   1 Komodo 8 1+1          :   26.7     330.0     600   55.0%
   2 Komodo 8 2+0          :    5.3     306.0     600   51.0%
   3 Komodo 8 40/1         :    2.2     302.5     600   50.4%
   4 Komodo 8 2sec_move    :  -34.3     261.5     600   43.6%

I am trying to figure out a way to extract the time data so that we can how it is distributed through the games.

Next will be Houdini 4 so that Uri's request is fulfilled.

[Laskos]

Komodo 8

Code: Select all

   # PLAYER                : RATING    POINTS  PLAYED    (%)
   1 Komodo 8 1+1          :   26.7     330.0     600   55.0%
   2 Komodo 8 2+0          :    5.3     306.0     600   51.0%
   3 Komodo 8 40/1         :    2.2     302.5     600   50.4%
   4 Komodo 8 2sec_move    :  -34.3     261.5     600   43.6%

I am trying to figure out a way to extract the time data so that we can how it is distributed through the games.

Next will be Houdini 4 so that Uri's request is fulfilled.

Very similar to SF. Curious about total time used. SF is tuned on very small increment, much smaller than you use. Your results seem to portray the testing framework, that's why efficiency of SF (as strength with total time used goes) seems to be somewhere between 1+1 or 2+0. Your tests might be important in determining the testing framework.

Also, Larry Kaufman's claims that one shouldn't test Komodo at sudden death, which favors SF, seem not to be grounded.
http://www.talkchess.com/forum/viewtopi ... ht=#598243

[Adam Hair]

Here is the output from ProTools

Code: Select all

Engine                 Depth       Time   Games     Moves  Average Forfeit
Komodo 8 1+1           18.89   19:38:23     600     34775    2.03     0
Komodo 8 2sec_move     20.00   19:14:03     600     34661    2.00     0
Komodo 8 40/1          18.74   16:37:53     600     33885    1.77     0
Komodo 8 2+0           17.59   16:46:38     600     34552    1.75     0


Time control comparison between engines

Depth     : Average search depth
Time      : Total time engine used
Moves     : Total moves engine played
Average   : Average time per move in centi-seconds
Forfeit   : Games engine lost due to time forfeit

List is sorted on Average Time indicating the engine that uses the most time tops.

[Adam Hair]

Houdini 4

Code: Select all

   # PLAYER                 : RATING    POINTS  PLAYED    (%)
   1 Houdini 4 1+1          :   24.5     327.5     600   54.6%
   2 Houdini 4 40/1         :    6.2     307.0     600   51.2%
   3 Houdini 4 2+0          :    3.1     303.5     600   50.6%
   4 Houdini 4 2sec_move    :  -33.9     262.0     600   43.7%

Code: Select all

Engine                 Depth       Time   Games     Moves  Average Forfeit
Houdini 4 2sec_move    19.21   20:05:22     599     36352    1.99     0
Houdini 4 1+1          18.67   18:37:01     599     38594    1.74     0
Houdini 4 40/1         18.90   17:56:07     599     39575    1.63     0
Houdini 4 2+0          17.67   13:59:08     599     39155    1.29     0


Time control comparison between engines

Depth     : Average search depth
Time      : Total time engine used
Moves     : Total moves engine played
Average   : Average time per move in centi-seconds
Forfeit   : Games engine lost due to time forfeit

List is sorted on Average Time indicating the engine that uses the most time tops.