Threads test incl. Komodo 9.3

[fastgm]

After Stockifsh 7, Komodo 9.3 follows in my threads test.

Same test conditions, see http://fastgm.de/threads5.html

Andreas





Comparison between Stockfish 7 and Komodo 9.3

[APassionForCriminalJustic]

After Stockifsh 7, Komodo 9.3 follows in my threads test.

Same test conditions, see http://fastgm.de/threads5.html

Andreas





Comparison between Stockfish 7 and Komodo 9.3

Komodo is still clearly the king with huge threads...

Great stuff.

[Jouni]

Are "diminishing returns" already in effect here? SF7 is 50 points stronger at these fast games, so it cannot gain as much as weaker Komodo? Seems to be quite difficult to draw conclusion: better to leave on Kai or Jesús .

[Ajedrecista]

Hello:

Thank you very much, Andreas! It is much appreciated.

I am not going to write about diminishing returns but I write about fittings of Elo(n threads) - Elo (1 thread) for both SF7 and K9.3 with this data. I fit scores:

Code: Select all

Scores against SF7 (1 thread):

SF7  (2 threads): 1831.0/3000
SF7  (4 threads): 2028.0/3000
SF7  (8 threads): 2214.0/3000
SF7 (16 threads): 2364.5/3000

Score'(threads) = 0.55425 + 0.05955*log2(threads)
R² ~ 0.99644
For threads > 1.
Score' > 1 for threads > 179.

-------------------------------

Scores against K9.3 (1 thread):

K9.3  (2 threads): 1872.5/3000
K9.3  (4 threads): 2205.5/3000
K9.3  (8 threads): 2409.5/3000
K9.3 (16 threads): 2532.0/3000

Score'(threads) ~ 0.62465 + 0.15982*ln[log2(threads)]
R² ~ 0.99950
For threads > 1.
Score' > 1 for threads > 1419.

I include the limits of score' > 1 because these fits can not be generalized for any number of threads. They are only two fits with four data points each one. Please take into account that I did not care about confidence intervals of the scores.

Then, converting into Elo differences against 1 thread and rounding to the nearest integers, I get:

Code: Select all

Threads    Measured (SF7)    SF7    K9.3    Measured (K9.3)

   2             78           80     88           88
   3                         106    146
   4            128          126    178          177
   5                         141    200
   6                         154    216
   7                         165    230
   8            180          175    241          244
   9                         184    251
  10                         193    259
  11                         200    267
  12                         208    274
  13                         214    280
  14                         221    286
  15                         227    291
  16            228          233    296          293
  17                         238    301
  18                         244    305
  19                         249    309
  20                         254    313
  21                         259    317
  22                         263    321
  23                         268    324
  24                         272    327
  25                         276    330
  26                         281    333
  27                         285    336
  28                         289    339
  29                         293    342
  30                         297    344
  31                         300    347
  32                         304    349

I hope no typos. It would be great that other tests would be made with 3, 5, 6... threads just for seeing the goodness (or not) of those fits.

Regards from Spain.

Ajedrecista.

[Laskos]

Jesus, something with the fits is fishy, at least for SF7. Did you fit using the same form of function for SF and K? For SF you seem to have increasing returns for doubling, not diminishing.

[Ajedrecista]

Hello Kai:

Jesus, something with the fits is fishy, at least for SF7. Did you fit using the same form of function for SF and K? For SF you seem to have increasing returns for doubling, not diminishing.

As you see, fits use different forms (one logarithm versus two). I simply looked for high R² fitting scores (not Elo) and that is all. You can be sure it is fishy, hocus-pocus or whatever you want to call it for threads > 16... but they maybe stand well as interpolations between 2 and 16 threads. Extrapolation is other world. I hope I could be proven right with threads < 17, just a wish. Thanks for your feedback!

As I side note, Andreas' results on SF7 show the same Elo gains from 2 to 4, 4 to 8 and 8 to 16 threads more less. Since I fitted scores and the derivative of logit function of Elo is quite high when score --> 1-, crazy things (big Elo jumps) can happen with score --> 1-. Just my opinion.

You surely have better fits that make more sense and want to share them. I stay tuned.

Regards from Spain.

Ajedrecista.