After Stockifsh 7, Komodo 9.3 follows in my threads test.
Same test conditions, see http://fastgm.de/threads5.html
Andreas
Comparison between Stockfish 7 and Komodo 9.3
Threads test incl. Komodo 9.3
Moderators: hgm, Rebel, chrisw
-
- Posts: 417
- Joined: Sat May 24, 2014 9:16 am
Re: Threads test incl. Komodo 9.3
Komodo is still clearly the king with huge threads...fastgm wrote:After Stockifsh 7, Komodo 9.3 follows in my threads test.
Same test conditions, see http://fastgm.de/threads5.html
Andreas
Comparison between Stockfish 7 and Komodo 9.3
Great stuff.
-
- Posts: 3285
- Joined: Wed Mar 08, 2006 8:15 pm
Re: Threads test incl. Komodo 9.3
Are "diminishing returns" already in effect here? SF7 is 50 points stronger at these fast games, so it cannot gain as much as weaker Komodo? Seems to be quite difficult to draw conclusion: better to leave on Kai or Jesús .
Jouni
-
- Posts: 1968
- Joined: Wed Jul 13, 2011 9:04 pm
- Location: Madrid, Spain.
Re: Threads test including Komodo 9.3.
Hello:
Thank you very much, Andreas! It is much appreciated.
I am not going to write about diminishing returns but I write about fittings of Elo(n threads) - Elo (1 thread) for both SF7 and K9.3 with this data. I fit scores:
I include the limits of score' > 1 because these fits can not be generalized for any number of threads. They are only two fits with four data points each one. Please take into account that I did not care about confidence intervals of the scores.
Then, converting into Elo differences against 1 thread and rounding to the nearest integers, I get:
I hope no typos. It would be great that other tests would be made with 3, 5, 6... threads just for seeing the goodness (or not) of those fits.
Regards from Spain.
Ajedrecista.
Thank you very much, Andreas! It is much appreciated.
I am not going to write about diminishing returns but I write about fittings of Elo(n threads) - Elo (1 thread) for both SF7 and K9.3 with this data. I fit scores:
Code: Select all
Scores against SF7 (1 thread):
SF7 (2 threads): 1831.0/3000
SF7 (4 threads): 2028.0/3000
SF7 (8 threads): 2214.0/3000
SF7 (16 threads): 2364.5/3000
Score'(threads) = 0.55425 + 0.05955*log2(threads)
R² ~ 0.99644
For threads > 1.
Score' > 1 for threads > 179.
-------------------------------
Scores against K9.3 (1 thread):
K9.3 (2 threads): 1872.5/3000
K9.3 (4 threads): 2205.5/3000
K9.3 (8 threads): 2409.5/3000
K9.3 (16 threads): 2532.0/3000
Score'(threads) ~ 0.62465 + 0.15982*ln[log2(threads)]
R² ~ 0.99950
For threads > 1.
Score' > 1 for threads > 1419.
Then, converting into Elo differences against 1 thread and rounding to the nearest integers, I get:
Code: Select all
Threads Measured (SF7) SF7 K9.3 Measured (K9.3)
2 78 80 88 88
3 106 146
4 128 126 178 177
5 141 200
6 154 216
7 165 230
8 180 175 241 244
9 184 251
10 193 259
11 200 267
12 208 274
13 214 280
14 221 286
15 227 291
16 228 233 296 293
17 238 301
18 244 305
19 249 309
20 254 313
21 259 317
22 263 321
23 268 324
24 272 327
25 276 330
26 281 333
27 285 336
28 289 339
29 293 342
30 297 344
31 300 347
32 304 349
Regards from Spain.
Ajedrecista.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Threads test including Komodo 9.3.
Jesus, something with the fits is fishy, at least for SF7. Did you fit using the same form of function for SF and K? For SF you seem to have increasing returns for doubling, not diminishing.
-
- Posts: 1968
- Joined: Wed Jul 13, 2011 9:04 pm
- Location: Madrid, Spain.
Re: Threads test including Komodo 9.3.
Hello Kai:
As I side note, Andreas' results on SF7 show the same Elo gains from 2 to 4, 4 to 8 and 8 to 16 threads more less. Since I fitted scores and the derivative of logit function of Elo is quite high when score --> 1-, crazy things (big Elo jumps) can happen with score --> 1-. Just my opinion.
You surely have better fits that make more sense and want to share them. I stay tuned.
Regards from Spain.
Ajedrecista.
As you see, fits use different forms (one logarithm versus two). I simply looked for high R² fitting scores (not Elo) and that is all. You can be sure it is fishy, hocus-pocus or whatever you want to call it for threads > 16... but they maybe stand well as interpolations between 2 and 16 threads. Extrapolation is other world. I hope I could be proven right with threads < 17, just a wish. Thanks for your feedback!Laskos wrote:Jesus, something with the fits is fishy, at least for SF7. Did you fit using the same form of function for SF and K? For SF you seem to have increasing returns for doubling, not diminishing.
As I side note, Andreas' results on SF7 show the same Elo gains from 2 to 4, 4 to 8 and 8 to 16 threads more less. Since I fitted scores and the derivative of logit function of Elo is quite high when score --> 1-, crazy things (big Elo jumps) can happen with score --> 1-. Just my opinion.
You surely have better fits that make more sense and want to share them. I stay tuned.
Regards from Spain.
Ajedrecista.