Elo points gain from doubling time

[hgm]

WinBoard does inform engines of the name of their opponent and, when known, of their rating. (I.e. when the opponent has an ICS rating.)

I knew this post was coming.

Well, you asked for it... Ratings when not on an ICS is a bigger problem. I don't see how a GUI could know that. Often the testers will not know it (e.g. for new versions, they test for the first time). Of course you could let the user specify a rating, when he installs the engine, through a GUI option. The GUI could then notify the engine of its own and the opponent rating. Of course we could also have the engine announce its own rating. But that would not really help introduction of the feature, as initially no engine would support it. Plus that it invites cheating.

Of course you have the problem that Komodo is UCI, and that UCI is a dead protocol, carved in stone. But if you introduce an option for setting the opponent Elo (e.g. option name UCI_OppoRating type spin default XXX) where the convention is that XXX is your own Elo, I could let Polyglot translate the ratings WinBoard sends it to a "setoption name OppoRating value YYY" where YYY is computed as XXX plus the difference of the GUI/ICS ratings, so that you ould know if you are up against a stronger or weaker opponent.

Like you remarked, someone has to make the first step. Well, it turns out that the first step was already made, on the GUI side, and it is just a matter of the engines making a second step to get it operational. Crafty already bases its contempt factor on opponent rating for decades...

[lkaufman]

I played a gauntlet with Komodo 5 at different time/move vs Houdini 3 at 1s/move. This took some time because the time controls are not very short, and I only saw such tests performed at ultra-short fixed time or fixed depth controls (by Don and Adam).

Code: Select all

    Program                            Games    Elo 

  1 Komodo 5 4s                    :   2000    3131
  2 Komodo 5 2s                    :   2000    3050
  3 Komodo 5 1s                    :   4016    2957
  4 Komodo 5 0.5s                  :   4017    2850

  5 Houdini 3 1s                   :  12033    3036

The scaling of Komodo 5 with doubling time is:

Code: Select all

From 2s/move to 4s/move (blitz)     +81 Elo points
From 1s/move to 2s/move             +93 Elo points
From 0.5s/move to 1s/move (bullet) +107 Elo points

The fit is:
107*(0.87)^{log2(time in seconds per move)} = 107*(time in seconds per move)^(-0.20) Elo points per doubling time (or cores, assuming perfect scaling).

Extrapolating to longer time controls, for 120min/40 moves on one core it gives 107*180^(-0.20) ~ 40 Elo points per doubling time. On eight cores for 120min/40moves LTC it's ~30 Elo points per doubling time. Of course, this is an extrapolation.

Further speculation: to the infinite time control, the improvement from 1s/move is 107/(1-0.87) ~ 820 Elo points, so that Komodo 5 is limited by something like 4000 Elo points strength (calibrated to the current lists) at infinite time control.

I think the formula 107*(time per move in seconds)^(-0.20) Elo points is useful as a rule of thumb for gain from doubling time. This is on one modern core, on several cores time should be multiplied by #cores.

Kai

Based on a study of the public blitz rating lists, I concluded that at that level the average value of a doubling for all Komodo versions was 90 elo. This is at a 6" average level (40/4'). Your results are lower than this but not drastically so. Part of the difference may be because as each Komodo version gets stronger than the previous one, the doubling value tends to decline as it is effectively moving up the curve to higher time limits. Put another way, the average Komodo on the lists may play about like Komodo 5 at the 3" level, for which you show 81 elo, only 9 less.

What you say seems in line to confirm results, as the cores on which I test are modern i7 3.5GHz, which are twice as fast as those cores in the lists like 40/4', so effectively their 6'' are equivalent to my 3''. Yes, 80-90 points at their 6'' (mine 3'') is very plausible. I was curious about extrapolation to 120min/40moves, my extrapolation gives some 40 points gain per doubling on one core, 30 points on 8 cores at this long time control. Do you have empirical data on this TC to confirm the prediction?

Kai

No data at 40/2, but quite good data at the CCRL time limit of 40/40', exactly ten times slower than their blitz level. My method is simply to compare the ratings for Komodo 64 bit with Komodo 32 bit, since with every version the 64 bit was identical but ran almost exactly twice as fast, perfect for this study! There are seven Komodo versions with 40/40 ratings for both 64 bit and 32 bit (eight if you count the predecessor Doch), and the average elo difference was 71 (72 counting Doch). Moreover there was surprisingly little spread, every value came out in the range 62 to 82! What does your formula predict for this, allowing for the hardware adjustment and also bearing in mind that the 32 bit data corresponds to half the stated time control? If you want to get real fancy, you can try to allow for the progression in elo of the Komodo versions; the 64 bit values (starting with the Doch value) were 2888, 2950, 2981, 3015, 3098, 3142, 3148, and 3158.
So in round numbers, my conclusion is that going from 3" to 6" per move on old hardware (so 1.5" to 3" on your hardware) was worth 90 elo for the average Komodo version, and going from 30" to 60" on old hardware (15" to 30" on your hardware) was worth 70 elo.

[Laskos]

No data at 40/2, but quite good data at the CCRL time limit of 40/40', exactly ten times slower than their blitz level. My method is simply to compare the ratings for Komodo 64 bit with Komodo 32 bit, since with every version the 64 bit was identical but ran almost exactly twice as fast, perfect for this study! There are seven Komodo versions with 40/40 ratings for both 64 bit and 32 bit (eight if you count the predecessor Doch), and the average elo difference was 71 (72 counting Doch). Moreover there was surprisingly little spread, every value came out in the range 62 to 82! What does your formula predict for this, allowing for the hardware adjustment and also bearing in mind that the 32 bit data corresponds to half the stated time control? If you want to get real fancy, you can try to allow for the progression in elo of the Komodo versions; the 64 bit values (starting with the Doch value) were 2888, 2950, 2981, 3015, 3098, 3142, 3148, and 3158.
So in round numbers, my conclusion is that going from 3" to 6" per move on old hardware (so 1.5" to 3" on your hardware) was worth 90 elo for the average Komodo version, and going from 30" to 60" on old hardware (15" to 30" on your hardware) was worth 70 elo.

So, I have to account for my hardware being 2 times faster, and 32 bit is 2 times slower, therefore for a total factor of 4 compared to 6'' and 60'' CCRL respectively, correct? In this case my rule of thumb formula gives 98 and 62 Elo points respectively. Pretty close to your empirical 90 and 70, a bit steeper slope in my case, but well within our error margins.

[lkaufman]

No data at 40/2, but quite good data at the CCRL time limit of 40/40', exactly ten times slower than their blitz level. My method is simply to compare the ratings for Komodo 64 bit with Komodo 32 bit, since with every version the 64 bit was identical but ran almost exactly twice as fast, perfect for this study! There are seven Komodo versions with 40/40 ratings for both 64 bit and 32 bit (eight if you count the predecessor Doch), and the average elo difference was 71 (72 counting Doch). Moreover there was surprisingly little spread, every value came out in the range 62 to 82! What does your formula predict for this, allowing for the hardware adjustment and also bearing in mind that the 32 bit data corresponds to half the stated time control? If you want to get real fancy, you can try to allow for the progression in elo of the Komodo versions; the 64 bit values (starting with the Doch value) were 2888, 2950, 2981, 3015, 3098, 3142, 3148, and 3158.
So in round numbers, my conclusion is that going from 3" to 6" per move on old hardware (so 1.5" to 3" on your hardware) was worth 90 elo for the average Komodo version, and going from 30" to 60" on old hardware (15" to 30" on your hardware) was worth 70 elo.

So, I have to account for my hardware being 2 times faster, and 32 bit is 2 times slower, therefore for a total factor of 4 compared to 6'' and 60'' CCRL respectively, correct? In this case my rule of thumb formula gives 98 and 62 Elo points respectively. Pretty close to your empirical 90 and 70, a bit steeper slope in my case, but well within our error margins.

Assuming that your formula is based on the time before the doubling, that is correct. So yes, we agree reasonably well. If we split the difference and round off, we can say that doubling for Komodo is worth 95 at bullet speeds on good hardware and 65 at 15" per move on good hardware.

[Adam Hair]

BTW in CSS forum is search depth test with Houdini 3. Result is weird:

Code: Select all

Tiefe   Elodiff.   Ergebnis
        
1  -  2   222    217,5-782,5  (+82=271-647)
2  -  3   185    256-744      (+75=362-563)
3  -  4   151    295,5-704,5  (+67=457-476)
4  -  5   128    323,5-676,5  (+69=509-422)
5  -  6   122    331,5-668,5  (+69=525-406)
6  -  7   107    350,5-649,5  (+72=557-371)
7  -  8   120    333,5-666,5  (+54=559-387)
8  -  9   175    268-732      (+28=480-492)
9  -  10  123    330,5-669,5  (+39=583-378)
10 -  11   92    370-630      (+29=682-289)
11 -  12   75    393,5-606,5  (+49=689-220)
12 -  13   63    410,5-589,5  (+41=739-220)
13  - 14   63    411-589      (+28=766-206)
14  - 15   46    433,5-566,5  (+36=795-169) 

What on earth happens in 8-9 ply match !?!?

I did a similar study with Houdini 2.0c.

Code: Select all

1 Houdini_ply17             : 2363.4  3792 (+1594,=1942,-256), 67.6 %

Houdini_ply15                 : 2000 (+997,=923,- 80), 72.9 %
Houdini_ply16                 : 1792 (+597,=1019,-176), 61.7 %

2 Houdini_ply16             : 2276.7  5792 (+1882,=3034,-876), 58.7 %

Houdini_ply14                 : 2000 (+1032,=895,- 73), 74.0 %
Houdini_ply15                 : 2000 (+674,=1120,-206), 61.7 %
Houdini_ply17                 : 1792 (+176,=1019,-597), 38.3 %

3 Houdini_ply15             : 2190.3  8000 (+2111,=3966,-1923), 51.2 %

Houdini_ply13                 : 2000 (+1110,=812,- 78), 75.8 %
Houdini_ply14                 : 2000 (+715,=1111,-174), 63.5 %
Houdini_ply16                 : 2000 (+206,=1120,-674), 38.3 %
Houdini_ply17                 : 2000 (+ 80,=923,-997), 27.1 %

4 Houdini_ply14             : 2092.2  8000 (+2166,=3832,-2002), 51.0 %

Houdini_ply12                 : 2000 (+1135,=786,- 79), 76.4 %
Houdini_ply13                 : 2000 (+784,=1040,-176), 65.2 %
Houdini_ply15                 : 2000 (+174,=1111,-715), 36.5 %
Houdini_ply16                 : 2000 (+ 73,=895,-1032), 26.0 %

5 Houdini_ply13             : 1988.2  8000 (+2351,=3508,-2141), 51.3 %

Houdini_ply11                 : 2000 (+1291,=663,- 46), 81.1 %
Houdini_ply12                 : 2000 (+806,=993,-201), 65.1 %
Houdini_ply14                 : 2000 (+176,=1040,-784), 34.8 %
Houdini_ply15                 : 2000 (+ 78,=812,-1110), 24.2 %

6 Houdini_ply12             : 1875.8  8000 (+2500,=3354,-2146), 52.2 %

Houdini_ply10                 : 2000 (+1354,=601,- 45), 82.7 %
Houdini_ply11                 : 2000 (+866,=974,-160), 67.7 %
Houdini_ply13                 : 2000 (+201,=993,-806), 34.9 %
Houdini_ply14                 : 2000 (+ 79,=786,-1135), 23.6 %

7 Houdini_ply11             : 1743.0  8000 (+2728,=2951,-2321), 52.5 %

Houdini_ply10                 : 2000 (+968,=888,-144), 70.6 %
Houdini_ply12                 : 2000 (+160,=974,-866), 32.4 %
Houdini_ply13                 : 2000 (+ 46,=663,-1291), 18.9 %
Houdini_ply09                 : 2000 (+1554,=426,- 20), 88.3 %

8 Houdini_ply10             : 1588.6  8000 (+2976,=2539,-2485), 53.1 %

Houdini_ply11                 : 2000 (+144,=888,-968), 29.4 %
Houdini_ply12                 : 2000 (+ 45,=601,-1354), 17.3 %
Houdini_ply08                 : 2000 (+1688,=293,- 19), 91.7 %
Houdini_ply09                 : 2000 (+1099,=757,-144), 73.9 %

9 Houdini_ply09             : 1398.1  8000 (+3097,=2123,-2780), 52.0 %

Houdini_ply10                 : 2000 (+144,=757,-1099), 26.1 %
Houdini_ply11                 : 2000 (+ 20,=426,-1554), 11.7 %
Houdini_ply07                 : 2000 (+1656,=321,- 23), 90.8 %
Houdini_ply08                 : 2000 (+1277,=619,-104), 79.3 %

10 Houdini_ply08             : 1162.9  8000 (+2765,=2054,-3181), 47.4 %

Houdini_ply10                 : 2000 (+ 19,=293,-1688),  8.3 %
Houdini_ply06                 : 2000 (+1564,=393,- 43), 88.0 %
Houdini_ply07                 : 2000 (+1078,=749,-173), 72.6 %
Houdini_ply09                 : 2000 (+104,=619,-1277), 20.7 %

11 Houdini_ply07             : 989.8  8000 (+2878,=2141,-2981), 49.4 %

Houdini_ply05                 : 2000 (+1569,=377,- 54), 87.9 %
Houdini_ply06                 : 2000 (+1113,=694,-193), 73.0 %
Houdini_ply08                 : 2000 (+173,=749,-1078), 27.4 %
Houdini_ply09                 : 2000 (+ 23,=321,-1656),  9.2 %

12 Houdini_ply06             : 814.7  8000 (+2976,=2105,-2919), 50.4 %

Houdini_ply04                 : 2000 (+1582,=359,- 59), 88.1 %
Houdini_ply05                 : 2000 (+1158,=659,-183), 74.4 %
Houdini_ply07                 : 2000 (+193,=694,-1113), 27.0 %
Houdini_ply08                 : 2000 (+ 43,=393,-1564), 12.0 %

13 Houdini_ply05             : 634.7  8000 (+3041,=1984,-2975), 50.4 %

Houdini_ply03                 : 2000 (+1619,=337,- 44), 89.4 %
Houdini_ply04                 : 2000 (+1185,=611,-204), 74.5 %
Houdini_ply06                 : 2000 (+183,=659,-1158), 25.6 %
Houdini_ply07                 : 2000 (+ 54,=377,-1569), 12.1 %

14 Houdini_ply04             : 451.5  8000 (+3296,=1626,-3078), 51.4 %

Houdini_ply02                 : 2000 (+1785,=145,- 70), 92.9 %
Houdini_ply03                 : 2000 (+1248,=511,-241), 75.2 %
Houdini_ply05                 : 2000 (+204,=611,-1185), 25.5 %
Houdini_ply06                 : 2000 (+ 59,=359,-1582), 11.9 %

15 Houdini_ply03             : 256.3  6000 (+1742,=1172,-3086), 38.8 %

Houdini_ply02                 : 2000 (+1457,=324,-219), 81.0 %
Houdini_ply04                 : 2000 (+241,=511,-1248), 24.8 %
Houdini_ply05                 : 2000 (+ 44,=337,-1619), 10.6 %

16 Houdini_ply02             : 0.0  4000 (+289,=469,-3242), 13.1 %

Houdini_ply03                 : 2000 (+219,=324,-1457), 19.1 %
Houdini_ply04                 : 2000 (+ 70,=145,-1785),  7.1 %

Code: Select all

Ply X+1 – Ply X	Elo Diff
3 – 2	          256.3
4 – 3	          195.2
5 – 4	          183.2
6 – 5	          180.0
7 – 6	          175.1
8 – 7	          173.1
9 – 8	          235.2
10 – 9	         190.5
11 – 10	        154.4
12 – 11	        132.8
13 - 12	        112.4
14 – 13	        104.0
15 – 14	         98.1
16 – 15	         86.4
17 - 16	         86.7

[Adam Hair]

It was Houdini 3 ply 14 vs ply 15, ultra-bullet, 80% draws are impossible. The highest I seen at very long time controls is 73% or so.

Correct, with Houdini 3 this is nearly impossible - it suggests that the opening positions for the test are not well chosen.
Even at long time control Houdini will have close to 50% decided games. See for example the 90 min+30 sec/move tests I played with the Houdini 3 beta against Houdini 2, Stockfish 2.3.1 and Komodo 5, which over-all was +135 -50 =175.

Robert

In my study with Houdini 2.0c, I had the following draw rates:

ply 16 vs ply 17: 56.9%
ply 15 vs ply 16: 56%
ply 14 vs ply 15: 55.6%

[Adam Hair]

BTW in CSS forum is search depth test with Houdini 3. Result is weird:

Code: Select all

Tiefe   Elodiff.   Ergebnis
        
1  -  2   222    217,5-782,5  (+82=271-647)
2  -  3   185    256-744      (+75=362-563)
3  -  4   151    295,5-704,5  (+67=457-476)
4  -  5   128    323,5-676,5  (+69=509-422)
5  -  6   122    331,5-668,5  (+69=525-406)
6  -  7   107    350,5-649,5  (+72=557-371)
7  -  8   120    333,5-666,5  (+54=559-387)
8  -  9   175    268-732      (+28=480-492)
9  -  10  123    330,5-669,5  (+39=583-378)
10 -  11   92    370-630      (+29=682-289)
11 -  12   75    393,5-606,5  (+49=689-220)
12 -  13   63    410,5-589,5  (+41=739-220)
13  - 14   63    411-589      (+28=766-206)
14  - 15   46    433,5-566,5  (+36=795-169) 

What on earth happens in 8-9 ply match !?!?

Very simple -- Singular Extension kicks in at 9 ply on all Rybka/Ippolit derived engines, as well as on many others.

Just as a confirmation to your assertion, I found the same sort of anomaly at ply 9 in my study with Houdini 2.0c.

[Houdini]

BTW in CSS forum is search depth test with Houdini 3. Result is weird:

What on earth happens in 8-9 ply match !?!?

Very simple -- Singular Extension kicks in at 9 ply on all Rybka/Ippolit derived engines, as well as on many others.

Just as a confirmation to your assertion, I found the same sort of anomaly at ply 9 in my study with Houdini 2.0c.

There is indeed a discontinuity in Houdini's search at depth 9, but the main reason is not singular extensions.
There's a second, smaller discontinuity at depth 10 (for yet another reason), so in this kind of analysis the jumps from 8 to 9 and from 9 to 10 should be discarded.

Robert

[lkaufman]

BTW in CSS forum is search depth test with Houdini 3. Result is weird:

What on earth happens in 8-9 ply match !?!?

Very simple -- Singular Extension kicks in at 9 ply on all Rybka/Ippolit derived engines, as well as on many others.

Just as a confirmation to your assertion, I found the same sort of anomaly at ply 9 in my study with Houdini 2.0c.

There is indeed a discontinuity in Houdini's search at depth 9, but the main reason is not singular extensions.
There's a second, smaller discontinuity at depth 10 (for yet another reason), so in this kind of analysis the jumps from 8 to 9 and from 9 to 10 should be discarded.

Robert

Does this comment refer only to Houdini 3, or to Houdini 2 and 3, or to 1.5, 2, and 3?

[lkaufman]

BTW in CSS forum is search depth test with Houdini 3. Result is weird:

Code: Select all

Tiefe   Elodiff.   Ergebnis
        
1  -  2   222    217,5-782,5  (+82=271-647)
2  -  3   185    256-744      (+75=362-563)
3  -  4   151    295,5-704,5  (+67=457-476)
4  -  5   128    323,5-676,5  (+69=509-422)
5  -  6   122    331,5-668,5  (+69=525-406)
6  -  7   107    350,5-649,5  (+72=557-371)
7  -  8   120    333,5-666,5  (+54=559-387)
8  -  9   175    268-732      (+28=480-492)
9  -  10  123    330,5-669,5  (+39=583-378)
10 -  11   92    370-630      (+29=682-289)
11 -  12   75    393,5-606,5  (+49=689-220)
12 -  13   63    410,5-589,5  (+41=739-220)
13  - 14   63    411-589      (+28=766-206)
14  - 15   46    433,5-566,5  (+36=795-169) 

What on earth happens in 8-9 ply match !?!?

I did a similar study with Houdini 2.0c.

Code: Select all

1 Houdini_ply17             : 2363.4  3792 (+1594,=1942,-256), 67.6 %

Houdini_ply15                 : 2000 (+997,=923,- 80), 72.9 %
Houdini_ply16                 : 1792 (+597,=1019,-176), 61.7 %

2 Houdini_ply16             : 2276.7  5792 (+1882,=3034,-876), 58.7 %

Houdini_ply14                 : 2000 (+1032,=895,- 73), 74.0 %
Houdini_ply15                 : 2000 (+674,=1120,-206), 61.7 %
Houdini_ply17                 : 1792 (+176,=1019,-597), 38.3 %

3 Houdini_ply15             : 2190.3  8000 (+2111,=3966,-1923), 51.2 %

Houdini_ply13                 : 2000 (+1110,=812,- 78), 75.8 %
Houdini_ply14                 : 2000 (+715,=1111,-174), 63.5 %
Houdini_ply16                 : 2000 (+206,=1120,-674), 38.3 %
Houdini_ply17                 : 2000 (+ 80,=923,-997), 27.1 %

4 Houdini_ply14             : 2092.2  8000 (+2166,=3832,-2002), 51.0 %

Houdini_ply12                 : 2000 (+1135,=786,- 79), 76.4 %
Houdini_ply13                 : 2000 (+784,=1040,-176), 65.2 %
Houdini_ply15                 : 2000 (+174,=1111,-715), 36.5 %
Houdini_ply16                 : 2000 (+ 73,=895,-1032), 26.0 %

5 Houdini_ply13             : 1988.2  8000 (+2351,=3508,-2141), 51.3 %

Houdini_ply11                 : 2000 (+1291,=663,- 46), 81.1 %
Houdini_ply12                 : 2000 (+806,=993,-201), 65.1 %
Houdini_ply14                 : 2000 (+176,=1040,-784), 34.8 %
Houdini_ply15                 : 2000 (+ 78,=812,-1110), 24.2 %

6 Houdini_ply12             : 1875.8  8000 (+2500,=3354,-2146), 52.2 %

Houdini_ply10                 : 2000 (+1354,=601,- 45), 82.7 %
Houdini_ply11                 : 2000 (+866,=974,-160), 67.7 %
Houdini_ply13                 : 2000 (+201,=993,-806), 34.9 %
Houdini_ply14                 : 2000 (+ 79,=786,-1135), 23.6 %

7 Houdini_ply11             : 1743.0  8000 (+2728,=2951,-2321), 52.5 %

Houdini_ply10                 : 2000 (+968,=888,-144), 70.6 %
Houdini_ply12                 : 2000 (+160,=974,-866), 32.4 %
Houdini_ply13                 : 2000 (+ 46,=663,-1291), 18.9 %
Houdini_ply09                 : 2000 (+1554,=426,- 20), 88.3 %

8 Houdini_ply10             : 1588.6  8000 (+2976,=2539,-2485), 53.1 %

Houdini_ply11                 : 2000 (+144,=888,-968), 29.4 %
Houdini_ply12                 : 2000 (+ 45,=601,-1354), 17.3 %
Houdini_ply08                 : 2000 (+1688,=293,- 19), 91.7 %
Houdini_ply09                 : 2000 (+1099,=757,-144), 73.9 %

9 Houdini_ply09             : 1398.1  8000 (+3097,=2123,-2780), 52.0 %

Houdini_ply10                 : 2000 (+144,=757,-1099), 26.1 %
Houdini_ply11                 : 2000 (+ 20,=426,-1554), 11.7 %
Houdini_ply07                 : 2000 (+1656,=321,- 23), 90.8 %
Houdini_ply08                 : 2000 (+1277,=619,-104), 79.3 %

10 Houdini_ply08             : 1162.9  8000 (+2765,=2054,-3181), 47.4 %

Houdini_ply10                 : 2000 (+ 19,=293,-1688),  8.3 %
Houdini_ply06                 : 2000 (+1564,=393,- 43), 88.0 %
Houdini_ply07                 : 2000 (+1078,=749,-173), 72.6 %
Houdini_ply09                 : 2000 (+104,=619,-1277), 20.7 %

11 Houdini_ply07             : 989.8  8000 (+2878,=2141,-2981), 49.4 %

Houdini_ply05                 : 2000 (+1569,=377,- 54), 87.9 %
Houdini_ply06                 : 2000 (+1113,=694,-193), 73.0 %
Houdini_ply08                 : 2000 (+173,=749,-1078), 27.4 %
Houdini_ply09                 : 2000 (+ 23,=321,-1656),  9.2 %

12 Houdini_ply06             : 814.7  8000 (+2976,=2105,-2919), 50.4 %

Houdini_ply04                 : 2000 (+1582,=359,- 59), 88.1 %
Houdini_ply05                 : 2000 (+1158,=659,-183), 74.4 %
Houdini_ply07                 : 2000 (+193,=694,-1113), 27.0 %
Houdini_ply08                 : 2000 (+ 43,=393,-1564), 12.0 %

13 Houdini_ply05             : 634.7  8000 (+3041,=1984,-2975), 50.4 %

Houdini_ply03                 : 2000 (+1619,=337,- 44), 89.4 %
Houdini_ply04                 : 2000 (+1185,=611,-204), 74.5 %
Houdini_ply06                 : 2000 (+183,=659,-1158), 25.6 %
Houdini_ply07                 : 2000 (+ 54,=377,-1569), 12.1 %

14 Houdini_ply04             : 451.5  8000 (+3296,=1626,-3078), 51.4 %

Houdini_ply02                 : 2000 (+1785,=145,- 70), 92.9 %
Houdini_ply03                 : 2000 (+1248,=511,-241), 75.2 %
Houdini_ply05                 : 2000 (+204,=611,-1185), 25.5 %
Houdini_ply06                 : 2000 (+ 59,=359,-1582), 11.9 %

15 Houdini_ply03             : 256.3  6000 (+1742,=1172,-3086), 38.8 %

Houdini_ply02                 : 2000 (+1457,=324,-219), 81.0 %
Houdini_ply04                 : 2000 (+241,=511,-1248), 24.8 %
Houdini_ply05                 : 2000 (+ 44,=337,-1619), 10.6 %

16 Houdini_ply02             : 0.0  4000 (+289,=469,-3242), 13.1 %

Houdini_ply03                 : 2000 (+219,=324,-1457), 19.1 %
Houdini_ply04                 : 2000 (+ 70,=145,-1785),  7.1 %

Code: Select all

Ply X+1 – Ply X	Elo Diff
3 – 2	          256.3
4 – 3	          195.2
5 – 4	          183.2
6 – 5	          180.0
7 – 6	          175.1
8 – 7	          173.1
9 – 8	          235.2
10 – 9	         190.5
11 – 10	        154.4
12 – 11	        132.8
13 - 12	        112.4
14 – 13	        104.0
15 – 14	         98.1
16 – 15	         86.4
17 - 16	         86.7

You show much higher numbers for each extra ply of Houdini 2 than the above quoted study showed for Houdini 3. This would suggest that Houdini 2 scales far better than Houdini 3, but clearly this is not true. Any idea how to reconcile these contradictory results?