Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

[Vinvin]

I think that the point was to play against the top programs of their time.

Rebel6 was a top program in 1994
Rebel12 was not a top program when it appeared.

Yes, good point !
In 1994-1995, Rebel 6 was on the top of the SSDF list, just after Chess Genius, the number 1.
1994/7 : https://groups.google.com/forum/#!msg/r ... UJwgSmzRoJ
1995/6 : https://groups.google.com/forum/#!topic ... GRnYtzTth4

Thanks !

[Vinvin]

Rebel 6 will be lost 100 times in a row .
This look like not a good idea math between both engine .

It may lose 100 - 0 even with a x3 time advantage. The match is not about the math. Anyone can look at rating numbers. I had a request of how do engines of today play better, from the ones in the past. One fun way to show this is to let them play.

So we are going to step threw time in a series of matches with the older engine always having a x3 time advantage, and both engines playing one core for consistency.

Rebel 6 (1994)
Fritz 6 (2000)
Fruit 2 (2005-7)
Rybka 3 (2008)
Houdini 1.5a (2010-11)

The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit

[Guenther]

Rebel 6 will be lost 100 times in a row .
This look like not a good idea math between both engine .

:shock:

It may lose 100 - 0 even with a x3 time advantage. The match is not about the math. Anyone can look at rating numbers. I had a request of how do engines of today play better, from the ones in the past. One fun way to show this is to let them play.

So we are going to step threw time in a series of matches with the older engine always having a x3 time advantage, and both engines playing one core for consistency.

Rebel 6 (1994)
Fritz 6 (2000)
Fruit 2 (2005-7)
Rybka 3 (2008)
Houdini 1.5a (2010-11)

The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit

Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.

[Vinvin]

Rebel 6 will be lost 100 times in a row .
This look like not a good idea math between both engine .

It may lose 100 - 0 even with a x3 time advantage. The match is not about the math. Anyone can look at rating numbers. I had a request of how do engines of today play better, from the ones in the past. One fun way to show this is to let them play.

So we are going to step threw time in a series of matches with the older engine always having a x3 time advantage, and both engines playing one core for consistency.

Rebel 6 (1994)
Fritz 6 (2000)
Fruit 2 (2005-7)
Rybka 3 (2008)
Houdini 1.5a (2010-11)

The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit

Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.

I followed some games during the live broadcast and everything seems OK to me.
May be @mwyoung can replay this experience ?

[Guenther]

The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit

Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.

I followed some games during the live broadcast and everything seems OK to me.
May be @mwyoung can replay this experience ? :-)

Are we talking about the same games, the CCRL games Houdini 1.5a-64 vs. SF 10-64?
From my experience any perf for more than 40/50 games outside ~170 err level might have a problem
and here it says the diff is 266.
Below is game one. 359 seconds for depth 22 looks already somehow strange in move 14.

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2019.05.06"]
[Round "666.1.622"]
[White "Houdini 1.5a 64-bit"]
[Black "Stockfish 10 64-bit"]
[Result "0-1"]
[WhiteElo "3110"]
[BlackElo "3387"]
[ECO "E00"]
[Opening "Catalan opening"]
[PlyCount "197"]

1. d4 {book} Nf6 {book}
2. c4 {book} e6 {book}
3. g3 {book} c5 {book}
4. Nf3 {book} cxd4 {book}
5. Nxd4 {book} d5 {book}
6. Bg2 {book} e5 {book}
7. Nf3 {book} d4 {book}
8. O-O {book} Nc6 {book}
9. e3 {book} Be7 {book}
10. exd4 {book} exd4 {book}
11. Re1 {+0.00/19 87} O-O {+0.09/25 58}
12. Bf4 {+0.04/19 74} Nh5 {+0.07/29 51}
13. Be5 {+0.06/20 89} Bb4 {+0.00/27 27}
14. Nbd2 {+0.15/22 359} f6 {+0.69/29 21}
15. Bxd4 {+0.23/21 94} Nxd4 {-0.13/35 79}
16. Nxd4 {+0.24/20 0} Qxd4 {+0.10/33 21}
17. Qxh5 {+0.25/22 38} Qxd2 {+0.00/35 69}
18. a3 {+0.25/21 0} Qxe1+ {+0.00/32 40}
19. Rxe1 {+0.27/20 5} Bxe1 {+0.00/34 28}
20. Be4 {+0.21/24 88} h6 {+0.00/35 26}
21. Qd1 {+0.41/21 40} Bxf2+ {+0.00/36 40}
22. Kxf2 {+0.42/19 0} Be6 {+0.00/37 68}
23. Bxb7 {+0.24/20 43} Rab8 {+0.00/34 32}
24. Qe2 {+0.23/23 47} Bf7 {+0.00/35 32}
25. Be4 {+0.36/23 85} Rfd8 {+0.00/35 31}
26. b4 {+0.19/22 54} Rd4 {+0.00/35 38}
27. Bd5 {+0.12/23 141} Bxd5 {+0.00/34 42}
28. cxd5 {+0.13/22 0} Rd8 {+0.00/34 39}
29. Qa6 {-0.03/24 126} Rd2+ {+0.00/37 101}
30. Kf3 {-0.03/24 41} R8xd5 {+0.00/34 32}
31. Qxa7 {-0.03/24 56} Kh7 {+0.00/34 67}
32. Qa6 {-0.03/22 37} h5 {+0.00/32 48}
33. a4 {-0.03/23 54} Rxh2 {+0.00/32 43}
34. Ke4 {+0.03/20 52} Rg5 {+0.00/34 149}
35. Kf3 {-0.02/21 37} h4 {+0.00/35 44}
36. gxh4 {-0.07/23 49} Rh3+ {+0.00/36 70}
37. Kf2 {-0.07/25 54} Rg4 {+0.00/37 53}
38. b5 {-0.07/22 52} Rb3 {+0.00/38 71}
39. Qa8 {-0.02/22 78} Rc4 {+0.14/35 123}
40. Ke2 {-0.07/23 26} Rxh4 {+0.03/40 188}
41. Kf2 {-0.07/24 61} Rh2+ {+1.30/33 72}
42. Kg1 {-0.07/23 6} Rd2 {+0.21/36 89}
43. Qh1+ {-0.07/25 40} Kg6 {+0.00/40 76}
44. Qe4+ {-0.07/23 0} Kg5 {+0.00/41 36}
45. Qe1 {-0.07/25 51} Rdb2 {+0.83/31 33}
46. Qc1+ {-0.07/24 41} Kg6 {+0.87/32 34}
47. Qf4 {-0.07/25 43} Rb4 {+0.77/34 87}
48. Qg3+ {-0.07/23 0} Kh5 {+0.80/36 49}
49. Qh3+ {-0.07/25 57} Kg5 {+0.72/35 65}
50. Qd7 {-0.07/25 74} Kh4 {+0.50/34 62}
51. a5 {-0.18/23 93} Rg4+ {+0.43/34 89}
52. Kf1 {-0.18/22 0} Rc4 {+0.73/35 112}
53. Qd1 {-0.19/21 0} Rxb5 {+0.77/32 23}
54. Qe1+ {-0.07/23 58} Kg5 {+0.84/31 31}
55. Qd2+ {-0.07/24 86} Kg6 {+0.84/35 41}
56. a6 {-0.07/23 57} Ra4 {+0.90/30 24}
57. Qd3+ {-0.13/24 46} Rf5+ {+1.21/32 26}
58. Ke2 {-0.14/26 75} Ra5 {+1.25/33 48}
59. Qc4 {-0.21/25 81} Rfe5+ {+1.23/34 90}
60. Kf2 {-0.30/25 195} Rec5 {+1.25/36 47}
61. Qg4+ {-0.25/24 41} Kh7 {+1.25/35 16}
62. Qe4+ {-0.28/24 45} Rf5+ {+1.30/33 23}
63. Kg1 {-0.29/24 31} g6 {+1.94/32 21}
64. Qh4+ {-0.28/24 61} Kg7 {+2.27/31 24}
65. Qc4 {-0.28/25 31} Rfc5 {+3.14/31 22}
66. Qb3 {-0.35/25 150} Ra1+ {+3.65/34 22}
67. Kf2 {-1.07/25 54} Rc7 {+4.12/36 37}
68. Qb6 {-1.07/24 1} Re7 {+4.37/35 31}
69. Kf3 {-1.07/26 33} Ra3+ {+4.97/33 46}
70. Kg2 {-1.16/27 70} Ra2+ {+5.10/35 41}
71. Kg3 {-1.15/25 90} Ra4 {+3.81/38 117}
72. Kg2 {-1.07/25 30} Rae4 {+5.16/35 39}
73. Qb8 {-1.15/26 28} Ra4 {+5.20/38 29}
74. Qb6 {-0.07/25 0} Ra2+ {+5.22/37 22}
75. Kg3 {-0.07/34 27} g5 {+5.43/39 91}
76. Kf3 {-0.97/24 22} Ra3+ {+5.77/32 23}
77. Kf2 {-0.97/27 37} Rf7 {+6.07/36 89}
78. Ke2 {-0.97/25 33} Rd7 {+6.27/36 99}
79. Kf2 {-0.97/25 38} Rd2+ {+6.42/33 26}
80. Ke1 {-0.65/20 3} Rda2 {+6.76/36 106}
81. Qb7+ {-1.73/24 289} Kg6 {+6.76/36 38}
82. Qe4+ {-1.76/24 31} Kh5 {+6.76/40 45}
83. a7 {-1.77/24 64} f5 {+10.88/40 207}
84. Qxf5 {-1.83/23 33} Re3+ {+11.08/33 25}
85. Kd1 {-1.85/24 33} Rxa7 {+11.08/40 39}
86. Kd2 {-1.89/25 52} Rae7 {+11.08/35 41}
87. Qd5 {-1.95/23 36} Re2+ {+11.08/31 45}
88. Kc3 {-3.09/25 252} R7e3+ {+11.18/29 79}
89. Kc4 {-3.18/24 95} Re4+ {+11.27/32 68}
90. Kc3 {-3.33/25 201} Kh4 {+12.09/32 106}
91. Qd8 {-3.33/21 168} Re5 {+12.18/27 20}
92. Kd3 {-3.41/21 29} Kg3 {+12.56/31 25}
93. Qb6 {-3.49/21 23} Kg2 {+73.15/30 52}
94. Qg6 {-3.73/21 19} R2e4 {+152.99/29 34}
95. Qh6 {-4.71/22 138} g4 {+153.06/30 25}
96. Qf6 {-4.88/20 18} g3 {+153.07/33 21}
97.Qd6 {-5.96/20 110} Kg1 {+153.12/37 49}
98. Kc2 {-6.40/17 14} g2 {+153.15/29 30}
99. Kb3 {-9.60/17 80}
0-1

[Vinvin]

The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit

Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.

I followed some games during the live broadcast and everything seems OK to me.
May be @mwyoung can replay this experience ?

Are we talking about the same games, the CCRL games Houdini 1.5a-64 vs. SF 10-64?
From my experience any perf for more than 40/50 games outside ~170 err level might have a problem
and here it says the diff is 266.
Below is game one. 359 seconds for depth 22 looks already somehow strange in move 14.
...

The time control is 40 moves in 40 minutes, so it's not crazy to use 6 minutes for a move in the opening.

I just ran a tournament SF_190726 vs H1.5 and I got a similar results : only 9 draws for H1.5 in 128 games (96.5%-3.5%) :

Code: Select all

Score of Houdini_15_x64 vs stockfish_19072603_x64_modern: 0 - 119 - 9 [0.035]
ELO difference: -575.38 +/- 132.50

128 of 128 games finished.

Interface : Cute Chess
TC : 3m+2s
No EGTB
1 core on i7@4GHz
Opening book : NoomenShortLinesTestsuite.pgn

128 games in the zip file.

[Guenther]

The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit

Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.

I followed some games during the live broadcast and everything seems OK to me.
May be @mwyoung can replay this experience ? :-)

Are we talking about the same games, the CCRL games Houdini 1.5a-64 vs. SF 10-64?
From my experience any perf for more than 40/50 games outside ~170 err level might have a problem
and here it says the diff is 266.
Below is game one. 359 seconds for depth 22 looks already somehow strange in move 14.
...

The time control is 40 moves in 40 minutes, so it's not crazy to use 6 minutes for a move in the opening.

But not with only reaching depth 22. Note that the real TC is 32/40 BTW. (I can deduct this with my stats tools)
I have already 2 draws after two games at 32/40 on my hardware which seems to be similar speed, as the used hardware...

The chance for a perf diff of 266 over 52 games is extremely unlikely...

You should be aware also that I have already found several times anomalies in CCRL matches and all those games had to be repeated
or cancelled. (all of those had err levels >150)

Stats for the first two games for Houdini:
(complete breakdown of depths in game 2 for Houdini in middlegame and exitus - each second or third game has that pattern...)

Code: Select all

game 1
+0.00/19 87;w;0-1
+0.04/19 74;w;0-1
+0.06/20 89;w;0-1
+0.15/22 359;w;0-1
+0.23/21 94;w;0-1
+0.24/20 0;w;0-1
+0.25/22 38;w;0-1
+0.25/21 0;w;0-1
+0.27/20 5;w;0-1
+0.21/24 88;w;0-1
+0.41/21 40;w;0-1
+0.42/19 0;w;0-1
+0.24/20 43;w;0-1
+0.23/23 47;w;0-1
+0.36/23 85;w;0-1
+0.19/22 54;w;0-1
+0.12/23 141;w;0-1
+0.13/22 0;w;0-1
-0.03/24 126;w;0-1
-0.03/24 41;w;0-1
-0.03/24 56;w;0-1
-0.03/22 37;w;0-1
-0.03/23 54;w;0-1
+0.03/20 52;w;0-1
-0.02/21 37;w;0-1
-0.07/23 49;w;0-1
-0.07/25 54;w;0-1
-0.07/22 52;w;0-1
-0.02/22 78;w;0-1
-0.07/23 26;w;0-1
-0.07/24 61;w;0-1
-0.07/23 6;w;0-1
-0.07/25 40;w;0-1
-0.07/23 0;w;0-1
-0.07/25 51;w;0-1
-0.07/24 41;w;0-1
-0.07/25 43;w;0-1
-0.07/23 0;w;0-1
-0.07/25 57;w;0-1
-0.07/25 74;w;0-1
-0.18/23 93;w;0-1
-0.18/22 0;w;0-1
-0.19/21 0;w;0-1
-0.07/23 58;w;0-1
-0.07/24 86;w;0-1
-0.07/23 57;w;0-1
-0.13/24 46;w;0-1
-0.14/26 75;w;0-1
-0.21/25 81;w;0-1
-0.30/25 195;w;0-1
-0.25/24 41;w;0-1
-0.28/24 45;w;0-1
-0.29/24 31;w;0-1
-0.28/24 61;w;0-1
-0.28/25 31;w;0-1
-0.35/25 150;w;0-1
-1.07/25 54;w;0-1
-1.07/24 1;w;0-1
-1.07/26 33;w;0-1
-1.16/27 70;w;0-1
-1.15/25 90;w;0-1
-1.07/25 30;w;0-1
-1.15/26 28;w;0-1
-0.07/25 0;w;0-1
-0.07/34 27;w;0-1
-0.97/24 22;w;0-1
-0.97/27 37;w;0-1
-0.97/25 33;w;0-1
-0.97/25 38;w;0-1
-0.65/20 3;w;0-1
-1.73/24 289;w;0-1
-1.76/24 31;w;0-1
-1.77/24 64;w;0-1
-1.83/23 33;w;0-1
-1.85/24 33;w;0-1
-1.89/25 52;w;0-1
-1.95/23 36;w;0-1
-3.09/25 252;w;0-1
-3.18/24 95;w;0-1
-3.33/25 201;w;0-1
-3.33/21 168;w;0-1
-3.41/21 29;w;0-1
-3.49/21 23;w;0-1
-3.73/21 19;w;0-1
-4.71/22 138;w;0-1
-4.88/20 18;w;0-1
-5.96/20 110;w;0-1
-6.40/17 14;w;0-1
-9.60/17 80;w;0-1
game 2
+0.02/20 141;b;1-0
-0.02/20 97;b;1-0
-0.06/19 97;b;1-0
-0.02/20 80;b;1-0
-0.07/20 58;b;1-0
-0.04/20 55;b;1-0
-0.16/18 48;b;1-0
-0.18/20 63;b;1-0
-0.17/18 0;b;1-0
-0.22/18 48;b;1-0
-0.22/19 62;b;1-0
-0.33/19 105;b;1-0
-0.27/20 240;b;1-0
-0.25/18 49;b;1-0
-0.44/19 46;b;1-0
-0.36/18 50;b;1-0
-0.44/19 156;b;1-0
-0.44/17 0;b;1-0
-0.69/19 140;b;1-0
-1.31/18 103;b;1-0
-1.41/17 55;b;1-0
-1.37/15 38;b;1-0
-1.39/16 21;b;1-0
-3.26/17 41;b;1-0
-3.58/17 18;b;1-0
-3.80/17 31;b;1-0
-4.30/16 12;b;1-0
-5.73/16 20;b;1-0
-6.97/16 23;b;1-0
-9.93/15 8;b;1-0
-11.48/17 99;b;1-0