TalkChess.com

Computer Chess Club
https://talkchess.com/

Testing Houdini 4.0 Pro B: 480 games.

https://talkchess.com/viewtopic.php?t=50249

Page 1 of 2

Testing Houdini 4.0 Pro B: 480 games.

Posted: Wed Nov 27, 2013 2:58 pm
by Tomcass
TESTING HOUDINI 4.0 PRO B = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 26.0 - 14.0 +17/=18/-5 65.00%
Houdini 4 Pro x64x6 - Komodo 6 64-bitx6NOB 20.0 - 20.0 +11/=18/-11 50.00%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 18.5 - 21.5 +6/=25/-9 46.25%

Time Control= 2+2

201311Houdini4.0ProB_2+2_120games 2013

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 23.5 - 16.5 +14/=19/-7 58.75%
Houdini 4 Pro x64x6 - Komodo 6 64-bitNOBx6 25.0 - 15.0 +14/=22/-4 62.50%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 22.0 - 18.0 +10/=24/-6 55.00%

Games (120+120)=
http://www.mediafire.com/view/16ac84...120games_1.pgn
http://www.mediafire.com/view/3vxcw3...120games_2.pgn
Score using 6 cores= 135.0 -105.0 = 56.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 22.0 - 18.0 +11/=22/-7 55.00%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 24.0 - 16.0 +15/=18/-7 60.00%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 20.0 - 20.0 +11/=18/-11 50.00%

Time Control= 2+2

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 20.5 - 19.5 +9/=23/-8 51.25%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 27.0 - 13.0 +18/=18/-4 67.50%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 16.0 - 24.0 +7/=18/-15 40.00%

Games (240)= http://www.mediafire.com/view/uw6cxu...40games_LP.pgn
Score using 4 cores= 129.5 – 110.5 = 53.96%

Fixed = 130.5 – 109.5 = 54.37%
Incremental = 134.0 -106.0 = 55.83%

Global Score: 264.5-225.5 = 55.10%
Against Houdini 3.0 Pro (3172): 57.50% Komodo 6 (3154): 60.00% Stockfish 111113SL Ipman (3224) 47.81%

Average ELO of opponents= 3.183
Estimated ELO Performance= 3.219

With only 480 games I obviously give only a first approach to the Houdini 4.0Pro B strength. Under my testing conditions and using 4 and 6 cores computers:

- The improvement of Houdini 4.0 Pro over Houdini 3.0 Pro is substantial: Around 47 Estimated Elo Points (EEP).
- Houdini 4.0 Pro is about 65 EEP ahead of Komodo 6
- Houdini 4.0 Pro is about the same strength of the best Stockfish compile –only 5 EEP below-: The SF 11113SL compile by Ipman. Within the statistical margin error.
- Houdini 4.0 Pro is about 100 EEP ahead of Stockfish 4.
- Houdini 4.0 Pro is about 15 EEP ahead of the latest Stockfish development I have tested: 191113.

Regards,

Tom.

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Wed Nov 27, 2013 5:23 pm
by Dr.Wael Deeb
Thanks Tom for the heads up :D

So Houdini's author must head toward the drawing sheet again unless he wants to be overpassed in less than 2 months by Stockfish....
Dr.D

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Wed Nov 27, 2013 6:32 pm
by ouachita
I agree but the challenge will be to use the strongest version of SF when it is compiled daily and varies in strength!

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Wed Nov 27, 2013 10:30 pm
by fauzi
I think the problem in the links of this test and the one before is that You don't write the full address , if You look at them there are 3 points between the address "..." and that shows obviously that the address is incomplete, please check again and post the full address so we are able to download the games , thank You for Your work Tom

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Wed Nov 27, 2013 10:57 pm
by majortom
fauzi wrote:I think the problem in the links of this test and the one before is that You don't write the full address , if You look at them there are 3 points between the address "..." and that shows obviously that the address is incomplete, please check again and post the full address so we are able to download the games , thank You for Your work Tom
Correct links:
TESTING HOUDINI 4.0 PRO B = 480 GAMES

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 26.0 - 14.0 +17/=18/-5 65.00%
Houdini 4 Pro x64x6 - Komodo 6 64-bitx6NOB 20.0 - 20.0 +11/=18/-11 50.00%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 18.5 - 21.5 +6/=25/-9 46.25%

Time Control= 2+2

201311Houdini4.0ProB_2+2_120games 2013

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 23.5 - 16.5 +14/=19/-7 58.75%
Houdini 4 Pro x64x6 - Komodo 6 64-bitNOBx6 25.0 - 15.0 +14/=22/-4 62.50%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 22.0 - 18.0 +10/=24/-6 55.00%

Games (120+120)=
http://www.mediafire.com/view/16ac84tk4 ... ames_1.pgn
http://www.mediafire.com/view/3vxcw33lw ... ames_2.pgn
Score using 6 cores= 135.0 -105.0 = 56.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 22.0 - 18.0 +11/=22/-7 55.00%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 24.0 - 16.0 +15/=18/-7 60.00%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 20.0 - 20.0 +11/=18/-11 50.00%

Time Control= 2+2

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 20.5 - 19.5 +9/=23/-8 51.25%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 27.0 - 13.0 +18/=18/-4 67.50%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 16.0 - 24.0 +7/=18/-15 40.00%

Games (240)= http://www.mediafire.com/view/uw6cxu1a1 ... mes_LP.pgn
Score using 4 cores= 129.5 – 110.5 = 53.96%

Fixed = 130.5 – 109.5 = 54.37%
Incremental = 134.0 -106.0 = 55.83%

Global Score: 264.5-225.5 = 55.10%
Against Houdini 3.0 Pro (3172): 57.50% Komodo 6 (3154): 60.00% Stockfish 111113SL Ipman (3224) 47.81%

Average ELO of opponents= 3.183
Estimated ELO Performance= 3.219

With only 480 games I obviously give only a first approach to the Houdini 4.0Pro B strength. Under my testing conditions and using 4 and 6 cores computers:

- The improvement of Houdini 4.0 Pro over Houdini 3.0 Pro is substantial: Around 47 Estimated Elo Points (EEP).
- Houdini 4.0 Pro is about 65 EEP ahead of Komodo 6
- Houdini 4.0 Pro is about the same strength of the best Stockfish compile –only 5 EEP below-: The SF 11113SL compile by Ipman. Within the statistical margin error.
- Houdini 4.0 Pro is about 100 EEP ahead of Stockfish 4.
- Houdini 4.0 Pro is about 15 EEP ahead of the latest Stockfish development I have tested: 191113.

Regards,

Tom.

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Wed Nov 27, 2013 11:01 pm
by Dr.Wael Deeb
majortom wrote:
fauzi wrote:I think the problem in the links of this test and the one before is that You don't write the full address , if You look at them there are 3 points between the address "..." and that shows obviously that the address is incomplete, please check again and post the full address so we are able to download the games , thank You for Your work Tom
Correct links:
TESTING HOUDINI 4.0 PRO B = 480 GAMES

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 26.0 - 14.0 +17/=18/-5 65.00%
Houdini 4 Pro x64x6 - Komodo 6 64-bitx6NOB 20.0 - 20.0 +11/=18/-11 50.00%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 18.5 - 21.5 +6/=25/-9 46.25%

Time Control= 2+2

201311Houdini4.0ProB_2+2_120games 2013

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 23.5 - 16.5 +14/=19/-7 58.75%
Houdini 4 Pro x64x6 - Komodo 6 64-bitNOBx6 25.0 - 15.0 +14/=22/-4 62.50%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 22.0 - 18.0 +10/=24/-6 55.00%

Games (120+120)=
http://www.mediafire.com/view/16ac84tk4 ... ames_1.pgn
http://www.mediafire.com/view/3vxcw33lw ... ames_2.pgn
Score using 6 cores= 135.0 -105.0 = 56.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 22.0 - 18.0 +11/=22/-7 55.00%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 24.0 - 16.0 +15/=18/-7 60.00%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 20.0 - 20.0 +11/=18/-11 50.00%

Time Control= 2+2

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 20.5 - 19.5 +9/=23/-8 51.25%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 27.0 - 13.0 +18/=18/-4 67.50%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 16.0 - 24.0 +7/=18/-15 40.00%

Games (240)= http://www.mediafire.com/view/uw6cxu1a1 ... mes_LP.pgn
Score using 4 cores= 129.5 – 110.5 = 53.96%

Fixed = 130.5 – 109.5 = 54.37%
Incremental = 134.0 -106.0 = 55.83%

Global Score: 264.5-225.5 = 55.10%
Against Houdini 3.0 Pro (3172): 57.50% Komodo 6 (3154): 60.00% Stockfish 111113SL Ipman (3224) 47.81%

Average ELO of opponents= 3.183
Estimated ELO Performance= 3.219

With only 480 games I obviously give only a first approach to the Houdini 4.0Pro B strength. Under my testing conditions and using 4 and 6 cores computers:

- The improvement of Houdini 4.0 Pro over Houdini 3.0 Pro is substantial: Around 47 Estimated Elo Points (EEP).
- Houdini 4.0 Pro is about 65 EEP ahead of Komodo 6
- Houdini 4.0 Pro is about the same strength of the best Stockfish compile –only 5 EEP below-: The SF 11113SL compile by Ipman. Within the statistical margin error.
- Houdini 4.0 Pro is about 100 EEP ahead of Stockfish 4.
- Houdini 4.0 Pro is about 15 EEP ahead of the latest Stockfish development I have tested: 191113.

Regards,

Tom.
Thanks Andrey....
Dr.D

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Thu Nov 28, 2013 10:10 am
by Tomcass
Thank you very much Andrey!. :D

Just now I have started the second 480 games leg with Houdini 4.0 Pro B. Now against Stockfish 191113 (No Large Pages, development version), Critter 1.6 and Deep Rybka 4. I expect a global 67% score for Houdini. Let's see.

Regards,

Tom.

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Thu Nov 28, 2013 10:28 am
by majortom
Thank you Tom for your amazing tests!

It will be very interesting to see Gull R600 in yr tests too!

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Thu Nov 28, 2013 10:31 am
by Vinvin
Tomcass wrote:Thank you very much Andrey!. :D

Just now I have started the second 480 games leg with Houdini 4.0 Pro B. Now against Stockfish 191113 (No Large Pages, development version), Critter 1.6 and Deep Rybka 4. I expect a global 67% score for Houdini. Let's see.

Regards,

Tom.
Hello Tom, is it possible to get ALL the games you played (I suppose it's around 5000) zipped somewhere ?

Thanks,
Vincent

Re: Testing Houdini 4.0 Pro B: 480 games.

Posted: Thu Nov 28, 2013 10:49 am
by Tomcass
Hi, Vincent.

I have a lot of games stored from my tests in the last five years. Around 110.000 games in my two computers. To be honest I think that most of them are useless or obsolete. The speed of improvement in computer chess is impressive.

I can zip the most recent ones ... but I am not very skilled doing it. (As you can realize from my difficulties to simply download files). :oops: In the Stockfish 480 games test trend you can find most of them. Perhaps 20.000 to 22.000 I guess.

Kind regards,

Tom.

... by the way I realized that this Stockfish thread has surpassed the 50.000 views (!!!). Thank you very much, my friends, for following so closely my tests. :D
All times are UTC+02:00
Page 1 of 2

Powered by phpBB® Forum Software © phpBB Limited