Testing Houdini 4.0 Pro B: 480 games.

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 7:09 pm

Testing Houdini 4.0 Pro B: 480 games.

Post by Tomcass » Wed Nov 27, 2013 1:58 pm

TESTING HOUDINI 4.0 PRO B = 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 26.0 - 14.0 +17/=18/-5 65.00%
Houdini 4 Pro x64x6 - Komodo 6 64-bitx6NOB 20.0 - 20.0 +11/=18/-11 50.00%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 18.5 - 21.5 +6/=25/-9 46.25%

Time Control= 2+2

201311Houdini4.0ProB_2+2_120games 2013

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 23.5 - 16.5 +14/=19/-7 58.75%
Houdini 4 Pro x64x6 - Komodo 6 64-bitNOBx6 25.0 - 15.0 +14/=22/-4 62.50%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 22.0 - 18.0 +10/=24/-6 55.00%

Games (120+120)=
http://www.mediafire.com/view/16ac84...120games_1.pgn
http://www.mediafire.com/view/3vxcw3...120games_2.pgn
Score using 6 cores= 135.0 -105.0 = 56.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 22.0 - 18.0 +11/=22/-7 55.00%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 24.0 - 16.0 +15/=18/-7 60.00%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 20.0 - 20.0 +11/=18/-11 50.00%

Time Control= 2+2

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 20.5 - 19.5 +9/=23/-8 51.25%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 27.0 - 13.0 +18/=18/-4 67.50%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 16.0 - 24.0 +7/=18/-15 40.00%

Games (240)= http://www.mediafire.com/view/uw6cxu...40games_LP.pgn
Score using 4 cores= 129.5 – 110.5 = 53.96%

Fixed = 130.5 – 109.5 = 54.37%
Incremental = 134.0 -106.0 = 55.83%

Global Score: 264.5-225.5 = 55.10%
Against Houdini 3.0 Pro (3172): 57.50% Komodo 6 (3154): 60.00% Stockfish 111113SL Ipman (3224) 47.81%

Average ELO of opponents= 3.183
Estimated ELO Performance= 3.219


With only 480 games I obviously give only a first approach to the Houdini 4.0Pro B strength. Under my testing conditions and using 4 and 6 cores computers:

- The improvement of Houdini 4.0 Pro over Houdini 3.0 Pro is substantial: Around 47 Estimated Elo Points (EEP).
- Houdini 4.0 Pro is about 65 EEP ahead of Komodo 6
- Houdini 4.0 Pro is about the same strength of the best Stockfish compile –only 5 EEP below-: The SF 11113SL compile by Ipman. Within the statistical margin error.
- Houdini 4.0 Pro is about 100 EEP ahead of Stockfish 4.
- Houdini 4.0 Pro is about 15 EEP ahead of the latest Stockfish development I have tested: 191113.

Regards,

Tom.

User avatar
Dr.Wael Deeb
Posts: 9772
Joined: Wed Mar 08, 2006 7:44 pm
Location: Amman,Jordan

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by Dr.Wael Deeb » Wed Nov 27, 2013 4:23 pm

Thanks Tom for the heads up :D

So Houdini's author must head toward the drawing sheet again unless he wants to be overpassed in less than 2 months by Stockfish....
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….

ouachita
Posts: 454
Joined: Tue Jan 15, 2013 3:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by ouachita » Wed Nov 27, 2013 5:32 pm

I agree but the challenge will be to use the strongest version of SF when it is compiled daily and varies in strength!
SIM, PhD, MBA, PE

fauzi
Posts: 61
Joined: Wed Nov 20, 2013 9:42 am

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by fauzi » Wed Nov 27, 2013 9:30 pm

I think the problem in the links of this test and the one before is that You don't write the full address , if You look at them there are 3 points between the address "..." and that shows obviously that the address is incomplete, please check again and post the full address so we are able to download the games , thank You for Your work Tom

majortom
Posts: 669
Joined: Mon Nov 04, 2013 9:19 pm
Contact:

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by majortom » Wed Nov 27, 2013 9:57 pm

fauzi wrote:I think the problem in the links of this test and the one before is that You don't write the full address , if You look at them there are 3 points between the address "..." and that shows obviously that the address is incomplete, please check again and post the full address so we are able to download the games , thank You for Your work Tom
Correct links:
TESTING HOUDINI 4.0 PRO B = 480 GAMES

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 26.0 - 14.0 +17/=18/-5 65.00%
Houdini 4 Pro x64x6 - Komodo 6 64-bitx6NOB 20.0 - 20.0 +11/=18/-11 50.00%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 18.5 - 21.5 +6/=25/-9 46.25%

Time Control= 2+2

201311Houdini4.0ProB_2+2_120games 2013

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 23.5 - 16.5 +14/=19/-7 58.75%
Houdini 4 Pro x64x6 - Komodo 6 64-bitNOBx6 25.0 - 15.0 +14/=22/-4 62.50%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 22.0 - 18.0 +10/=24/-6 55.00%

Games (120+120)=
http://www.mediafire.com/view/16ac84tk4 ... ames_1.pgn
http://www.mediafire.com/view/3vxcw33lw ... ames_2.pgn
Score using 6 cores= 135.0 -105.0 = 56.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 22.0 - 18.0 +11/=22/-7 55.00%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 24.0 - 16.0 +15/=18/-7 60.00%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 20.0 - 20.0 +11/=18/-11 50.00%

Time Control= 2+2

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 20.5 - 19.5 +9/=23/-8 51.25%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 27.0 - 13.0 +18/=18/-4 67.50%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 16.0 - 24.0 +7/=18/-15 40.00%

Games (240)= http://www.mediafire.com/view/uw6cxu1a1 ... mes_LP.pgn
Score using 4 cores= 129.5 – 110.5 = 53.96%

Fixed = 130.5 – 109.5 = 54.37%
Incremental = 134.0 -106.0 = 55.83%

Global Score: 264.5-225.5 = 55.10%
Against Houdini 3.0 Pro (3172): 57.50% Komodo 6 (3154): 60.00% Stockfish 111113SL Ipman (3224) 47.81%

Average ELO of opponents= 3.183
Estimated ELO Performance= 3.219


With only 480 games I obviously give only a first approach to the Houdini 4.0Pro B strength. Under my testing conditions and using 4 and 6 cores computers:

- The improvement of Houdini 4.0 Pro over Houdini 3.0 Pro is substantial: Around 47 Estimated Elo Points (EEP).
- Houdini 4.0 Pro is about 65 EEP ahead of Komodo 6
- Houdini 4.0 Pro is about the same strength of the best Stockfish compile –only 5 EEP below-: The SF 11113SL compile by Ipman. Within the statistical margin error.
- Houdini 4.0 Pro is about 100 EEP ahead of Stockfish 4.
- Houdini 4.0 Pro is about 15 EEP ahead of the latest Stockfish development I have tested: 191113.

Regards,

Tom.

User avatar
Dr.Wael Deeb
Posts: 9772
Joined: Wed Mar 08, 2006 7:44 pm
Location: Amman,Jordan

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by Dr.Wael Deeb » Wed Nov 27, 2013 10:01 pm

majortom wrote:
fauzi wrote:I think the problem in the links of this test and the one before is that You don't write the full address , if You look at them there are 3 points between the address "..." and that shows obviously that the address is incomplete, please check again and post the full address so we are able to download the games , thank You for Your work Tom
Correct links:
TESTING HOUDINI 4.0 PRO B = 480 GAMES

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control= 4+0.

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 26.0 - 14.0 +17/=18/-5 65.00%
Houdini 4 Pro x64x6 - Komodo 6 64-bitx6NOB 20.0 - 20.0 +11/=18/-11 50.00%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 18.5 - 21.5 +6/=25/-9 46.25%

Time Control= 2+2

201311Houdini4.0ProB_2+2_120games 2013

Houdini 4 Pro x64x6 - Houdini 3 Pro x64_6 23.5 - 16.5 +14/=19/-7 58.75%
Houdini 4 Pro x64x6 - Komodo 6 64-bitNOBx6 25.0 - 15.0 +14/=22/-4 62.50%
Houdini 4 Pro x64x6 - Stockfish 111113SL 64 SSE4.2L 22.0 - 18.0 +10/=24/-6 55.00%

Games (120+120)=
http://www.mediafire.com/view/16ac84tk4 ... ames_1.pgn
http://www.mediafire.com/view/3vxcw33lw ... ames_2.pgn
Score using 6 cores= 135.0 -105.0 = 56.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases. No RTB used.
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control= 4+0

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 22.0 - 18.0 +11/=22/-7 55.00%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 24.0 - 16.0 +15/=18/-7 60.00%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 20.0 - 20.0 +11/=18/-11 50.00%

Time Control= 2+2

Houdini 4 Pro x64x4 - Houdini 3 Pro x64_4 20.5 - 19.5 +9/=23/-8 51.25%
Houdini 4 Pro x64x4 - Komodo 6 64-bitx4_NOB 27.0 - 13.0 +18/=18/-4 67.50%
Houdini 4 Pro x64x4 - Stockfish 111113SL 64 SSE4.2I 16.0 - 24.0 +7/=18/-15 40.00%

Games (240)= http://www.mediafire.com/view/uw6cxu1a1 ... mes_LP.pgn
Score using 4 cores= 129.5 – 110.5 = 53.96%

Fixed = 130.5 – 109.5 = 54.37%
Incremental = 134.0 -106.0 = 55.83%

Global Score: 264.5-225.5 = 55.10%
Against Houdini 3.0 Pro (3172): 57.50% Komodo 6 (3154): 60.00% Stockfish 111113SL Ipman (3224) 47.81%

Average ELO of opponents= 3.183
Estimated ELO Performance= 3.219


With only 480 games I obviously give only a first approach to the Houdini 4.0Pro B strength. Under my testing conditions and using 4 and 6 cores computers:

- The improvement of Houdini 4.0 Pro over Houdini 3.0 Pro is substantial: Around 47 Estimated Elo Points (EEP).
- Houdini 4.0 Pro is about 65 EEP ahead of Komodo 6
- Houdini 4.0 Pro is about the same strength of the best Stockfish compile –only 5 EEP below-: The SF 11113SL compile by Ipman. Within the statistical margin error.
- Houdini 4.0 Pro is about 100 EEP ahead of Stockfish 4.
- Houdini 4.0 Pro is about 15 EEP ahead of the latest Stockfish development I have tested: 191113.

Regards,

Tom.
Thanks Andrey....
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 7:09 pm

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by Tomcass » Thu Nov 28, 2013 9:10 am

Thank you very much Andrey!. :D

Just now I have started the second 480 games leg with Houdini 4.0 Pro B. Now against Stockfish 191113 (No Large Pages, development version), Critter 1.6 and Deep Rybka 4. I expect a global 67% score for Houdini. Let's see.

Regards,

Tom.

majortom
Posts: 669
Joined: Mon Nov 04, 2013 9:19 pm
Contact:

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by majortom » Thu Nov 28, 2013 9:28 am

Thank you Tom for your amazing tests!

It will be very interesting to see Gull R600 in yr tests too!

Vinvin
Posts: 4754
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by Vinvin » Thu Nov 28, 2013 9:31 am

Tomcass wrote:Thank you very much Andrey!. :D

Just now I have started the second 480 games leg with Houdini 4.0 Pro B. Now against Stockfish 191113 (No Large Pages, development version), Critter 1.6 and Deep Rybka 4. I expect a global 67% score for Houdini. Let's see.

Regards,

Tom.
Hello Tom, is it possible to get ALL the games you played (I suppose it's around 5000) zipped somewhere ?

Thanks,
Vincent

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 7:09 pm

Re: Testing Houdini 4.0 Pro B: 480 games.

Post by Tomcass » Thu Nov 28, 2013 9:49 am

Hi, Vincent.

I have a lot of games stored from my tests in the last five years. Around 110.000 games in my two computers. To be honest I think that most of them are useless or obsolete. The speed of improvement in computer chess is impressive.

I can zip the most recent ones ... but I am not very skilled doing it. (As you can realize from my difficulties to simply download files). :oops: In the Stockfish 480 games test trend you can find most of them. Perhaps 20.000 to 22.000 I guess.

Kind regards,

Tom.

... by the way I realized that this Stockfish thread has surpassed the 50.000 views (!!!). Thank you very much, my friends, for following so closely my tests. :D

Post Reply