Testing Houdini 2.0 with one and six cores.

[Tomcass]

This is a long post. Apologies.

HOUDINI 2.0 PRO

1.- TEST USING ONE CORE. 400 games

Time control: 2 min. + 1 sec.
GUI: Fritz 12
Book: HS Masterbook 3.0
Quad i7 using one core each engine.
Ponder: Off

201109Houdini2.0b 2011
Houdini 2.0 Pro x64(x1) - Komodo64 3 (x1) 31.5 - 18.5 +24/=15/-11 63.00%
Houdini 2.0 Pro x64(x1) - Rybka 4 x64 (x1) 31.5 - 18.5 +21/=21/-8 63.00%
Houdini 2.0 Pro x64(x1) - Critter 1.2 64-bit SSE4 (x1) 32.0 - 18.0 +17/=30/-3 64.00%
Games: http://www.megaupload.com/?d=R983QNRY

201109Houdini2.0c-1 2011
Houdini 2.0 Pro x64 (x1) - Ivanhoe B47e x64 (x1) PP 28.0 - 22.0 +14/=28/-8 56.00%
Houdini 2.0 Pro x64 (x1) - RobboLito 0.09 x64 34.5 - 15.5 +22/=25/-3 69.00%
Games: http://www.megaupload.com/?d=85UUU7LM

201109Houdini2.0Final-4 2011
Houdini 2.0 Pro x64 (x1) F - Komodo64 3 (x1) 26.5 - 23.5 +16/=21/-13 53.00%
Houdini 2.0 Pro x64 (x1) F - RobboLito 0.09 x64 34.5 - 15.5 +24/=21/-5 69.00%
Houdini 2.0 Pro x64 (x1) F - Ivanhoe B47e x64 (x1) PP 33.5 - 16.5 +20/=27/-3 67.00%
Games: http://www.megaupload.com/?d=ZM57VL3X


GLOBAL SCORE:

400 GAMES

Houdini 2.0 Pro x64(x1) - Komodo64 3 (x1) 58.0 - 42.0 +40/=36/-24 58.00%
Houdini 2.0 Pro x64 (x1) - Ivanhoe B47e x64 (x1) PP 61.5 - 38.5 +34/=55/-11 61.50%
Houdini 2.0 Pro x64 (x1) - RobboLito 0.09 x64 69.0 - 31.0 +46/=46/-8 69.00%
Houdini 2.0 Pro x64(x1) - Rybka 4 x64 (x1) 31.5 - 18.5 +21/=21/-8 63.00%
Houdini 2.0 Pro x64(x1) - Critter 1.2 64-bit SSE4 (x1) 32.0 - 18.0 +17/=30/-3 64.00%

Overall Houdini 2.0 Pro gets 252.0 / 400 that is 63.00%.

Comments:
- No crashes.
- Some increase in speed in relation to H 1.5. ( around 4% increase)
- I feel an improvement in strength in relation to H 1.5a. Difficult to quantify in ELO terms at this moment, but I think is a good step ahead. Using one core I find around 26 to 30 ELO points increase when I compare this test with the previous ones with H 1.5a (this is a mere estimate, tests not 100% homogeneous) .

2.- TEST USING 6 CORES WITH AND WITHOUT 6 HT. 400 games

Time control: 2 min. + 1 sec.
GUI: Fritz 12
Book: Perfect 2011
Quad i7 X980 3.33 Mhz 6 cores (HT 6 cores)
Ponder: Off

201104. Houdini 2.0 2011
Houdini 2.0 Pro x64 (x12) - Critter 1.2 64-bit SSE4(x12) 31.5 - 19.5 +20/=23/-8 61.76%
Houdini 2.0 Pro x64 (x12) - Deep Rybka 4.1 SSE42 x64(x12) 30.5 - 19.5 +20/=21/-9 61.00%

201104. Houdini 2.0b-1 2011
Houdini 2.0 Pro x64(x6) - Critter 1.2 64-bit SSE4(x12) 27.5 - 22.5 +13/=29/-8 55.00%
Houdini 2.0 Pro x64(x6) - Deep Rybka 4.1 SSE42 x64(x12) 28.0 - 21.0 +14/=28/-7 57.14%

201104. Houdini 2.0.Fcbtourn 2011
Houdini 2.0 Pro x64 (x12) F - Critter 1.2 64-bit SSE4(x12) 27.0 - 23.0 +15/=24/-11 54.00%
Houdini 2.0 Pro x64 (x12) F - Deep Rybka 4.1 SSE42 x64(x12) 32.0 - 18.0 +23/=18/-9 64.00%

201104. Houdini 2.0bF 2011
Houdini 2.0 Pro x64 (x6) F - Deep Rybka 4.1 SSE42 x64(x12) 33.0 - 17.0 +22/=22/-6 66.00%
Houdini 2.0 Pro x64 (x6) F - Critter 1.2 64-bit SSE4(x12) 30.5 - 19.5 +19/=23/-8 61.00%

Houdini 2.0 Pro WITHOUT HT: SCORE 119 / 199 = 59.80%

Houdini 2.0 Pro x64(x6) - Critter 1.2 64-bit SSE4(x12) 58.0 -42.0 +32/=52/-16 58.00%
Houdini 2.0 Pro x64(x6) - Deep Rybka 4.1 SSE42 x64(x12) 61.0 -38.0 +36/=50/-13 61.62%

Houdini 2.0 Pro WITH HT: SCORE 121/201 = 60.20%

Houdini 2.0 Pro x64 (x12)- Critter 1.2 64-bit SSE4(x12) 58.5 -42.5 +35/=47/-19 57.92%
Houdini 2.0 Pro x64 (x12)-Deep Rybka 4.1 SSE42 x64(x12) 62.5 -37.5 +43/=39/-18 62.50%

GLOBAL SCORE 400 games:
240 / 400 = 60.00%

Against Critter 1.2 64-bit SSE4(x12) : 116,5/201 57,96%
Against Deep Rybka 4.1 SSE42 x64(x12) : 123.5/199 62.06%

Games 6 cores: http://www.megaupload.com/?d=EEYZR4U9

Comments:
- One crash against Deep Rybka 4.1. (Probably this was a fault of my computer, not the engine).
- The increase in ELO using six cores seems to be a bit lower than with one core: Around 25 ELO points. (Please note that this is a mere estimate: again tests not 100% homogeneous) .

Congratulations to Robert Houdart for this new improvement in Houdini. Well done!.

Regards from Barcelona,

Tom.

[Tomcass]

and now:


After a 10 days x 24 hours non stop test, here you have the results with 119 games:

GUI: Fritz 12
Time control: 60 min + 1 sec.
Book: Perfect 2011
No Tablebases, Nalimov or Gaviota.
Ponder: Off.
6 real cores, no HT.


1 Houdini 2.0 Pro x64 (x6) F +11/ =12/ -6 58.62% 17.0 / 29
2 Critter 1.2 64-bit SSE4x6bo +6/ =12/ -11 41.38% 12.0 / 29
( One game dissapeared from my database. Sorry!)

1 Houdini 2.0 Pro x64 (x6) F +12/ =17/ -1 68.33% 20.5/30
2 Deep Rybka 4.1 SSE42 x64 (x6) -1/ =17/ -12 31.47% 9.5/30

1 Houdini 2.0 Pro x64 (x6) F +12/=18/-0 70.00% 21.0/30
2 Ivanhoe B47e x64(x6) PP +0/ =18/-12 30.00% 9.0/30

1 Houdini 2.0 Pro x64 (x6) F +10/=14/-6 56.67% 17.0/30
2 Ivanhoe B47f0.2 x64(x6) +6/=14/-10 43.33% 13.0/30

Overall after 119 games at this TC, Houdini 2.0 Pro has got:

+ 45/ = 61/ - 13, that is 75.5/119 or 63.44%

Games (in three parts):

http://www.megaupload.com/?d=AZK0YPE0
http://www.megaupload.com/?d=27ZSJ7H4
http://www.megaupload.com/?d=W4R3VPWO

Some brief comments:

1.- Also at this longer time control, the superiority of Houdini 2.0 Pro is simply impressive, especially against Deep Rybka 4.1 and Ivanhoe B47e.

2.- Critter 1.2 has resisted well and produced some nice games against Houdini.

3.- The latest PeterPan's compile, Ivanhoe B47f0.2 x64 has been the strongest opponent for Houdini in this test. Only 30 games, but the improvement over the previous version, Ivanhoe B47e x64, seems to be huge.

4.- What a pitty I have not been able yet to test the promising Komodo 3 with 6 real cores!.

Best regards from Barcelona.

Tom.

[rvida]

3.- The latest PeterPan's compile, Ivanhoe B47f0.2 x64 has been the strongest opponent for Houdini in this test. Only 30 games, but the improvement over the previous version, Ivanhoe B47e x64, seems to be huge.

Honestly, I think you fell victim of the statistical 'magics'

The difference between Ivan B47f-whatever and B47e-whatever is maybe (and I am hugely exaggerating) 4 lines of code and 3-4 constants? Or are they just compiled by a different compiler?

Sorry, but real progress has to be made elsewhere...

[Tomcass]

3.- The latest PeterPan's compile, Ivanhoe B47f0.2 x64 has been the strongest opponent for Houdini in this test. Only 30 games, but the improvement over the previous version, Ivanhoe B47e x64, seems to be huge.

Honestly, I think you fell victim of the statistical 'magics'

The difference between Ivan B47f-whatever and B47e-whatever is maybe (and I am hugely exaggerating) 4 lines of code and 3-4 constants? Or are they just compiled by a different compiler?

Sorry, but real progress has to be made elsewhere...

Who knows, Richard?. Although with only 30 games (x2), the difference between both Ivanhoes is substantial. If you have time to give a look to the games, I am sure you will perceive the difference in strength.

The compiler is the same for both engines: PeterPan. IMHO one of the most competent ones in the world. I am simply an amateur tester, trying to get some preliminary conclusions of what I see and feel in this enjoyable activity.

Tom.

[Izak Pretorius]

Hi Richard Vida

First of all,I have a lot of respect for you as a professional and original chess programmer.
I can also understand if you think it's just a few values changes and 4 lines of code changed or less as you stated.

But I added the code,so I should know it's more than 4 lines
Yes compilers can also learn to add value to chess programs.

I totally understand this assumption made by you,but this time you judged too soon

No hard feelings and I hope Critter will become nr.1 engine in the future

Regards
Izak

[Izak Pretorius]

hi Tom

Thanks for pointing out to me that this Ivanhoe release has potential especially for longer time controls.

I am running a match myself and so far the results are encouraging.

Same time control,60 minutes + 1 second
Unfortunately i am not as lucky as you to have a 6 core machine,only have a 4 core intel pc.

Here are the results so far...

Code: Select all

                     
1   Ivanhoe B47f0.2 x64        +25   ½½½1½½½½½½½½10   7.5/14
2   Critter 1.2 64-bit         -25   ½½½0½½½½½½½½01   6.5/14

[Tomcass]

In order to minimize the statistical effect mentioned by Richard Vida in a previous post, I have run 60 more games at a 1 hour + time control, trying to know better the real strength of Ivanhoe B47f0.2 in relation to Critter 1.2 and Deep Rybka 4.1:

Time control: 60 min + 1 sec.
Book: Perfect 2011
Hash : 1024
Hardware: i7 x980 3.33Ghz RAM 24.0 GB
6 real cores, no HT
Ponder: Off.
No Robbo , Triple Bases or Nalimov.
For Ivanhoe about 48 BNG and for Critter about 52 BNG (Billion Nodes per Game) approx.

1 2 3
1 Ivanhoe B47f0.2 x64(x6)NOU ********** ½½½1½½½½00 ½1½½½½½½½1 10.5/20
2 Critter 1.2 64-bit SSE4x6bo ½½½0½½½½11 ********** 1½0½1½½0½0 10.0/20
3 Deep Rybka 4.1 SSE42 x64 ½0½½½½½½½0 0½1½0½½1½1 ********** 9.5/20

Games: megaupload.com UN86HFZ1


Time control: 60 min + 1 sec.
Book: Masterbook 3.0
Hash : 256
Hardware: i7 x975 3.33Ghz RAM 12.0 GB
4 real cores, no HT
Ponder: Off.
No Robbo , Triple Bases or Nalimov.
For Ivanhoe about 48 BNG and for Critter about 34 BNG (Billion Nodes per Game) approx.

1 Ivanhoe B47f02 XP x64(x4) +3/=24/-3 50.00% 15.0/30 0.00
2 Critter 1.2 64-bit SSE4(x4) +3/=24/-3 50.00% 15.0/30 0.00

Games: megaupload.com HCJ1P1Y7

Regards from Barcelona.

Tom.