Testing DamirsRK60 and 61

Discussion of computer chess matches and engine tournaments.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 7:09 pm

Testing DamirsRK60 and 61

Post by Tomcass » Sun Jun 27, 2010 5:17 pm

TESTING DAMIRSRK 60 AND 61
300 games


TEST 1

Quad 2.33
Gui: Fritz 12
Ponder: Off
No Triple or Robbo Bases
Book: HS Masterbook 2
Time Control: 4 min+ 2 sec.
DRybka4, Naum 4.2 and Fire 1.31 4 cores
DRK60 and 61 and Mishas 1 core. Speed han
dicap: 290%


2010-06DRKiller60 2010

DamirsRybkaKiller60_w32 - Deep Rybka 4 w32 6.0 - 14.0 +1/=10/-9 30.00%
DamirsRybkaKiller60_w32 - Fire 1.31 w32KLO 7.0 - 13.0 +0/=14/-6 35.00%
DamirsRybkaKiller60_w32 - Naum 4.2 (x4) 12.0 - 8.0 +6/=12/-2 60.00%

2010-06DRKiller61 2010

DamirsRybkaKiller61_w32 - Deep Rybka 4 w32 5.5 - 14.5 +1/=9/-10 27.50%
DamirsRybkaKiller61_w32 - Fire 1.31 w32KLO 8.0 - 12.0 +0/=16/-4 40.00%
DamirsRybkaKiller61_w32 - Naum 4.2 (x4) 10.5 - 9.5 +6/=9/-5 52.50%
DamirsRybkaKiller61_w32 - MishasMauler12T_w32 8.0 - 12.0 +1/=14/-5 40.00%

TEST 2

I7 975
Gui: Fritz 12
Ponder: Off
No Triple or Robbo Bases
Book: HS Masterbook 2
Time Control: 4 min+ 2 sec.
DRybka4, Naum 4.2 and Fire 1.2: 8 cores
DRK60 and 61 and Mishas:1 core. (Speed hand
icap: 710%)



201006DRKiller60-2 2010
DamirsRybkaKiller60_w32(ok) - Deep Rybka 4 x64 6.0 - 14.0 +2/=8/-10 30.00%
DamirsRybkaKiller60_w32(ok) - FireBird 1.2 x64 (x8) 7.5 - 12.5 +1/=13/-6 37.50%
DamirsRybkaKiller60_w32(ok) - Naum 4.2 64(x8) 8.5 - 11.5 +3/=11/-6 42.50%
DamirsRybkaKiller60_w32(ok) - MishasMauler12T_w32 11.0 - 9.0 +5/=12/-3 55.00%



201006DRKiller61 2010
DamirsRybkaKiller61_w32 - Deep Rybka 4 x64 4.0 - 16.0 +1/=6/-13 20.00%
DamirsRybkaKiller61_w32 - FireBird 1.2 x64 (x8) 7.5 - 12.5 +0/=15/-5 37.50%
DamirsRybkaKiller61_w32 - Naum 4.2 64(x8) 9.5 - 10.5 +4/=11/-5 47.50%
DamirsRybkaKiller61_w32 - MishasMauler12T_w32 10.0 - 10.0 +3/=14/-3 50.00%


GLOBAL RESULTS

DamirsRybkaKiller60_w32(ok)

Against Deep Rybka 4= 12,0 - 28,0 +3/=18/-19 30%
Against Firebird= 14,5 - 25,5 +1/=27/-12 36,25%
Against Naum 4.2= 23,0 – 17,0 +9/=23/-8 51,25%
Against Mishas12T = 11.0 - 9.0 +5/=12/-3 55.00%

DamirsRybkaKiller61_w32

Against Deep Rybka 4= 9,5 – 30,5 +2/=15/-23 23,75
Against Firebird= 15,5 – 24,5 +0/=31/-9 38,75
Against Naum 4.2= 20,0 – 20,0 +10/=20/-10 50,0%
Against Mishas12T = 18,0 - 22,0 +4/=28/-8 45,0%

After these results, I think that the time management introduced in DRK61 is not good enough. For me, DRK60, with such an excellent performance (with a global speed handicap of 500% against DR4, Fire and Naum) is the best DRK available. Even slightly better than Mishas12T. My conclusion: PERHAPS DRK60 is today the best engine in the world using only one core.

Regards from Barcelona.

Tom.

User avatar
Houdini
Posts: 1471
Joined: Mon Mar 15, 2010 11:00 pm
Contact:

Re: Testing DamirsRK60 and 61

Post by Houdini » Sun Jun 27, 2010 5:37 pm

Tom,

A couple of observations.

1) A core i7-975 only has 4 cores with hyper-threading. Running engines with 8 threads will in the best case produce a speed-up of about 20% compared to 4 threads, most of which will not translate into Elo strength (8 threads have more overhead than 4 threads). Your results provide a clear demonstration of this point.

2) If you want to find out which engine is the best single-core, why don't you play single-core matches? This would give more useful results that are not influenced by unknown scaling effects.

3) Many more games are needed, but of course you know that.

4) Why don't you include Houdini in your test? ;)

Robert

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 7:09 pm

Re: Testing DamirsRK60 and 61

Post by Tomcass » Sun Jun 27, 2010 6:14 pm

Thanks for your suggestions, Robert.

I have two computers -one quad 2,33 (not very fast indeed) and one i7 975-. The difference in NPS between both of them is, on average, 245% for the second one. I have checked the real difference in terms of time to reach a certain depth and it is always above 220%. This is the reason for me to give some validity to my results and conclussions.

I know that I am not a scientist of testing. For instance, I use HS Masterbooks 1.0 and 2.0 with no limit in the number of moves. I don't like test suites. Since I follow as many games as I can, I enjoy testing this way, with a lot of variety in games. My point is that if the number of games is high enough the randomness effect of books becomes less important.

On the other hand I have tested all Houdinis. My tests show -to me!- that your latest version is almost even with DR4. What a GREAT engine!. I am anxious to discover what the next version (Did you mention mid-july?:wink: ) will offer to us. Please do not hesitate to contact me, should you want me to test it.

Congratulations and thanks for your excellent work, Robert!.

Regards,

Tom.

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 7:09 pm

Re: Testing DamirsRK60, 61 ... and 69!

Post by Tomcass » Mon Jul 12, 2010 9:32 am

New long test with DRK69.

Having been away from home for one week, I let my quad testing DRK69 against Deep Rybka 4. I was lucky enough to finish this 450 games test:

Quad 2,33 w32
Book: HS Masterbook 2.0
Time control 10 min + 0
Ponder: Off
No Robbo, Triple or Nalimov.
DRK using ONE CORE and Deep Rybka, Ivan and Houdini using FOUR CORES

450 games: http://www.megaupload.com/?d=JEI1TZBY


2010-07DamirRKiller69-1 2010

DamirsRybkaKiller69_w32 - Deep Rybka 4 w32 56.0 - 94.0 +15/=82/-53 37.33%
DamirsRybkaKiller69_w32 - IvanHoe 9.55b w32 (x4) 57.5 - 92.5 +4/=107/-39 38.33%
DamirsRybkaKiller69_w32 - Houdini 1.02 w32 4_CPU 57.5 - 92.5 +5/=105/-40 38.33%

This is the best result I have got for a DRK among all versions. Well done, Bill, please keep working!!.

Regards,

Tom.

Post Reply