Fritz 11 - 4'+2" matches, Noomen 2007 Testset

[Erik Roggenburg]

First of all, I'd like to thank Jeroen Noomen for providing me with an updated "Noomen Testset."

Second, Fritz 11 has not played many games yet, it appears to be quite a leap forward in strength - +50 Elo vs. DF10, +152 Elo vs. F9. Whatever your view of Chessbase, you have to admit that their programmers have done a fine job. If DF11 picks up another 30-50 Elo, plus it maintains its balanced to aggressive playing style, that will be another feather in CB's hat.

Yes, yes, it hasn't surpassed Rybka, but does anyone really use ONLY ONE engine for analysis?

Time Control: 4+2
Test Set(s): Noomen 2007 (current), Noomen 2006, Silver Suite Full
Hardware: 4400+, 2GB RAM
Other specifics: EGTBs used (CB Turbobases + a few other 6 pc.), 128 MB RAM for each engine.

Code: Select all

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Rybka 2.3.1 mp 32-bit          : 2824   13  13  2080    73.1 %   2650   29.4 %
  2 Rybka 2.2 mp 32-bit            : 2815   15  15  1580    72.7 %   2645   30.4 %
  3 Rybka 2.3 LK mp 32-bit         : 2810   18  18  1140    73.1 %   2637   28.8 %
  4 Rybka 2.3 mp 32-bit            : 2810   18  18  1140    73.0 %   2637   30.4 %
  5 Fritz 11                       : 2774   40  40   180    56.1 %   2731   38.9 %
  6 Deep Shredder 11 UCI nobk      : 2740   20  19   900    64.2 %   2638   29.6 %
  7 Zappa Mexico 2T                : 2731   18  18   960    63.2 %   2637   34.5 %
  8 HIARCS 11.1 MP UCI             : 2725   12  12  2060    60.1 %   2654   36.3 %
  9 Deep Fritz 10                  : 2724   12  12  2160    59.3 %   2659   31.2 %
 10 Loop 13.5.32 2CPU              : 2723   13  13  1860    61.3 %   2643   36.9 %
 11 Naum 2.2 2T                    : 2707   18  18   900    59.8 %   2638   40.6 %
 12 LoopMP 12.32 2CPU              : 2702   13  13  1680    56.9 %   2654   36.5 %
 13 HIARCS 11 MP UCI               : 2695   14  14  1480    58.1 %   2638   37.2 %
 14 Deep Shredder 10 UCI           : 2673   12  12  2100    52.3 %   2657   30.4 %
 15 Naum 2.1 MP                    : 2663   12  12  1860    49.8 %   2664   38.5 %
 16 HIARCS 11 UCI                  : 2653   15  15  1380    52.7 %   2634   35.7 %
 17 Toga II 1.3x4                  : 2649   26  26   480    52.4 %   2632   31.9 %
 18 Fruit 2.3.1                    : 2648   18  18   900    51.4 %   2638   35.7 %
 19 Chess Tiger 2007.1 UCI         : 2647   73  75    60    32.5 %   2773   35.0 %
 20 Toga II 1.2.1a                 : 2642   12  12  2040    47.6 %   2658   34.2 %
 21 Deep Junior 10.1               : 2636   12  12  2100    46.9 %   2658   30.6 %
 22 Spike 1.2 Turin                : 2632   12  12  2040    46.2 %   2659   37.0 %
 23 Hiarcs X54 UCI                 : 2628   16  16  1140    51.2 %   2620   35.7 %
 24 Hiarcs X50 UCI                 : 2627   16  16  1140    51.0 %   2620   36.7 %
 25 Fritz 9                        : 2622   15  15  1380    48.1 %   2635   29.9 %
 26 Deep Sjeng 2.7 2T              : 2621   18  18   960    46.8 %   2644   33.8 %
 27 Fruit 2.2.1                    : 2618   12  12  2100    44.2 %   2659   33.5 %
 28 Glaurung 1.2 SMP               : 2584   13  13  2040    39.2 %   2660   28.5 %
 29 Scorpio 2.0 2T                 : 2572   19  19   900    40.6 %   2638   31.7 %
 30 Chess Tiger 2007 UCI           : 2562   12  12  2100    36.1 %   2660   32.3 %
 31 Naum 2.0                       : 2547   16  16  1200    38.9 %   2626   36.2 %
 32 Scorpio 1.91 2CPU              : 2545   14  14  1800    32.4 %   2672   32.0 %
 33 Chess Tiger 15.0               : 2524   17  17  1140    35.8 %   2625   34.5 %
 34 Deep Pharaon 3.5.1             : 2523   13  13  2100    31.1 %   2662   29.5 %
 35 Scorpio 1.8 2CPU               : 2502   16  16  1380    31.1 %   2640   30.0 %
 36 Deep Frenzee 3.0               : 2500   14  14  2100    28.3 %   2662   24.6 %

Code: Select all

5 Fritz 11                  : 2774  180 (+ 66,= 70,- 44), 56.1 %

Deep Fritz 10                 :  60 (+ 28,= 17,- 15), 60.8 %
Rybka 2.3.1 mp 32-bit         :  60 (+  8,= 32,- 20), 40.0 %
Chess Tiger 2007.1 UCI        :  60 (+ 30,= 21,-  9), 67.5 %

[ernest]

First of all, I'd like to thank Jeroen Noomen for providing me with an updated "Noomen Testset."

Hi Erik,
I also like a lot the Noomen2006 testsuite.
Is there a way to get the updated "Noomen Testset" (which I believe you call Noomen2007)?

[Erik Roggenburg]

I don't think that would be a problem, but I've asked Jeroen - just to make sure he's OK with passing it along. I do not want to overstep my bounds.

Erik

[Erik Roggenburg]

First of all, I'd like to thank Jeroen Noomen for providing me with an updated "Noomen Testset."

Hi Erik,
I also like a lot the Noomen2006 testsuite.
Is there a way to get the updated "Noomen Testset" (which I believe you call Noomen2007)?

Send me a PM, and I'll reply with the testset.

If anyone else would like the testset, let me know. Jeroen said I could send it to anyone who asks.

[Erik Roggenburg]

Fritz 11 has dropped off a bit. I knew there was the possibility of a wild swing due to the low initial number of games.

Code: Select all

6 Fritz 11                  : 2739  420 (+201,=141,- 78), 64.6 %

Deep Shredder 10 UCI          :  60 (+ 28,= 19,- 13), 62.5 %
Deep Fritz 10                 :  60 (+ 28,= 17,- 15), 60.8 %
Deep Pharaon 3.5.1            :  60 (+ 38,= 17,-  5), 77.5 %
Deep Frenzee 3.0              :  60 (+ 40,= 17,-  3), 80.8 %
Chess Tiger 2007.1 UCI        :  60 (+ 30,= 21,-  9), 67.5 %
Deep Junior 10.1              :  60 (+ 29,= 18,- 13), 63.3 %
Rybka 2.3.1 mp 32-bit         :  60 (+  8,= 32,- 20), 40.0 %

Code: Select all

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Rybka 2.3.1 mp 32-bit          : 2823   13  13  2080    73.1 %   2649   29.4 %
  2 Rybka 2.2 mp 32-bit            : 2815   15  15  1580    72.7 %   2645   30.4 %
  3 Rybka 2.3 LK mp 32-bit         : 2810   18  18  1140    73.1 %   2637   28.8 %
  4 Rybka 2.3 mp 32-bit            : 2810   18  18  1140    73.0 %   2637   30.4 %
  5 Deep Shredder 11 UCI nobk      : 2740   20  19   900    64.2 %   2638   29.6 %
  6 Fritz 11                       : 2739   28  28   420    64.6 %   2634   33.6 %
  7 Zappa Mexico 2T                : 2731   18  18   960    63.2 %   2637   34.5 %
  8 HIARCS 11.1 MP UCI             : 2725   12  12  2060    60.1 %   2654   36.3 %
  9 Deep Fritz 10                  : 2724   12  12  2160    59.3 %   2658   31.2 %
 10 Loop 13.5.32 2CPU              : 2723   13  13  1860    61.3 %   2643   36.9 %
 11 Naum 2.2 2T                    : 2707   18  18   900    59.8 %   2638   40.6 %
 12 LoopMP 12.32 2CPU              : 2702   13  13  1680    56.9 %   2654   36.5 %
 13 HIARCS 11 MP UCI               : 2695   14  14  1480    58.1 %   2638   37.2 %
 14 Deep Shredder 10 UCI           : 2673   12  12  2160    51.9 %   2660   30.4 %
 15 Naum 2.1 MP                    : 2663   12  12  1860    49.8 %   2664   38.5 %
 16 HIARCS 11 UCI                  : 2653   15  15  1380    52.7 %   2634   35.7 %
 17 Toga II 1.3x4                  : 2649   26  26   480    52.4 %   2632   31.9 %
 18 Fruit 2.3.1                    : 2648   18  18   900    51.4 %   2638   35.7 %
 19 Toga II 1.2.1a                 : 2642   12  12  2040    47.6 %   2659   34.2 %
 20 Deep Junior 10.1               : 2637   12  12  2160    46.6 %   2661   30.6 %
 21 Spike 1.2 Turin                : 2632   12  12  2040    46.2 %   2659   37.0 %
 22 Hiarcs X54 UCI                 : 2628   16  16  1140    51.2 %   2620   35.7 %
 23 Hiarcs X50 UCI                 : 2627   16  16  1140    51.0 %   2620   36.7 %
 24 Fritz 9                        : 2622   15  15  1380    48.1 %   2635   29.9 %
 25 Deep Sjeng 2.7 2T              : 2621   18  18   960    46.8 %   2644   33.8 %
 26 Fruit 2.2.1                    : 2618   12  12  2100    44.2 %   2659   33.5 %
 27 Glaurung 1.2 SMP               : 2584   13  13  2040    39.2 %   2660   28.5 %
 28 Scorpio 2.0 2T                 : 2572   19  19   900    40.6 %   2638   31.7 %
 29 Chess Tiger 2007.1 UCI         : 2563   12  12  2160    36.0 %   2663   32.4 %
 30 Naum 2.0                       : 2548   16  16  1200    38.9 %   2626   36.2 %
 31 Scorpio 1.91 2CPU              : 2545   14  14  1800    32.4 %   2672   32.0 %
 32 Chess Tiger 15.0               : 2524   17  17  1140    35.8 %   2625   34.5 %
 33 Deep Pharaon 3.5.1             : 2524   13  13  2160    30.8 %   2664   29.4 %
 34 Scorpio 1.8 2CPU               : 2502   16  16  1380    31.1 %   2641   30.0 %
 35 Deep Frenzee 3.0               : 2501   14  14  2160    28.0 %   2664   24.7 %

[Erik Roggenburg]

Updated list. I am currently testing Toga 1.3x4. Also of interest: I'm starting to run some "Old Warhorses" just for fun. First Old Warhorse up: Shredder 7.04 UCI running on two threads. Remember that Shredder? What an awesome program at the time!

I just looked at this list. The amount of $$ spent on chess engines... Oh, well. Got to support the programming community, I guess.

Code: Select all

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Rybka 2.3.1 mp 32-bit          : 2822   13  13  2080    73.1 %   2649   29.4 %
  2 Rybka 2.2 mp 32-bit            : 2815   15  15  1580    72.7 %   2645   30.4 %
  3 Rybka 2.3 LK mp 32-bit         : 2810   18  18  1140    73.1 %   2637   28.8 %
  4 Rybka 2.3 mp 32-bit            : 2810   18  18  1140    73.0 %   2637   30.4 %
  5 Deep Shredder 11 UCI           : 2739   19  19   930    64.4 %   2636   29.7 %
  6 Zappa Mexico 2T                : 2731   18  18   960    63.2 %   2637   34.5 %
  7 Fritz 11                       : 2728   18  18   930    62.9 %   2636   35.5 %
  8 HIARCS 11.1 MP UCI             : 2727   12  12  2180    60.1 %   2655   36.6 %
  9 Deep Fritz 10                  : 2723   12  12  2180    59.4 %   2657   31.1 %
 10 Loop 13.5.32 2T                : 2723   12  12  1980    61.0 %   2645   37.4 %
 11 Naum 2.2 2T                    : 2707   18  18   900    59.8 %   2638   40.6 %
 12 LoopMP 12.32 2T                : 2702   13  13  1680    56.9 %   2654   36.5 %
 13 HIARCS 11 MP UCI               : 2695   14  14  1480    58.1 %   2638   37.2 %
 14 Deep Shredder 10 UCI           : 2672   12  12  2190    52.0 %   2658   30.7 %
 15 Naum 2.1 MP                    : 2664   12  12  1920    49.6 %   2666   38.6 %
 16 HIARCS 11 UCI                  : 2653   15  15  1380    52.7 %   2634   35.7 %
 17 Fruit 2.3.1                    : 2648   18  18   900    51.4 %   2638   35.7 %
 18 Toga II 1.2.1a                 : 2643   12  12  2100    47.5 %   2660   34.3 %
 19 Toga II 1.3x4                  : 2640   22  22   660    49.3 %   2644   34.1 %
 20 Deep Junior 10.1               : 2637   12  12  2180    46.7 %   2659   30.6 %
 21 Spike 1.2 Turin                : 2633   12  12  2100    46.1 %   2661   36.9 %
 22 Hiarcs X54 UCI                 : 2628   16  16  1140    51.2 %   2620   35.7 %
 23 Hiarcs X50 UCI                 : 2627   16  16  1140    51.0 %   2620   36.7 %
 24 Deep Sjeng 2.7 2T              : 2623   18  18   990    47.2 %   2642   33.4 %
 25 Fritz 9                        : 2621   15  15  1410    48.2 %   2634   29.6 %
 26 Fruit 2.2.1                    : 2616   12  12  2160    43.7 %   2661   33.4 %
 27 Glaurung 1.2 SMP               : 2586   13  13  2160    39.4 %   2661   28.8 %
 28 Shredder 7.04 2T               : 2584   36  36   260    42.5 %   2637   28.1 %
 29 Scorpio 2.0 2T                 : 2572   19  19   900    40.6 %   2638   31.7 %
 30 Chess Tiger 2007.1 UCI         : 2562   12  12  2180    36.1 %   2661   32.4 %
 31 Naum 2.0                       : 2547   16  16  1200    38.9 %   2626   36.2 %
 32 Scorpio 1.91 2T                : 2544   13  13  1860    32.2 %   2674   32.0 %
 33 Chess Tiger 15.0               : 2524   17  17  1140    35.8 %   2625   34.5 %
 34 Deep Pharaon 3.5.1             : 2524   13  13  2190    31.1 %   2662   29.3 %
 35 Scorpio 1.8 2T                 : 2502   16  16  1380    31.1 %   2640   30.0 %
 36 Deep Frenzee 3.0               : 2499   14  14  2180    28.0 %   2663   24.7 %