SWCR after 17.480 games, final results from Naum 4.2

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

SWCR after 17.480 games, final results from Naum 4.2

Post by Frank Quisinsky »

Hi there,

the Naum 4.2 test was horrible.
Better the statistics, the engine is great ... runs without any problem, very nice games, very strong endgame.

Naum lost from game number 135 to 760 51 ELO points.

Better, from game number 300 up to 410 32 ELO points.
Never I saw such a statistic and I made it now a long time.

OK, here are the actual SWCR:

- each one vs. each one 40 games
- 40 moves in 10 minutes up to mate, remis. One game need around 40 minutes.
- only 32-Bit, only 1 Core, all with ponder = on
- 4-pieces table-bases.

This is the main ratinglist list.
Normaly it isn't a rating list, more or less a big tournament.

Code: Select all

Rank Name                      Elo    +    - games score oppo. draws 
   1 Rybka 3                  2852   22   21   760   75%  2673   29% 
   2 Naum 4.2                 2822   21   21   760   71%  2675   34% 
   3 Stockfish 1.6.3 JA       2820   21   21   760   71%  2675   35% 
   4 Shredder 12              2800   21   20   760   68%  2676   34% 
   5 Deep Fritz 12            2773   20   20   760   64%  2677   41% 
   6 Komodo 1.0 JA            2710   20   19   760   55%  2681   41% 
   7 Thinker 5.4d Inert       2706   20   20   760   54%  2681   39% 
   8 Protector 1.3.2          2675   20   20   760   49%  2682   41% 
   9 Hiarcs 12.1 Sharpen PV   2662   20   20   760   47%  2683   38% 
  10 Sjeng WC-2008            2660   20   20   760   47%  2683   38% 
  11 Spark 0.3a               2655   19   20   760   46%  2683   42% 
  12 Zappa Mexico II          2651   19   19   760   45%  2684   43% 
  13 Junior 11.1a             2649   20   20   760   45%  2684   38% 
  14 Onno 1.1.1               2628   20   20   760   42%  2685   41% 
  15 Critter 0.52b            2611   20   20   760   39%  2686   38% 
  15 Fruit 05/11/03           2611   20   20   760   39%  2686   41% 
  17 Ktulu 9.03               2605   20   21   760   39%  2686   30% 
  18 Loop 13.6                2601   20   20   760   38%  2686   39% 
  19 SmarThink 1.20           2583   20   20   760   35%  2687   39% 
  20 Twisted Logic 20100131x  2567   21   21   760   33%  2688   32%


And here the list after all played games (17.640 so far):

Code: Select all

Rank Name                      Elo    +    - games score oppo. draws 
   1 Rybka 3                  2858   17   17  1320   74%  2687   31% 
   2 Naum 4.2                 2826   21   21   760   71%  2679   34% 
   3 Stockfish 1.6.3 JA       2824   19   19   960   70%  2684   36% 
   - Naum 4.1                 2810   19   19   920   68%  2684   35% 
   - Stockfish 1.6.0 JA       2801   18   18   960   68%  2686   39% 
   4 Shredder 12              2800   16   16  1320   66%  2689   36% 
   - Naum 4.0                 2784   18   18   960   65%  2681   38% 
   5 Deep Fritz 12            2779   19   19   840   64%  2687   41% 
   - Fritz 12                 2745   16   16  1160   59%  2687   44% 
   - Stockfish 1.5.1 JA       2730   21   21   680   57%  2684   43% 
   6 Komodo 1.0 JA            2715   18   17  1000   54%  2693   40% 
   7 Thinker 5.4d Inert       2711   15   15  1320   53%  2691   43% 
   - Doch 1.3.4 JA            2689   19   19   840   51%  2685   44% 
   - Cyclone xTreme Wrath     2675   17   17  1080   47%  2696   41% 
   8 Protector 1.3.2          2673   16   16  1160   47%  2694   41% 
   9 Junior 2010              2668   16   16  1240   47%  2690   36% 
   - Doch 1.2 JA              2667   20   21   720   47%  2689   40% 
  10 Hiarcs 12.1              2666   19   18   880   47%  2688   41% 
   - Protector 1.3.1b         2666   19   19   840   47%  2689   42% 
   - Hiarcs 12.1 Sharpen PV   2665   17   17  1080   45%  2698   40% 
  11 Sjeng WC-2008            2657   15   15  1320   45%  2693   37% 
  11 Zappa Mexico II          2657   15   15  1320   45%  2693   44% 
   - Doch 09.980 JA           2655   21   21   680   45%  2689   43% 
  13 Spark 0.3a               2655   17   17  1080   44%  2698   41% 
   - Junior 11.1a             2653   20   20   760   45%  2688   38% 
   - Spark 0.3                2642   19   19   880   43%  2689   42% 
  14 Onno 1.1.1               2631   15   15  1320   41%  2694   43% 
  15 Fruit 05/11/03           2616   16   16  1240   38%  2694   41% 
  16 Loop 13.6                2615   15   16  1320   38%  2694   40% 
  17 Critter 0.52b            2613   19   19   880   38%  2697   37% 
   - Glaurung 2.2 JA          2609   17   17  1080   37%  2699   36% 
  18 Ktulu 9.03               2607   20   20   840   38%  2695   29% 
  19 SmarThink 1.20           2597   16   16  1320   36%  2695   38% 
  20 Twisted Logic 20100131x  2570   20   20   880   32%  2699   31%
At last the results from Naum 4.2 (latest engine I tested):

Code: Select all

2 Naum 4.2                  : 2826  760 (+411,=257,- 92), 71.0 %

Onno 1.1.1                    :  40 (+ 21,= 15,-  4), 71.2 %
Rybka 3                       :  40 (+ 12,= 16,- 12), 50.0 %
Zappa Mexico II               :  40 (+ 26,= 13,-  1), 81.2 %
Thinker 5.4d Inert            :  40 (+ 21,= 15,-  4), 71.2 %
SmarThink 1.20                :  40 (+ 24,= 15,-  1), 78.8 %
Shredder 12                   :  40 (+ 16,= 17,-  7), 61.3 %
Sjeng WC-2008                 :  40 (+ 22,= 12,-  6), 70.0 %
Loop 13.6                     :  40 (+ 26,= 11,-  3), 78.8 %
Deep Fritz 12                 :  40 (+ 13,= 23,-  4), 61.3 %
Critter 0.52b                 :  40 (+ 28,= 12,-  0), 85.0 %
Ktulu 9.03                    :  40 (+ 29,=  8,-  3), 82.5 %
Spark 0.3a                    :  40 (+ 23,= 14,-  3), 75.0 %
Twisted Logic 20100131x       :  40 (+ 32,=  7,-  1), 88.8 %
Protector 1.3.2               :  40 (+ 18,= 16,-  6), 65.0 %
Fruit 05/11/03                :  40 (+ 26,= 11,-  3), 78.8 %
Hiarcs 12.1 Sharpen PV        :  40 (+ 22,= 13,-  5), 71.2 %
Komodo 1.0 JA                 :  40 (+ 18,= 14,-  8), 62.5 %
Stockfish 1.6.3 JA            :  40 (+  9,= 16,- 15), 42.5 %
Junior 11.1a                  :  40 (+ 25,=  9,-  6), 73.8 %
Next engines:

1. Protector 1.3.4 JA
2. Engine 21: Spike 1.2 Turin

Best
Frank