Ordo 1.0.9 (new features for testers)

[michiguel]

https://github.com/michiguel/Ordo/releases/tag/1.0.9

You can select columns to display, you can specify synonyms, and you can input many files at the same time.

ordo -QWD -Y syn.txt -U "0,1,2,3,4,5,6,7,8,9,10,11" -s100 -- games1.pgn games2.pgn

outputs

Code: Select all

   
   # PLAYER      : RATING  ERROR    POINTS  PLAYED     (%)   CFS(next)       W       D       L    D(%)  OppAvg
   1 engine_A    : 2583.3   95.2      53.0      60   88.3%      97          50       6       4    10.0  2205.6
   2 engine_B    : 2434.3   93.0      41.0      60   68.3%     100          38       6      16    10.0  2255.2
   3 engine_D    : 2108.2   97.4      15.5      60   25.8%      67          12       7      41    11.7  2363.9
   4 engine_C    : 2074.2   86.8      10.5      60   17.5%     ---           7       7      46    11.7  2375.3

but this
ordo -QWD -Y syn.txt -U "0,1,11" -s100 -- games1.pgn games2.pgn

will output

Code: Select all

   # PLAYER      : RATING  OppAvg
   1 engine_A    : 2583.3  2205.6
   2 engine_B    : 2434.3  2255.2
   3 engine_D    : 2108.2  2363.9
   4 engine_C    : 2074.2  2375.3

The switch -Y tells that the file syn.txt contains a list of synonyms. In this case the file contains

"engine_A","A"
"engine_B","B"
"engine_C","C"
"engine_D","D","Dd"

Which means that every time the name of a engine is "A", it will be translated into engine_A. This is very important when the same engine has been use in different databases with different names and we want to merge both files. No need to replace the name in any file.

In addition, the example uses two different input files, game1.pgn and game2.pgn. No need to join them beforehand. You can include a long list of files after --

Miguel

[IWB]

Hello Miguel,

That is nice, especially the multiple file possibility. I allways feard that the size of my pgn file will be too big sooner or later.

Btw, is there a limit of the file size Ordo can handle and what is the limit for the number of individual PGNs it can include into one list?

The alias feature for engines will become quite complicated if many names are used. I would guess it is easier to rename them in the original PGN files to one standard - some might see that different

Thanks again
Ingo

[Modern Times]

The alias feature for engines will become quite complicated if many names are used. I would guess it is easier to rename them in the original PGN files to one standard - some might see that different

Thanks again
Ingo

Our processing scripts use an alias file. Essential given that testers often submit games with variations of the same name. You set up the mapping once, and then it is done.

[Frank Quisinsky]

Hi Migual,

I wish me to see ...

Example

Code: Select all

    # PLAYER      : RATING  ERROR    POINTS  PLAYED     (%)   Q-OPP  ERROR OPP  CFS(next)       W       D       L    D(%)  OppAvg 
    1 engine_A    : 2583.3   95.2      53.0      60   88.3%      42       97.4     97          50       6       4    10.0  2205.6 	

Q-OPP
= Quantity opponents

ERROR OPP
= Average of ERROR by opponents

I can faster see how strong is a Rating List because after quantity of games most important is quantity of opponents for a more exactly rating.

Congratulation ... your changes are good to me.

THANKS and

Best
Frank

PS:
The more opponents and the lower the ERROR the opponents have the better (lower) must be the own ERROR. Missed in each calculation. So it's a good evidence / indication for a rating list to see how many opponents and the average on ERROR the opponents have.

Example 1:
Eng A vs. Eng B plays 1000 games = 1000 games.

Example 2:
Eng A vs. 40 different other engines plays 25 games = 1000 games.

Rating of Eng A is more exactly with Example 2. and the ERROR must be lower.

We can't compare ERROR with ERROR in different Rating Lists if the Quantity of Opponents is quit different. In my humble opinion the quantity of games (5 stars) and the quantity of opponents (4 stars). Means a bit more important is of course quantity of games but we forget in the calculation the very important quantity of opponents. I know that isn't easy to calculate but we can view in the table "Quantity of opponents" and average of ERROR the opponents have.

[Frank Quisinsky]

An easy example:

Shredder 12 and Cheng 4.39 have the same Rating = ~ 2.800!

Shredder 12 is bad in king safety and Cheng is strong in king saftety. Shredder lose many games very fast, Cheng wins many games very fast.

Now ...
Shredder 12 and Cheng have to play vs. Frenzee.

Frenzee is bad in king safety.

What will be the result if both ... Cheng & Shredder ... have to play vs. Frenzee.

At the moment Frenzee is playing the games for my Rating List.
Current results:

Code: Select all

Frenzee 3.5.19 x64 - Shredder 12 x64 (2802)  13.0 - 26.0  33.33%  Perf=2682 
Frenzee 3.5.19 x64 - Cheng 4.39 x64 (2801)  5.5 - 34.5  13.75%  Perf=2483

I can build such examples with the playing style discription on my webpage very often very easy without to know the results between the engines.

That's what I mean ...
Shredder will have with more opponents and engines bad in endgames and other final rating ... compare to Shredder have opponents only strong in the first game phase.

Shredder will makes the points vs. engines weak in endgames, Cheng can't make.

Quantity of opponents is very very important if I have interest to see how strong and exactly is a Rating List.

1000 games Cheng vs. Frenzee ...
You will never get the exactly results the ERROR views.

Best
Frank


In my humble opinion two ways:

1.
Rating List should not viewed ERROR (most uninteresting without more information).

2.
More information about opponents and we can build after viewing a Rating List better the own opinion about the ERROR information.

[Ozymandias]

You can select columns to display, you can specify synonyms, and you can input many files at the same time.

All three are quite useful, but the last one in particular, is a time saver.

Code: Select all

┈┈┈┈┈┈▕▔╲
┈┈┈┈┈┈┈▏▕
┈┈┈┈┈┈┈▏▕▂▂▂
▂▂▂▂▂▂╱┈▕▂▂▂▏
▉▉▉▉▉┈┈┈▕▂▂▂▏
▉▉▉▉▉┈┈┈▕▂▂▂▏
▔▔▔▔▔▔╲▂▕▂▂▏

[michiguel]

Hi Migual,

I wish me to see ...

Example

Code: Select all

    # PLAYER      : RATING  ERROR    POINTS  PLAYED     (%)   Q-OPP  ERROR OPP  CFS(next)       W       D       L    D(%)  OppAvg 
    1 engine_A    : 2583.3   95.2      53.0      60   88.3%      42       97.4     97          50       6       4    10.0  2205.6 	

Q-OPP
= Quantity opponents

ERROR OPP
= Average of ERROR by opponents

I think I can incorporate both.

Miguel

I can faster see how strong is a Rating List because after quantity of games most important is quantity of opponents for a more exactly rating.

Congratulation ... your changes are good to me.

THANKS and

Best
Frank

PS:
The more opponents and the lower the ERROR the opponents have the better (lower) must be the own ERROR. Missed in each calculation. So it's a good evidence / indication for a rating list to see how many opponents and the average on ERROR the opponents have.

Example 1:
Eng A vs. Eng B plays 1000 games = 1000 games.

Example 2:
Eng A vs. 40 different other engines plays 25 games = 1000 games.

Rating of Eng A is more exactly with Example 2. and the ERROR must be lower.

We can't compare ERROR with ERROR in different Rating Lists if the Quantity of Opponents is quit different. In my humble opinion the quantity of games (5 stars) and the quantity of opponents (4 stars). Means a bit more important is of course quantity of games but we forget in the calculation the very important quantity of opponents. I know that isn't easy to calculate but we can view in the table "Quantity of opponents" and average of ERROR the opponents have.

[michiguel]

Hello Miguel,

That is nice, especially the multiple file possibility. I allways feard that the size of my pgn file will be too big sooner or later.

Btw, is there a limit of the file size Ordo can handle and what is the limit for the number of individual PGNs it can include into one list?

The alias feature for engines will become quite complicated if many names are used. I would guess it is easier to rename them in the original PGN files to one standard - some might see that different

Thanks again
Ingo

The file size has no limits, which means it is determined by the memory of the computer. The number of pgn files max is 1024, but I can easily change that (I figure that is big enough). I will make it also limitless, but first I wanted to see if this works properly. Anyway, there are already limits in the OS regarding how long the command line could be. For that reason, the input may need to change with a new switch -P <file>, in which that file will contain a list of pgns. For big number of files, that would be more efficient.

Miguel

[Frank Quisinsky]

Hi Miguel,

-Q is great
-j is fantastic

This one ...
Ordo_109-x64.exe -a 2890 -A "Naum 4.6 x64" -p 1.pgn -o ordo109.txt -j programs.txt -QWDV -U "0,1,4,5,10,2,11,6" -s1000

I need any longer -E (ELOstat) option.

And I got ... (sorted with for me most important information at first)

Code: Select all

   # PLAYER                           : RATING  PLAYED     (%)    D(%)  ERROR  OppAvg   CFS(next)
   1 Komodo 9.3 x64                   : 3178.1    2100   84.1%    27.7   15.7  2854.9      65    
   2 Komodo 9.2 x64                   : 3173.8    2400   84.1%    27.3   15.0  2852.0      68    
   3 Stockfish 02Dec2015 BMI2 x64     : 3168.5    2100   83.4%    30.0   15.9  2855.4      70    
   4 Stockfish 20Aug2015 BMI2 x64     : 3162.7    2400   83.3%    29.9   13.7  2852.5      60    
   5 Stockfish 26Apr2015 BMI2 x64     : 3160.1    2250   82.9%    31.0   14.0  2856.8      55    
   6 Komodo 9 x64                     : 3158.8    2250   82.8%    29.3   14.5  2856.9      92    
   7 Stockfish 6 BMI2 x64             : 3143.2    1700   83.8%    28.9   16.6  2832.3      99    
   8 Stockfish 02Oct2014 BMI2 x64     : 3114.0    1150   79.5%    34.6   19.1  2854.3      56    
   9 Komodo 8 x64                     : 3112.3    2100   79.8%    32.8   14.0  2846.0      50    
  10 Stockfish 03Aug2014 BMI2 x64     : 3112.2    2100   81.1%    32.5   14.6  2832.8      97    
  11 Houdini 4 STD B x64              : 3093.3    2650   75.7%    32.2   12.4  2867.9      52    
  12 Stockfish 26Mar2014 SSE42 x64    : 3092.6    1050   81.8%    31.5   21.8  2809.0      63    
  13 Komodo 7a x64                    : 3087.7    1250   79.6%    31.9   17.9  2825.8      54    
  14 Stockfish 5 SSE42 x64            : 3086.3    1000   79.6%    35.4   19.6  2828.1      98    
  15 Komodo TCECr x64                 : 3056.9    1050   78.5%    33.0   18.9  2810.7      72    
  16 Fire 4 x64                       : 3050.5    3400   71.2%    37.2   10.5  2868.1      84    
  17 GullChess 3.0 BMI2 x64           : 3043.3    3550   69.1%    41.7    9.8  2881.6     100    
  18 GullChess 2.8 Beta BMI2 x64      : 2997.5    1050   72.4%    39.3   18.9  2813.5      56    
  19 Equinox 3.30 x64                 : 2996.0    3350   65.0%    43.0    9.7  2872.4      99    
  20 Critter 1.6a x64                 : 2979.8    2650   62.9%    45.8   10.3  2878.2      99    
  21 Rybka 4.1 SSE42 x64              : 2962.4    2750   62.3%    43.1   10.6  2865.3      54    
  22 Fire 3.0 AVX x64                 : 2961.6    2100   64.8%    41.7   12.2  2841.7      83    
  23 Protector 1.9.0 x64              : 2952.9    2200   61.1%    45.6   11.4  2867.0      75    
  24 Nirvanachess 2.2 POP x64         : 2946.9    2250   60.2%    45.0   12.1  2868.7      53    
  25 Sting SF 4.8.4 x64               : 2946.3    3100   58.8%    44.6   10.1  2877.8      90    
  26 Protector 1.8.0 x64              : 2935.7    1850   56.4%    48.1   13.2  2886.4      93    
  27 Andscacs 0.83 POP x64            : 2922.4    2150   57.4%    43.6   11.4  2867.3      65    
  28 Protector 1.7.0 x64              : 2919.3    2150   55.8%    45.1   11.8  2877.0      89    
  29 Critter 0.90 SSE4 x64            : 2909.1    2150   55.9%    44.6   11.1  2865.7      64    
  30 Hannibal 1.5 x64                 : 2906.5    3400   56.3%    45.0    9.3  2859.5      52    
  31 Texel 1.05 x64                   : 2906.1    3400   56.0%    41.3    8.9  2861.6      64    
  32 Chiron 2.0 x64                   : 2904.0    5750   59.7%    39.2    7.2  2829.1      55    
  33 Nirvanachess 2.1c POP x64        : 2903.1    2750   59.0%    44.6   10.4  2834.8      91    
  34 Protector 1.6.0 x64              : 2891.8    1600   58.7%    44.4   13.6  2826.4      58    
  35 Naum 4.6 x64                     : 2890.0    4250   58.4%    42.1    8.5  2825.8      69    
  36 Andscacs 0.81 POP x64            : 2886.2    2100   53.9%    39.7   12.0  2855.6      56    
  37 Andscacs 0.82 POP x64            : 2884.9    2050   55.0%    42.2   11.6  2849.4      96    
  38 Hannibal 1.4b x64                : 2871.0    3550   56.6%    42.8    9.0  2822.1      98    
  39 Texel 1.04 x64                   : 2856.9    3000   55.8%    41.1    9.9  2815.8      54    
  40 Protector 1.5.0 JA x64           : 2855.8    1050   55.0%    45.3   16.3  2820.3      73    
  41 Nirvanachess 2.0a x64            : 2849.5    2400   52.2%    44.1   11.0  2837.1      92    
  42 Hakkapeliitta 3.0 x64            : 2839.7    4650   49.2%    38.3    7.8  2850.6      72    
  43 Senpai 1.0 SSE42 x64             : 2836.5    6050   52.3%    40.8    6.8  2823.6      89    
  44 Hiarcs 14 WCSC w32               : 2830.6    6050   51.6%    41.7    6.9  2823.7      95    
  45 Andscacs 0.72 POP x64            : 2819.7    2500   48.1%    42.7   10.8  2840.0      92    
  46 Fizbo 1.5 x64                    : 2809.8    3250   50.0%    44.0    9.3  2815.3      63    
  47 Sjeng c't 2010 w32               : 2807.6    4150   48.9%    40.5    8.4  2823.1      83    

...
and the others!

programs.txt I created with -j (compare to ELOstat but much better)

Code: Select all

Head to head statistics:

  1) Komodo 9.3 x64                3178.1 :   2100 (+1475,=582,-43),  84.1 %

     vs.                                  :  games (    +,   =,  -),   (%) :    Diff,    SD, CFS (%)
     Stockfish 02Dec2015 BMI2 x64         :    100 (   15,  69, 16),  49.5 :    +9.6,  10.9,   81.0
     Houdini 4 STD B x64                  :     50 (   21,  24,  5),  66.0 :   +84.8,   9.8,  100.0
     Fire 4 x64                           :     50 (   21,  26,  3),  68.0 :  +127.6,   9.5,  100.0
     GullChess 3.0 BMI2 x64               :     50 (   15,  32,  3),  62.0 :  +134.8,   9.0,  100.0
     Equinox 3.30 x64                     :     50 (   30,  19,  1),  79.0 :  +182.0,   9.4,  100.0
     Critter 1.6a x64                     :     50 (   27,  22,  1),  76.0 :  +198.2,   9.6,  100.0
     Rybka 4.1 SSE42 x64                  :     50 (   26,  23,  1),  75.0 :  +215.6,   9.7,  100.0
     Protector 1.9.0 x64                  :     50 (   24,  23,  3),  71.0 :  +225.2,   9.8,  100.0
     Nirvanachess 2.2 POP x64             :     50 (   32,  17,  1),  81.0 :  +231.1,  10.2,  100.0
     Sting SF 4.8.4 x64                   :     50 (   32,  18,  0),  82.0 :  +231.7,   9.4,  100.0
     Andscacs 0.83 POP x64                :     50 (   35,  13,  2),  83.0 :  +255.7,   9.7,  100.0
     Hannibal 1.5 x64                     :     50 (   29,  20,  1),  78.0 :  +271.6,   9.3,  100.0
     Texel 1.05 x64                       :     50 (   38,  11,  1),  87.0 :  +272.0,   9.0,  100.0
     Chiron 2.0 x64                       :     50 (   34,  16,  0),  84.0 :  +274.1,   8.7,  100.0
     Naum 4.6 x64                         :     50 (   39,  11,  0),  89.0 :  +288.1,   9.1,  100.0
     Hakkapeliitta 3.0 x64                :     50 (   37,  11,  2),  85.0 :  +338.4,   9.0,  100.0
     Senpai 1.0 SSE42 x64                 :     50 (   42,   7,  1),  91.0 :  +341.5,   8.8,  100.0
     Hiarcs 14 WCSC w32                   :     50 (   41,   9,  0),  91.0 :  +347.5,   8.6,  100.0
     Fizbo 1.5 x64                        :     50 (   40,  10,  0),  90.0 :  +368.3,   9.2,  100.0
     Sjeng c't 2010 w32                   :     50 (   42,   8,  0),  92.0 :  +370.5,   9.3,  100.0
     Shredder 12 x64                      :     50 (   40,  10,  0),  90.0 :  +376.0,   9.0,  100.0
     Cheng 4.39 x64                       :     50 (   43,   7,  0),  93.0 :  +377.5,   9.3,  100.0
     Vajolet2 2.0 POP x64                 :     50 (   36,  14,  0),  86.0 :  +386.4,   9.2,  100.0
     Junior 13.3.00 x64                   :     50 (   38,  12,  0),  88.0 :  +388.0,   8.6,  100.0
     Spike 1.4 Leiden w32                 :     50 (   37,  13,  0),  87.0 :  +391.4,   8.7,  100.0
     Booot 5.2.0 x64                      :     50 (   43,   7,  0),  93.0 :  +398.6,   9.3,  100.0
     DiscoCheck 5.2.1 x64                 :     50 (   41,   9,  0),  91.0 :  +404.6,   9.1,  100.0
     iCE 2.0 v2240 POP x64                :     50 (   47,   3,  0),  97.0 :  +411.0,   8.4,  100.0
     Quazar 0.4 x64                       :     50 (   41,   9,  0),  91.0 :  +411.3,   8.8,  100.0
     Deuterium 14.3.34.130 POP x64        :     50 (   46,   4,  0),  96.0 :  +413.5,   9.0,  100.0
     Spark 1.0 x64                        :     50 (   39,  11,  0),  89.0 :  +417.6,   8.7,  100.0
     SmarThink 1.70 SSE3 x64              :     50 (   36,  14,  0),  86.0 :  +419.8,   8.9,  100.0
     Alfil 15.04 C# Beta 24 x64           :     50 (   39,  11,  0),  89.0 :  +422.9,   9.2,  100.0
     Zappa Mexico II x64                  :     50 (   37,  13,  0),  87.0 :  +428.1,   8.8,  100.0
     Thinker 5.4d Inert x64               :     50 (   40,   9,  1),  89.0 :  +433.1,   9.2,  100.0
     TogaII 280513 Intel w32              :     50 (   39,  11,  0),  89.0 :  +445.2,   9.5,  100.0
     Gaviota 1.0 AVX x64                  :     50 (   43,   7,  0),  93.0 :  +448.6,   8.6,  100.0
     Arasan 18.1 POP x64                  :     50 (   39,  10,  1),  88.0 :  +449.6,   9.4,  100.0
     Atlas 3.80 x64                       :     50 (   43,   7,  0),  93.0 :  +450.1,   9.1,  100.0
     MinkoChess 1.3 JA POP x64            :     50 (   41,   9,  0),  91.0 :  +455.5,   9.1,  100.0
     EXchess 7.71b x64                    :     50 (   47,   3,  0),  97.0 :  +464.5,   9.4,  100.0

and the others ...

And now ... after OppAvg
The Information with Quantity of Opp and after this one the Information with Average Error of Opp and all is perfect for me.

Very nice work you do with your last changes.
I am really happy with your really gigantic great work here !!!

FCP Rating List will be in the future displayed with my first table and the programs.txt for my database download with the others important files.

Best
Frank

[Frank Quisinsky]

And for my FCT League 1-3 I need only this one ...
Most important at first ...

Example:
FCT League 2 (10.500 games).

Ordo_109-x64.exe -a 2785 -A "Spike 1.4 Leiden w32" -p 1.pgn -o ordo109.txt -j programs.txt -QWDV -U "0,1,10,4,3,7,8,9,6" -s1000

Code: Select all

   # PLAYER                           : RATING    D(%)  PLAYED    POINTS       W       D       L   CFS(next)
   1 Shredder 12 x64                  : 2802.6    44.3    1000     549.5     328     443     229      61    
   2 Cheng 4.39 x64                   : 2799.2    46.3    1000     544.5     313     463     224      50    
   3 Vajolet2 2.0 POP x64             : 2799.2    42.9    1000     544.5     330     429     241      80    
   4 Junior 13.3.00 x64               : 2789.7    44.3    1000     530.5     309     443     248      58    
   5 DiscoCheck 5.2.1 x64             : 2787.4    43.2    1000     527.0     311     432     257      56    
   6 Quazar 0.4 x64                   : 2785.7    44.9    1000     524.5     300     449     251      52    
   7 Spike 1.4 Leiden w32             : 2785.0    45.9    1000     523.5     294     459     247      61    
   8 Booot 5.2.0 x64                  : 2781.7    44.5    1000     518.5     296     445     259      54    
   9 iCE 2.0 v2240 POP x64            : 2780.6    44.2    1000     517.0     296     442     262      75    
  10 Deuterium 14.3.34.130 POP x64    : 2772.5    45.6    1000     505.0     277     456     267      66    
  11 Alfil 15.04 C# Beta 24 x64       : 2767.8    37.8    1000     498.0     309     378     313      64    
  12 SmarThink 1.70 SSE3 x64          : 2763.4    40.3    1000     491.5     290     403     307      51    
  13 Zappa Mexico II x64              : 2763.0    45.6    1000     491.0     263     456     281      57    
  14 Spark 1.0 x64                    : 2761.0    45.8    1000     488.0     259     458     283      58    
  15 Doch 1.3.4 JA x64                : 2758.7    48.3    1000     484.5     243     483     274      72    
  16 Crafty 25.0 DC x64               : 2751.9    46.3    1000     474.5     243     463     294      67    
  17 Thinker 5.4d Inert x64           : 2747.2    45.3    1000     467.5     241     453     306      60    
  18 Gaviota 1.0 AVX x64              : 2744.4    41.7    1000     463.5     255     417     328      60    
  19 Atlas 3.80 x64                   : 2741.7    44.9    1000     459.5     235     449     316      67    
  20 TogaII 280513 Intel w32          : 2736.6    42.6    1000     452.0     239     426     335      65    
  21 Arasan 18.1 POP x64              : 2732.2    44.7    1000     445.5     222     447     331     ---    

White advantage = 30.65 +/- 2.55
Draw rate (equal opponents) = 44.86 % +/- 0.49

Not sure with CFS ...
Should I display CFS for League 1-3 / FCP Rating List or not?

ERROR not important here ...
OppAvg not important here ...

I can created the perfect table for each constellation!
Again ... very nice changes you do!

Biggest compliment to you and of course your tester Adam too.

Best
Frank