BoCC -- Beauty of Computer Chess

[pohl4711]

Ed, please, I just really believe, that using results of Patricia/Cereberus is a great opportunity to verify, if our tools are working or not.
Of course, you dont have to confirm anything of my testings. You could easily do own testruns. Or build a Cerberus with CSTal 2.1 EAS as first engine (instead of Patricia) and compare the results of this Cerberus with CSTal 2.1 EAS results. And I just realized, you have to do own testruns, because I made the stupid mistake to not copy the games of Cerberus 21124081r81 into my achive-files, so that the evals of these games are lost (in my full ratinglist, the games have no comments, to save space). And your tool needs the evals.
Because Cerberus and Patricia (or CSTal if you wish) are identical in the early stage of the game but not after that. That is great for doing such comparisons and see, if the tools show valid results. And Cerberus is just an empty adapter, you can use any UCI-egines you want and switch between them at any number of pieces (using movenumber for switching is possible, too, but this is a stupid idea, IMO). And only 1-2 minutes are needed to set up a working Cerberus. This is just perfect for testing the tools, dont you agree?

The strange results of an old EAS-Tool, you posted here, I cannot explain. For this, I need the data, the tool was feeded with. Because the avg. win move of Patricia 5 is 60 in both cases, this should not happen. Getting the data would be very helpful. And seeing, if the latest version of the EAS-Tool (5.8) shows the same strange output.
I am not bothered if I get bug reports. Definitly not. But I need the data, not just strange outputs, if you want the bug to get fixed. Please be kind.

And I never doubted, you are bigger than me. But I can say it clearly here, everybody can read, the 3 truths about Ed Schröder and myself: Ed: You are smarter than myself, you are a much better programmer than myself, you have achieved much more in computerchess than myself.

This is the truth, now and forever.

[Rebel]

These are the games, 11 engines matches collected in one file BoCC-3578.pgn, create a full EAS list from it and then do each engine separately and notice the sometime gigantic differences. I am almost certain the gauntlet runs are correct.

In retrospect there was another sign something was wrong, when I made one file of your 7 archives I was surprised to see Cstal on top while this was not so on your pages. What both (BoCC-3578-pgn and the 7 archives) have in common is that they have multiple Patricia's and Rebel's and that maybe (I am guessing) PGN-EXTRACT gets confused by the names.

Good luck with the fix.

Meanwhile I am working on a new page without EAS.

https://rebel7775.wixsite.com/rebel/bea ... uter-chess

[Rebel]

https://rebel7775.wixsite.com/rebel/bea ... uter-chess

NEW VERSION

Users are advised to re-download
. BoCC-3400
. BoCC-3500
. BoCC-3578

[Rebel]

@Stefan

Downloaded your latest EAS version
. EAS_Tool_V5.8
. Gauntlet_EAS_Tool_V5.7

Unfortunately it's still there.

Code: Select all

EAS_Tool_V5.8
                                 bad  avg.win 
Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    418009  50.85%  52.75%  09.39%   53   Rebel-Extreme-1.1-BETA  
   2    381831  48.83%  50.78%  09.51%   55   CSTal-2.1-EAS  
   3    374393  53.59%  35.95%  07.18%   60   Patricia-5.0  
   4    351240  47.57%  50.49%  05.82%   58   Patricia-3.01  

Code: Select all

Gauntlet_EAS_Tool_V5.7
                                 bad  avg.win 
Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   x    415009  50.85%  52.75%  09.39%   53   Rebel-Extreme-1.1-BETA  
   x    378831  48.83%  50.78%  09.51%   55   CSTal-2.1-EAS  
   x    466812  53.59%  56.86%  07.18%   60   Patricia-5.0  
   x    296644  47.57%  33.98%  05.82%   58   Patricia-3.01  

Again, it's in the shortie part the Patricia's are hurt. It's like bonus points are given to wrong Patricia versions.

Hope it helps.

[pohl4711]

Thank you for the games. I am looking into it.
First step, I see no bug:
I made EAS-calculation of the Patricia 5 games alone (Gauntlet-EAS-Tool V5.7, fixed to 60 moves short wins bonus limit):

Code: Select all

                                 bad  avg.win 
Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    466812  53.59%  56.86%  07.18%   60   Patricia-5.0  

The I tried all Patricia.pgn merged into 1 file (using EAS-Tool V5.8 fixed to 60 moves short wins bonus limit):

Code: Select all

                                 bad  avg.win 
Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    451812  53.59%  56.86%  07.18%   60   Patricia-5.0  
   2    445995  47.57%  70.87%  05.82%   58   Patricia-3.01  
   3    422109  49.01%  57.40%  06.25%   59   Patricia-4  
   4    419228  49.30%  56.65%  07.56%   59   Patricia-4-dev  
   5     74082  01.07%  34.08%  47.37%   68   seer_v2.8  
   6     73849  01.40%  35.89%  47.90%   68   viridithas-14.0.1  
   7     67817  01.86%  30.80%  37.99%   70   Obsidian130  
   8     67021  01.54%  31.38%  41.89%   69   Clover.8.0.2  
   9     58127  01.95%  27.90%  38.84%   72   Lizard-11_0  
  10     54990  01.34%  27.83%  43.48%   70   berserk-13  
  11     47716  01.38%  25.17%  41.89%   72   PlentyChess-2.1.0  
  12     46402  01.50%  26.26%  44.26%   73   Alexandria-7.0  
  13     46331  01.52%  23.41%  41.27%   73   caissa-1.20  
  14     40238  01.83%  22.03%  44.81%   75   Titan-1.1  

Until here, all looks good. shorts-stat of Patricia 5 is 56.86% in both EAS-calculations

[pohl4711]

I think, I understand whats happend. Its not a bug in the tools, you mismatched results, using Gauntlet-EAS tool and/or EAS-tool with the hardcoded movelimit for short-wins bonus and the normal EAS-tool (here this movelimit is calculated out of the average length of all won games of the source.pgn). This leads (of course) to completely different values for the shorts-stat. But not only Patricia, this affects of course, all engines in a source.pgn.
From the ReadMe:
"Since V5.2, the move-limit is no longer fixed to 40-60 moves, but the average length of all won games in the source.pgn is calculated, rounded to 5 or 10 and -15."
In the Bocc.pgn (full database) the average length of the won games is 69 moves. So, shortwins movelimit is 65-15= 50 if the EAS-Tool calculates it.
This is a huge difference to the hardcoded value of 60.

Here the EAS-calculation with shortwins=60 binary (from the "for engine developers"-folder of my EAS-tool):

Code: Select all

                                 bad  avg.win 
Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    511259  50.85%  71.40%  09.39%   53   Rebel-Extreme-1.1-BETA  
   2    475394  48.83%  68.36%  09.51%   55   CSTal-2.1-EAS  
   3    448812  53.59%  56.86%  07.18%   60   Patricia-5.0  
   4    442995  47.57%  70.87%  05.82%   58   Patricia-3.01  
   5    419109  49.01%  57.40%  06.25%   59   Patricia-4  
   6    416228  49.30%  56.65%  07.56%   59   Patricia-4-dev  
   7    413684  42.36%  61.31%  11.21%   57   Rebel-Extreme  
   8    295411  36.21%  50.03%  10.93%   62   Velvet-8.1-risky  
   9    258327  31.13%  41.95%  13.78%   67   Rebel-EAS-2.0  
  10    220725  24.02%  42.61%  13.79%   65   SF-17.1  
  11    201674  20.95%  44.25%  14.54%   65   Fritz-20-AVX512  
  12     77794  01.98%  34.12%  31.93%   68   Clover.8.0.2  
  13     77488  01.65%  37.21%  37.22%   67   viridithas-14.0.1  
  14     76293  02.30%  31.39%  28.48%   69   Obsidian130  
  15     73617  01.77%  34.53%  37.01%   68   seer_v2.8  
  16     70117  02.76%  28.67%  27.97%   71   Lizard-11_0  
  17     59817  01.88%  28.04%  31.69%   71   berserk-13  
  18     58506  02.37%  26.97%  30.57%   72   PlentyChess-2.1.0  
  19     57396  02.23%  27.85%  32.38%   72   Alexandria-7.0  
  20     53124  01.78%  25.52%  33.86%   72   caissa-1.20  
  21     46129  01.94%  23.03%  34.76%   75   Titan-1.1  
  22     45706  02.08%  22.92%  34.67%   75   Titan  

And here the "normal" EAS-Tool:

Code: Select all

                                 bad  avg.win 
Rank  EAS-Score  sacs   shorts  draws  moves  Engine/player 
-------------------------------------------------------------------
   1    418009  50.85%  52.75%  09.39%   53   Rebel-Extreme-1.1-BETA  
   2    381831  48.83%  50.78%  09.51%   55   CSTal-2.1-EAS  
   3    374393  53.59%  35.95%  07.18%   60   Patricia-5.0  
   4    351240  47.57%  50.49%  05.82%   58   Patricia-3.01  
   5    341646  49.01%  37.91%  06.25%   59   Patricia-4  
   6    339734  49.30%  38.03%  07.56%   59   Patricia-4-dev  
   7    334572  42.36%  42.46%  11.21%   57   Rebel-Extreme  
   8    231346  36.21%  30.17%  10.93%   62   Velvet-8.1-risky  
   9    205463  31.13%  24.54%  13.78%   67   Rebel-EAS-2.0  
  10    167282  24.02%  21.72%  13.79%   65   SF-17.1  
  11    148092  20.95%  22.12%  14.54%   65   Fritz-20-AVX512  
  12     39989  02.30%  12.73%  28.48%   69   Obsidian130  
  13     38692  01.98%  13.54%  31.93%   68   Clover.8.0.2  
  14     37711  02.76%  10.89%  27.97%   71   Lizard-11_0  
  15     35155  01.65%  14.75%  37.22%   67   viridithas-14.0.1  
  16     32714  01.77%  15.13%  37.01%   68   seer_v2.8  
  17     28565  01.88%  09.89%  31.69%   71   berserk-13  
  18     28417  02.37%  09.71%  30.57%   72   PlentyChess-2.1.0  
  19     26283  02.23%  10.26%  32.38%   72   Alexandria-7.0  
  20     23912  01.78%  09.83%  33.86%   72   caissa-1.20  
  21     20368  01.94%  08.40%  34.76%   75   Titan-1.1  
  22     20095  02.08%  08.34%  34.67%   75   Titan  

So, the EAS-tool works fine. But you have to avoid matching results of EAS tool-runs using the hardcoded movelimit and EAS tool-runs using the normal movelimit-calculation !!! This is the reason, the hardcoded EAS-tool versions are "hidden" in the for_engine_developers-folder.
Great relief for me, no bug in the EAS-Tool.

[pohl4711]

By the way: The stats of your new Rebel Extreme beta are looking very promising!
A clear improvement in all single-stats and besides Patricia 5 the only engine with more than 50% sacs (of all won games) in your gamebase. Really impressive. Cant wait to test this one!

[Rebel]

I think, I understand whats happend. Its not a bug in the tools, you mismatched results, using Gauntlet-EAS tool and/or EAS-tool with the hardcoded movelimit for short-wins bonus and the normal EAS-tool (here this movelimit is calculated out of the average length of all won games of the source.pgn). This leads (of course) to completely different values for the shorts-stat. But not only Patricia, this affects of course, all engines in a source.pgn.

LOL, now it's my fault

I am glad you tagged the problem, just tell me step by step which version to download and how to run it.

[Rebel]

By the way: The stats of your new Rebel Extreme beta are looking very promising!
A clear improvement in all single-stats and besides Patricia 5 the only engine with more than 50% sacs (of all won games) in your gamebase. Really impressive. Cant wait to test this one!

I will mail you the version, please keep it private. Meanwhile I will try to lower the similarity, I am not so sure if that is possible.

[chrisw]

By the way: The stats of your new Rebel Extreme beta are looking very promising!
A clear improvement in all single-stats and besides Patricia 5 the only engine with more than 50% sacs (of all won games) in your gamebase. Really impressive. Cant wait to test this one!

I will mail you the version, please keep it private. Meanwhile I will try to lower the similarity, I am not so sure if that is possible.

presumably the similarity is because of unchanged search?
funny how these things are sticky in the sim testing