Fischerle 0.9.80 SE 64-bit Gauntlet for CCRL 40/40

[RolandoFurioso]

I did it for CCRL 40/4.

Thanks, Gabor! These results - as far as yet reflected in the CCRL - are pretty in line with the 40/40 evaluation. I'm really curious to see the next update; the current lists are from mid of April.

[RolandoFurioso]

The problem is that it crashes even with the proper jar. 0.9.70 crashes as well, the 32-bit also. Can it be a mistake with invoking java? The batch file 'finds' the proper java because its path is in the system path. Maybe the exe does it otherwise?

It would be helpful to obtain more information about why it crashes. Could you open a command line window, navigate to the distribution folder, and invoke the .exe manually? The error message should then be displayed in the window, which remains open.

Which Windows version are you using?

[sandermvdb]

If I were you I would get rid of the .exe file (or batch-file). I think it causes more problems than that it solves:
- Initial or max jvm memory settings are NOT important unless you want to use a really large transposition table. I agree that setting the initial memory setting higher will be a bit faster because it does not need to this this on-the-fly, but this performance improvement will be negligible!
- Write your chessengine in such a way that it does not need to do any garbage collection (avoid new keyword). Mine does 1 small gc (<10 msec) every couple of minutes. Again, negligible. This can nicely be monitored with the VisualGC plugin for VisualVM. Setting a specific gc-method will then become unnecessary.
- jar-files are multi-platform, exe-files are not

Another tip is to write a small logfile when the engine crashes in the tmp folder of the system with relevant information: stacktrace, initial fen, current fen, statistics, etc... If you need an example, look at the sourcecode of chess22k.

Goodluck!

[RolandoFurioso]

What I am still hoping for is that somehow I can get to the root of why Arena does this. To show you how strange the results are here is the tournament cross table:

Code: Select all

Fischerle 0.9.80 SE 64-bit - Adam 3.3                   : 0,0/6 0-6-0 (000000)   0% -1200
Fischerle 0.9.80 SE 64-bit - AliChess 4.25              : 0,0/6 0-6-0 (000000)   0% -1200
Fischerle 0.9.80 SE 64-bit - Anechka 0.08               : 0,5/6 0-5-1 (00=000)   8%  -424
Fischerle 0.9.80 SE 64-bit - ANT 2006-F                 : 0,5/6 0-5-1 (000=00)   8%  -424
Fischerle 0.9.80 SE 64-bit - Atak 6.8                   : 1,5/6 1-4-1 (100=00)  25%  -191
Fischerle 0.9.80 SE 64-bit - Beowulf 2.4a               : 0,5/6 0-5-1 (0000=0)   8%  -424
Fischerle 0.9.80 SE 64-bit - Buzz 0.08 64-bit           : 2,5/6 0-1-5 (=====0)  42%   -56
Fischerle 0.9.80 SE 64-bit - Chispa 4.03                : 3,0/6 3-3-0 (100011)  50%    ±0
Fischerle 0.9.80 SE 64-bit - EveAnn 1.72                : 0,0/6 0-6-0 (000000)   0% -1200
Fischerle 0.9.80 SE 64-bit - Firefly 2.7.0 64-bit       : 2,0/6 2-4-0 (100001)  33%  -123
Fischerle 0.9.80 SE 64-bit - Madeleine 0.2              : 3,0/6 2-2-2 (==1001)  50%    ±0
Fischerle 0.9.80 SE 64-bit - Natwarlal 0.14             : 2,0/6 1-3-2 (=010=0)  33%  -123
Fischerle 0.9.80 SE 64-bit - NGplay 9.86 64-bit         : 2,0/6 1-3-2 (001=0=)  33%  -123
Fischerle 0.9.80 SE 64-bit - Paladin 0.1 64-bit         : 1,5/6 1-4-1 (00=100)  25%  -191
Fischerle 0.9.80 SE 64-bit - RamJet 0.13                : 1,5/6 1-4-1 (00100=)  25%  -191
Fischerle 0.9.80 SE 64-bit - RattateChess 1.0 Nosferatu : 1,0/6 0-4-2 (=0=000)  17%  -275
Fischerle 0.9.80 SE 64-bit - Uralochka 1.1b             : 0,0/6 0-6-0 (000000)   0% -1200
Fischerle 0.9.80 SE 64-bit - Zeus 1.29                  : 1,5/6 1-4-1 (000=10)  25%  -191

Yes - something seems to be wrong here.

But the 40/4 results as published so far(2271 ELO) seem to be consistent with expectations. This seems to imply that the problem crept in while the test was running?!

Just an idea: maybe Fischerle 0.9.80 has been accidentally run with a very small hashtable in the latter part of the evaluation? Could you check whether it is still set to 256 MB (or the actual standard value that you're employing)? You could check this as well by looking at Fischerle's native GUI (to be made visibly by selecting "Display Fischerle GUI" in the engine configuration menu) - the proper hashtable size should be displayed in one of the GUI elements (just right to the garbage bin button).

[RolandoFurioso]

Thanks!

OK - so this might indicate that the problem is related to the bat-to-exe converter that I'm employing. Possibly it doesn't generat output suitable for Win10. (Still using Win7 here!). So please stick to the .bat starter; I'll see whether I can find a solution for the next distribution.

[Guenther]

Well, this is what happened.

I selected an engine from the CCRL 40/4 list against which Fischerle 0.9.70 64-bit had drawn (16-16). It was Chezzz.
Now I played a gauntlet with Chezzz against 0.9.70 again and 0.9.80. I installed the engines using the jar files directly, as Sanders suggested.
TC was 40/1 so that it does not take very long.
Here are the results:

Code: Select all

Chezzz 1.0.3 - Fischerle 0.9.70 SE 64-bit            : 17,0/20
Chezzz 1.0.3 - Fischerle 0.9.80 SE 64-bit            : 18,5/20

As can be seen, even 0.9.70 underperformed seriously which confirms to me that it is not that 0.9.80 is way worse than 0.9.80, they are in fact about equal.
I don't know what has changed since my previous test with 0.9.70. Owing to an incident which erased my chess-related material from my HDD I had to reinstall all my chess stuff 3 months ago. Can't see how that could affect things.

Well, I have run out of ideas. Sorry Roland.

Well, haven't you changed to Win10 after that?

[sandermvdb]

You could run a quick tournament with another java engine to see if that one also under performs.

[sandermvdb]

Does fischerle depend on an openingbook and endgame tablebases? Could there something be wrong with this configuration?

[RolandoFurioso]

If I were you I would get rid of the .exe file (or batch-file). I think it causes more problems than that it solves:
- Initial or max jvm memory settings are NOT important unless you want to use a really large transposition table. I agree that setting the initial memory setting higher will be a bit faster because it does not need to this this on-the-fly, but this performance improvement will be negligible!
- Write your chessengine in such a way that it does not need to do any garbage collection (avoid new keyword). Mine does 1 small gc (<10 msec) every couple of minutes. Again, negligible. This can nicely be monitored with the VisualGC plugin for VisualVM. Setting a specific gc-method will then become unnecessary.
- jar-files are multi-platform, exe-files are not
Goodluck!

Thanks - but this is nothing new to me. Of course, Fischerle avoids usage of "new" ans much as possible, i. e., pre-allocates the required menory during the startup phase as far as possible. Moreover - once again: it does *not* hurt to specify thes options, and we could leave it up to the Java implementation what it makes out of "UseParallelGC" & Co. if they are specified.

Of course, I would be very happy to go without an ".exe" (Thanks a ton for this enlightenment, but I'm of course perfectly aware that it's platform specific!) ; but this is a requirementexplicitly stated by some CCRL testers (using environments such as ChessGUI) - hence, I try to comply with it.