Crafty 23.5 mod

[jarkkop]

I think we can be certain of these things:

1.) Evaluation function parameters have multiple local optimal points.
2.) Only one global optimum which we probably never reach
3.) By tuning one parameter there is no guarantee that we get out of the local optimum valley.

[Peter Skinner]

Robert please consider releasing a UCI version.

I have asked Robert for a UCI version and the long and short of it is, it would require a complete re-write of how the engine works. So I wouldn't be expecting it anytime soon.

Add on the fact Robert doesn't like how the UCI protocol handles an engine, and you will REALLY have to wait a while

Being the source is fully published, someone could take a stab and implementing it.

Peter

[Werewolf]

oh....

[Adam Hair]

I got a lot better performance for Crafty with these parameters
against earlier Rybka engines.
Against standard Crafty score was quite even.
Can anyone verify this?

personality 11 -92 92
personality 12 -302 302
personality 13 -315 315
personality 14 -514 514
personality 15 -1006 1006

Unfortunately, I found that those parameters made Crafty a little worse in my testing. I matched both versions of Crafty against many of the top engines in time odds gauntlets and combined the results with my database for testing Gaviota. This is what I found (40/x means 40 moves in x seconds):

Code: Select all

# ENGINE                            : RATING  ERROR   POINTS  PLAYED    (%)
   1 Critter_1.6a_64bit     (40/2)  :   85.8    4.2   4075.0    6637   61.4%
   2 Stockfish_2.2.2_64-bit (40/4)  :   76.2    4.2   3866.0    6448   60.0%
   3 Crafty_23.5            (40/16) :   75.2    4.1   4077.5    6596   61.8%
   4 Hiarcs_11              (40/16) :   62.7    4.2   3828.5    6587   58.1%
   5 Crafty_23.5_Jarkko     (40/16) :   60.4    4.2   4055.5    6765   59.9%
   6 Protector_1.4.0_64-bit (40/8)  :   25.7    4.0   3517.0    6598   53.3%
   7 Gaviota_0.86_wrn2      (40/16) :   17.2    2.0  13912.5   25830   53.9%
   8 Houdini_2.0c_64-bit    (40/1)  :    8.7    4.1   3368.0    6641   50.7%
   9 Nemo_1.01_beta_64-bit  (40/8)  :  -12.3    4.1   3250.5    6880   47.2%
  10 Quazar_0.4_64-bit      (40/8)  :  -16.3    4.1   3106.0    6680   46.5%
  11 Gaviota_0.85.1         (40/16) :  -24.4    2.1  12411.5   25839   48.0%
  12 Rybka_4.1_64-bit       (40/2)  :  -26.2    4.2   3102.0    6838   45.4%
  13 Gull_1.1_64-bit        (40/4)  :  -37.8    4.1   2995.5    6790   44.1%
  14 Spark_1.0_64-bit       (40/4)  :  -57.1    4.3   2733.5    6662   41.0%
  15 Hannibal_1.2_64-bit    (40/4)  :  -62.4    4.0   2720.5    6788   40.1%
  16 Zappa                  (40/16) :  -86.8    4.4   2474.5    6686   37.0%
  17 Spike_1.4              (40/8)  :  -88.6    4.3   2410.0    6543   36.8%

[jarkkop]

Oh well

Thanks for helping out.
I must continue the quest.

[jefk]

[quote="Adam Hair"][quote="jarkkop"]I got a lot better performance for Crafty with these parameters against earlier Rybka engines. Against standard Crafty score was quite even.
[/code][/quote]

well i suspect for Crafty tuning only will not do the job, i often noticed
the eval is completely different than the top engines like Houdini.
sometimes negative where Houdini is positive and vice versa eg +/- 0.15
No wonder 'backsolving' ie minimax to find opening positions doesnt
work well for Crafty. The eval of the endnodes is crucial with such
a method (which in itself i believe is correct although not always highly robust but it can be combined with other methods like in Aquarium ).
Resorting to statistics only in the opening phase is a bit like using
a Monte Carlo method and not believing in your engine altogether.
Well if thats the case Monte Carlo might be used as well for
slow games for the middle game eval as well i suppose