Crystal 8

[jshriver]

Private Testsuite

Mind I ask what format your testsuite is in and how do you run them to get these results? Sounds like a fun project and would be interested in creating my own.

Congrats on the Crystal 8 release.

[jhellis3]

I can answer your previous question regarding development: maybe.

I have done some work on Crystal since 8 and recently pulled & merged upstream into Crystal locally.

Unfortunately, there has been a quite significant regression in performance. I am currently attempting to track down the source(s) of the issue, but if I am unable to, development may come to an end. Currently, I do not know whether it is something I have done (most likely), or if the searches have diverged so much that many of the tuned values in Stockfish are simply much worse for Crystal. Hopefully, it is the former and I am able to track down the issues without much trouble. If it is the latter, development may have to come to an end as I don't really have the time or resources to produce a fully "tuned" version of Crystal.

[jhellis3]

Crystal 9 is released here: https://github.com/jhellis3/Stockfish/releases/tag/v9

It is not quite where I wanted it to be, but a number of small practical issues have been resolved and it has been over a year since Crystal 8. There is a fair amount of Elo gain, though how this translates to other aspects of the engine I can't say. I was tempted to delay the release further in order to continue working on some outstanding issues, but one shouldn't let an ideal be the enemy of progress... To that end, people can always compile dev versions.

To be honest, I am not sure if there will be further iterations of Crystal. I find I have less and less time to devote to the project, and the development pace of Stockfish remains swift as ever. Switching to another highly rated open source engine (with fewer authors / slower development pace) might be a possibility, but the upfront investment would not be insignificant either...

In any event, it is here for now for those who find value in it.

[peter]

Thanks for the new version.
A little match against latest dev.- version, I had a compile of, from Dez. last year, with 3'+1", single thread (3.5GHz), UHO 2024 900-990cp 6mvs:

Score of Crystal9 vs Crystal241216: 136 - 106 - 258 [0.530]
Elo difference: 20.9 +/- 21.2, LOS: 97.3 %, DrawRatio: 51.6 %
500 of 500 games finished.

More or less accidentally with 128 anti engine puzzles I get similiar EloStatTS (Frank Schubert)- difference with 8 threads and 30"/pos.

Code: Select all

    Program                                    Elo   +/-  Matches  Score   Av.Op.   S.Pos.   MST1    MST2   RIndex

  2 Crystal9-8t-MuPV4                        : 3583    6   2691    62.7 %   3493    96/128    5.1s   11.3s   0.69
  5 Crystal241216-8t-MuPV4                   : 3554    7   2618    58.5 %   3495    91/128    6.2s   13.1s   0.54

MST1  : Mean solution time (solved positions only)
MST2  : Mean solution time (solved and unsolved positions)
RIndex: Score according to solution time ranking for each position

With a MEA- suite of 359 similiar positions (first 128 are the same) and 1 minute/pos., Crystal9 solves single threaded 10 positions and gets 81 points more than 241216 (Crystal9 =2154, Crystal241216=2073 MEA- points out of a max. of 3590), error bar of this one suite is about 2% of max. of points to be scored, regards

[kasinp]

Thank you for the new version. Always one of my favorite engines, both for quick analysis and for digging deeper, especially into endgame positions.

Peter

[Paloma]

So, is 9er better than Crystal 8?
I heard it's not as good as Crystal 8

[peter]

So, is 9er better than Crystal 8?
I heard it's not as good as Crystal 8

It depends on the way you test it. The examples of results I gave above were compared to an in between version from Dez. 24, testing against Crystal 8 don't differ much, as far as I saw till now, even if Crystal 8 has the edge with certain positions, suites and hardware- TC, especially compared to the compile from Dez. I have, maybe it's not as well compiled as the ones of the major updates.
Anyhow you won't get out of error bar with 500 games in head to head eng-eng-match only, neither with suites, if you know and mind their error bars too and don't look at VSTC for bigger ones or too little amounts of positions and or runs for longer TCs only, regards