I think Crystal must have an eval bug

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Dann Corbit
Posts: 12121
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

I think Crystal must have an eval bug

Post by Dann Corbit » Sun Jun 14, 2020 8:52 am

If you look at the current TCEC games, in each and every game, Crystal draws a sawtooth pattern while no other engine does that.
Possible explanation:
A term that should be applied to black's score is applied to white or vice versa.
Just examine any game. Something is not what it ought to be.

There could be some other explanation, but that is the only one that occurs to me.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

jhellis3
Posts: 488
Joined: Fri Aug 16, 2013 10:36 pm

Re: I think Crystal must have an eval bug

Post by jhellis3 » Sun Jun 14, 2020 12:52 pm

No...

Crystal does not touch eval except to reduce it for shuffling. What you are perceiving is the result of mcts eval being very stable and Crystal exploring suboptimal paths, correcting the score, and switching off of them.

Dann Corbit
Posts: 12121
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: I think Crystal must have an eval bug

Post by Dann Corbit » Sun Jun 14, 2020 1:04 pm

The other AB engines (even SF) do not appear to behave in the same way.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

jhellis3
Posts: 488
Joined: Fri Aug 16, 2013 10:36 pm

Re: I think Crystal must have an eval bug

Post by jhellis3 » Sun Jun 14, 2020 1:20 pm

Yes, Crystal will often have the most unstable eval out of all the engines, by design. At least when the position is not a draw :wink: .

The other AB engines do not have progress detection (or at least not to the level of Crystal) either.

There are also many situations where Crystal has had both the most stable and most accurate eval for the vast majority of the game.

People classifying any unexpected behavior as a bug is tiresome.

If one desires a more stable eval from Crystal, try zooming out on the graph, works wonders.

User avatar
Ovyron
Posts: 4415
Joined: Tue Jul 03, 2007 2:30 am

Re: I think Crystal must have an eval bug

Post by Ovyron » Sun Jun 14, 2020 9:38 pm

Yeah, a "bug" is an unexpected behavior that appeared and wasn't intended by the programmer. No matter how weird it looks, if it's there by design it can't be a "bug".

Dann Corbit
Posts: 12121
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: I think Crystal must have an eval bug

Post by Dann Corbit » Mon Jun 15, 2020 12:02 am

jhellis3 wrote:
Sun Jun 14, 2020 1:20 pm
Yes, Crystal will often have the most unstable eval out of all the engines, by design. At least when the position is not a draw :wink: .

The other AB engines do not have progress detection (or at least not to the level of Crystal) either.

There are also many situations where Crystal has had both the most stable and most accurate eval for the vast majority of the game.

People classifying any unexpected behavior as a bug is tiresome.

If one desires a more stable eval from Crystal, try zooming out on the graph, works wonders.
My mistake, and I do repent on bended knee in dust and ashes.
:oops:
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Post Reply