Dog & SYZYGY

[flok]

Regarding syzygy in Dog.

What I do in the main search: I check if there are 5 (depending on the tb size) pieces on the board or less for the current position and then use tb_probe_wdl() to return max_eval for win etc.
the result is horrible:

Code: Select all

Results of Dog-syzygy vs Dog-plain (8+0.08, 1t, 256MB, 8mvs_big_+80_+109.epd):
Elo: -114.90 +/- 34.64, nElo: -145.16 +/- 40.55

Dog-syzygy is with the 5-piece syzygy.

I believe that I have all the + and -s correctly, because if I return -max_eval instead of max_eval, then the elo difference is even worse

The code is at https://github.com/folkertvanheusden/Do ... h.cpp#L426

Anyone willing to take look or who has an idea what is going wrong?
Note that searching only in the root does help the rating positively.

[syzygy]

What I do in the main search: I check if there are 5 (depending on the tb size) pieces on the board or less for the current position and then use tb_probe_wdl() to return max_eval for win etc.

I assume you probe them in the search (immediately after a capture into a 5-men position), not in the eval.

You should not return max_eval but treat it like you treat mate, i.e. correct for the distance to the root. Just pick a TB_WIN value that is below your positive "mate in x" values.
E.g. if mate-in-1 is 31999, then TB_WIN-in-1 could be 31499, leaving enough room for all reasonable mate-in-x scores.

At the root, do a regular search without TB probing in the search, but first remove all the root moves that do not lead to a win (or do not preserve the draw, etc). For optimal results, probe DTZ and compare with the 50-move counter to see which moves are optimal.

[flok]

What I do in the main search: I check if there are 5 (depending on the tb size) pieces on the board or less for the current position and then use tb_probe_wdl() to return max_eval for win etc.

I assume you probe them in the search (immediately after a capture into a 5-men position), not in the eval.

Oh! I checked it in every search-node (not qs).

You should not return max_eval but treat it like you treat mate, i.e. correct for the distance to the root. Just pick a TB_WIN value that is below your positive "mate in x" values.
E.g. if mate-in-1 is 31999, then TB_WIN-in-1 could be 31499, leaving enough room for all reasonable mate-in-x scores.

Ah I did that different indeed. I thought I should do max_eval - tree_height.

These two fixes (thus only checking in search when a capture-move led to 5 pieces or less and then scoring fix) gives me:

Code: Select all

Results of Dog-syzygy vs Dog-plain (8+0.08, 1t, 256MB, 8mvs_big_+80_+109.epd):
Elo: -36.07 +/- 16.81, nElo: -58.32 +/- 26.96

(from -114!)

At the root, do a regular search without TB probing in the search, but first remove all the root moves that do not lead to a win (or do not preserve the draw, etc). For optimal results, probe DTZ and compare with the 50-move counter to see which moves are optimal.

Oh what I did was at the root of the tree (before checking the first moves) check what the best moves are and then just pick the win/draw with the shortest distance.

[syzygy]

What I do in the main search: I check if there are 5 (depending on the tb size) pieces on the board or less for the current position and then use tb_probe_wdl() to return max_eval for win etc.

I assume you probe them in the search (immediately after a capture into a 5-men position), not in the eval.

Oh! I checked it in every search-node (not qs).

Effectively it should be the same, assuming you always take the cutoff if the TB probe is successful. The only way to get from more than 6 to 5 pieces is by a capture. So if it is quicker to check that the number of pieces is <= 5 than first checking whether the previous move was a capture, then just directly check the number of pieces.

You should not return max_eval but treat it like you treat mate, i.e. correct for the distance to the root. Just pick a TB_WIN value that is below your positive "mate in x" values.
E.g. if mate-in-1 is 31999, then TB_WIN-in-1 could be 31499, leaving enough room for all reasonable mate-in-x scores.

Ah I did that different indeed. I thought I should do max_eval - tree_height.

It seems you do this:

Code: Select all

				int score = syzygy_score.value();
				if (score < 0)
					score -= -max_eval + csd;
				else if (score > 0)
					score = max_eval - csd;
				return score;

Is syzygy_score.value() one of -1,0,1? (Or one of -2,-1,0,1,2, with -1/1 being 50-move draws?)

Why do you substract -max_eval, i.e add a huge value, if the position is a loss?
I guess "-=" should be "=".

Earlier you set csd = max_depth - depth. But depth is the depth of the current search iteration, so you are assigning the same winning positions with the same distance to the root different values in different search iterations. That is "wrong".
And it seems your code for dealing with mates is the same, so also wrong.

I would suggest you watch or inspect some games to see if what the engine does makes any sense.

Oh what I did was at the root of the tree (before checking the first moves) check what the best moves are and then just pick the win/draw with the shortest distance.

That is fine for a start (it will win the games that are won, provided you deal correctly with zeroing moves!!), but it leads to very ugly play. With DTZ-optimal play in KQBNvK, white will force black to takes its queen if that is the fastest way to a zeroing move. This can most easily be avoided by doing a short search on those moves that preserve the win.

[syzygy]

Earlier you set csd = max_depth - depth. But depth is the depth of the current search iteration, so you are assigning the same winning positions with the same distance to the root different values in different search iterations. That is "wrong".
And it seems your code for dealing with mates is the same, so also wrong.

What the engine should do is assign to a mate or TB win the proper value from the point of view of the root node.

If, counted from the root, you can reach a mate (or TB win) in 15 ply, then it should be scored as mate-in-15-ply (or TBwin-in-15-ply). That way, the search will pick a mate in 11 ply over a mate in 15 ply. And counted in this way, you are not changing the value in different iterations.

However, you then have to be careful when storing and retrieving mate and TBwin values to/from the TT (they should be stored not relative to the root position but relative to the TT node).
Your code already seems to do this in eval_from/to_tt() (which should also make the +/-ply correction to TBwin/loss scores).

[flok]

Why do you substract -max_eval, i.e add a huge value, if the position is a loss?
I guess "-=" should be "=".

That's a horrible bug Indeed when fixed, elo goes to +18 (from -36).

Earlier you set csd = max_depth - depth. But depth is the depth of the current search iteration, so you are assigning the same winning positions with the same distance to the root different values in different search iterations. That is "wrong".
And it seems your code for dealing with mates is the same, so also wrong.

Are you sure? Because if I change https://github.com/folkertvanheusden/Do ... h.cpp#L375 and https://github.com/folkertvanheusden/Do ... h.cpp#L600 to -max_eval + depth, I get a -15 elo drop.

I would suggest you watch or inspect some games to see if what the engine does makes any sense.

I barely know how to play chess so I check my changes by sprt-tests. And complaining on-line

Thank you for your elaborate replies by the way, much appreciated.

[syzygy]

Earlier you set csd = max_depth - depth. But depth is the depth of the current search iteration, so you are assigning the same winning positions with the same distance to the root different values in different search iterations. That is "wrong".
And it seems your code for dealing with mates is the same, so also wrong.

Are you sure? Because if I change https://github.com/folkertvanheusden/Do ... h.cpp#L375 and https://github.com/folkertvanheusden/Do ... h.cpp#L600 to -max_eval + depth, I get a -15 elo drop.

No, "depth" should stay out of this. The value of depth changes per search iteration. You should use the distance to the root, which seems to be "ply", with the proper sign.

I would suggest you watch or inspect some games to see if what the engine does makes any sense.

I barely know how to play chess so I check my changes by sprt-tests. And complaining on-line

Of course I did not see your engine play, but I suspect it is not able to reliably report and play out mates that it finds.

Do you understand why eval_to_tt() and eval_from_tt() in your engine are the way they are?

[flok]

Earlier you set csd = max_depth - depth. But depth is the depth of the current search iteration, so you are assigning the same winning positions with the same distance to the root different values in different search iterations. That is "wrong".
And it seems your code for dealing with mates is the same, so also wrong.

Are you sure? Because if I change https://github.com/folkertvanheusden/Do ... h.cpp#L375 and https://github.com/folkertvanheusden/Do ... h.cpp#L600 to -max_eval + depth, I get a -15 elo drop.

No, "depth" should stay out of this. The value of depth changes per search iteration. You should use the distance to the root, which seems to be "ply", with the proper sign.

That's what I thought yes. Then I don't understand the comment about the code dealing with mates?

I would suggest you watch or inspect some games to see if what the engine does makes any sense.

I barely know how to play chess so I check my changes by sprt-tests. And complaining on-line

Of course I did not see your engine play, but I suspect it is not able to reliably report and play out mates that it finds.

That's interesting! I mean it is around 3k elo, this more sounds like a sub-2k problem?

Do you understand why eval_to_tt() and eval_from_tt() in your engine are the way they are?

I believe that they are there to "compensate" for search-depth so that the same-position but at a different depth would not neccessarily be chosen unless same or better depth.

[syzygy]

Earlier you set csd = max_depth - depth. But depth is the depth of the current search iteration, so you are assigning the same winning positions with the same distance to the root different values in different search iterations. That is "wrong".
And it seems your code for dealing with mates is the same, so also wrong.

Are you sure? Because if I change https://github.com/folkertvanheusden/Do ... h.cpp#L375 and https://github.com/folkertvanheusden/Do ... h.cpp#L600 to -max_eval + depth, I get a -15 elo drop.

No, "depth" should stay out of this. The value of depth changes per search iteration. You should use the distance to the root, which seems to be "ply", with the proper sign.

That's what I thought yes. Then I don't understand the comment about the code dealing with mates?

Ok, it seems max_depth is also changed per iteration, so csd is in fact distance to the root.
I had assumed max_depth is simply the max supported depth.

[syzygy]

That's what I thought yes. Then I don't understand the comment about the code dealing with mates?

Ok, it seems max_depth is also changed per iteration, so csd is in fact distance to the root.
I had assumed max_depth is simply the max supported depth.

However, note that "max_depth - depth" is not the same as "distance to the root" if you do any search extensions or reductions.
E.g. it seems that a mate position encountered at 10 ply from the root may be scored in the search as, say, a mate in 14 ply because search reductions caused the position to be searched with reduced depth (i.e. csd == max_depth - depth is higher than the distance to the root).

Since you seem to use the same the same "csd" correction when you score the position in the search and when you store it in the TT, the value in the TT seems correct. And if you don't reduce on the PV, then in practice things will probably work out. So even though theoretically not the most accurate this might not have any measurable effect on strength.