Felicity EGTB generators and probers

[noobpwnftw]

While one can argue that such "optimal play" may be optinal, in reality getting the accurate game outcome is dependent on doing so, as the result of a mutual check or chase situation is depending on the intened sequences both sides will play which will then become its own history.

[phhnguyen]

Code: Select all

wtm max dtm: 48
btm max dtm: 28

As I have guessed, "wtm max dtm" and "btm max dtm" mean the max DTMs for White and Black. Is it correct?

The above stats confused me since the gap between max DTMs for White and Black is too large. Suppose you are in a position with max DTM for White. You should make 48 moves to mate. After the first move, it is the Black turn and Black should make 48 or 47 moves to mate. Thus the max DTM for Black must be equal or larger than 47, not 28. By my understanding, the gap between those numbers should be equal to or less than 1.

What am I wrong? Can you explain? Thanks

[noobpwnftw]

How is it confusing? One is for the rook side to win and other for the knight and cannon side to win.

[phhnguyen]

How is it confusing? One is for the rook side to win and other for the knight and cannon side to win.

OK, I got it. They are not max-DTMs as I have understood but wining-max-DTMs. Thanks.

[phhnguyen]

After fixing some bugs and changing the way to get statistics, the new one for krkcn changed a bit:

Code: Select all

Name                        : krkcn
Total positions             : 32805000 (each side)
Legal positions             : 36856968 (56%, 2 sides)
Total draws                 : 649207 (1% of legals)
Draws + perpetuations       : 1476790 (4% of legals)
White to move (WDL)         : 97%, 0%, 0%
 perpetual checks (WL)      : 41784, 45
 perpetual chases (WL)      : 695, 6
 max winning DTM            : 115
Black to move (WDL)         : 13%, 2%, 80%
 perpetual checks (WL)      : 695, 6
 perpetual chases (WL)      : 42617, 45
 max winning DTM            : 27
Total perpetuations         : 827583, 56% of draws + perpetuations. #checks: 42530, #chases: 43363
Total forward perpetuations : 741690, #wins: 97625, #losses: 644065
Max forward distance        : 29


Samples of max DTM for White:
9/3k5/9/9/4c4/9/6n2/5K3/1R7/9 w 0 1
9/3k5/9/9/4c4/9/6n2/5K3/R8/9 w 0 1
9/3k5/9/9/4c4/9/6n2/1R3K3/9/9 w 0 1
9/3k5/9/9/4c4/9/6n2/5K3/9/1R7 w 0 1
9/3k5/9/9/4c4/9/6n2/R4K3/9/9 w 0 1
9/3k5/R8/9/4c4/9/6n2/5K3/9/9 w 0 1
9/3k5/9/9/4c4/9/6n2/5K3/9/R8 w 0 1
1R7/3k5/9/9/4c4/9/6n2/5K3/9/9 w 0 1
9/3k5/1R7/9/4c4/9/6n2/5K3/9/9 w 0 1
R8/3k5/9/9/4c4/9/6n2/5K3/9/9 w 0 1


Samples of max DTM for Black:
1c7/4k4/2R6/9/9/6n2/9/9/9/5K3 b 0 1
8c/4k4/R8/9/9/6n2/9/9/9/5K3 b 0 1
3c5/4k4/2R6/9/9/6n2/9/9/9/5K3 b 0 1
5c3/3k5/2R6/9/9/6n2/9/9/9/5K3 b 0 1
6c2/4k4/R8/9/9/6n2/9/9/9/5K3 b 0 1
2c6/4k4/R8/9/9/6n2/9/9/9/5K3 b 0 1
1c7/4k4/R8/9/9/6n2/9/9/9/5K3 b 0 1
5c3/9/R3k4/9/9/6n2/9/9/9/5K3 b 0 1
c8/4k4/R8/9/9/6n2/9/9/9/5K3 b 0 1
5c3/3k5/R8/9/9/6n2/9/9/9/5K3 b 0 1


Samples of perpetual checks:
9/9/5k3/9/9/4c4/2n6/3K5/9/4R4 b 0 1
9/4c4/5k3/9/9/9/9/3K5/1n7/4R4 w 0 1
5k3/9/9/9/9/9/2n6/3K5/c8/4R4 b 0 1
9/9/5k3/1c7/9/9/9/3K5/6n2/3R5 b 0 1
9/9/2c1k4/9/9/2n6/9/3K5/9/3R5 w 0 1
9/9/4k4/9/9/9/9/3K5/1cn6/3R5 b 0 1
4k4/9/9/9/9/2n6/9/3K4c/9/3R5 w 0 1
9/9/4k4/9/9/9/9/1n1K5/2c6/3R5 b 0 1
9/4k4/9/9/9/1c7/4n4/3K5/9/2R6 b 0 1
9/5k3/1c7/9/9/9/9/3K5/6n2/3R5 b 0 1


Samples of perpetual chases:
4c4/4k4/9/2n6/9/2R6/9/5K3/9/9 b 0 1
4c4/9/4k4/2n6/9/2R6/9/9/5K3/9 b 0 1
9/4k4/4c4/9/4R4/5n3/9/9/5K3/9 b 0 1
4c4/9/4k4/2n6/9/2R6/9/9/9/5K3 b 0 1
9/4k4/4c4/9/9/6n2/4R4/9/9/3K5 w 0 1
4c4/9/4k4/9/n8/1R7/9/9/9/5K3 b 0 1
4c4/4k4/9/9/n8/1R7/9/9/9/5K3 b 0 1
4c4/4k4/9/2n6/9/1R7/9/9/9/5K3 w 0 1
9/4k4/4c4/9/4R4/9/5n3/9/9/5K3 w 0 1
4c4/4k4/9/9/n8/2R6/9/9/9/5K3 w 0 1


Samples of max forward perpetuations:
1c7/3k5/9/R8/9/9/9/9/2n2K3/9 b 0 1
3c5/9/R2k5/9/9/9/9/9/2n2K3/9 b 0 1
8c/3k5/R8/9/9/9/9/9/2n2K3/9 b 0 1
7c1/3k5/R8/9/9/9/9/9/2n2K3/9 b 0 1
6c2/3k5/R8/9/9/9/9/9/2n2K3/9 b 0 1
5c3/3k5/R8/9/9/9/9/9/2n2K3/9 b 0 1
c8/3k5/9/R8/9/9/9/9/2n2K3/9 b 0 1
2c6/3k5/R8/9/9/9/9/9/2n2K3/9 b 0 1
1c7/3k5/R8/9/9/9/9/9/2n2K3/9 b 0 1
c8/3k5/R8/9/9/9/9/9/2n2K3/9 b 0 1

[phhnguyen]

How to count them in DTM is appearently the million dollar question.

Obviously whether one side should commit to a perpetual is dependent on the game outcome, therefore if they are not considered together then adjudication will be wrong in some mutual check or chase cases. They can be nested, some more complicated ones are not seen until there are more pieces on board. You might not know the upperbound of the actual distance to mate due to how many repetitions happened in history or about to happen, but there is certainly a determistic lowerbound before which no mate is possible with perfect play, all things considered.

So far we (you and me) have been using two different ways to store values for perpetual check/chase (PCC) positions:
1. standard metrics, say, DTM/DTC
2. special metric. I named it (temporarily) as perpetuation metric (PM) (any suggestion on a better name?)


To be honest, I have considered using (1) for a while but then chose (2). I don’t mean my selection is better since we need some studies before making conclusions. IMHO, anyone has its advantages and disadvantages. For example, (1) has the advantage of being simple and quick while (2) has the advantage of having max/full information.

I didn’t choose (1) because of worried about losing some information. With your implementation, it is the first time I can verify that. Below is an example. The FEN is 5k3/1C7/9/9/9/9/n8/4K4/4A4/9 w - - 0 0. With your endgame, the losing side managed to last the game after 6 full moves. The last move enclosed a repeat of a perpetual check and then the game was ruled as a loss.

The information from my EGTB could make that endgame significantly longer. One of its paths is similar to yours. However, instead of jumping directly into a perpetual check (losing immediately), the losing side could alternate the last move (before enclosing the repetition) to keep the game longer to at least 15 full moves - more than double yours. There may be some other alternative moves that are longer than that game too.

That also explains why your endgames may have surprised short max-DTMs, shorter than the original ones (I mean the max-DTMs without considering perpetual checks/chases). IMHO, in general, a game with PCC should be longer than the one without PCC because the losing player could first follow the path of losing by PCC then change the last move (before being ruled as a loss) to follow the path of losing by being mated.

Sometimes, the situation may be more complicated if you want to convert undermining values into fixed ones. Based on values, I guessed your endgames count only 1-folk repetition for PCC. But if you consider the draw rule of 60 moves, you should count 3 folks instead because a PCC repetition with over 20 moves will become a draw after 3 laps.

FEN is 5k3/1C7/9/9/9/9/n8/4K4/4A4/9 w - - 0 0

[noobpwnftw]

Not really, my default move pick in GUI for the losing side avoids perpetuals in DTM unless there is no other way to make it look more natural.

The other reason it appears longer in yours is because the winning side did not play the strongest minimizing move. Sometimes trying to do repetitions from the defender ends up having to avoid an immediate rule loss but the other non-repeating move has significantly shorter DTM, since you have seperate metrics, the winning side does not know about this. This situation may not happen in those trivial tables but you should try playing with my moves from the winning side and see if your defender can last just as long.

Of course it is not possible to assume how long the n-fold repetitions will last with the n changing, therefore I don't count them at all, but from what I understand in order to simply have the move counters last longer, one should use DTC instead, as any captures will reset the counter and total game length does not matter.

[noobpwnftw]

In fact, your metric is somewhere in between my DTM and DTC where the latter has a count of reptitions as a secondary metric. As a consequence of that, of course, what happens after crossing dimention boundaries is not forseeable. There are merits to use such a secondary metric in DTM instead of DTC, so as you did, and at least your game outcome numbers looks correct now. I just don't like the situation that a metric intends to provide distance to 'game' information(instead of distance to 'state') ends up having intermediate non-continuous boundaries.

Good thing is these numbers are facts, we can easily see about what is the best method that works in practice.

[phhnguyen]

The other reason it appears longer in yours is because the winning side did not play the strongest minimizing move. Sometimes trying to do repetitions from the defender ends up having to avoid an immediate rule loss but the other non-repeating move has significantly shorter DTM, since you have seperate metrics, the winning side does not know about this. This situation may not happen in those trivial tables but you should try playing with my moves from the winning side and see if your defender can last just as long.

You can’t say about optimised moves because the first 27 half-moves are from your game: they are the first suggested moves (with the highest scores) from your website. I have verified that my EGTB accepted those moves too. It alternated only the last move to make the battle longer.

In fact, your metric is somewhere in between my DTM and DTC where the latter has a count of reptitions as a secondary metric. As a consequence of that, of course, what happens after crossing dimention boundaries is not forseeable. There are merits to use such a secondary metric in DTM instead of DTC, so as you did, and at least your game outcome numbers looks correct now. I just don't like the situation that a metric intends to provide distance to 'game' information(instead of distance to 'state') ends up having intermediate non-continuous boundaries.

The perpetuation metric (PM) is somewhat closer to DTZ50 of Syzygy since my perpetual values (PERPETUAL_CHECK/CHASE) and curve values of Syzygy are undetermined. It is not between DTM and DTC when their values are determined. You are family with DTZ50, aren’t you? Syzygy’s DTZ50 provides more information than the pure DTC and it requires applications to understand and process correctly with some extra work. So does the PM.

In fact, you have to convert perpetual check/chase (PCC) information into DTM/DTC when generating. In contrast, I didn’t convert but keep that information as much as possible. That means that applications using my EGTB have more choices to deal with PCC, from converting them into DTM/DTC (as your generator) or selecting moves because it has more information than yours. Once you convert PCC, you lose information about whether a position is a PCC or not.

Good thing is these numbers are facts, we can easily see about what is the best method that works in practice.

Yes, it is good for us all. So far there are so few people working in that field. In this forum, looks like there are only three: you, me and @HGM. Hope some more people join us soon.

My design has been not finalized yet, all may be changed seriously. It is good that you published your EGTB. I am going to publish all my work (EGTB, code, as well as some documents I have been writing). Thus we have chances to verify all methods and make all things work.

[phhnguyen]

The code for processing perpetuations has been pushed into the branch "perpetuation".
It is still a working code. All could be changed later.