mmt Vs. Ovyron (G4 D5 BG2)

[Alayan]

50 move rule and the hash having no 50mr counter can lead to false mate announcements in many engines.This issue is irrelevant in most situations with a mate claim, however.

[Uri Blass]

Please just let others respond, if they are happy to. I doubt that in the entire history of chess engines, they've all always verified mate announcements. When, almost a quarter of a century ago, an engine reported a mate in 150 (as was reported on this board), it would have had to have made TBs do most of the work for that to be possible (and maybe it did, but who knows?).

It's funny, because I do not remember SF announcing mates in 150 or 250 moves in the last few years (and Sesse, for example, does have TB access). Maybe, just maybe, it's because SF does try to verify its announcements (so, TB or not, it's not easy to get to 250 moves) and other engines do not. (There are other possible explanations, but surely this is one.)

I mean, to make such a sweeping claim that all engines have always verified, when most of them are not open source, is more you wanting to believe something than having strong evidence for it.

The reason is simple.
sf does not use the nalimov tablabases that have a distance to mate but tablebases that have distance to conversion so it does not have information about long mate in the tablebases when the mate may be a draw by the 50 move rule.

[jdart]

May I ask what Arasan does? If it makes mate announcements, does it switch off pruning for the defence (not the attack) and verify every defensive line before making the announcement? (Can you point us to the relevant section of the source code?)

It does not do anything different from other engines, and I did not mean to imply that it does.

I don't believe it will report incorrect mate-in-n, in the sense that if there is a mate in 4 reported, then the side to move has mate in 4 moves or less. If this were not the case then a mate score should not have been propagated to the root. On the other hand, that does not mean there is not also a shorter mate, say, mate in 3, with optimal play.

--Jon

[Uri Blass]

May I ask what Arasan does? If it makes mate announcements, does it switch off pruning for the defence (not the attack) and verify every defensive line before making the announcement? (Can you point us to the relevant section of the source code?)

It does not do anything different from other engines, and I did not mean to imply that it does.

I don't believe it will report incorrect mate-in-n, in the sense that if there is a mate in 4 reported, then the side to move has mate in 4 moves or less. If this were not the case then a mate score should not have been propagated to the root. On the other hand, that does not mean there is not also a shorter mate, say, mate in 3, with optimal play.

--Jon

1)I stopped to work about my engine movei many years ago but I know that in tablebases positions it can report shorter mate than the shortest possible mate and it was not important for me to find the reason and fix the problem because I did not see a case when it could not win inspite of a mate score.

I remember that in the past some Fritz versions also reported wrong mate scores in tablebases position but had no problem to win from them.

2)I think that engines should report mate in at most n moves and not mate in n moves if we are sure that it is really mate in at most n moves
but do not know that n is the shortest mate.

Other possible scores can be mate in exactly n moves(if the engine verified that there is no mate in n-1 moves after finding mate in n moves
and it is possible to decide to verify that there is no shorter mate at depth that is big enough after finding mate without losing elo
or guessing the distance to mate based on playing against itself

The last option can be used by some interface with every engine that support fixed depth search and the output may be something like

guess depth 1 draw some pv
guess depth 2 mate in 30 for black some pv
guess depth 3 mate in 50 for white some pv
guess depth 4 mate in 40 for white some pv
guess depth 5 mate in 45 for white some pv

The pv's are basically the moves of the games and
the output means that
1)The engine drew against itself at fixed depth 1
2)The engine won with 30 moves with black at fixed depth 2
3)The engine won with 50 moves with white at fixed depth 3
4)The engine won with 40 moves with white at fixed depth 4
5)The engine won with 45 moves with white at fixed depth 5

[mmt]

1. g4 d5 2. Bg2 Bxg4 3. c4 c6 4. Qb3 e6 5. Qxb7 Nd7 6. Nc3 Ne7 7. cxd5 exd5 8. d4 Rb8 9. Qa6 Rb6 10. Qd3 Ng6 11. h3 Be6 12. Nf3 Bd6 13. h4 h5 14. b3 Nf6 15. Bg5 O-O 16. e3 Re8 17. Kf1 Bg4 18. Ne1 Bb4 19. Na4 Rb8 20. Nc2 Be7 21. f3 Be6 22. Nc5 Bc8 23. Kf2 Nd7 24. Ne6 Qa5 25. Bxe7 Rxe7 26. b4 Qb6 27. Ng5 Ba6 28. Qa3

[d]1r4k1/p2nrpp1/bqp3n1/3p2Np/1P1P3P/Q3PP2/P1N2KB1/R6R b - - 1 1
This was not LC0's first choice but faced with SF's responses its score fell more. It's already down to -1.93.

[Alayan]

2)I think that engines should report mate in at most n moves and not mate in n moves if we are sure that it is really mate in at most n moves
but do not know that n is the shortest mate.

Mate in N announcement means that N is the upperbound, it says nothing about the lowerbound. This is widely understood and definitely not a bug.

[jp]

When, almost a quarter of a century ago, an engine reported a mate in 150 (as was reported on this board), it would have had to have made TBs do most of the work for that to be possible (and maybe it did, but who knows?).

It's funny, because I do not remember SF announcing mates in 150 or 250 moves in the last few years (and Sesse, for example, does have TB access). Maybe, just maybe, it's because SF does try to verify its announcements (so, TB or not, it's not easy to get to 250 moves) and other engines do not. (There are other possible explanations, but surely this is one.)

The reason is simple.
sf does not use the nalimov tablabases that have a distance to mate but tablebases that have distance to conversion so it does not have information about long mate in the tablebases when the mate may be a draw by the 50 move rule.

It could be. When there was talk on this board about these mates in 150 or 250 a decade ago, there was definitely debate about which TBs were being used, etc. I guess some of those announced mates in 150 or 250 could have been cursed mates.
If an engine is using DTC/DTZ TBs, it can of course still be made to give long announcements (upper bound) if that's what the author wants, by just adding all the DTCs.

50 move rule and the hash having no 50mr counter can lead to false mate announcements in many engines.

So no TB needed to give problems...

[jp]

2)I think that engines should report mate in at most n moves and not mate in n moves if we are sure that it is really mate in at most n moves but do not know that n is the shortest mate.

Mate in N announcement means that N is the upperbound, it says nothing about the lowerbound. This is widely understood and definitely not a bug.

Yes, after all, the output just says something like "+M25", so it's for us to interpret that as "mate in at most 25 moves".

But I think Uri's main point is in the rest of his point "2)". That might be good to implement as a non-playing mode, like the mate-finding mode some engines have.

1)I stopped to work about my engine movei many years ago but I know that in tablebases positions it can report shorter mate than the shortest possible mate and it was not important for me to find the reason and fix the problem because I did not see a case when it could not win inspite of a mate score.

I remember that in the past some Fritz versions also reported wrong mate scores in tablebases position but had no problem to win from them.

Uri, does this mean you suspect all false mates (false upper bounds) announced were/are due to tablebase issues?

[Ovyron]

I'll be honest and say that the game is really difficult as I have no idea about what mmt's choices will be, and those positions that are -2.60 but fall back to -2.00 as you go deeper are really scary. I still haven't managed to break through the -2.30s, if mmt has a drawn trap prepared it's possible I don't see it, but I'll do my best (this here in case people are wondering why it's taking me so long to move.)

[mmt]

I'll be honest and say that the game is really difficult as I have no idea about what mmt's choices will be, and those positions that are -2.60 but fall back to -2.00 as you go deeper are really scary. I still haven't managed to break through the -2.30s, if mmt has a drawn trap prepared it's possible I don't see it, but I'll do my best (this here in case people are wondering why it's taking me so long to move.)

LC0 with the older Sergio net 384x30-t40-swa-1808000 almost thought it had one in the Qf5 Nf6 Bf1 Bc8 Qd3 Ne8 line with a queen sacrifice!

[d]1rb1n1k1/p3rpp1/1qp3n1/3p2Np/1P1P3P/3QPP2/P1N2K2/R4B1R w - - 6 4
Nxf7 Rxf7 Qxg6 Bf5 Qxf5?! Unfortunately, it wasn't solid and other nets don't play it. For several following moves LC0 thought it's only down by -2.0 but SF11 already had it at like -9. Other newer nets have it at -5.2 immediately.