Stockfish can't evaluate the KP vs K endgame?

[Zhu.Jianzhao]

So this is an analysis bug only, IMO. Most engines give a draw score in the search after one repetition to (barely) improve speed, right? I wonder if Bob would do a cluster run to see if one repetition has any noticeable ELO improvement compared to requiring two reps?

Anyway, my engine does the same thing. Seems like the easiest way to resolve this is to require two reps when in analysis mode, and only one rep when playing a game (on the assumption that the speed increase has value). "Bug" fixed.

jm

How do you know you are analyzing? you can't. If the engine is used with another tool to annotate the game, I cannot see how that could be done. IMHO, this is not a bug, but a flawed optimization.

Miguel

Winboard's "analyze" command puts the engine in analysis mode. Does UCI not provide a similar command?

jm

If making two versions of stockfish , the one without optimization to be used as an analyze engine, problem solved:)

By the way, Houdini 3 doesn't suffer from this repetition "bug", maybe it's good for us to kindly ask Robert how he handle this.

[JVMerlino]

So this is an analysis bug only, IMO. Most engines give a draw score in the search after one repetition to (barely) improve speed, right? I wonder if Bob would do a cluster run to see if one repetition has any noticeable ELO improvement compared to requiring two reps?

Anyway, my engine does the same thing. Seems like the easiest way to resolve this is to require two reps when in analysis mode, and only one rep when playing a game (on the assumption that the speed increase has value). "Bug" fixed.

jm

How do you know you are analyzing? you can't. If the engine is used with another tool to annotate the game, I cannot see how that could be done. IMHO, this is not a bug, but a flawed optimization.

Miguel

Winboard's "analyze" command puts the engine in analysis mode. Does UCI not provide a similar command?

jm

If making two versions of stockfish , the one without optimization to be used as an analyze engine, problem solved:)

By the way, Houdini 3 doesn't suffer from this repetition "bug", maybe it's good for us to kindly ask Robert how he handle this.

The simplest way, of course, is to require two reps for a draw. It's quite possible this is what Houdini does. Hopefully somebody who has done a thorough comparison of the ELO difference between using 1 and 2 reps will chime in.

jm

[michiguel]

So this is an analysis bug only, IMO. Most engines give a draw score in the search after one repetition to (barely) improve speed, right? I wonder if Bob would do a cluster run to see if one repetition has any noticeable ELO improvement compared to requiring two reps?

Anyway, my engine does the same thing. Seems like the easiest way to resolve this is to require two reps when in analysis mode, and only one rep when playing a game (on the assumption that the speed increase has value). "Bug" fixed.

jm

How do you know you are analyzing? you can't. If the engine is used with another tool to annotate the game, I cannot see how that could be done. IMHO, this is not a bug, but a flawed optimization.

Miguel

Winboard's "analyze" command puts the engine in analysis mode. Does UCI not provide a similar command?

jm

go infinite rather than a specific time, but you can use the engines with other annotating tools and that is a problem.

Miguel

[Michel]

So this is an analysis bug only, IMO. Most engines give a draw score in the search after one repetition to (barely) improve speed, right? I wonder if Bob would do a cluster run to see if one repetition has any noticeable ELO improvement compared to requiring two reps?

Anyway, my engine does the same thing. Seems like the easiest way to resolve this is to require two reps when in analysis mode, and only one rep when playing a game (on the assumption that the speed increase has value). "Bug" fixed.

jm

How do you know you are analyzing? you can't. If the engine is used with another tool to annotate the game, I cannot see how that could be done. IMHO, this is not a bug, but a flawed optimization.

Miguel

Winboard's "analyze" command puts the engine in analysis mode. Does UCI not provide a similar command?

jm

If making two versions of stockfish , the one without optimization to be used as an analyze engine, problem solved:)

By the way, Houdini 3 doesn't suffer from this repetition "bug", maybe it's good for us to kindly ask Robert how he handle this.

In the same way of course that GNU Chess handles this. Read my post above.

[Eelco de Groot]

So this is an analysis bug only, IMO. Most engines give a draw score in the search after one repetition to (barely) improve speed, right? I wonder if Bob would do a cluster run to see if one repetition has any noticeable ELO improvement compared to requiring two reps?

Anyway, my engine does the same thing. Seems like the easiest way to resolve this is to require two reps when in analysis mode, and only one rep when playing a game (on the assumption that the speed increase has value). "Bug" fixed.

jm

How do you know you are analyzing? you can't. If the engine is used with another tool to annotate the game, I cannot see how that could be done. IMHO, this is not a bug, but a flawed optimization.

Miguel

Winboard's "analyze" command puts the engine in analysis mode. Does UCI not provide a similar command?

jm

If making two versions of stockfish , the one without optimization to be used as an analyze engine, problem solved:)

By the way, Houdini 3 doesn't suffer from this repetition "bug", maybe it's good for us to kindly ask Robert how he handle this.

The simplest way, of course, is to require two reps for a draw. It's quite possible this is what Houdini does. Hopefully somebody who has done a thorough comparison of the ELO difference between using 1 and 2 reps will chime in.

jm

Winboard's "analyze" command puts the engine in analysis mode. Does UCI not provide a similar command?

Code: Select all

// if the engine supports the "UCI_AnalyseMode" option and the next search is supposted to
// be an analysis, the GUI should set "UCI_AnalyseMode" to true if it is currently
// set to false with this engine
setoption name UCI_AnalyseMode value true

True three fold repetition was just thrown out as far as I know, because it was actually a regression. Which was more or less to be expected I suppose as all programmers say it simply will decrease performance, for game play its just not so good, although I have not studied the actual code. Threefold repetition was only recognized in PV nodes but that is probably enough? It can be easily found on GitHub. I don't see why that particular patch could not be used in UCI analysis mode only, I think it would be a nice idea to try, especially if it would solve this, I don't know.

The patch was reversed here: Revert "Extend full 3 fold detection to PvNodes"

Eelco

[JVMerlino]

So this is an analysis bug only, IMO. Most engines give a draw score in the search after one repetition to (barely) improve speed, right? I wonder if Bob would do a cluster run to see if one repetition has any noticeable ELO improvement compared to requiring two reps?

Anyway, my engine does the same thing. Seems like the easiest way to resolve this is to require two reps when in analysis mode, and only one rep when playing a game (on the assumption that the speed increase has value). "Bug" fixed.

jm

How do you know you are analyzing? you can't. If the engine is used with another tool to annotate the game, I cannot see how that could be done. IMHO, this is not a bug, but a flawed optimization.

Miguel

Winboard's "analyze" command puts the engine in analysis mode. Does UCI not provide a similar command?

jm

go infinite rather than a specific time, but you can use the engines with other annotating tools and that is a problem.

Miguel

Yes, indeed. There's no way to know for sure how a tool tells an engine to "think" on a position. For example, GradualTest does not put the engine in analysis mode, but instead tells the engine that it has a very long amount of time to think (several hours, IIRC), then uses the "go" and "?" commands to get the move from the engine after the specified time. The only way that I determined this was by looking at Myrddin's logfiles.

jm

[Michel]

Nobody seems to understand here that this issue is trivial to solve.

I'll repeat: _only_ use the 2 fold repetion rule when both positions occur in search. Do not use it for positions that occur in the game history.

With this rule the first Kf6 will not be scored as a draw since the original position has become part of the game history. So repeating it during search is not enough to enough to declare draw.

The second Kf6 however will be scored as a draw since now there is
a threefold repetion in search (with the first two positions in the game history).

In UCI this is even easier to implement than in the xboard protocol since
the game history is sent with every move.

[michiguel]

Nobody seems to understand here that this issue is trivial to solve.

I'll repeat: _only_ use the 2 fold repetion rule when both positions occur in search. Do not use it for positions that occur in the game history.

With this rule the first Kf6 will not be scored as a draw since the original position has become part of the game history. So repeating it during search is not enough to enough to declare draw.

The second Kf6 however will be scored as a draw since now there is
a threefold repetion in search (with the first two positions in the game history).

In UCI this is even easier to implement than in the xboard protocol since
the game history is sent with every move.

Nobody is a pretty low number

Yes, it is trivial. I do the same in Gaviota and Bob was teaching about it for two decades. SF team seems to think that there is a faster search if this is detection is "accelerated". I think the price to pay in terms of analysis is too high, IMHO.

Miguel

[bob]

[d] 6k1/8/6K1/6P1/8/8/8/8 w - - 0 1

Code: Select all

New game
6k1/8/6K1/6P1/8/8/8/8 w - - 0 1

Analysis by Stockfish 20-02-13 64bit:

1.Kh6 Kf7 2.g6+ Kf8 3.Kh7 Ke7 4.g7 Kd6 5.g8Q Kc5 6.Kg6 Kd4 7.Kf6 Kd3 8.Ke5 Kc3 9.Qd5 Kc2 10.Kd4 Kb2 11.Qc4 Ka1 12.Kc3 Kb1 13.Qb5+ Ka2 14.Qb2# 
+-  (#14)    Depth: 46/32   00:00:36  168mN
(User,  23.02.2013)

I continued to input the move, after 1.Kf6 ...

[d] 6k1/8/5K2/6P1/8/8/8/8 b - - 0 1

Code: Select all

New game
6k1/8/5K2/6P1/8/8/8/8 b - - 0 1

Analysis by Stockfish 20-02-13 64bit:

1...Kh7 2.Kf7 Kh8 3.Kf8 Kh7 4.Kf7 
=  (0.00)    Depth: 100/7   00:00:02  1047kN
(User,  23.02.2013)

following 1...Kh7 2.Kf7, the evaluation still returns 0.00

[d] 8/5K1k/8/6P1/8/8/8/8 b - - 0 2

Code: Select all

New game
8/5K1k/8/6P1/8/8/8/8 b - - 0 1

Analysis by Stockfish 20-02-13 64bit:

2...Kh8 3.Kf8 Kh7 4.Kf7 
=  (0.00)    Depth: 100/5   00:00:00  4kN
(User,  23.02.2013)

after I input 2...Kh8 3.Kg6, again the evaluation is 0.00, but we can see the mate from the analysis.

[d] 7k/8/6K1/6P1/8/8/8/8 b - - 0 3

Code: Select all

New game
7k/8/6K1/6P1/8/8/8/8 b - - 0 1

Analysis by Stockfish 20-02-13 64bit:

3...Kg8 4.Kh6 Kf7 5.g6+ Kf8 6.Kh7 Ke7 7.g7 Kd6 8.g8Q Kc5 9.Kg6 Kd4 10.Kf6 Kd3 11.Ke5 Kc3 12.Qd5 Kc2 13.Kd4 Kb2 14.Qc4 Ka1 15.Kc3 Kb1 16.Qb5+ Ka2 17.Qb2# 
=  (0.00)    Depth: 100/2   00:00:00  6kN
(User,  23.02.2013)

Have tried other engines like Hiarcs, Komodo, Fritz, only Houdini 3 don't have such problem.

There are plenty that have no problems with this. Crafty finds a mate in 18 after a couple of seconds, and a mate in 15 by 5 seconds, no EGTBs used at all...

e23-> 1.59 13.84 1. Kh6 Kf7 2. g6+ Ke6 3. g7 Kd6 4.
g8=Q Kc5 5. Qb3 Kd4 6. Qb4+ Kd3 7.
Kg6 Ke2 8. Qc4+ Kf2 9. Qd3 Kg1 10.
Qe4 Kf2 11. Kh6 Kf1 12. Kg5 Kf2 13.
Kf4 Kg1 14. Qf3 Kh2 15. Qg3+ Kh1
e24 1.76 ++ 1. Kh6! (>+14.09)
e24 1.78 ++ 1. Kh6! (>+14.25)
e24 1.78 ++ 1. Kh6! (>+14.57)
e24 1.78 ++ 1. Kh6! (>+15.21)
e24 1.78 ++ 1. Kh6! (>+16.49)
e24 1.79 ++ 1. Kh6! (>+19.05)
e24 1.79 ++ 1. Kh6! (>+24.17)
e24 3.57 Mat18 1. Kh6 Kf7 2. g6+ Kf8 3. Kh7 Ke7 4.
g7 Kd6 5. g8=Q Kc5 6. Qb8 Kd4 7. Kh6
Kc4 8. Qb6 Kc3 9. Qb5 Kd4 10. Kg5 Ke4
11. Qc6+ Kd3 12. Kf4 Kd4 13. Qb5 Kc3
14. Ke4 Kc2 15. Qb4 Kd1 16. Qb2 Ke1
17. Ke3 Kf1 18. Qf2#
e24-> 4.11 Mat18 1. Kh6 Kf7 2. g6+ Kf8 3. Kh7 Ke7 4.
g7 Kd6 5. g8=Q Kc5 6. Qb8 Kd4 7. Kh6
Kc4 8. Qb6 Kc3 9. Qb5 Kd4 10. Kg5 Ke4
11. Qc6+ Kd3 12. Kf4 Kd4 13. Qb5 Kc3
14. Ke4 Kc2 15. Qb4 Kd1 16. Qb2 Ke1
17. Ke3 Kf1 18. Qf2#
e25 4.67 Mat18 1. Kh6 Kf7 2. g6+ Kf8 3. Kh7 Ke7 4.
g7 Kd6 5. g8=Q Kc5 6. Qb8 Kc4 7. Qb6
Kd3 8. Kh6 Kd2 9. Qb3 Ke2 10. Kg5 Kd2
11. Kf4 Kc1 12. Qb5 Kc2 13. Qb4 Kd3
14. Qc5 Kd2 15. Qc4 Ke1 16. Qc2 Kf1
17. Kf3 Ke1 18. Qe2#
e25-> 5.01 Mat18 1. Kh6 Kf7 2. g6+ Kf8 3. Kh7 Ke7 4.
g7 Kd6 5. g8=Q Kc5 6. Qb8 Kc4 7. Qb6
Kd3 8. Kh6 Kd2 9. Qb3 Ke2 10. Kg5 Kd2
11. Kf4 Kc1 12. Qb5 Kc2 13. Qb4 Kd3
14. Qc5 Kd2 15. Qc4 Ke1 16. Qc2 Kf1
17. Kf3 Ke1 18. Qe2#
e26 5.74 Mat15 1. Kh6 Kf7 2. g6+ Kf8 3. Kh7 Ke7 4.
g7 Kd6 5. g8=Q Kc5 6. Qb3 Kc6 7. Qc4+
Kb6 8. Qb4+ Kc7 9. Kg6 Kc6 10. Kf6
Kc7 11. Qb5 Kc8 12. Qb6 Kd7 13. Qc5
Kd8 14. Ke6 Ke8 15. Qc8#
e26-> 5.93 Mat15 1. Kh6 Kf7 2. g6+ Kf8 3. Kh7 Ke7 4.
g7 Kd6 5. g8=Q Kc5 6. Qb3 Kc6 7. Qc4+
Kb6 8. Qb4+ Kc7 9. Kg6 Kc6 10. Kf6
Kc7 11. Qb5 Kc8 12. Qb6 Kd7 13. Qc5
Kd8 14. Ke6 Ke8 15. Qc8#

[bob]

I tired Stockfish 2.3.1 JA just now, got the same problem, it returns 0.00 after I entered Kf6.

Yes Miguel is right - it's a repetion problem.

Code: Select all

position fen 6k1/8/6K1/6P1/8/8/8/8 w - - 0 1 moves g6f6

However, SF would never play Kf6 in the first position anyway. Not a bug then.

It does not matter whether the engine plays Kf6 or not, if it is used to analyzed a game, it will give wrong results for positions with a repetition in its path.

Miguel

So this is an analysis bug only, IMO. Most engines give a draw score in the search after one repetition to (barely) improve speed, right? I wonder if Bob would do a cluster run to see if one repetition has any noticeable ELO improvement compared to requiring two reps?

Anyway, my engine does the same thing. Seems like the easiest way to resolve this is to require two reps when in analysis mode, and only one rep when playing a game (on the assumption that the speed increase has value). "Bug" fixed.

jm

I did this years ago. It was not a huge win, but it was certainly significant. The problem is that a program will then convert wins to draws, because they don't realize that if the opponent can force a 2-fold repetition, he can also force a 3-fold repetition, so it voluntarily enters a line where a 2-fold is forced but it doesn't realize it is now drawing, not winning...