Repetition detection structure.

[bob]

From what I have seen over what is now 40 years of computer chess activities, the only "proof" that would be acceptable for a game would be to actually play the game out to a recognized final position. I have _never_ liked adjudications. Even should both programs agree on the outcome, it is always in doubt...

[gladius]

From what I have seen over what is now 40 years of computer chess activities, the only "proof" that would be acceptable for a game would be to actually play the game out to a recognized final position. I have _never_ liked adjudications. Even should both programs agree on the outcome, it is always in doubt...

I don't think anyone is arguing that adjudication based on score is going to get it wrong some of the time. The question is, is the time saved by not playing out those (mostly lost) endgames worth the error introduced by the few times when it is not lost?

Or, perhaps better phrased as, do you get more accuracy by playing more games in the same time then you would be able to otherwise. That's a tricky question to answer.

[hgm]

I think this is all pretty interesting in light of the discussion we had about winboard_F and its apparently policy of terminating a game when the rules of chess do not allow it. In the case we were discussing, false draw claims, and possibly illegal moves were grounds for terminating the game on the spot.

I think you got that wrong. Those were cases when the _engine_ terminated the game, not the GUI. It was only a matter of what the GUI should write on the score sheet. And a matter of how to handle engines that refuse to respect the protocol. And unlike what you seem to think, even in Human Chess people that refuse to obey protocol forfeit the game. The handshake incident is a recent example of that.

Even though a program _could_ recover, either by playing on if the draw is disallowed, or play a different move it ifs first move was somehow illegal.

Apparently my heart bleeds less for rogue programs than yours. And I am pretty sure that it would have been no problem to 'recover' from the refusal to shake hands as well, by giving the offender a warning and a second chance. Yet the zero did stand.

My take has always been that the GUI enforces rules, and only rules. Claiming a game is lost just because both programs say -5 is ridiculous, because I have seen _many_ games where Crafty was at -9.0 and still drew, and some where it was at +9.0 and also drew. So the scores are really "private data" that is displayed for the benefit of observers, but should not be used in adjudicating games.

I think we are in full agreement about that. Scores are private information, and if GUIs tend to abuse that information, we should not give it to them. Note, though, that in my opinion the communication protocol is just as much part of the rules that must be obeyed, just as shaking hands is in Human Chess. So if programs send 'resign', they should not later come back to complain that they unjustly lost the game, because for them 'resign' actually is their way of expressing that they want to castle...

[hgm]

Or, perhaps better phrased as, do you get more accuracy by playing more games in the same time then you would be able to otherwise. That's a tricky question to answer.

You would be surprised how easily the balance would swing in favor of playing fewer games, then. The sqrt(N) law means that the statistical noise only decreases extremely slowly with number of games. But the erroneous adjudications do add extra noise, which has to be averaged away by this painstaking process.

But what is even more important: I don't think that the two goals are at odds with each other. You can both save time (and thus have more games and higher accuracy in same time interval), and have accurate game results. Just by allowing the engines in a 40/40' game 1 minute to play out the game at points where you would otherwise adjuicate.

[Uri Blass]

Or, perhaps better phrased as, do you get more accuracy by playing more games in the same time then you would be able to otherwise. That's a tricky question to answer.

You would be surprised how easily the balance would swing in favor of playing fewer games, then. The sqrt(N) law means that the statistical noise only decreases extremely slowly with number of games. But the erroneous adjudications do add extra noise, which has to be averaged away by this painstaking process.

But what is even more important: I don't think that the two goals are at odds with each other. You can both save time (and thus have more games and higher accuracy in same time interval), and have accurate game results. Just by allowing the engines in a 40/40' game 1 minute to play out the game at points where you would otherwise adjuicate.

I think that the best procedure is probably to allow the engine in 1 40/40 game 1 minute to play and in case the result is not the same as the adjudication to continue the original game in 40/40 because a program that is weak at blitz should not suffer from adjudication.

There is no problem to do it in ponder off games and you can use a copy of the engine so the engine even do not forget the information in the hash.

There may be a problem in ponder on games because I see no way to tell engines that time stops.

The only way in ponder on games is to give the position to some different computer that does not use the engines and play bullet game and continue the engine-engine game at the same time when you adjudicate only after the bullet game.

Uri

[Spock]

You would be surprised how easily the balance would swing in favor of playing fewer games, then. The sqrt(N) law means that the statistical noise only decreases extremely slowly with number of games. But the erroneous adjudications do add extra noise, which has to be averaged away by this painstaking process.

I would hope that the erroneous adjudications are such a minute proportion of the games that the effect is almost immeasurable. Firstly it is only ever Arena that has been reported to do this, and that is a tiny portion of our games. Most widely used are chessbase and Shredder GUIs and they don't suffer from this problem. But in principle I don't like any GUI making any such decisions, and this very isolated incident has lead me to the conclusion personally that I'll turn adjudication off totally.

[bob]

I think this is all pretty interesting in light of the discussion we had about winboard_F and its apparently policy of terminating a game when the rules of chess do not allow it. In the case we were discussing, false draw claims, and possibly illegal moves were grounds for terminating the game on the spot.

I think you got that wrong. Those were cases when the _engine_ terminated the game, not the GUI. It was only a matter of what the GUI should write on the score sheet. And a matter of how to handle engines that refuse to respect the protocol. And unlike what you seem to think, even in Human Chess people that refuse to obey protocol forfeit the game. The handshake incident is a recent example of that.

Even though a program _could_ recover, either by playing on if the draw is disallowed, or play a different move it ifs first move was somehow illegal.

Apparently my heart bleeds less for rogue programs than yours. And I am pretty sure that it would have been no problem to 'recover' from the refusal to shake hands as well, by giving the offender a warning and a second chance. Yet the zero did stand.

Note that was apparantely a FIDE rule, which is OK by me. But if a program says "this is a draw" and it isn't, the game should continue unless the program refuses and loses on time.

My take has always been that the GUI enforces rules, and only rules. Claiming a game is lost just because both programs say -5 is ridiculous, because I have seen _many_ games where Crafty was at -9.0 and still drew, and some where it was at +9.0 and also drew. So the scores are really "private data" that is displayed for the benefit of observers, but should not be used in adjudicating games.

I think we are in full agreement about that. Scores are private information, and if GUIs tend to abuse that information, we should not give it to them. Note, though, that in my opinion the communication protocol is just as much part of the rules that must be obeyed, just as shaking hands is in Human Chess. So if programs send 'resign', they should not later come back to complain that they unjustly lost the game, because for them 'resign' actually is their way of expressing that they want to castle...

[bob]

From what I have seen over what is now 40 years of computer chess activities, the only "proof" that would be acceptable for a game would be to actually play the game out to a recognized final position. I have _never_ liked adjudications. Even should both programs agree on the outcome, it is always in doubt...

I don't think anyone is arguing that adjudication based on score is going to get it wrong some of the time. The question is, is the time saved by not playing out those (mostly lost) endgames worth the error introduced by the few times when it is not lost?

Or, perhaps better phrased as, do you get more accuracy by playing more games in the same time then you would be able to otherwise. That's a tricky question to answer.

to follow up, would you get to play more games if you stopped each one after move 30 and declared a winner if both programs liked the same side?

[Ovyron]

Claiming a game is lost just because both programs say -5 is ridiculous, because I have seen _many_ games where Crafty was at -9.0 and still drew, and some where it was at +9.0 and also drew.

I've even seen games with an engine showing +6 and losing the game, and I think that adjudicating that lose as a win would have been worse than not running the game at all (So, I only adjudicate games when they reach a tablebase solution, otherwise I can't be sure of the result.)

[hgm]

Note that was apparantely a FIDE rule, which is OK by me. But if a program says "this is a draw" and it isn't, the game should continue unless the program refuses and loses on time.

Yes, it was a FIDE rule, and by that FIDE rule all computer programs would forfeit their game even before it would start, as they have no hands and do not greet their opponent in any alternative way. That just illustrates how silly it is to require that Human rules apply without any adaptation to computers.

Furthermore note that the engines under WinBoard_F (if the user sets this option) do not forfeit because they say "this is a draw", but because they say "I won't play on" in a position that is not won or drawn by any rule. And if they don't mean that, they'd just better not say it. This is also completely similar to Human Chess, where there is also no way back if you say "I resign", and later claim that you meant something else... Even if you think this is undesirable (which I don't), in practice there is simply no way to accomodate engines that substitute arbitry commands for arbitrary other commands, by having a GUI second-guess what they really mean. Because where would it stop? You might as well have the GUI say, when a program plays 1. a2a3 "Well, but this is such a poor move, it cannot possibly mean that. Let's assume it is just bad pronunciation, and that it means 1. e2e3"... If you don't stick to the protocol, the consequences are entirely for you. That is true if you number the board files differently, and it is true when you decide 'resign' means castling. So why should it be any different when you say "this game is definitely over" when you mean you want to play on?