adjudication of games before they are finished

[Uri Blass]

The stockfish team use rules to adjudicate games and it can distort the results.

Of course they do it to save testing time but my opinion about it is bad.

Recently they decided to change
cutechess-cli adjudication rules from
movecount=2 to movecount=8 but resign is still at -4.

I have no proof that these adjudication rules are worse than no adjudication
but my feeling is that it may be better even not to have adjudication.

I do not post about it in the stockfish forum because this forum is relatively hostile to posting ideas without code but I can imagine that some counter productive change may be accepted because of adjudication.

Imagine that some change cause a bug that cause the program not to be able to win in won positions when the program often draw by the 50 move rule or by repetition in positions that it has a forced mate.

Adjudication is not going to find the bug.
Even without finding bugs like that there are also fortress positions with evaluation lower than -4 and even if they do not happen often considering the fact that they test small changes it can still cause distortion in the results.

My opinion is that adjudication can be good only if the number of games that their result is modified by adjudication is very small relative to the time that people save by adjudication.

Note that most of the matchs need significantly more than 1000 games so doing no adjudication in the first 1000 games is not going to add a lot of machine time.

I think that after people decide about adjudication rules
it may be better simply to run the first 1000 games of every match with no adjudication and calculate also the number of results that are different in case of adjudication in these games and the time that you could save by adjudication and use the information in order to decide if to do adjudication in the rest of the games.

[jdart]

I use -7 as a threshold for adjudication. I've seen games online where the score got below -5 and the game was eventually drawn. I don't think this is frequent but I use -7 in testing as a precaution. Also, -5 is the default resign threshold for Arasan, but I set it to -7 when playing on the chess servers.

--Jon

[Don]

Those are reasonable adjudication rules which could be wrong once in a great while. I personally don't like adjudication rules based on the opinions of the players themselves, even if the opponent agrees. The rules should be ones that could be consistently applied without knowledge of either players opinion.

One way to do this is just count the material on the board using a scoring system such as the classic 1,3,3,5,9 values of the pieces and have the rule that the material should be stable for N moves in a row - and even better also have pawn moves reset this counter.

A rule does not have to be absolutely perfect though - when you consider that the rule is the same for both sides. So it doesn't impose a significant advantage to one player or the other although there are positional reasons it could create a very tiny advantage or disadvantage for a certain style of player. But when the players are different versions of the same program that becomes lost in the insignificant bits.

Larry and I have no adjudication rules. You don't really need them if you play at sudden death time controls or have an increment that is very small in relation to your main time control. If you are in a long drawn out ending the play speeds up in sudden death with each move. If you are crushing the opponent the game will terminate quickly with the help of sudden death and if you have a big advantage but you cannot finish the game quickly then perhaps the "real" advantage is not so great and you shouldn't adjudicate. So very little is gained with complicated adjudication rules when playing sudden death. But I have to agree that very little is lost too.