xboard 4.3.15 mishandles Crafty draw agreement

[zullil]

In a game between Crafty-22.9 and Crafty-22.8 played under xboard 4.3.15 the engines apparently agreed to a draw, but the result was recorded as 0-1 in xboard:

Code: Select all

63... Re4+ {False draw claim: 'Draw agreed'} 0-1

Here's the end of the Crafty-22.9 (White) log of the game:

Code: Select all

Black(60): time 8470
time remaining:   1:24 (Crafty).
Black(60): otim 10971
time remaining:   1:49 (opponent).
Black(60): Rd4
              time used:   2.45
              time limit 1.92 (+0.00) (17.92)
              depth   time  score   variation (1)
starting thread 1
               16     1.05   0.30   Ra5 Rxf4 Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+
                                    Ke3 Ra4 Ra6+ Kf5 Ra8 Ra3+ Kd4 Ra4+
                                    Kc5 Ra2 a6 Rxg2 a7 Rc2+ Kd6
               16->   1.21   0.30   Ra5 Rxf4 Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+
                                    Ke3 Ra4 Ra6+ Kf5 Ra8 Ra3+ Kd4 Ra4+
                                    Kc5 Ra2 a6 Rxg2 a7 Rc2+ Kd6
               17     2.30   0.16   Ra5 Rxf4 Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+
                                    Ke1 Ra4 a6 Ke5 Ra8 Kf4 a7 Kg3 Ke2 Ra3
                                    <HT>
              time=2.64  mat=0  n=12085929  fh=90%  nps=4.6M
              ext-> check=518K qcheck=899K reduce=5.0M/1.3M
              predicted=19  evals=7.1M  50move=1  EGTBprobes=1K  hits=1K
              SMP->  splits=155  aborts=13  data=5/128  elap=2.64
terminating SMP processes.White(61): Ra5
              time used:   2.64
Black(61): time 8206
time remaining:   1:22 (Crafty).
Black(61): otim 10736
time remaining:   1:47 (opponent).
Black(61): Rxf4
              time used:   2.35
              time limit 1.86 (+0.00) (17.36)
              depth   time  score   variation (1)
starting thread 1
               16     1.38   0.07   Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+ Ke3 Ra4
                                    a6 Kf5 Ra8 Kg4 Kd2 Kg3 a7 Kxg2 Kd3
                                    g5 Ke3 <HT>
               16->   1.73   0.07   Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+ Ke3 Ra4
                                    a6 Kf5 Ra8 Kg4 Kd2 Kg3 a7 Kxg2 Kd3
                                    g5 Ke3 <HT>
              time=4.16  mat=-1  n=18899191  fh=89%  nps=4.5M
              ext-> check=798K qcheck=1.5M reduce=7.5M/1.9M
              predicted=19  evals=11.1M  50move=0  EGTBprobes=2K  hits=2K
              SMP->  splits=175  aborts=19  data=6/128  elap=4.16
terminating SMP processes.White(62): Rxa7+
              time used:   4.16

       +---+---+---+---+---+---+---+---+
    8  |   | . |   | . |   | . |   | . |
       +---+---+---+---+---+---+---+---+
    7  |-R-|   | . |   | . |<K>| . |   |
       +---+---+---+---+---+---+---+---+
    6  |   | . |   | . |   | . |<P>| . |
       +---+---+---+---+---+---+---+---+
    5  | . |   | . |   | . |   | . |   |
       +---+---+---+---+---+---+---+---+
    4  |-P-| . |   | . |   |<R>|   |<P>|
       +---+---+---+---+---+---+---+---+
    3  | . |   | . |   | . |   | . |   |
       +---+---+---+---+---+---+---+---+
    2  |   | . |   | . |-K-| . |-P-| . |
       +---+---+---+---+---+---+---+---+
    1  | . |   | . |   | . |   | . |   |
       +---+---+---+---+---+---+---+---+
         a   b   c   d   e   f   g   h

Black(62): time 7790
time remaining:   1:17 (Crafty).
Black(62): otim 10507
time remaining:   1:45 (opponent).
Black(62): Kf6
              time used:   2.29
              time limit 1.77 (+0.00) (16.52)
              depth   time  score   variation (1)
starting thread 1
               14->   0.08   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
               15     0.08   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
               15->   0.18   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
               16     0.18   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
               16->   0.45   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
              time=1.89  mat=0  n=8474115  fh=89%  nps=4.5M
              ext-> check=370K qcheck=663K reduce=3.4M/929K
              predicted=19  evals=4.9M  50move=1  EGTBprobes=1K  hits=1K
              SMP->  splits=206  aborts=14  data=6/128  elap=1.89
terminating SMP processes.White(63): a5
              time used:   1.89
Black(63): draw
Black(63): time 7602
time remaining:   1:16 (Crafty).
Black(63): otim 10271
time remaining:   1:42 (opponent).
Black(63): Re4+
              time used:   2.35
              time limit 1.72 (+0.00) (16.10)
              depth   time  score   variation (1)
starting thread 1
               15     0.48   0.01   Kf3 Ra4 Ke3 g5 a6 Ke5 Ra8 Ra3+ Kf2
                                    Kf4 Rf8+ Ke4 Re8+ Kf4 Rf8+ <HT>
               15->   0.48   0.01   Kf3 Ra4 Ke3 g5 a6 Ke5 Ra8 Ra3+ Kf2
                                    Kf4 Rf8+ Ke4 Re8+ Kf4 Rf8+ <HT>
              time=1.78  mat=0  n=8265014  fh=89%  nps=4.6M
              ext-> check=352K qcheck=706K reduce=3.2M/847K
              predicted=19  evals=4.8M  50move=1  EGTBprobes=1K  hits=1K
              SMP->  splits=228  aborts=24  data=5/128  elap=1.78
terminating SMP processes.tellics draw
1/2-1/2 {Draw agreed}
White(64): Kf3
              time used:   1.78
Black(64): result 0-1 {False draw claim: 'Draw agreed'}
LearnBook() executed
Black(64): force
Black(64): ping 2
pong 2

Not sure who's "bug" this is, but I don't recall it occurring in older versions of Xboard.

[hgm]

It seems Crafty commits a protocol violation here: the RESULT command (1/2-1/2) should not be used in WinBoard protocol to accept a draw offer. This is not an invention of mine, but already clearly specified in Tim Mann's description of the protocol. The reason is that even if the opponent offered a draw, the draw offer might be no longer valid, and ending the game through a RESULT command is not safe:

draw
The engine's opponent offers the engine a draw. To accept the draw, send "offer draw". To decline, ignore the offer (that is, send nothing). If you're playing on ICS, it's possible for the draw offer to have been withdrawn by the time you accept it, so don't assume the game is over because you accept a draw offer. Continue playing until xboard tells you the game is over. See also "offer draw" below.

The reason that Crafty can get away with this in old WB versions is that those versions do not check the claims engines make at all. Such versions would not even complain if Crafty would say, on reception of a draw offer: 1-0 {opponent resigns}, and count it as a win for Crafty. To reproduce that old behaviour, you can simply run with the option /testClaims=false.

But if you run WinBoard 4.3 with claim verification, it flags such incorrect or unfounded RESULT claims.

Now I admit that declaring a forfeit for Crafty in this situation is a bit harsh, if the opponent indeed offered a draw, as the protocol also specifies that a draw offer remains valid for two moves in engine-engine mode. But the protocol specifies nothing for ICS mode, and as in general we don't want engines to behave differently for ICS and loal play, I see little benefit in adding code to ameliorate the consequences of this protocol violation.

Of necessity the consequences of violating protocol are unpredictable, and might even lead to forfeit. People that think it is a good idea to have their engines claim a draw after a 3-fold repeat by sending the message How about a game of Space Invaders? to the GUI, in stead of the prescribed 1/2-1/2, should expect to forfeit on time in a drawn situation. This is what protocol violations get you, no matter how unfair it seems. So why make an exception for this particular one? It would merely provide an incentive for people to violate protocol, which is in no ones interest.

[zullil]

Thanks for the quick and thorough reply. I assume Bob Hyatt will respond later.

[Jim Ablett]

'offer draw' code is there. It's in 'resign.c'

Code: Select all

if (offer_draws && result == 2) {
    draw_offered = 1;
    if (!ics && !xboard) {
      Print(128, "\nI offer a draw.\n\n");
      if (audible_alarm)
        printf("%c", audible_alarm);
      if (speech) {
        char announce[128];

        strcpy(announce, SPEAK);
        strcat(announce, "Drawoffer");
        system(announce);
      }
    } else if (xboard)
      Print(4095, "offer draw\n");
    else
      Print(4095, "\n*draw\n");
  } else
    draw_offered = 0;
}

Jim.

[Jim Ablett]

in 'main.c'

Code: Select all

  if (draw_offer_pending) {
      int drawsc = abs_draw_score;

      draw_offer_pending = 0;
      if (move_number < 40 || !accept_draws)
        drawsc = -300;
      if (value <= drawsc && (tc_increment != 0 ||
              tc_time_remaining_opponent >= 1000)) {
        if (xboard)
          Print(4095, "tellics draw\n");
        else {
          Print(128, "Draw accepted.\n");
          if (audible_alarm)
            printf("%c", audible_alarm);
          if (speech) {
            char announce[128];

            strcpy(announce, SPEAK);
            strcat(announce, " Drawaccept");
            system(announce);
          }
        }
        Print(4095, "1/2-1/2 {Draw agreed}\n");
        strcpy(pgn_result, "1/2-1/2");
      } else {
        if (xboard) {
          Print(4095, "tellics decline\n");
          Print(128, "Decline\n");
        } else
          Print(4095, "Draw declined.\n");
      }
    }
/*

Jim.

[hgm]

I guess this should have been:

Code: Select all

  if (draw_offer_pending) {
      int drawsc = abs_draw_score;

      draw_offer_pending = 0;
      if (move_number < 40 || !accept_draws)
        drawsc = -300;
      if (value <= drawsc && (tc_increment != 0 ||
              tc_time_remaining_opponent >= 1000)) {
        if (xboard) }
          Print(4095, "offer draw\n"); // should also relay the offer to ICS!
          Print(4095, "tellics draw\n"); // So not sure if this is needed.
        } else {
          Print(128, "Draw accepted.\n");
          Print(4095, "1/2-1/2 {Draw agreed}\n");
          strcpy(pgn_result, "1/2-1/2");
          if (audible_alarm)
            printf("%c", audible_alarm);
          if (speech) {
            char announce[128];

            strcpy(announce, SPEAK);
            strcat(announce, " Drawaccept");
            system(announce);
          }
        }
      } else {
        if (xboard) {
          Print(4095, "tellics decline\n");
          Print(128, "Decline\n");
        } else
          Print(4095, "Draw declined.\n");
      }
    }
/*

[bob]

In a game between Crafty-22.9 and Crafty-22.8 played under xboard 4.3.15 the engines apparently agreed to a draw, but the result was recorded as 0-1 in xboard:

Code: Select all

63... Re4+ {False draw claim: 'Draw agreed'} 0-1

Here's the end of the Crafty-22.9 (White) log of the game:

Code: Select all

Black(60): time 8470
time remaining:   1:24 (Crafty).
Black(60): otim 10971
time remaining:   1:49 (opponent).
Black(60): Rd4
              time used:   2.45
              time limit 1.92 (+0.00) (17.92)
              depth   time  score   variation (1)
starting thread 1
               16     1.05   0.30   Ra5 Rxf4 Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+
                                    Ke3 Ra4 Ra6+ Kf5 Ra8 Ra3+ Kd4 Ra4+
                                    Kc5 Ra2 a6 Rxg2 a7 Rc2+ Kd6
               16->   1.21   0.30   Ra5 Rxf4 Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+
                                    Ke3 Ra4 Ra6+ Kf5 Ra8 Ra3+ Kd4 Ra4+
                                    Kc5 Ra2 a6 Rxg2 a7 Rc2+ Kd6
               17     2.30   0.16   Ra5 Rxf4 Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+
                                    Ke1 Ra4 a6 Ke5 Ra8 Kf4 a7 Kg3 Ke2 Ra3
                                    <HT>
              time=2.64  mat=0  n=12085929  fh=90%  nps=4.6M
              ext-> check=518K qcheck=899K reduce=5.0M/1.3M
              predicted=19  evals=7.1M  50move=1  EGTBprobes=1K  hits=1K
              SMP->  splits=155  aborts=13  data=5/128  elap=2.64
terminating SMP processes.White(61): Ra5
              time used:   2.64
Black(61): time 8206
time remaining:   1:22 (Crafty).
Black(61): otim 10736
time remaining:   1:47 (opponent).
Black(61): Rxf4
              time used:   2.35
              time limit 1.86 (+0.00) (17.36)
              depth   time  score   variation (1)
starting thread 1
               16     1.38   0.07   Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+ Ke3 Ra4
                                    a6 Kf5 Ra8 Kg4 Kd2 Kg3 a7 Kxg2 Kd3
                                    g5 Ke3 <HT>
               16->   1.73   0.07   Rxa7+ Kf6 a5 Re4+ Kf2 Rf4+ Ke3 Ra4
                                    a6 Kf5 Ra8 Kg4 Kd2 Kg3 a7 Kxg2 Kd3
                                    g5 Ke3 <HT>
              time=4.16  mat=-1  n=18899191  fh=89%  nps=4.5M
              ext-> check=798K qcheck=1.5M reduce=7.5M/1.9M
              predicted=19  evals=11.1M  50move=0  EGTBprobes=2K  hits=2K
              SMP->  splits=175  aborts=19  data=6/128  elap=4.16
terminating SMP processes.White(62): Rxa7+
              time used:   4.16

       +---+---+---+---+---+---+---+---+
    8  |   | . |   | . |   | . |   | . |
       +---+---+---+---+---+---+---+---+
    7  |-R-|   | . |   | . |<K>| . |   |
       +---+---+---+---+---+---+---+---+
    6  |   | . |   | . |   | . |<P>| . |
       +---+---+---+---+---+---+---+---+
    5  | . |   | . |   | . |   | . |   |
       +---+---+---+---+---+---+---+---+
    4  |-P-| . |   | . |   |<R>|   |<P>|
       +---+---+---+---+---+---+---+---+
    3  | . |   | . |   | . |   | . |   |
       +---+---+---+---+---+---+---+---+
    2  |   | . |   | . |-K-| . |-P-| . |
       +---+---+---+---+---+---+---+---+
    1  | . |   | . |   | . |   | . |   |
       +---+---+---+---+---+---+---+---+
         a   b   c   d   e   f   g   h

Black(62): time 7790
time remaining:   1:17 (Crafty).
Black(62): otim 10507
time remaining:   1:45 (opponent).
Black(62): Kf6
              time used:   2.29
              time limit 1.77 (+0.00) (16.52)
              depth   time  score   variation (1)
starting thread 1
               14->   0.08   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
               15     0.08   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
               15->   0.18   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
               16     0.18   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
               16->   0.45   0.01   a5 Re4+ Kf2 Rf4+ Ke3 Ra4 a6 Kf5 Rf7+
                                    Ke6 Rg7 Kf6 Ra7 <HT>
              time=1.89  mat=0  n=8474115  fh=89%  nps=4.5M
              ext-> check=370K qcheck=663K reduce=3.4M/929K
              predicted=19  evals=4.9M  50move=1  EGTBprobes=1K  hits=1K
              SMP->  splits=206  aborts=14  data=6/128  elap=1.89
terminating SMP processes.White(63): a5
              time used:   1.89
Black(63): draw
Black(63): time 7602
time remaining:   1:16 (Crafty).
Black(63): otim 10271
time remaining:   1:42 (opponent).
Black(63): Re4+
              time used:   2.35
              time limit 1.72 (+0.00) (16.10)
              depth   time  score   variation (1)
starting thread 1
               15     0.48   0.01   Kf3 Ra4 Ke3 g5 a6 Ke5 Ra8 Ra3+ Kf2
                                    Kf4 Rf8+ Ke4 Re8+ Kf4 Rf8+ <HT>
               15->   0.48   0.01   Kf3 Ra4 Ke3 g5 a6 Ke5 Ra8 Ra3+ Kf2
                                    Kf4 Rf8+ Ke4 Re8+ Kf4 Rf8+ <HT>
              time=1.78  mat=0  n=8265014  fh=89%  nps=4.6M
              ext-> check=352K qcheck=706K reduce=3.2M/847K
              predicted=19  evals=4.8M  50move=1  EGTBprobes=1K  hits=1K
              SMP->  splits=228  aborts=24  data=5/128  elap=1.78
terminating SMP processes.tellics draw
1/2-1/2 {Draw agreed}
White(64): Kf3
              time used:   1.78
Black(64): result 0-1 {False draw claim: 'Draw agreed'}
LearnBook() executed
Black(64): force
Black(64): ping 2
pong 2

Not sure who's "bug" this is, but I don't recall it occurring in older versions of Xboard.

This is probably something that needs cleaning up. I need to look carefully at current xboard to see whether the tellics stuff is still needed or not. For example, if I send xboard "offer draw" will it (a) relay it to the other engine if this is an engine-engine match? (b) relay it to ICC if I am playing an opponent there? (c) are all versions of xboard going to be compatible in this regard so that one way of doing things will work on old and new versions since there are tons of linux distributions that include both Crafty and xboard...

OK, I had two places where "tellics" was used. I have eliminated both and am going to play some games to see what happens. One case was where Crafty offered a draw, the other was where it resigns. If both work on ICC without the "tellics" then I'll release a 22.10 that fixes this...

[hgm]

This is probably something that needs cleaning up. I need to look carefully at current xboard to see whether the tellics stuff is still needed or not. For example, if I send xboard "offer draw" will it (a) relay it to the other engine if this is an engine-engine match? (b) relay it to ICC if I am playing an opponent there? (c) are all versions of xboard going to be compatible in this regard so that one way of doing things will work on old and new versions since there are tons of linux distributions that include both Crafty and xboard...

I think this is what "offer draw" specifies in WB protocol. Tim Mann's last version (4.2.7) certainly relayed it to the engine in Two-Machines Mode, and to the ICS in ICS Mode. In 4.2.7 XBoard relayed in, and in Two-Machines Mode set a flag for 2 moves to indicate a draw offer was pending. And a draw offer when the opponent had one pending resulted in an immediate draw. In ICS mode the offer is merely relayed, and the ICS is responsible for ending the game in reaction to it.

The only thing I changed there in 4.3.14 was that a draw offer in Two-Machines mode is not relayed immediately to the opponent engine, but is delayed to be handed tot he opponent together with the move of the engine making the offer (just before it), so that it is not possible to disrupt the opponent's pondering or thinking by bombarding him with draw offers. Basically you have to make the offer while pressing the clock.

OK, I had two places where "tellics" was used. I have eliminated both and am going to play some games to see what happens. One case was where Crafty offered a draw, the other was where it resigns. If both work on ICC without the "tellics" then I'll release a 22.10 that fixes this...

I have been careful not to change anything in ICS mode (as initially I did not understand the protocol for that at all), but what I understand from the code by now is that resigns and draw claims, which in Two-Machines Mode would make XBoard terminate the game, are relayed to the ICS in ICS mode without further action, while win claims are simply ignored in ICS Mode. This was true in 4.2.7, I have not looked at the code for any earlier versions. But as WB protocol on Tim's website specifies that things should be done this way, I suspect that this has been so for already quite a long time.

[bob]

This is probably something that needs cleaning up. I need to look carefully at current xboard to see whether the tellics stuff is still needed or not. For example, if I send xboard "offer draw" will it (a) relay it to the other engine if this is an engine-engine match? (b) relay it to ICC if I am playing an opponent there? (c) are all versions of xboard going to be compatible in this regard so that one way of doing things will work on old and new versions since there are tons of linux distributions that include both Crafty and xboard...

I think this is what "offer draw" specifies in WB protocol. Tim Mann's last version (4.2.7) certainly relayed it to the engine in Two-Machines Mode, and to the ICS in ICS Mode. In 4.2.7 XBoard relayed in, and in Two-Machines Mode set a flag for 2 moves to indicate a draw offer was pending. And a draw offer when the opponent had one pending resulted in an immediate draw. In ICS mode the offer is merely relayed, and the ICS is responsible for ending the game in reaction to it.

The only thing I changed there in 4.3.14 was that a draw offer in Two-Machines mode is not relayed immediately to the opponent engine, but is delayed to be handed tot he opponent together with the move of the engine making the offer (just before it), so that it is not possible to disrupt the opponent's pondering or thinking by bombarding him with draw offers. Basically you have to make the offer while pressing the clock.

OK, I had two places where "tellics" was used. I have eliminated both and am going to play some games to see what happens. One case was where Crafty offered a draw, the other was where it resigns. If both work on ICC without the "tellics" then I'll release a 22.10 that fixes this...

I have been careful not to change anything in ICS mode (as initially I did not understand the protocol for that at all), but what I understand from the code by now is that resigns and draw claims, which in Two-Machines Mode would make XBoard terminate the game, are relayed to the ICS in ICS mode without further action, while win claims are simply ignored in ICS Mode. This was true in 4.2.7, I have not looked at the code for any earlier versions. But as WB protocol on Tim's website specifies that things should be done this way, I suspect that this has been so for already quite a long time.

I've fixed this and tested for 15 games or so on ICC. Only thing I have not yet had happen is for my opponent to offer me a draw that Crafty accepts.

Can you post a link for your latest? I can run it on ICS to see if I have any sort of problems...

[hgm]

Latest stable WinBoard / XBoard version is hosted on WinBoard forum:

http://www.open-aurec.com/wbforum/viewt ... 19&t=49439

The actual download links are in that post; for Linux you would have to install it from source, and thus pick the source download. The direct link for this source is:

http://www.open-aurec.com/wbforum/WinBo ... 4.3.15.zip

(At the time of this writing I notice that none of the download links there currently works, and probably got mutilated in the recent upgrade of that forum to phpBB3. I notivied the admins and hope they correct that soon. Otherwise I will organize a mirror.)