Crafty problem with cutechess-cli

bob · Post by **bob** » Sat Jul 11, 2009 8:28 pm

Here is what I see when I put log=off in the .craftyrc:

scrappy% rm log.* game.*
rm: No match.
scrappy% crafty
found computer opening book file [./bookc.bin].
hash table memory =  192M bytes.
pawn hash table memory =   32M bytes.
show book statistics
pondering disabled.
noise level set to 0.


Crafty v23.1 (1 cpus)

White(1): e4
              time used:   1.09
  after screening, the following moves can be played
  move     played    %  score    learn    sortv   P%  P
    e5!!    18893   25  -0.05     0.07   13573.1   0  Y
    c5      38389   52   0.30     0.00    5011.0   0  Y
    e6       7187    9  -0.02     0.00    4262.2   0  Y
    c6       4527    6   0.35     0.00    4118.9   0  Y
   Nf6?       888    1  -0.35     0.00     164.1   0  N
   Nc6?       308    0  -0.35     0.00     149.0   0  N
    d6?      1560    2  -0.02     0.00     115.6   0  N
    d5?       898    1  -0.05     0.00     104.4   0  N
   Na6?         5    0  -0.11     0.00      93.1   0  N
    g6?       697    0   0.33     0.00      23.2   0  N
    a6?         7    0   0.29     0.00      13.2   0  N
    g5?        22    0   0.31     0.00       9.6   0  N
    b6?        31    0   0.33     0.00       5.8   0  N
               book moves {e5}
               moves considered {e5}
  move     played    %  score     sortv  P%  P
    e5!!    18893   25  -0.05   13573.0   0  Y
               book   0.0s     25%    e5!! Nf3
Black(1): e5
              time used:   0.01
note: scores are for the white side
                        +-----------white----------+-----------black----------+
material.......   0.05  |    comp     mg      eg   |    comp     mg      eg   |
pawns..........   0.00  |    0.29    0.29    0.57  |   -0.29   -0.29   -0.57  |
passed pawns...   0.00  |    0.00    0.00    0.00  |    0.00    0.00    0.00  |
knights........   0.00  |   -0.60   -0.60   -0.60  |    0.60    0.60    0.60  |
bishops........   0.00  |    0.43    0.43    0.79  |   -0.43   -0.43   -0.79  |
rooks..........   0.00  |   -0.54   -0.54   -0.30  |    0.54    0.54    0.30  |
queens.........   0.00  |    0.00    0.00    0.00  |    0.00    0.00    0.00  |
kings..........   0.00  |   -0.45   -0.45   -0.40  |    0.45    0.45    0.40  |
development....   0.00  |   -0.84   -0.84    0.00  |    0.84    0.84    0.00  |
pawn races.....   0.00  +--------------------------+--------------------------+
total..........   0.05
White(2): book off
book file disabled.
White(2): Nf3
              time used:   6.54
              time surplus  29.99  time limit 22.87 (+0.00) (3:33)
              depth   time  score   variation (1)
                1     0.00   0.05   2. ... Nc6     (13Knps)             
                1->   0.00   0.05   2. ... Nc6     (15Knps)             
                2     0.00     +1   2. ... Nc6?                         
                2     0.00   0.57   2. ... Nc6 3. Nc37Knps)             
                2->   0.00   0.57   2. ... Nc6 3. Nc35Knps)             
                3     0.00     -1   2. ... Nc6!                         
                3     0.01   0.05   2. ... Nc6 3. Nc3 Nf6s)             
                3->   0.01   0.05   2. ... Nc6 3. Nc3 Nf6s)             
                4     0.01     +1   2. ... Nc6?                         
                4     0.01   0.51   2. ... Nc6 3. Nc3 Nf6 4. Bc4        
                4->   0.01   0.51   2. ... Nc6 3. Nc3 Nf6 4. Bc4         
                5     0.02   0.20   2. ... Nc6 3. Bd3 Nf6 4. O-O Bc5     
                5->   0.02   0.20   2. ... Nc6 3. Bd3 Nf6 4. O-O Bc5     
                6     0.02     +1   2. ... Nc6?                          
                6     0.03   0.72   2. ... Nc6 3. Bd3 Nf6 4. O-O Bc5 5.  
                                    Nc3
                6     0.04   0.57   2. ... Nf6 3. Bd3 Bd6 4. Nc3 O-O 5.  
                                    O-O
                6->   0.04   0.57   2. ... Nf6 3. Bd3 Bd6 4. Nc3 O-O 5.  
                                    O-O
                7     0.05   0.28   2. ... Nf6 3. Nc3 Bd6 4. Bc4 O-O 5.  
                                    O-O Nc6
                7     0.06     -1   2. ... Nc6!                          
                7     0.06   0.62   2. ... Nc6 3. Nc3 Nf6 4. Bb5 Bc5 5.  
                                    Bxc6 dxc6 6. Nxe5
                7     0.07   0.28   2. ... Bd6 3. Nc3 Nf6 4. Bc4 O-O 5.  
                                    O-O Nc6 <HT>
                7->   0.08   0.28   2. ... Bd6 3. Nc3 Nf6 4. Bc4 O-O 5.  
                                    O-O Nc6 <HT>
                8     0.08   0.52   2. ... Bd6 3. Nc3 Nf6 4. Bc4 O-O 5.  
                                    O-O Nc6 6. d4 exd4 7. Nxd4
                8     0.11   0.09   2. ... Nf6 3. Nxe5 d6 4. Nc4 Nxe4 5. 
                                    Nc3 Qh4 6. Nxe4 Qxe4+ 7. Qe2
                8->   0.12   0.09   2. ... Nf6 3. Nxe5 d6 4. Nc4 Nxe4 5. 
                                    Nc3 Qh4 6. Nxe4 Qxe4+ 7. Qe2
                9     0.18   0.14   2. ... Nf6 3. Nc3 Nc6 4. Bb5 Bb4 5.  
                                    d3 O-O 6. Bxc6 dxc6 7. Nxe5
                9->   0.22   0.14   2. ... Nf6 3. Nc3 Nc6 4. Bb5 Bb4 5.  
                                    d3 O-O 6. Bxc6 dxc6 7. Nxe5
               10     0.40   0.14   2. ... Nf6 3. Nc3 Nc6 4. Bb5 Bb4 5.  
                                    d3 O-O 6. Bxc6 dxc6 7. Nxe5
               10->   0.42   0.14   2. ... Nf6 3. Nc3 Nc6 4. Bb5 Bb4 5.  
                                    d3 O-O 6. Bxc6 dxc6 7. Nxe5
               11     0.53   0.31   2. ... Nf6 3. Nc3 Nc6 4. Bc4 Bd6 5.  
                                    O-O O-O 6. d3 Na5 7. Bb5 Nc6
               11->   0.67   0.31   2. ... Nf6 3. Nc3 Nc6 4. Bc4 Bd6 5.  
                                    O-O O-O 6. d3 Na5 7. Bb5 Nc6
?              12     0.67   1/29*  2. ... Nf6     (1.5Mnps)             
               12     1.41   0.18   2. ... Nf6 3. Nc3 Nc6 4. Bc4 Bc5 5.
                                    O-O O-O 6. d3 d6 7. Bg5 Be6 8. Nd5
               12->   1.52   0.18   2. ... Nf6 3. Nc3 Nc6 4. Bc4 Bc5 5.  
                                    O-O O-O 6. d3 d6 7. Bg5 Be6 8. Nd5
Black(1): ?    13     1.52   1/29*  2. ... Nf6     (1.5Mnps)             
              time=1.97  mat=0  n=3082396  fh=89%  nps=1.6M
              extensions=53K qchecks=82K reduced=291K pruned=905K
              predicted=1  evals=1.9M  50move=1  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=1.97
Black(2): Nf6
              time used:   1.97
note: scores are for the white side
                        +-----------white----------+-----------black----------+
material.......   0.05  |    comp     mg      eg   |    comp     mg      eg   |
pawns..........   0.00  |    0.29    0.29    0.57  |   -0.29   -0.29   -0.57  |
passed pawns...   0.00  |    0.00    0.00    0.00  |    0.00    0.00    0.00  |
knights........   0.00  |   -0.16   -0.16   -0.16  |    0.16    0.16    0.16  |
bishops........   0.00  |    0.43    0.43    0.79  |   -0.43   -0.43   -0.79  |
rooks..........   0.00  |   -0.48   -0.48   -0.27  |    0.48    0.48    0.27  |
queens.........   0.00  |    0.00    0.00    0.00  |    0.00    0.00    0.00  |
kings..........   0.00  |   -0.45   -0.45   -0.40  |    0.45    0.45    0.40  |
development....   0.00  |   -0.72   -0.72    0.00  |    0.72    0.72    0.00  |
pawn races.....   0.00  +--------------------------+--------------------------+
total..........   0.05
White(3): end
scrappy% ls log.* game.*
ls: No match.
scrappy%

No log files of any kind, assuming none were present prior to the start of the game. It will not remove old log files, but will not create new ones. Here is a game where I used log=off typed in as a command:

Code: Select all

scrappy% ls log.*
ls: No match.
scrappy% crafty
found computer opening book file [./bookc.bin].
hash table memory =  192M bytes.
pawn hash table memory =   32M bytes.
show book statistics
pondering disabled.
noise level set to 0.


Crafty v23.1 (1 cpus)

White(1): log=off
White(1): book off
book file disabled.
White(1): st=1
search time set to 1.00.
White(1): e4
              time used:  14.39
              time surplus   0.00  time limit 1.00 (+0.00) (1.00)
              depth   time  score   variation (1)
                1     0.00  -0.25   1. ... Nc6     (5Knps)             
                1->   0.00  -0.25   1. ... Nc6     (7Knps)             
                2     0.00     +1   1. ... Nc6?                        
                2     0.00   0.27   1. ... Nc6 2. Nf3Knps)             
                2->   0.01   0.27   1. ... Nc6 2. Nf34Knps)             
                3     0.01     -1   1. ... Nc6!                         
                3     0.01  -0.27   1. ... Nc6 2. Nc3 Nf6s)             
                3->   0.01  -0.27   1. ... Nc6 2. Nc3 Nf6s)             
                4     0.01     +1   1. ... Nc6?                         
                4     0.01   0.25   1. ... Nc6 2. Nc3 Nf6 3. Nf3        
                4     0.01   0.13   1. ... Nf6 2. Nc3 d5 3. Bd3         
                4->   0.01   0.13   1. ... Nf6 2. Nc3 d5 3. Bd3          
                5     0.01   0.15   1. ... Nf6 2. e5 Ne4 3. Nc3 d5 4. Nxe4
                                    dxe4
                5     0.01   0.04   1. ... Nc6 2. Nc3 Nf6 3. Nf3 d6      
                5->   0.01   0.04   1. ... Nc6 2. Nc3 Nf6 3. Nf3 d6      
                6     0.01   0.32   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bc4
                6     0.02   0.27   1. ... Nf6 2. Nc3 d5 3. exd5 Nxd5 4. 
                                    Nf3 Nxc3 5. bxc3
                6->   0.02   0.27   1. ... Nf6 2. Nc3 d5 3. exd5 Nxd5 4. 
                                    Nf3 Nxc3 5. bxc3
                7     0.02   0.15   1. ... Nf6 2. Nc3 d5 3. e5 Ne4 4. Nxe4
                                    dxe4
                7     0.03   0.03   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    d4 Bd6
                7->   0.03   0.03   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    d4 Bd6
                8     0.03   0.14   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    d4 d5 5. Qd3 dxe4 6. Nxe4
                8->   0.04   0.14   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    d4 d5 5. Qd3 dxe4 6. Nxe4
                9     0.04   0.13   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bc4 Bb4 5. O-O O-O
                9->   0.05   0.13   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bc4 Bb4 5. O-O O-O
               10     0.07   0.13   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bb5 Bb4 5. O-O Bxc3 6. dxc3 Nxe4
               10->   0.11   0.13   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bb5 Bb4 5. O-O Bxc3 6. dxc3 Nxe4
               11     0.20   0.13   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bb5 Bb4 5. O-O Bxc3 6. dxc3 Nxe4
               11->   0.22   0.13   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bb5 Bb4 5. O-O Bxc3 6. dxc3 Nxe4
               12     0.33   0.16   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Qe2
                                    Bxc3 8. bxc3 dxe4 9. Bxc6 bxc6 10.
                                    dxe4
               12->   0.44   0.16   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Qe2
                                    Bxc3 8. bxc3 dxe4 9. Bxc6 bxc6 10.
                                    dxe4
               13     0.75   0.13   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bb5 Bb4 5. O-O O-O 6. d3 d5 7. e5 Ng4
               13->   0.95   0.13   1. ... Nc6 2. Nc3 Nf6 3. Nf3 e6 4.   
                                    Bb5 Bb4 5. O-O O-O 6. d3 d5 7. e5 Ng4
              time=1.04  mat=0  n=1500803  fh=88%  nps=1.4M)             
              extensions=19K qchecks=29K reduced=183K pruned=370K
              predicted=0  evals=984K  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=1.04
Black(1): Nc6
              time used:   1.04
note: scores are for the white side
                        +-----------white----------+-----------black----------+
material.......   0.05  |    comp     mg      eg   |    comp     mg      eg   |
pawns..........   0.21  |    0.29    0.29    0.57  |   -0.08   -0.08   -0.40  |
passed pawns...   0.00  |    0.00    0.00    0.00  |    0.00    0.00    0.00  |
knights........  -0.44  |   -0.60   -0.60   -0.60  |    0.16    0.16    0.16  |
bishops........   0.11  |    0.43    0.43    0.79  |   -0.32   -0.32   -0.68  |
rooks..........  -0.06  |   -0.54   -0.54   -0.30  |    0.48    0.48    0.27  |
queens.........   0.00  |    0.00    0.00    0.00  |    0.00    0.00    0.00  |
kings..........   0.00  |   -0.45   -0.45   -0.40  |    0.45    0.45    0.40  |
development....  -0.12  |   -0.84   -0.84    0.00  |    0.72    0.72    0.00  |
pawn races.....   0.00  +--------------------------+--------------------------+
total..........  -0.25
White(2): end
scrappy% ls log.*
ls: No match.
scrappy%

I run with log=off on my cluster to avoid creating a million log files over a weekend. And I tested this with xboard putting log=off in the .craftyrc and there it also worked just fine, as it has for as long as I can remember.

However, the problem does show up if you run crafty in a way I have always recommended avoiding. I always run with the "-xreuse" option so that the engine is completely stopped and restarted for each game. Trying to carry the executable across more than one game is a bit of a stretch as there is a lot of code that needs to get reset back to starting values, and the best way to make sure everything gets set is to restart which goes back to the original values stored on disk.

The solution is to tell the gui to execute a new instance of Crafty for each game being played. It is likely it will play better since there are no values that do not get reset and therefore start off at odd points. And you will get no log files either.

Gian-Carlo Pascutto · Sun Jul 12, 2009 12:25 pm

Unfortunately there seems to be no way to tell cutechess-cli that the engine doesn't support playing more than 1 game.

bob · Post by **bob** » Sun Jul 12, 2009 7:25 pm

Gian-Carlo Pascutto wrote:Unfortunately there seems to be no way to tell cutechess-cli that the engine doesn't support playing more than 1 game.

Sounds like an incorrect implementation of the winboard protocol which has always supported the -xreuse, and even allows the program to tell the GUI about this in protocol version 2.

hgm · Post by **hgm** » Sun Jul 12, 2009 9:51 pm

This is not really a WB protocol matter, but pertains to the GUI implementation. -xreuse is a command-line option of XBoard, not a command sent to the engine. Considering the purpose of cutechess-cli (ultra-fast games) it is not surprising that it is not interested in offering this service.

An engine that implements all commands of WB protocol should be able to play many games in a row, though. An engine that can't correctly do this cannot be said to be fully WB compliant.

Of course WB protocol specs don't say anything about making of log files. (This would be a nice extension of the protocol, btw...) So no matter how erratic and illogical engines behave with respect to log writing, this cannot be interpreted as them being non-compliant. Just user-unfriendly and poorly implemented.

bob · Post by **bob** » Mon Jul 13, 2009 6:50 am

hgm wrote:This is not really a WB protocol matter, but pertains to the GUI implementation. -xreuse is a command-line option of XBoard, not a command sent to the engine. Considering the purpose of cutechess-cli (ultra-fast games) it is not surprising that it is not interested in offering this service.

An engine that implements all commands of WB protocol should be able to play many games in a row, though. An engine that can't correctly do this cannot be said to be fully WB compliant.

Of course WB protocol specs don't say anything about making of log files. (This would be a nice extension of the protocol, btw...) So no matter how erratic and illogical engines behave with respect to log writing, this cannot be interpreted as them being non-compliant. Just user-unfriendly and poorly implemented.

I was talking about the wb2 protocol "reuse=0/1" an engine can send, which is supposed to implement this very feature. reuse=0 is supposed to say "restart the engine after each game" while "reuse=1" (feature reuse=1) is supposed to say "new" will start a new game without restarting the engine."

Matthias Gemuh · Post by **Matthias Gemuh** » Mon Jul 13, 2009 9:39 am

bob wrote:
Gian-Carlo Pascutto wrote:Unfortunately there seems to be no way to tell cutechess-cli that the engine doesn't support playing more than 1 game.
Sounds like an incorrect implementation of the winboard protocol which has always supported the -xreuse, and even allows the program to tell the GUI about this in protocol version 2.

ChessGUI does not allow reuse.
Every engine is killed after every game.

Matthias.

hgm · Post by **hgm** » Mon Jul 13, 2009 4:22 pm

bob wrote:I was talking about the wb2 protocol "reuse=0/1" an engine can send, which is supposed to implement this very feature. reuse=0 is supposed to say "restart the engine after each game" while "reuse=1" (feature reuse=1) is supposed to say "new" will start a new game without restarting the engine."

Ah, that is a more tricky point. I guess protocol v2 does allow the GUI to reply rejected reuse. So it is difficult to say if the engine or the GUI is at fault here.

bob · Post by **bob** » Mon Jul 13, 2009 8:52 pm

Matthias Gemuh wrote:
bob wrote:
Gian-Carlo Pascutto wrote:Unfortunately there seems to be no way to tell cutechess-cli that the engine doesn't support playing more than 1 game.
Sounds like an incorrect implementation of the winboard protocol which has always supported the -xreuse, and even allows the program to tell the GUI about this in protocol version 2.
ChessGUI does not allow reuse.
Every engine is killed after every game.

Matthias.

I personally think that is the way it should be done. Many like the reuse because the 5-6 piece EGTB initialization can take way long on slow disk drives. I'd prefer to waste the initialization time (or even better, play without the tables entirely) as opposed to possibly having unexpected bugs due to lack of initializing something critical.

hgm · Post by **hgm** » Mon Jul 13, 2009 9:20 pm

That is a bit like a double-edged sword. Because you always do it this way you will never become aware of such bugs, and thus never correct them. You convict yourself to a life sentence of inefficient running out of fear for bugs that might not even exist, or would be trivially corrected when they do...

michiguel · Post by **michiguel** » Tue Jul 14, 2009 12:54 am

hgm wrote:
bob wrote:I was talking about the wb2 protocol "reuse=0/1" an engine can send, which is supposed to implement this very feature. reuse=0 is supposed to say "restart the engine after each game" while "reuse=1" (feature reuse=1) is supposed to say "new" will start a new game without restarting the engine."
Ah, that is a more tricky point. I guess protocol v2 does allow the GUI to reply rejected reuse. So it is difficult to say if the engine or the GUI is at fault here.

The GUI.

If the engine is allowed to have a choice, the GUI should honor it.

Miguel

Crafty problem with cutechess-cli

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"

Re: Crafty ignores "log off"