Comparison data of Crafty vs Rybka that I promised 2 days ag

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
vb4
Posts: 154
Joined: Sat Mar 11, 2006 4:45 am
Location: NY

Comparison data of Crafty vs Rybka that I promised 2 days ag

Post by vb4 » Fri Jun 01, 2007 3:12 am

Code: Select all

The following EPD was the position I started with.

4r1k1/p1qr1p2/2pb1Bp1/1p3Q1p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;

Crafty-2014   and Rybka v1.2.w32    Analysis time was set to 1 minute and Dann Corbit ran these on the same Hardware. (Thx Dann)

What I did was seeing if there were any differences between these 2 engines by not only looking at reflected positions but also all legal slider positions.  Since this position is a guaranteed mate position the slider positions are all legal derivations from the original EPD string.  As I mnetiones in a post here several days ago a slider is placing the piece to move on any square that still allows the piece to move to hit the target square.  The thinking behind this is that all positions should be close to the same across the 4 way rotations for each of the slider positions.  The table below appears to show Rybka much better across the reflections compared to Crafty.  And for almost all the 11 slider positions holds true.  Perhaps Crafty is having a bit of trouble depending as to how Bobs engine works.  I am no expert when it comes to chess engines but I thought Uri especially and others might be interested in this.  In case the table is not clear the 4 way rotations are represented across the columns 1-4 and for this particular position there are 11 slider positions for each of the reflected boards.

Reflections 1 and 2 was WTM and 3 and 4 was BTM

                        Crafty                                 Rybka
                 1     2     3     4                    1     2     3     4
Slider
1               54    23    22    57                    7     7     7     7
2                2     3     3     3                    2     2     2     2
3                0     0     0     0                    0     0     0     0
4               11    13    11    13                    4     4     4     4
5               22    29    40    21                    4     4     4     4
6               45    51    41    16                    8     8     8     8
7               56    16    39     8                    4     4     4     4
8               56    43    10     8                    8     8     6     6
9               44    12    48     9                   10     6    10     9
10              13    37    49    22                    5     5     5     5
11              15    44    28    12                    8     8     4     4

Notice how much less deviation there is across the reflections in almost all the slider positions comparing Rybka to Crafty!  Can someone perhaps provide me with some explanation as to what could be causing this?  Bob may be interested in this data to improving this part of Crafty.

HTH

Les

Dann Corbit
Posts: 9827
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Comparison data of Crafty vs Rybka that I promised 2 day

Post by Dann Corbit » Fri Jun 01, 2007 3:14 am

The elements of the grid are seconds

vb4
Posts: 154
Joined: Sat Mar 11, 2006 4:45 am
Location: NY

Movei added to Crafty vs Rybka comaprison data

Post by vb4 » Fri Jun 01, 2007 4:06 am

Dann Corbit has just finished running the data set we used for the Crafty vs Rybka data using Uri Movei. I have combiend all 3 comparisons for your review. Again all data in the tables represents seconds.
I will post a New post for the 44 positions that were used to generated all this data for any of you that are interested in it.

Code: Select all

The following EPD was the position I started with.

4r1k1/p1qr1p2/2pb1Bp1/1p3Q1p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;

Crafty-2014   and Rybka v1.2.w32  and Movei  Analysis time was set to 1 minute and Dann Corbit ran these on the same Hardware. (Thx Dann)

What I did was seeing if there were any differences between these 2 engines by not only looking at reflected positions but also all legal slider positions.  Since this position is a guaranteed mate position the slider positions are all legal derivations from the original EPD string.  As I mnetiones in a post here several days ago a slider is placing the piece to move on any square that still allows the piece to move to hit the target square.  The thinking behind this is that all positions should be close to the same across the 4 way rotations for each of the slider positions.  The table below appears to show Rybka much better across the reflections compared to Crafty.  And for almost all the 11 slider positions holds true.  Perhaps Crafty is having a bit of trouble depending as to how Bobs engine works.  I am no expert when it comes to chess engines but I thought Uri especially and others might be interested in this.  In case the table is not clear the 4 way rotations are represented across the columns 1-4 and for this particular position there are 11 slider positions for each of the reflected boards.  All data in all tables represent seconds to solution.

Reflections 1 and 2 was WTM and 3 and 4 was BTM

              Crafty            Rybka                Movei
          1   2   3   4      1   2   3   4       1   2   3   4
Slider
1        54  23  22  57      7   7   7   7      19  17  19  21
2         2   3   3   3      2   2   2   2       1   1   1   1
3         0   0   0   0      0   0   0   0       0   0   0   0
4        11  13  11  13      4   4   4   4       3   3   4   3
5        22  29  40  21      4   4   4   4      16  15  15  16
6        45  51  41  16      8   8   8   8       3   3   3   3
7        56  16  39   8      4   4   4   4       3   3  12   2
8        56  43  10   8      8   8   6   6       9   8   7   8
9        44  12  48   9     10   6  10   9       3   3  15   3
10       13  37  49  22      5   5   5   5       5   5   6   6
11       15  44  28  12      8   8   4   4      10  11   2   3

Notice how much less deviation there is across the reflections in almost all the slider positions comparing Rybka to Crafty!  Can someone perhaps provide me with some explanation as to what could be causing this?  Bob may be interested in this data to improving this part of Crafty.

HTH

Les

bob
Posts: 20390
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Movei added to Crafty vs Rybka comaprison data

Post by bob » Sat Jun 02, 2007 1:41 am

I am not sure I understand what you are doing. One common "sanity test" I do in crafty is to feed it several thousand FEN positions, and have it evaluate each one 4 times. First, as the position is entered, then I flip the board vertically and swap the colors of all pieces, to give me a second position. I then flip the original position horizontally which just swaps the a file with the h-file and so forth, for a third position. And I take the second position (the vertically flipped position) and also flip it horizontally for a 4th position. Each position must produce the _same_ evaluation except that the sign must change when I invert white and black pieces. Any single failure causes a massive bug-hunt until it goes away...

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Movei added to Crafty vs Rybka comaprison data

Post by sje » Sat Jun 02, 2007 2:04 am

bob wrote:One common "sanity test" I do in crafty is to feed it several thousand FEN positions, and have it evaluate each one 4 times. First, as the position is entered, then I flip the board vertically and swap the colors of all pieces, to give me a second position. I then flip the original position horizontally which just swaps the a file with the h-file and so forth, for a third position. And I take the second position (the vertically flipped position) and also flip it horizontally for a 4th position. Each position must produce the _same_ evaluation except that the sign must change when I invert white and black pieces. Any single failure causes a massive bug-hunt until it goes away...
Apparently the test set doesn't include positions with non empty castling availability as I don't see how these would survive a reflection about the Y axis.

--------

Symbolic has a routine that reflects the board about the X axis and swaps the colors. During some testing modes, this is connected to the move path enumerator (perft) along with calls to the position evaluator before and after the exchange. This enables testing of huge numbers of positions in an orderly fashion. The enumerator has similar hooks for other complex or tricky routines, like the fast check marker and the checks only generator.

Dann Corbit
Posts: 9827
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Movei added to Crafty vs Rybka comaprison data

Post by Dann Corbit » Sat Jun 02, 2007 4:00 am

The involved both search and evaluation.
Each row (we would expect) should have about the same solution time, and none of the rows should be worse than the first.

These are the actual positions referenced in the test. For any solutions found, none should be worse than 32756 (though there are closer mates in a few instances).

This is a test set that is automatically generated from a tool Les made using WAC.141.

Code: Select all

4r1k1/p1qr1p2/2pb1Bp1/1p3Q1p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p4Qp/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1BpQ/1p5p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1nQR/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3PQ1/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PKQ/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B2QP2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP1Q1PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PK1/2Q4R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3PQn1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p2Q2p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p1Q3p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/pQ4p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/QpB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/RQn1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/1QP3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/QKP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2PQ2B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP1Q1PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R4Q2 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1nQP3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p2Q2p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P3q1P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P4qP/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bPq/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1Nqr/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3pq1/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pkq/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b2qp2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp1q1pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
2q4r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3pqN1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P2q2P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P1q3P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/Pq4P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P5P1/qPb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/rqN1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/1qp3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/qkp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2pq2b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp1q1pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r4q2/1kp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1Nqp3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P2q2P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;

Dann Corbit
Posts: 9827
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Movei added to Crafty vs Rybka comaprison data

Post by Dann Corbit » Sat Jun 02, 2007 4:48 am

These are the positions for slider 9:
[D]4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PK1/2Q4R w - - pv Qxf4; ce 32756;
[D]1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R4Q2 w - - pv Qxc4; ce 32756;
[D]2q4r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
[D]r4q2/1kp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;

As you can see, they are all exactly the same position, but with roles reverse or the board reflected.

bob
Posts: 20390
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Movei added to Crafty vs Rybka comaprison data

Post by bob » Sat Jun 02, 2007 5:56 am

sje wrote:
bob wrote:One common "sanity test" I do in crafty is to feed it several thousand FEN positions, and have it evaluate each one 4 times. First, as the position is entered, then I flip the board vertically and swap the colors of all pieces, to give me a second position. I then flip the original position horizontally which just swaps the a file with the h-file and so forth, for a third position. And I take the second position (the vertically flipped position) and also flip it horizontally for a 4th position. Each position must produce the _same_ evaluation except that the sign must change when I invert white and black pieces. Any single failure causes a massive bug-hunt until it goes away...
Apparently the test set doesn't include positions with non empty castling availability as I don't see how these would survive a reflection about the Y axis.

--------

Symbolic has a routine that reflects the board about the X axis and swaps the colors. During some testing modes, this is connected to the move path enumerator (perft) along with calls to the position evaluator before and after the exchange. This enables testing of huge numbers of positions in an orderly fashion. The enumerator has similar hooks for other complex or tricky routines, like the fast check marker and the checks only generator.
ok. I assumed that much was obvious, in that you can't mirror (left-to-right) and preserve castling status. That would certainly change the scoring when in one case a side could castle and in another case it could not. Crafty explicitly scores this as having lost the right to castle without castling, which drops the king safety score significantly with the king stuck in the center after mirroring. My test positions don't have castling as an opportunity.

bob
Posts: 20390
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Movei added to Crafty vs Rybka comaprison data

Post by bob » Sat Jun 02, 2007 5:58 am

Dann Corbit wrote:The involved both search and evaluation.
Each row (we would expect) should have about the same solution time, and none of the rows should be worse than the first.

These are the actual positions referenced in the test. For any solutions found, none should be worse than 32756 (though there are closer mates in a few instances).

This is a test set that is automatically generated from a tool Les made using WAC.141.

Code: Select all

4r1k1/p1qr1p2/2pb1Bp1/1p3Q1p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p4Qp/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1BpQ/1p5p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1nQR/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3PQ1/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PKQ/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B2QP2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP1Q1PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PK1/2Q4R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3PQn1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p2Q2p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p1Q3p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/pQ4p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/QpB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/RQn1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/1QP3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/QKP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2PQ2B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP1Q1PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R4Q2 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1nQP3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p2Q2p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P3q1P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P4qP/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bPq/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1Nqr/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3pq1/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pkq/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b2qp2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp1q1pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
2q4r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3pqN1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P2q2P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P1q3P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/Pq4P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P5P1/qPb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/rqN1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/1qp3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/qkp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2pq2b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp1q1pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r4q2/1kp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1Nqp3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P2q2P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
Actually that assumption is wrong. When you mirror or reflect the board, then move ordering will change since crafty uses bitboards and FirstOne() will produce different results for white or black. So you could get a different PV. I would think the score could also change since now the hashing signatures change which will also affect the tree shape. And then there's the castling issue where reflected positions (swapping white and black pieces) keeps castling status, but mirroring (exchanging a with h, etc) can not.

bob
Posts: 20390
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Movei added to Crafty vs Rybka comaprison data

Post by bob » Sat Jun 02, 2007 6:00 am

yes. In fact, crafty has commands "flip" and "flop" to do that to the current position. "evtest" takes a FEN suite and does the sanity test using those commands.

I ran all 4. all four reported a mate in 6.

the scores were +2.18 and -2.18, + when it was black to move, - when it was white to move.

So I am still not sure what this is all about since all 4 report a forced mate (the times vary because of the move ordering issue I mentioned). And the eval is correct for each position...

so where does his table of numbers come from and what do the numbers represent???

Post Reply