Page 1 of 3
Comparison data of Crafty vs Rybka that I promised 2 days ag
Posted: Fri Jun 01, 2007 5:12 am
by vb4
Code: Select all
The following EPD was the position I started with.
4r1k1/p1qr1p2/2pb1Bp1/1p3Q1p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
Crafty-2014 and Rybka v1.2.w32 Analysis time was set to 1 minute and Dann Corbit ran these on the same Hardware. (Thx Dann)
What I did was seeing if there were any differences between these 2 engines by not only looking at reflected positions but also all legal slider positions. Since this position is a guaranteed mate position the slider positions are all legal derivations from the original EPD string. As I mnetiones in a post here several days ago a slider is placing the piece to move on any square that still allows the piece to move to hit the target square. The thinking behind this is that all positions should be close to the same across the 4 way rotations for each of the slider positions. The table below appears to show Rybka much better across the reflections compared to Crafty. And for almost all the 11 slider positions holds true. Perhaps Crafty is having a bit of trouble depending as to how Bobs engine works. I am no expert when it comes to chess engines but I thought Uri especially and others might be interested in this. In case the table is not clear the 4 way rotations are represented across the columns 1-4 and for this particular position there are 11 slider positions for each of the reflected boards.
Reflections 1 and 2 was WTM and 3 and 4 was BTM
Crafty Rybka
1 2 3 4 1 2 3 4
Slider
1 54 23 22 57 7 7 7 7
2 2 3 3 3 2 2 2 2
3 0 0 0 0 0 0 0 0
4 11 13 11 13 4 4 4 4
5 22 29 40 21 4 4 4 4
6 45 51 41 16 8 8 8 8
7 56 16 39 8 4 4 4 4
8 56 43 10 8 8 8 6 6
9 44 12 48 9 10 6 10 9
10 13 37 49 22 5 5 5 5
11 15 44 28 12 8 8 4 4
Notice how much less deviation there is across the reflections in almost all the slider positions comparing Rybka to Crafty! Can someone perhaps provide me with some explanation as to what could be causing this? Bob may be interested in this data to improving this part of Crafty.
HTH
Les
Re: Comparison data of Crafty vs Rybka that I promised 2 day
Posted: Fri Jun 01, 2007 5:14 am
by Dann Corbit
The elements of the grid are seconds
Movei added to Crafty vs Rybka comaprison data
Posted: Fri Jun 01, 2007 6:06 am
by vb4
Dann Corbit has just finished running the data set we used for the Crafty vs Rybka data using Uri Movei. I have combiend all 3 comparisons for your review. Again all data in the tables represents seconds.
I will post a New post for the 44 positions that were used to generated all this data for any of you that are interested in it.
Code: Select all
The following EPD was the position I started with.
4r1k1/p1qr1p2/2pb1Bp1/1p3Q1p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
Crafty-2014 and Rybka v1.2.w32 and Movei Analysis time was set to 1 minute and Dann Corbit ran these on the same Hardware. (Thx Dann)
What I did was seeing if there were any differences between these 2 engines by not only looking at reflected positions but also all legal slider positions. Since this position is a guaranteed mate position the slider positions are all legal derivations from the original EPD string. As I mnetiones in a post here several days ago a slider is placing the piece to move on any square that still allows the piece to move to hit the target square. The thinking behind this is that all positions should be close to the same across the 4 way rotations for each of the slider positions. The table below appears to show Rybka much better across the reflections compared to Crafty. And for almost all the 11 slider positions holds true. Perhaps Crafty is having a bit of trouble depending as to how Bobs engine works. I am no expert when it comes to chess engines but I thought Uri especially and others might be interested in this. In case the table is not clear the 4 way rotations are represented across the columns 1-4 and for this particular position there are 11 slider positions for each of the reflected boards. All data in all tables represent seconds to solution.
Reflections 1 and 2 was WTM and 3 and 4 was BTM
Crafty Rybka Movei
1 2 3 4 1 2 3 4 1 2 3 4
Slider
1 54 23 22 57 7 7 7 7 19 17 19 21
2 2 3 3 3 2 2 2 2 1 1 1 1
3 0 0 0 0 0 0 0 0 0 0 0 0
4 11 13 11 13 4 4 4 4 3 3 4 3
5 22 29 40 21 4 4 4 4 16 15 15 16
6 45 51 41 16 8 8 8 8 3 3 3 3
7 56 16 39 8 4 4 4 4 3 3 12 2
8 56 43 10 8 8 8 6 6 9 8 7 8
9 44 12 48 9 10 6 10 9 3 3 15 3
10 13 37 49 22 5 5 5 5 5 5 6 6
11 15 44 28 12 8 8 4 4 10 11 2 3
Notice how much less deviation there is across the reflections in almost all the slider positions comparing Rybka to Crafty! Can someone perhaps provide me with some explanation as to what could be causing this? Bob may be interested in this data to improving this part of Crafty.
HTH
Les
Re: Movei added to Crafty vs Rybka comaprison data
Posted: Sat Jun 02, 2007 3:41 am
by bob
I am not sure I understand what you are doing. One common "sanity test" I do in crafty is to feed it several thousand FEN positions, and have it evaluate each one 4 times. First, as the position is entered, then I flip the board vertically and swap the colors of all pieces, to give me a second position. I then flip the original position horizontally which just swaps the a file with the h-file and so forth, for a third position. And I take the second position (the vertically flipped position) and also flip it horizontally for a 4th position. Each position must produce the _same_ evaluation except that the sign must change when I invert white and black pieces. Any single failure causes a massive bug-hunt until it goes away...
Re: Movei added to Crafty vs Rybka comaprison data
Posted: Sat Jun 02, 2007 4:04 am
by sje
bob wrote:One common "sanity test" I do in crafty is to feed it several thousand FEN positions, and have it evaluate each one 4 times. First, as the position is entered, then I flip the board vertically and swap the colors of all pieces, to give me a second position. I then flip the original position horizontally which just swaps the a file with the h-file and so forth, for a third position. And I take the second position (the vertically flipped position) and also flip it horizontally for a 4th position. Each position must produce the _same_ evaluation except that the sign must change when I invert white and black pieces. Any single failure causes a massive bug-hunt until it goes away...
Apparently the test set doesn't include positions with non empty castling availability as I don't see how these would survive a reflection about the Y axis.
--------
Symbolic has a routine that reflects the board about the X axis and swaps the colors. During some testing modes, this is connected to the move path enumerator (perft) along with calls to the position evaluator before and after the exchange. This enables testing of huge numbers of positions in an orderly fashion. The enumerator has similar hooks for other complex or tricky routines, like the fast check marker and the checks only generator.
Re: Movei added to Crafty vs Rybka comaprison data
Posted: Sat Jun 02, 2007 6:00 am
by Dann Corbit
The involved both search and evaluation.
Each row (we would expect) should have about the same solution time, and none of the rows should be worse than the first.
These are the actual positions referenced in the test. For any solutions found, none should be worse than 32756 (though there are closer mates in a few instances).
This is a test set that is automatically generated from a tool Les made using WAC.141.
Code: Select all
4r1k1/p1qr1p2/2pb1Bp1/1p3Q1p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p4Qp/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1BpQ/1p5p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1nQR/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3PQ1/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PKQ/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B2QP2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP1Q1PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PK1/2Q4R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3PQn1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p2Q2p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p1Q3p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/pQ4p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/QpB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/RQn1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/1QP3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/QKP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2PQ2B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP1Q1PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R4Q2 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1nQP3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p2Q2p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P3q1P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P4qP/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bPq/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1Nqr/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3pq1/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pkq/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b2qp2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp1q1pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
2q4r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3pqN1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P2q2P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P1q3P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/Pq4P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P5P1/qPb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/rqN1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/1qp3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/qkp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2pq2b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp1q1pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r4q2/1kp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1Nqp3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P2q2P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
Re: Movei added to Crafty vs Rybka comaprison data
Posted: Sat Jun 02, 2007 6:48 am
by Dann Corbit
These are the positions for slider 9:
[D]4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PK1/2Q4R w - - pv Qxf4; ce 32756;
[D]1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R4Q2 w - - pv Qxc4; ce 32756;
[D]2q4r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
[D]r4q2/1kp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
As you can see, they are all exactly the same position, but with roles reverse or the board reflected.
Re: Movei added to Crafty vs Rybka comaprison data
Posted: Sat Jun 02, 2007 7:56 am
by bob
sje wrote:bob wrote:One common "sanity test" I do in crafty is to feed it several thousand FEN positions, and have it evaluate each one 4 times. First, as the position is entered, then I flip the board vertically and swap the colors of all pieces, to give me a second position. I then flip the original position horizontally which just swaps the a file with the h-file and so forth, for a third position. And I take the second position (the vertically flipped position) and also flip it horizontally for a 4th position. Each position must produce the _same_ evaluation except that the sign must change when I invert white and black pieces. Any single failure causes a massive bug-hunt until it goes away...
Apparently the test set doesn't include positions with non empty castling availability as I don't see how these would survive a reflection about the Y axis.
--------
Symbolic has a routine that reflects the board about the X axis and swaps the colors. During some testing modes, this is connected to the move path enumerator (perft) along with calls to the position evaluator before and after the exchange. This enables testing of huge numbers of positions in an orderly fashion. The enumerator has similar hooks for other complex or tricky routines, like the fast check marker and the checks only generator.
ok. I assumed that much was obvious, in that you can't mirror (left-to-right) and preserve castling status. That would certainly change the scoring when in one case a side could castle and in another case it could not. Crafty explicitly scores this as having lost the right to castle without castling, which drops the king safety score significantly with the king stuck in the center after mirroring. My test positions don't have castling as an opportunity.
Re: Movei added to Crafty vs Rybka comaprison data
Posted: Sat Jun 02, 2007 7:58 am
by bob
Dann Corbit wrote:The involved both search and evaluation.
Each row (we would expect) should have about the same solution time, and none of the rows should be worse than the first.
These are the actual positions referenced in the test. For any solutions found, none should be worse than 32756 (though there are closer mates in a few instances).
This is a test set that is automatically generated from a tool Les made using WAC.141.
Code: Select all
4r1k1/p1qr1p2/2pb1Bp1/1p3Q1p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p4Qp/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1BpQ/1p5p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1nQR/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3PQ1/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PKQ/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B2QP2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP1Q1PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PK1/2Q4R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3PQn1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
4r1k1/p1qr1p2/2pb1Bp1/1p2Q2p/3P1n1R/1B3P2/PP3PK1/7R w - - pv Qxf4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p1Q3p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/pQ4p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/QpB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/RQn1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/1QP3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/QKP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2PQ2B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP1Q1PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1n1P3/2P3B1/1KP3PP/R4Q2 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p5p1/R1nQP3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
1k1r4/2p1rq1p/1pB1bp2/p2Q2p1/R1n1P3/2P3B1/1KP3PP/R7 w - - pv Qxc4; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P3q1P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P4qP/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bPq/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1Nqr/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3pq1/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pkq/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b2qp2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp1q1pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
2q4r/pp3pk1/1b3p2/3p1N1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3pqN1r/1P5P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
7r/pp3pk1/1b3p2/3p1N1r/1P2q2P/2PB1bP1/P1QR1P2/4R1K1 b - - pv Qxf5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P1q3P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/Pq4P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P5P1/qPb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/rqN1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/1qp3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/qkp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2pq2b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp1q1pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r4q2/1kp3pp/2p3b1/r1N1p3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1Nqp3/P5P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
r7/1kp3pp/2p3b1/r1N1p3/P2q2P1/1Pb1BP2/2P1RQ1P/1K1R4 b - - pv Qxc5; ce 32756;
Actually that assumption is wrong. When you mirror or reflect the board, then move ordering will change since crafty uses bitboards and FirstOne() will produce different results for white or black. So you could get a different PV. I would think the score could also change since now the hashing signatures change which will also affect the tree shape. And then there's the castling issue where reflected positions (swapping white and black pieces) keeps castling status, but mirroring (exchanging a with h, etc) can not.
Re: Movei added to Crafty vs Rybka comaprison data
Posted: Sat Jun 02, 2007 8:00 am
by bob
yes. In fact, crafty has commands "flip" and "flop" to do that to the current position. "evtest" takes a FEN suite and does the sanity test using those commands.
I ran all 4. all four reported a mate in 6.
the scores were +2.18 and -2.18, + when it was black to move, - when it was white to move.
So I am still not sure what this is all about since all 4 report a forced mate (the times vary because of the move ordering issue I mentioned). And the eval is correct for each position...
so where does his table of numbers come from and what do the numbers represent???