Weird behaviour of Pigeon 1.51 - Help requested

[Guenther]

I noticed some strange phenomena with Pigeon 1.51, which makes it untestable the way I usually do current testing.
It seems it was never reported due to certain circumstances. (also never seen here for an engine in 20+ years)

https://github.com/StuartRiffle/pigeon

I will add more infos later (but probably in the programmers forum).
A little confirmation about a few things happening would be welcome before.

So if you have a Pigeon 1.51 available and are in the mood for help debugging an interesting problem,
please start it :)

The best would be just to do this from cmd as suggested below.
Please run it with those commands sequentially one by one - most will be answered or acknowledged.

Code: Select all

uci
ucinewgame
isready
setoption name OwnBook value false
position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4
go wtime 120000 btime 120000 movestogo 37

If you are acquainted doing such stuff you could also try adding the first move you received now
plus some times

e.g.

Code: Select all

position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4 <move> g4f3 (except the Knight was moved of course)
go wtime 119000 btime 112430 movestogo 36
(if you know what you are doing you can subtract the real time from White otherwise the value given is ok)

then in a new run after closing try the same commands as in step one but replace
the position command with this

Code: Select all

position fen rn1qkb1r/ppp1pppp/5n2/3p4/6b1/3P1NP1/PPP1PP1P/RNBQKB1R w KQkq - 1 4

Please post your output. (You can also try by not changing internal book usage - default = true, hash change is also
possible but I don't expect relevant changes about the issue)

A last hint: If you are also used to work with Inbetween you could let it log all for you and you don't need to copy paste
each time after closing the console. The beginning will look like this.

Code: Select all

0.000: - Starting server: Pigeon151-64.exe
0.032: - Priority class set to normal
0.032: - Thread priority set to normal
0.047: < 
0.047: <      /O_    Pigeon Chess Engine 1.5.1
0.047: <      ||     UCI/SSE2
0.047: <     / \\    
0.047: <   =/__//    pigeonengine.com
0.047: <      ^^     
0.047: < 
3.979: > uci
3.979: < id name Pigeon 1.5.1
3.979: < id author Stuart Riffle 
3.979: < option name Hash type spin min 64 max 16384 default 512
3.979: < option name OwnBook type check default true
3.979: < uciok
8.456: > ucinewgame
30.093: > setoption name OwnBook value false
79.655: > position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4
119.575: > go wtime 120000 btime 120000 movestogo 37

Edit:
And if you still are interested and patient enough, you could also try a run with startposition
once with

Code: Select all

position startpos

and

Code: Select all

go infinite

and once with

Code: Select all

position fen rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1

and

Code: Select all

go infinite

Thanks for helping!

[Gabor Szots]

Hi Günther,

I'm not sure I have understood what you had written but here is my output (and input as well):

/O_ Pigeon Chess Engine 1.5.1
|| UCI/SSE4/POPCNT
/ \\
=/__// pigeonengine.com
^^

uci
id name Pigeon 1.5.1
id author Stuart Riffle
option name Hash type spin min 64 max 16384 default 512
option name OwnBook type check default true
uciok
ucinewgame
isready
readyok
setoption name OwnBook value false
position fen rn1qkb1r/ppp1pppp/5n2/3p4/6b1/3P1NP1/PPP1PP1P/RNBQKB1R w KQkq - 1 4
go wtime 120000 btime 120000 movestogo 37
info depth 1 seldepth 3 score cp 31 hashfull 0 nodes 43 time 0 nps 43000 pv f3e5
info depth 2 seldepth 11 score cp -16 hashfull 0 nodes 738 time 0 nps 738000 pv f3e5 b8c6
info depth 3 seldepth 15 score cp 27 hashfull 0 nodes 5387 time 15 nps 359133 pv h2h3 g4f3 e2f3
info depth 4 seldepth 21 score cp -16 hashfull 0 nodes 21753 time 31 nps 1359562 pv c1e3 b8c6 e3c5 b7b6
info depth 5 seldepth 23 score cp 29 hashfull 0 nodes 73675 time 53 nps 3348863 pv h2h3 g4f3 e2f3 b8c6 b1c3
info depth 6 seldepth 24 score cp -2 hashfull 0 nodes 300845 time 213 nps 1979243 pv f3e5 b8c6 c1f4 c6e5 f4e5 c7c6
info depth 7 seldepth 25 score cp 29 hashfull 0 nodes 1086058 time 576 nps 3008470 pv f3e5 b8c6 e5g4 f6g4 h2h3 g4e5 c1g5
info depth 8 seldepth 27 score cp 6 hashfull 0 nodes 4194313 time 2520 nps 2166483 pv f3e5 b8c6 e5g4 f6g4 b1c3 d8d7 c1g5 c6d4
bestmove f3e5

Is that what you asked for?

[Guenther]

Hi Günther,

I'm not sure I have understood what you had written but here is my output (and input as well):

/O_ Pigeon Chess Engine 1.5.1
|| UCI/SSE4/POPCNT
/ \\
=/__// pigeonengine.com
^^

uci
id name Pigeon 1.5.1
id author Stuart Riffle
option name Hash type spin min 64 max 16384 default 512
option name OwnBook type check default true
uciok
ucinewgame
isready
readyok
setoption name OwnBook value false
position fen rn1qkb1r/ppp1pppp/5n2/3p4/6b1/3P1NP1/PPP1PP1P/RNBQKB1R w KQkq - 1 4
go wtime 120000 btime 120000 movestogo 37
info depth 1 seldepth 3 score cp 31 hashfull 0 nodes 43 time 0 nps 43000 pv f3e5
info depth 2 seldepth 11 score cp -16 hashfull 0 nodes 738 time 0 nps 738000 pv f3e5 b8c6
info depth 3 seldepth 15 score cp 27 hashfull 0 nodes 5387 time 15 nps 359133 pv h2h3 g4f3 e2f3
info depth 4 seldepth 21 score cp -16 hashfull 0 nodes 21753 time 31 nps 1359562 pv c1e3 b8c6 e3c5 b7b6
info depth 5 seldepth 23 score cp 29 hashfull 0 nodes 73675 time 53 nps 3348863 pv h2h3 g4f3 e2f3 b8c6 b1c3
info depth 6 seldepth 24 score cp -2 hashfull 0 nodes 300845 time 213 nps 1979243 pv f3e5 b8c6 c1f4 c6e5 f4e5 c7c6
info depth 7 seldepth 25 score cp 29 hashfull 0 nodes 1086058 time 576 nps 3008470 pv f3e5 b8c6 e5g4 f6g4 h2h3 g4e5 c1g5
info depth 8 seldepth 27 score cp 6 hashfull 0 nodes 4194313 time 2520 nps 2166483 pv f3e5 b8c6 e5g4 f6g4 b1c3 d8d7 c1g5 c6d4
bestmove f3e5

Is that what you asked for?

Yes!, but that is the second part. Thanks Gabor!
Could you please try the first part too? (no fen but moves)

Ah and if it's not to tiresome you could try a non popcount version too, or does it detect it on runtime?
I don't remember now, if there were more compilations?

[Gabor Szots]

Here is the first part:

/O_ Pigeon Chess Engine 1.5.1
|| UCI/SSE4/POPCNT
/ \\
=/__// pigeonengine.com
^^

uci
id name Pigeon 1.5.1
id author Stuart Riffle
option name Hash type spin min 64 max 16384 default 512
option name OwnBook type check default true
uciok
ucinewgame
isready
readyok
setoption name OwnBook value false
position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4
go wtime 120000 btime 120000 movestogo 37
info depth 1 seldepth 3 score cp 61 hashfull 0 nodes 48 time 2 nps 24000 pv b1c3
info depth 2 seldepth 8 score cp -8 hashfull 0 nodes 556 time 2 nps 556000 pv b1c3 b8c6
info depth 3 seldepth 12 score cp 40 hashfull 0 nodes 3471 time 10 nps 3471000 pv b1c3 b8c6 c1e3
info depth 4 seldepth 36 score cp -47 hashfull 0 nodes 30185 time 33 nps 1588684 pv a2a4 b8c6 b1c3 d8d6
info depth 5 seldepth 36 score cp 29 hashfull 0 nodes 129744 time 103 nps 1853485 pv h2h3 g4f3 e2f3 b8c6 b1c3
info depth 6 seldepth 44 score cp -3 hashfull 0 nodes 2126119 time 1555 nps 1480584 pv b1c3 c7c6 a2a4 d8b6 f3e5 b6d4 e5g4 f6g4
bestmove b1c3
position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4 b1c3 g4f3
go wtime 119000 btime 112430 movestogo 36
info depth 1 seldepth 1 score cp 87 hashfull 0 nodes 28 time 0 nps 28000 pv e2f3
info depth 2 seldepth 4 score cp 37 hashfull 0 nodes 97 time 0 nps 97000 pv e2f3 b8c6
info depth 3 seldepth 9 score cp 77 hashfull 0 nodes 318 time 8 nps 318000 pv e2f3 d5d4 c3b5
info depth 4 seldepth 9 score cp 29 hashfull 0 nodes 1266 time 8 nps 1266000 pv e2f3 d5d4 c3b5 b8c6
info depth 5 seldepth 41 score cp 47 hashfull 0 nodes 2758253 time 1994 nps 1390248 pv e2f3 d5d4 f3f4 d8d7 c3e4
info depth 6 seldepth 41 score cp 17 hashfull 0 nodes 3036971 time 2182 nps 16154101 pv e2f3 d5d4 c3e4 b8c6 c1f4 e7e5
bestmove e2f3

[Gabor Szots]

I have only one Pigeon exe, the popcnt one.

[Guenther]

Do you see the difference? (pos moves tried two times - fen one time below)
Is your compilation from the download from github. I checked now there is only one download and one compilation.
If it is from there it detects popcount by runtime.

Code: Select all

0.000: - Starting server: Pigeon151-64.exe
0.031: - Priority class set to normal
0.031: - Thread priority set to normal
0.031: < 
0.031: <      /O_    Pigeon Chess Engine 1.5.1
0.031: <      ||     UCI/SSE2
0.031: <     / \\    
0.046: <   =/__//    pigeonengine.com
0.046: <      ^^     
0.046: < 
40.622: > uci
40.622: < id name Pigeon 1.5.1
40.622: < id author Stuart Riffle 
40.622: < option name Hash type spin min 64 max 16384 default 512
40.622: < option name OwnBook type check default true
40.622: < uciok
45.302: > ucinewgame
49.374: > isready
49.483: < readyok
58.812: > position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4
66.908: > go wtime 120000 btime 120000 movestogo 37
66.908: < info depth 1 seldepth 3 score cp 61 hashfull 0 nodes 48 time 0 nps 48000 pv b1c3
66.908: < info depth 2 seldepth 9 score cp 0 hashfull 0 nodes 578 time 0 nps 578000 pv b1c3 b8c6
66.908: < info depth 3 seldepth 17 score cp -11591 hashfull 0 nodes 4428 time 10 nps 442800 pv c1f4 c7c6 f4b8
66.940: < info depth 4 seldepth 31 score cp -3 hashfull 0 nodes 24498 time 40 nps 816600 pv f3e5 b8c6 c1f4 e7e6
67.018: < info depth 5 seldepth 31 score cp -12004 hashfull 0 nodes 87379 time 110 nps 1248271 pv b1c3 c7c6 f3e5 d5d4 e5f7
67.860: < info depth 6 seldepth 37 score cp 1 hashfull 0 nodes 671130 time 960 nps 789564 pv h2h3 g4f3 e2f3 b8c6 b1c3 d8d6
67.860: < bestmove h2h3
87.423: > position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4 h2h3 g4f3
96.252: > go wtime 119030 btime 112430 movestogo 36
96.252: < info depth 1 seldepth 1 score cp 29 hashfull 0 nodes 26 time 0 nps 26000 pv e2f3
96.252: < info depth 2 seldepth 4 score cp -23 hashfull 0 nodes 121 time 0 nps 121000 pv e2f3 b8c6
96.268: < info depth 3 seldepth 25 score cp -198 hashfull 0 nodes 3680 time 10 nps 368000 pv e2f3 h7h6 c1h6 g7h6
96.268: < info depth 4 seldepth 25 score cp -8 hashfull 0 nodes 9840 time 20 nps 984000 pv e2f3 b8c6 b1c3 c6d4
96.361: < info depth 5 seldepth 25 score cp -4811 hashfull 0 nodes 80425 time 110 nps 893611 pv d1d2 c7c6 d2a5 d5d4 a5d8
96.580: < info depth 6 seldepth 33 score cp -71 hashfull 0 nodes 227661 time 320 nps 1084100 pv e2f3 c7c6 c1e3 d8a5 e1e2 a5a2
98.811: < info depth 7 seldepth 35 score cp -4440 hashfull 15 nodes 2185428 time 2560 nps 975637 pv d1d2 c7c6 d2b4 d8a5 b4a5 d5d4 e2f3
98.811: < bestmove d1d2

Code: Select all

0.000: - Starting server: Pigeon151-64.exe
0.047: - Priority class set to normal
0.047: - Thread priority set to normal
0.062: < 
0.062: <      /O_    Pigeon Chess Engine 1.5.1
0.062: <      ||     UCI/SSE2
0.062: <     / \\    
0.062: <   =/__//    pigeonengine.com
0.062: <      ^^     
0.062: < 
32.807: > uci
32.807: < id name Pigeon 1.5.1
32.807: < id author Stuart Riffle 
32.807: < option name Hash type spin min 64 max 16384 default 512
32.807: < option name OwnBook type check default true
32.807: < uciok
37.518: > ucinewgame
50.638: > isready
50.747: < readyok
79.763: > setoption name OwnBook value false
91.526: > position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4
107.859: > go wtime 120000 btime 120000 movestogo 37
107.859: < info depth 1 seldepth 3 score cp 20674 hashfull 0 nodes 48 time 0 nps 48000 pv b1c3
107.859: < info depth 2 seldepth 9 score cp 0 hashfull 0 nodes 578 time 0 nps 578000 pv b1c3 b8c6
107.875: < info depth 3 seldepth 17 score cp -11591 hashfull 0 nodes 4428 time 10 nps 442800 pv c1f4 c7c6 f4b8
107.906: < info depth 4 seldepth 31 score cp -3 hashfull 0 nodes 24498 time 40 nps 816600 pv f3e5 b8c6 c1f4 e7e6
107.968: < info depth 5 seldepth 31 score cp -12004 hashfull 0 nodes 87379 time 110 nps 1248271 pv b1c3 c7c6 f3e5 d5d4 e5f7
108.826: < info depth 6 seldepth 37 score cp 1 hashfull 0 nodes 671130 time 960 nps 789564 pv h2h3 g4f3 e2f3 b8c6 b1c3 d8d6
108.826: < bestmove h2h3
128.623: > position startpos moves g1f3 d7d5 d2d3 g8f6 g2g3 c8g4 h2h3 g4f3
141.555: > go wtime 119030 btime 112430 movestogo 36
141.555: < info depth 1 seldepth 1 score cp 6902 hashfull 0 nodes 26 time 0 nps 26000 pv e2f3
141.555: < info depth 2 seldepth 6 score cp 28598 hashfull 0 nodes 290 time 0 nps 290000 pv b2b4 f3e2
141.555: < info depth 3 seldepth 7 score cp 30 hashfull 0 nodes 978 time 0 nps 978000 pv e2f3 b8c6 b1c3
141.555: < info depth 4 seldepth 9 score cp 235 hashfull 0 nodes 4237 time 10 nps 423700 pv e2f3 f6h5 c2c3 h5g3 f2g3
141.586: < info depth 5 seldepth 15 score cp 29 hashfull 0 nodes 25801 time 30 nps 1290050 pv e2f3 b8c6 b1c3 c6d4 c1g5
142.008: < info depth 6 seldepth 27 score cp 3109 hashfull 0 nodes 414418 time 450 nps 986709 pv c2c3 d8d6 d1a4 e8d8 a4e8 d8e8
142.008: < bestmove c2c3

Code: Select all

0.000: - Starting server: Pigeon151-64.exe
0.047: - Priority class set to normal
0.047: - Thread priority set to normal
0.078: < 
0.078: <      /O_    Pigeon Chess Engine 1.5.1
0.078: <      ||     UCI/SSE2
0.078: <     / \\    
0.078: <   =/__//    pigeonengine.com
0.078: <      ^^     
0.093: < 
3.463: > uci
3.463: < id name Pigeon 1.5.1
3.463: < id author Stuart Riffle 
3.463: < option name Hash type spin min 64 max 16384 default 512
3.463: < option name OwnBook type check default true
3.463: < uciok
6.879: > ucinewgame
14.617: > isready
14.742: < readyok
32.978: > setoption name OwnBook value false
46.036: > position fen rn1qkb1r/ppp1pppp/5n2/3p4/6b1/3P1NP1/PPP1PP1P/RNBQKB1R w KQkq - 1 4
71.479: > go wtime 120000 btime 120000 movestogo 37
71.479: < info depth 1 seldepth 3 score cp 31 hashfull 0 nodes 43 time 0 nps 43000 pv f3e5
71.479: < info depth 2 seldepth 11 score cp -16 hashfull 0 nodes 738 time 10 nps 73800 pv f3e5 b8c6
71.479: < info depth 3 seldepth 15 score cp 27 hashfull 0 nodes 5387 time 10 nps 5387000 pv h2h3 g4f3 e2f3
71.511: < info depth 4 seldepth 21 score cp -16 hashfull 0 nodes 21753 time 40 nps 725100 pv c1e3 b8c6 e3c5 b7b6
71.573: < info depth 5 seldepth 23 score cp 29 hashfull 0 nodes 73675 time 100 nps 1227916 pv h2h3 g4f3 e2f3 b8c6 b1c3
71.901: < info depth 6 seldepth 24 score cp -2 hashfull 0 nodes 300845 time 420 nps 940140 pv f3e5 b8c6 c1f4 c6e5 f4e5 c7c6
72.696: < info depth 7 seldepth 25 score cp 29 hashfull 0 nodes 1086058 time 1220 nps 1374756 pv f3e5 b8c6 e5g4 f6g4 h2h3 g4e5 c1g5
72.696: < bestmove f3e5

[Gabor Szots]

I use the exe from github.

[Guenther]

My first thought was it had to do with the internal book and even when told to not use it would still
read some wrong poskeys from it.
The bug appeared with OwnBook true and false. Even more surprising is, it only happens to a certain ply length
If the pgn moves are 12 plies the bad effect already disappears.
With plain start position and my current 6 plies pgn it would show may be in 1/3 of the games exorbitant strange evals
in the first two moves after bookend and sometimes do a straight blunder.

According to Gabors test it seems hardware popcount is replaced by some code at runtime, which is buggy
at least with mentioned conditions. It would still be helpful if someone could do the test Gabor did for me
on a non-popcount machine for a last verification.

Game 1 shows what can happen with the bug.
Game 2 is the same startposition but fed with FEN.

[pgn][Event "Debugging"]
[Site "RWBC-CAPPUCCINO Win7U64 Q8200 2.33Ghz + Nvidia GT 710"]
[Date "2021.02.19"]
[Round "1"]
[White "Pigeon_151-64"]
[Black "BikJump_13"]
[Result "0-1"]
[ECO "A06"]
[GameDuration "00:02:36"]
[GameEndTime "2021-02-19T14:59:38.337 Mitteleuropäische Zeit"]
[GameStartTime "2021-02-19T14:57:01.564 Mitteleuropäische Zeit"]
[Opening "Reti"]
[PlyCount "66"]
[TimeControl "40/120"]
[Variation "Old Indian attack"]

1. Nf3 {book} d5 {book} 2. d3 {book} Nf6 {book} 3. g3 {book} Bg4 {book}
4. h3 {+0.01/6 0.95s} Bxf3 {+0.15/10 3.16s} 5. c3 {+31.09/6 0.45s}
Bxh1 {+8.01/9 3.19s} 6. g4 {-7.52/6 0.41s} e5 {+8.09/8 3.17s}
7. Qb3 {-7.43/6 0.38s} Qc8 {+8.13/8 3.17s} 8. g5 {-7.46/6 0.50s}
Nfd7 {+8.15/9 3.16s} 9. f3 {-7.29/7 1.89s} c6 {+8.22/9 3.18s}
10. d4 {-7.36/7 1.50s} Be7 {+8.28/9 3.17s} 11. Kf2 {-7.06/6 0.98s}
exd4 {+7.38/8 3.18s} 12. Kg1 {-6.44/7 3.89s} dxc3 {+7.38/9 3.21s}
13. Kxh1 {-6.08/8 2.90s} cxb2 {+7.36/9 3.17s} 14. Qxb2 {-5.91/7 0.84s}
O-O {+7.36/9 3.17s} 15. h4 {-5.96/7 2.91s} Qc7 {+7.36/8 3.19s}
16. h5 {-6.07/7 1.92s} Qg3 {+8.16/8 3.17s} 17. Qxb7 {-7.20/8 3.24s}
Bc5 {+11.28/8 3.16s} 18. e3 {-7.85/7 1.10s} Qxf3+ {+11.28/7 3.16s}
19. Kg1 {-8.12/8 4.36s} Bxe3+ {+11.28/7 3.18s} 20. Bxe3 {-9.62/9 3.17s}
Qxe3+ {+11.37/8 3.16s} 21. Kh1 {-9.64/7 0.46s} Nb6 {+11.56/8 3.22s}
22. Nc3 {-10.39/7 2.85s} Qxc3 {+12.42/7 3.18s} 23. Rb1 {-10.84/8 2.02s}
Qf3+ {+13.08/7 3.19s} 24. Kg1 {-11.24/8 4.87s} Qe3+ {+13.08/7 3.16s}
25. Kh2 {-11.04/8 1.75s} Qe5+ {+13.08/7 3.15s} 26. Kg1 {-11.33/8 5.11s}
Re8 {+13.47/7 3.20s} 27. g6 {-12.55/8 2.33s} Re7 {+15.03/6 3.14s}
28. gxh7+ {-12.71/9 2.11s} Kxh7 {+15.33/9 3.18s} 29. Rxb6 {-14.38/9 4.19s}
Qg3+ {+17.05/6 3.19s} 30. Kh1 {-13.06/9 2.21s} Qf3+ {+25.11/7 3.15s}
31. Kg1 {-12.79/9 3.02s} Re1 {+M11/6 3.21s} 32. Qxf7 {-325.11/9 2.27s}
Rxf1+ {+M3/1 0.003s} 33. Kh2 {-325.11/9 1.56s} Rh1# {+M1/1 0.002s, Black mates}
0-1[/pgn]

[pgn][Event "My Tournament"]
[Site "RWBC-CAPPUCCINO Win7U64 Q8200 2.33Ghz + Nvidia GT 710"]
[Date "2021.02.19"]
[Round "1"]
[White "Pigeon_151-64"]
[Black "BikJump_13"]
[Result "0-1"]
[FEN "rn1qkb1r/ppp1pppp/5n2/3p4/6b1/3P1NP1/PPP1PP1P/RNBQKB1R w KQkq - 0 1"]
[GameDuration "00:08:11"]
[GameEndTime "2021-02-19T14:16:43.039 Mitteleuropäische Zeit"]
[GameStartTime "2021-02-19T14:08:31.359 Mitteleuropäische Zeit"]
[PlyCount "172"]
[SetUp "1"]
[TimeControl "40/120"]

1. Ne5 {+0.29/7 1.20s} c6 {-0.02/8 2.96s} 2. Bg5 {+0.41/6 0.61s}
Nbd7 {-0.02/8 2.92s} 3. Nxg4 {+0.59/7 3.11s} Nxg4 {-0.05/8 2.93s}
4. h3 {+0.31/6 0.61s} Ngf6 {+0.02/8 2.93s} 5. Nc3 {+0.24/7 2.12s}
e5 {+0.12/9 2.94s} 6. Bg2 {+0.10/7 2.67s} Bb4 {+0.11/8 2.95s}
7. a3 {+0.22/7 1.69s} Bxc3+ {+0.14/9 2.93s} 8. bxc3 {+0.28/7 0.84s}
Qa5 {+0.10/9 2.95s} 9. Qd2 {+0.18/6 0.92s} O-O {+0.10/8 2.94s}
10. c4 {+0.16/6 1.27s} Qxd2+ {+0.19/10 2.97s} 11. Bxd2 {+0.06/9 2.33s}
dxc4 {+0.19/10 2.93s} 12. dxc4 {+0.14/8 2.42s} Nb6 {+0.20/9 2.93s}
13. Rb1 {-0.05/8 2.49s} Nxc4 {+0.90/9 2.94s} 14. Rxb7 {-0.14/8 2.81s}
Nxa3 {+0.90/9 2.92s} 15. Bc1 {-0.11/7 3.64s} Nxc2+ {+1.15/10 2.94s}
16. Kd2 {-0.68/8 1.17s} Nd4 {+1.22/8 2.92s} 17. e3 {-0.71/7 1.10s}
Ne6 {+2.12/8 2.94s} 18. Bb2 {-1.00/7 0.98s} Nc5 {+2.20/8 2.97s}
19. Rc7 {-1.09/8 2.66s} Rfd8+ {+1.91/7 2.94s} 20. Ke2 {-1.18/8 1.69s}
Nfe4 {+1.92/8 2.96s} 21. Bxe4 {-1.20/8 1.63s} Nxe4 {+1.92/9 2.92s}
22. Bxe5 {-1.18/8 1.32s} Rd2+ {+1.94/8 2.91s} 23. Kf3 {-1.26/9 2.59s}
Nxf2 {+1.92/9 2.91s} 24. Rh2 {-0.86/8 2.90s} Ne4 {+0.96/9 2.93s}
25. Rxd2 {-0.68/9 1.66s} Nxd2+ {+1.21/9 2.92s} 26. Kg2 {-0.54/9 3.69s}
a5 {+1.24/9 2.92s} 27. Rxc6 {-0.90/8 0.97s} f6 {+1.24/9 2.90s}
28. Bc3 {-0.69/8 1.21s} Nb1 {+1.29/10 2.92s} 29. Be1 {-0.72/9 3.82s}
a4 {+1.29/10 2.93s} 30. Rb6 {-0.77/9 5.79s} Na3 {+1.32/10 2.91s}
31. Rc6 {-1.05/9 2.20s} Nb5 {+1.32/9 2.92s} 32. Rb6 {-1.12/9 2.80s}
Nc7 {+1.39/10 2.91s} 33. Rb7 {-1.60/10 5.79s} Ne6 {+1.37/10 2.91s}
34. Re7 {-1.19/9 6.71s} Nf8 {+1.57/10 2.93s} 35. Bc3 {-1.81/10 3.00s}
a3 {+1.59/10 2.94s} 36. Ba1 {-1.68/9 2.08s} a2 {+1.59/10 2.96s}
37. Rb7 {-1.68/9 5.61s} Ne6 {+1.62/10 2.91s} 38. Rb5 {-1.90/8 2.77s}
Kf8 {+1.61/9 2.90s} 39. h4 {-1.81/8 2.79s} Kg8 {+1.62/9 2.89s}
40. h5 {-1.81/10 20.63s} Ng5 {+1.62/9 2.89s} 41. Rb4 {-1.55/7 1.13s}
Ne6 {+1.62/9 3.00s} 42. Rb5 {-1.58/7 3.13s} Ra3 {+1.61/9 3.02s}
43. Rb8+ {-1.76/8 2.84s} Kf7 {+1.62/10 3.02s} 44. Rb7+ {-1.76/8 1.36s}
Kg8 {+1.62/10 3.00s} 45. Kf3 {-1.75/8 1.51s} h6 {+1.64/10 3.04s}
46. Rb8+ {-1.71/8 1.16s} Kh7 {+1.64/10 3.00s} 47. Rb7 {-1.77/8 1.74s}
Ra4 {+1.64/9 3.03s} 48. g4 {-1.75/8 1.45s} Ng5+ {+2.55/9 3.00s}
49. Ke2 {-1.85/9 1.71s} Rxg4 {+2.55/9 3.03s} 50. Ra7 {-1.79/9 2.22s}
Ne4 {+2.48/9 3.01s} 51. Kf3 {-1.69/9 3.51s} Rh4 {+1.98/11 3.00s}
52. Rxa2 {-1.75/9 1.96s} Ng5+ {+1.99/10 3.00s} 53. Kg2 {-1.77/10 2.44s}
Rxh5 {+1.97/9 3.02s} 54. Ra7 {-1.79/8 0.74s} Kg6 {+2.00/9 2.98s}
55. Re7 {-1.80/8 0.63s} Rh4 {+2.03/9 2.98s} 56. Bd4 {-2.04/10 3.16s}
h5 {+2.28/9 3.02s} 57. Rd7 {-2.07/9 2.22s} Rg4+ {+2.34/10 3.01s}
58. Kf2 {-2.60/10 3.42s} h4 {+2.35/9 3.00s} 59. Bb6 {-2.58/9 1.84s}
h3 {+2.59/9 3.00s} 60. Bc7 {-3.73/9 1.35s} Rg2+ {+4.09/10 3.02s}
61. Kf1 {-4.22/10 4.20s} Ra2 {+4.11/10 3.00s} 62. Bf4 {-4.15/7 0.38s}
h2 {+4.12/10 3.01s} 63. Bxh2 {-4.29/10 2.67s} Rxh2 {+4.12/10 3.00s}
64. Ke1 {-4.31/10 2.70s} Kf5 {+4.13/9 3.00s} 65. Rxg7 {-4.27/9 0.90s}
Ke4 {+4.13/10 3.00s} 66. Ra7 {-4.33/9 0.93s} Kxe3 {+4.13/10 3.01s}
67. Ra3+ {-4.37/11 2.63s} Kf4 {+4.15/11 2.99s} 68. Ra7 {-4.36/9 5.36s}
f5 {+4.39/10 3.00s} 69. Kd1 {-4.49/8 0.48s} Nf3 {+4.42/10 2.99s}
70. Re7 {-4.60/10 2.37s} Ne5 {+4.48/10 2.98s} 71. Ra7 {-4.70/10 6.01s}
Ke3 {+4.51/10 2.99s} 72. Re7 {-4.95/11 2.75s} Ke4 {+4.68/10 2.99s}
73. Rb7 {-5.15/10 3.85s} f4 {+4.70/10 2.99s} 74. Rb4+ {-5.39/10 1.50s}
Ke3 {+10.97/11 2.99s} 75. Rb7 {-7.38/10 4.22s} f3 {+14.80/10 2.96s}
76. Rb6 {-8.50/9 0.99s} f2 {+17.22/9 2.96s} 77. Rf6 {-12.51/12 7.92s}
Nf3 {+18.34/10 2.98s} 78. Re6+ {-9.48/14 6.30s} Kd3 {+21.06/11 2.98s}
79. Rd6+ {-14.77/13 13.92s} Nd4 {+23.61/11 3.00s} 80. Rxd4+ {-325.11/17 9.82s}
Kxd4 {+23.61/12 2.96s} 81. Ke2 {-325.11/12 1.09s} f1=Q+ {+M11/10 1.28s}
82. Kxf1 {-325.11/14 2.04s} Ke3 {+M9/8 0.028s} 83. Kg1 {-325.11/14 1.93s}
Rf2 {+M7/6 0.007s} 84. Kh1 {-325.11/15 3.22s} Kf3 {+M5/4 0.003s}
85. Kg1 {-325.11/15 2.05s} Kg3 {+M3/2 0.002s} 86. Kh1 {-325.11/16 2.00s}
Rf1# {+M1/1 0.002s, Black mates} 0-1[/pgn]

From a 200 games test match, which was incidentally done w/o disabling the internal book
there are 67 of those evals available after pgn book end.
This lead me to investigating this and I realized it also happens with internal book disabled.

Evals in the move after book end: (only filtered the weird ones from 200)

Code: Select all

----------------------------------------
Suche nach '%4^. ' in 'C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn' :
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/75: 4. Qd4 {+1.00/5 4.94} Qh4 {+166.81/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/248: 4. exf5 {+1.12/5 4.88} Nd4 {+91.54/6 1.12}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/657: 4. O-O {+0.37/5 7.58} Ne4 {+273.78/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/862: 4. d5 {-161.72/7 1.74} Nxd5 {+2.50/5 7.50}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/1290: 4. Bb5+ {-4.56/5 3.24} Nc6 {+0.25/5 1.21}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/1495: 4. Nc3 {+0.75/5 3.64} Qb6 {-119.76/7 2.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/1880: 4. Qe2 {0.00/5 4.84} Be6 {-95.98/7 0.93}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/2023: 4. Nc3 {+0.37/5 2.83} c6 {+254.14/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/2088: 4. Bh3 {-220.61/7 1.41} d5 {+1.25/5 7.51}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/2666: 4. a3 {+0.75/5 7.57} Qg5 {-119.86/7 1.90}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/3232: 4. c3 {-278.06/7 1.48} gxh4 {+3.12/5 2.78}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/3408: 4. g3 {+202.15/6 0.92} Nc6 {-0.62/4 4.06}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/3491: 4. Nf3 {+0.75/5 6.03} c6 {+204.61/6 0.60}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/3712: 4. f4 {+283.94/6 3.24} Bg4 {+0.62/5 4.91}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/3815: 4. Nf3 {-152.04/7 2.30} Nc6 {+0.75/5 2.70}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/4121: 4. Bd2 {-1.25/6 6.64} Ba3 {-194.59/7 3.07}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/4242: 4. c5 {+283.99/6 3.24} Bxc5 {+0.50/4 4.05}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/4536: 4. Nc3 {+0.75/5 4.40} Bxd4 {-6.79/5 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/4625: 4. Nd5 {-150.81/7 1.92} Nf6 {+0.75/5 7.57}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/4692: 4. Bb5 {+0.37/5 7.52} Qa5 {-223.65/7 1.13}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/6026: 4. Nc3 {+0.75/5 7.57} c6 {+305.03/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/6084: 4. d4 {+10.11/6 3.24} Qa5 {-0.25/5 5.08}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/6252: 4. Nc3 {+0.75/5 4.45} Ba3 {-11.86/5 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/6351: 4. Nd5 {+221.87/6 1.71} Nf6 {-0.62/5 7.53}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/6401: 4. Bb5+ {+0.75/5 3.98} Bd7 {+188.97/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/6517: 4. Bb5+ {-3.99/5 0.50} c6 {+1.25/5 7.58}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/7226: 4. Nc3 {+0.50/5 2.76} d5 {+192.74/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/7261: 4. e4 {-6.52/5 0.52} dxe4 {+1.62/5 3.52}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/7367: 4. c3 {+205.63/6 3.24} Bf5 {+0.75/5 7.60}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/7574: 4. Na3 {+0.25/5 7.58} a6 {+147.66/6 0.43}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/8086: 4. Nf3 {0.00/5 4.24} Ba3 {-88.18/5 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/8151: 4. Ne5 {+63.61/6 3.24} Nxe5 {+4.25/5 3.82}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/8275: 4. Qa4+ {-118.59/7 1.42} Bd7 {-0.50/6 5.23}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/8322: 4. dxc5 {+1.12/5 4.25} b5 {+91.94/6 1.04}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/8384: 4. Qd3 {-34.36/7 3.24} d5 {0.00/5 2.70}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/8586: 4. Qb3 {+0.75/5 7.59} a4 {+118.12/8 2.59}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/8752: 4. Qb3 {+232.94/6 3.24} e6 {+0.75/5 4.53}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/8921: 4. Bd3 {+0.62/5 7.54} a6 {+122.47/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/9261: 4. d5 {+132.83/6 0.50} Nb6 {+1.12/5 7.60}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/9378: 4. Qa4 {+0.75/5 7.50} Nf6 {+49.15/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/9564: 4. Nd5 {+219.88/6 3.24} Nxe4 {+0.75/5 4.50}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/9613: 4. Nf3 {+0.75/5 5.00} d5 {+119.92/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/9680: 4. Bb5+ {-123.10/7 1.64} Bd7 {+0.75/5 7.55}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/9922: 4. Bd3 {+0.37/5 7.58} Qb6 {+255.44/6 1.06}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/10010: 4. Nc3 {+0.75/5 5.73} Bf5 {-109.18/5 1.16}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/10491: 4. Nc3 {-117.24/5 3.24} Na6 {+0.62/5 5.29}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/10534: 4. Bxf6 {+0.12/5 4.26} Be4 {+118.00/6 0.51}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/10815: 4. Bg5 {+0.75/5 3.78} Qxg5 {+257.92/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/11120: 4. Bh3 {-130.03/7 1.48} Nf6 {+0.75/5 4.23}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/11242: 4. c3 {-168.81/7 2.22} Qb6 {+0.75/5 7.53}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/11369: 4. Qe2 {-221.80/7 3.24} Nc6 {0.00/4 7.58}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/11502: 4. Nc3 {+0.62/5 4.09} b6 {+145.18/6 0.40}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/11580: 4. Bb5+ {+160.73/6 3.24} c6 {+0.50/5 4.00}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/11737: 4. e4 {+0.75/5 7.51} Qh4 {-110.23/3 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/11963: 4. c3 {-74.04/7 2.48} f6 {-0.25/5 7.53}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/12098: 4. Nd5 {+239.78/6 3.24} Nc6 {-0.75/5 7.57}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/12222: 4. Bc4 {-184.81/7 1.26} d6 {-0.12/5 7.50}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/12371: 4. Nd5 {-2.41/5 0.78} Nf6 {+0.62/5 4.32}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/12450: 4. d3 {+0.75/5 7.53} a6 {+263.73/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/12591: 4. Nc3 {+0.75/5 3.43} a6 {+174.06/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/12627: 4. Qd2 {-262.86/7 0.99} Ndf6 {-0.25/5 3.00}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/12751: 4. c5 {-2.87/5 3.24} d6 {+1.25/5 3.64}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/13003: 4. Qf5 {+242.13/6 3.24} Qf6 {+0.37/5 3.04}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/13056: 4. Nf3 {+0.75/5 5.56} d6 {+235.23/6 3.24}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/13146: 4. Ndf3 {+0.37/5 2.44} Nd5 {-121.87/7 1.73}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/13212: 4. d5 {-183.55/7 2.99} Bd6 {+1.25/5 7.51}
C:\ChessTools\QualityControl\Pigeon_Test\PGN\20210208.pgn/13282: 4. e5 {+0.75/5 7.57} Qb6 {-217.71/7 2.06}
Fundstellen: '%4^. ' 200 Einträge.

[Guenther]

It is solved now! Martin Sedlak (Cheng) was so nice to look at it and found something with UB sanitizer.
SquareBit seemed to have the wrong size somewhere and indexing of bits was wrecked up.

He created a quick fix and also made it easier compiling on Windows (failed here before, due to posix stuff).

I compiled a version for my hardware and did a testmatch again.
The problem with weird evals in 1/3 of games with short start pgn and even some straight piece blunders
for the very first plies out of the small book is totally gone!

I made compilations available at my download site including the fixed source and a readme.
(two w/o popcount and one for avx2+bmi2, it is still unclear to me if popcount is affected,
but I added it at least)

https://rwbc-chess.de/download.htm

The test match between the fixed version and the old version ended 81:47.
tc 40/2, both 128MB hash
(If someone reallls is interested all ganmes are available as pgn too)

I did not check how much games were directly decided to to piece blunders but there is a sheet available
which shows all evals between move 4 and 7 for both.

Code: Select all

Pigeon
RWBC-CAPPUCCINO Win7U64 Q8200 2.33Ghz + Nvidia GT 710, 2021.02.20
                        Score     12345678901234567890123456789012345678901234567890                                                                              
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
 1: Pigeon_151fix-64  81.0 / 128   1==11==00=01100111=11111=110=01=1=111=0=11=01111=11==11=110=000=1111=01111011=1011==101=0010011=0101=1=101=01=0=10=110111==11=00  (+64 -30 =34)
 2: Pigeon_151-64     47.0 / 128   0==00==11=10011000=00000=001=10=0=000=1=00=10000=00==00=001=111=0000=10000100=0100==010=1101100=1010=0=010=10=1=01=001000==00=11  (+30 -64 =34)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
128 games: +45 -49 =34

Thanks for helping out Martin!

[Gabor Szots]

The test match between the fixed version and the old version ended 81:47.
tc 40/2, both 128MB hash
(If someone reallls is interested all ganmes are available as pgn too)

Thanks Günther and Martin!

Our scripts remove very short games and games with an abnormally biased opening so I hope CCRL ranking of Pigeon is not distorted (although bugs are part of the engine).
I'm going to use the fix in my future tournaments and will see if there is significant difference.