In what file I can find stockfish's evaluation?

[smatovic]

[...]
I read that Weiss is the best engine with HCE that still play in TCEC and I do not like the fact that somebody hopes that it is going to go down.
I basically prefer engines that humans can understand what they do and not engines that people tell me that humans cannot understand what they do.

Others might know more, but as far as I got it, Xiphos was the last top10 member on CCRL wtih classic eval before the NNUE engines took over.

https://computerchess.org.uk/ccrl/404/c ... 4-bit_8CPU

--
Srdja

[syzygy]

I wonder if every evaluation that humans can understand is called HCE (hand craft evaluation) even if people did optimizations to have better weights in the evaluation.

HCE just means hand-crafted, it does not mean that humans can "understand" it.

And what is not to understand about a simple neural network? Just 1 type of calculation, whereas a complex HCE has dozens or hundreds of different kinds of calculation. And in all cases all weights are the result not of human "intuition" but of hundreds of thousands of hours of calculation.

But do you understand how Carlsen evaluates? Not even Carlsen does.

[Uri Blass]

I wonder if every evaluation that humans can understand is called HCE (hand craft evaluation) even if people did optimizations to have better weights in the evaluation.

HCE just means hand-crafted, it does not mean that humans can "understand" it.

And what is not to understand about a simple neural network? Just 1 type of calculation, whereas a complex HCE has dozens or hundreds of different kinds of calculation. And in all cases all weights are the result not of human "intuition" but of hundreds of thousands of hours of calculation.

But do you understand how Carlsen evaluates? Not even Carlsen does.

I do not expect to be able to understand how humans evaluates so the question how carlsen evaluates is not relevant for the discussion.

HCE is a lot of calculation but at least I can understand something of what the thing is doing if I look at the code.

I understand nothing of this NNUE file that I can download in
https://tests.stockfishchess.org/nns

People gave no explanation how to read and understand them.

I have no idea what material the nnue file contain and
when I try to open nn-9bc3b650e90f.nnue by tools that I have
I see things like the following that seems to me a language that I cannot understand.



c8gׂ !»^jPsybblx»p~
½LH`YE@m&qה ¿a»׃ ֲ  €…„¿Žd׳ִ ׀o`ּ -ֶ k•YCn:¢~;9Š±awqQJPSwנ ַּ~¿Pž 6'e °OXd‡X€9fFIb±©¹צ~s×g6OqW¹h¬sת~RU'"/N½
÷ƒtֵ ֵ ~>:“ק~bMDufפ ©כ <™ֽ ¢Q^A}‡ץ~ֹ ky o¢h·01½Uְ [ $±¨~‎~WSke.Z{eXx÷šLQ¿e¼G$O…ן~_µֱ ÷ַ ־ yfš»X¢jUšn~
€Ug°Qם~•Lk]š"ֱ †£·o?÷T ’H»nyס~qֿ~Szuֶ´ּ ¡B—Œ!״ ל~ƒY[KŠdת~´_~ם~w´"¶~ cuM ע knj_w.

[hgm]

Looking at a binary file like it is text usually isn't very elucidating. If you would print the eval.o file of a HCE you would see something similar, and learn as little.

I suppose the NNUE file is just a list of numbers, without any indication for how these numbers are to be used. Most of the numbers will actually be elements of piece-square tables. A popular form of NNUE uses 256 sets of piece-square tables for each location of a King. That is a lot of numbers (256 x 10 x 64 x 64).

[RubiChess]

You may have a look at my "nnuedebug" branch in https://github.com/Matthies/Stockfish.
It adds debug output of all the stages of NNUE evaluation. The branch is not in sync with Stockfish master and uses older network archtiecture but for your needs this should be okay.
Quick HowTo:

Code: Select all

git clone https://github.com/Matthies/Stockfish.git
cd Stockfish/src
git checkout nnuedebug
make build ARCH=x64-64-... (don't compile a profile-build, the debug output will make it run forever)
./stockfish
eval

This will give you the detailed informations of how the evaluation of startpos is done (and the evaluation of several other positions removing single pieces from the board so that eval can give a summary for values of each piece).

Edit: Of course this output is not enough to understand NNUE evaluation in detail. Maybe it could be improved by adding output of the weights and biases that are used for the different steps.

Code: Select all

# ./stockfish.exe
Stockfish dev-20230210-4bd928f2 by the Stockfish developers (see AUTHORS file)
eval
info string NNUE evaluation using nn-1337b1adec5b.nnue enabled


accumulation (c=0):
0046 0040 ff43 ff03 ff91 fe57 fe93 004e 01de ff75 004f 000c fed8 ff8a 00f7 001f    000
0020 018c 009c ff15 ff29 fe64 003d fef3 ff8c 0021 fde5 ffd4 ffad fffd ff1f fff0    010
ffba fdd9 0040 ffda ff32 ffb9 fee1 ffa1 ff4c ffd4 ffbe ffb5 fcb0 ff47 fe0d fe41    020
fff2 ff57 ffce 002e 0042 007c ff7f 004c ffc6 ff49 00b4 ffe3 ffce ff59 fefa ffe7    030
0027 ff41 ffb8 fe70 ffba 0229 0064 fff4 001c fe00 00de ff69 fe0a 0080 002d 031b    040
ff82 ff4b fcb3 ff94 ff98 ff70 ff8b 001b ff6a ff03 ffec fffd 0052 ffcd fffd 0066    050
0033 ff6a ffe5 016b fe08 004b fba4 feb0 000d ffd9 ffd5 ff95 faa0 0071 ffdb ff3f    060
ff2b fe89 ff28 ff3e 000d ffae fffb fff5 0027 0048 ffd1 fe6d 01a3 fe93 006d ffe6    070
ff5e 00e7 fff2 ffd3 00b2 ffee fefc ff39 0005 ff4c ffdc fffd ffb0 03c2 ffe8 fe22    080
0060 fffe 0064 fbf2 001f ffdd fff7 fdc7 0022 ffc2 fefc ff40 0017 ffa7 006c 0008    090
001b 005a fe4b 0001 0105 000a 001c 02e6 002c 0025 ff84 003f 006f 0002 fee2 ff99    0a0
fff9 0036 ff6e ff98 ff68 001e 00f5 ff1a ff9d ff28 0126 ff3a fff7 0098 fff0 fff1    0b0
feed fd9e ffdc 002c ff50 ff05 ffba ffdd fddf 005c ff7a ffe5 fe1c ff6e 007f 0050    0c0
ffe7 fea0 fe38 00ca ffa1 ffc6 007f fe0c ff2b ffa9 ffaa 0064 feff 003c fe69 fe4e    0d0
ffaa 008f ff86 0017 fced ff7f ffbc ff39 0028 fff0 ff34 0025 0007 fe91 ff7b 001b    0e0
fda9 002e 014f 0031 ffef fff9 ff2a ffa3 ff55 ff85 fe99 0017 ff6f ff5f fff1 ffb8    0f0
0079 0006 fe80 ff84 fdbf ffd9 fffa 0078 ffbe ff4f ff7b fc5d ffc8 feb4 ff48 ffc1    100
ffb8 ff76 00eb 001b 00b7 febc fd4e fefd 0018 001b 0070 0030 003a 00af 0040 0059    110
ffe0 fff9 fee5 0005 0088 ff1d ffd7 ff42 007a ffa4 ffd9 ff09 fdb8 0002 fe24 00ea    120
02ee 000b 006f 00d9 ffa4 feb2 fffe ffc8 0059 fe84 00bf ff50 002f ff90 0033 ffc9    130
fff6 ffe0 032d 008d ffeb fffe 0015 ffcf ff82 ff9b ff11 0012 0047 0063 0071 00ec    140
000f ff3d 0000 0033 fec3 ffa3 feb0 0033 fa82 fff5 0018 ffba 0144 ffb6 fec3 0184    150
ff3d fa4e 0012 fefb fde8 00ed ffb3 0055 ff08 fea2 003d ff6d ffc1 003d ff3b 02cb    160
0047 0001 ffc7 0042 ffe9 fd8f ffcd ffd9 fde1 02e8 0026 003d ffef 0037 0002 fed9    170
ff6e ffeb 004f fff6 ff56 0003 fef5 ff1b ff23 fffa ffda 000e 0033 ffe0 fef5 fff2    180
0033 ff74 fff5 fe0a ffab fed9 0189 0048 009e fea2 ffce fffc 0095 fef3 0053 fe18    190
00ca fbd6 ffe3 fd8f ff51 ffe9 0024 fef3 ff18 0011 002b fe2e fe6d 0080 ffee 00ce    1a0
ff56 0020 ff79 ffc1 0012 ffee 0050 ff0f 00cd fe7a ffea ffe5 0007 ffc6 ff09 ffa8    1b0
ffa8 fef9 ff42 ff2c 0006 ffab ff4c fff2 ffe4 002d 002f 0017 ffef ffa7 0065 002d    1c0
0009 0039 00f8 fe0a ffd0 0069 0031 ffcb ff27 ff80 0013 000c ffe7 ffe7 00d4 ff5e    1d0
016e ff6b 005e 009e 0074 010d ffde ffee fdd3 ffa3 fff1 ff8a 0282 ff16 0050 ff54    1e0
0005 fea7 fc74 ff34 004a ff67 0253 009a ffb7 ff5c ffe9 0007 0006 ff4f 0039 0003    1f0

psqtaccumulation (c=0):
 28956 -11692  -1795  25743  27708  14233   6152  -1070    000

accumulation (c=1):
0046 0040 ff43 ff03 ff91 fe57 fe93 004e 01de ff75 004f 000c fed8 ff8a 00f7 001f    000
0020 018c 009c ff15 ff29 fe64 003d fef3 ff8c 0021 fde5 ffd4 ffad fffd ff1f fff0    010
ffba fdd9 0040 ffda ff32 ffb9 fee1 ffa1 ff4c ffd4 ffbe ffb5 fcb0 ff47 fe0d fe41    020
fff2 ff57 ffce 002e 0042 007c ff7f 004c ffc6 ff49 00b4 ffe3 ffce ff59 fefa ffe7    030
0027 ff41 ffb8 fe70 ffba 0229 0064 fff4 001c fe00 00de ff69 fe0a 0080 002d 031b    040
ff82 ff4b fcb3 ff94 ff98 ff70 ff8b 001b ff6a ff03 ffec fffd 0052 ffcd fffd 0066    050
0033 ff6a ffe5 016b fe08 004b fba4 feb0 000d ffd9 ffd5 ff95 faa0 0071 ffdb ff3f    060
ff2b fe89 ff28 ff3e 000d ffae fffb fff5 0027 0048 ffd1 fe6d 01a3 fe93 006d ffe6    070
ff5e 00e7 fff2 ffd3 00b2 ffee fefc ff39 0005 ff4c ffdc fffd ffb0 03c2 ffe8 fe22    080
0060 fffe 0064 fbf2 001f ffdd fff7 fdc7 0022 ffc2 fefc ff40 0017 ffa7 006c 0008    090
001b 005a fe4b 0001 0105 000a 001c 02e6 002c 0025 ff84 003f 006f 0002 fee2 ff99    0a0
fff9 0036 ff6e ff98 ff68 001e 00f5 ff1a ff9d ff28 0126 ff3a fff7 0098 fff0 fff1    0b0
feed fd9e ffdc 002c ff50 ff05 ffba ffdd fddf 005c ff7a ffe5 fe1c ff6e 007f 0050    0c0
ffe7 fea0 fe38 00ca ffa1 ffc6 007f fe0c ff2b ffa9 ffaa 0064 feff 003c fe69 fe4e    0d0
ffaa 008f ff86 0017 fced ff7f ffbc ff39 0028 fff0 ff34 0025 0007 fe91 ff7b 001b    0e0
fda9 002e 014f 0031 ffef fff9 ff2a ffa3 ff55 ff85 fe99 0017 ff6f ff5f fff1 ffb8    0f0
0079 0006 fe80 ff84 fdbf ffd9 fffa 0078 ffbe ff4f ff7b fc5d ffc8 feb4 ff48 ffc1    100
ffb8 ff76 00eb 001b 00b7 febc fd4e fefd 0018 001b 0070 0030 003a 00af 0040 0059    110
ffe0 fff9 fee5 0005 0088 ff1d ffd7 ff42 007a ffa4 ffd9 ff09 fdb8 0002 fe24 00ea    120
02ee 000b 006f 00d9 ffa4 feb2 fffe ffc8 0059 fe84 00bf ff50 002f ff90 0033 ffc9    130
fff6 ffe0 032d 008d ffeb fffe 0015 ffcf ff82 ff9b ff11 0012 0047 0063 0071 00ec    140
000f ff3d 0000 0033 fec3 ffa3 feb0 0033 fa82 fff5 0018 ffba 0144 ffb6 fec3 0184    150
ff3d fa4e 0012 fefb fde8 00ed ffb3 0055 ff08 fea2 003d ff6d ffc1 003d ff3b 02cb    160
0047 0001 ffc7 0042 ffe9 fd8f ffcd ffd9 fde1 02e8 0026 003d ffef 0037 0002 fed9    170
ff6e ffeb 004f fff6 ff56 0003 fef5 ff1b ff23 fffa ffda 000e 0033 ffe0 fef5 fff2    180
0033 ff74 fff5 fe0a ffab fed9 0189 0048 009e fea2 ffce fffc 0095 fef3 0053 fe18    190
00ca fbd6 ffe3 fd8f ff51 ffe9 0024 fef3 ff18 0011 002b fe2e fe6d 0080 ffee 00ce    1a0
ff56 0020 ff79 ffc1 0012 ffee 0050 ff0f 00cd fe7a ffea ffe5 0007 ffc6 ff09 ffa8    1b0
ffa8 fef9 ff42 ff2c 0006 ffab ff4c fff2 ffe4 002d 002f 0017 ffef ffa7 0065 002d    1c0
0009 0039 00f8 fe0a ffd0 0069 0031 ffcb ff27 ff80 0013 000c ffe7 ffe7 00d4 ff5e    1d0
016e ff6b 005e 009e 0074 010d ffde ffee fdd3 ffa3 fff1 ff8a 0282 ff16 0050 ff54    1e0
0005 fea7 fc74 ff34 004a ff67 0253 009a ffb7 ff5c ffe9 0007 0006 ff4f 0039 0003    1f0

psqtaccumulation (c=1):
 28956 -11692  -1795  25743  27708  14233   6152  -1070    000

input layer:
45 3f 00 00 00 00 00 00 00 00 45 00 00 00 00 07    000
04 33 27 00 00 00 00 00 00 0b 00 00 00 00 00 00    010
00 00 17 00 00 00 00 00 00 00 00 00 00 00 00 00    020
00 00 00 00 15 00 00 15 00 00 00 00 00 00 00 00    030
18 00 00 00 00 00 32 00 09 00 00 00 00 2a 00 2b    040
00 00 00 00 00 00 00 00 00 00 00 00 0c 00 00 0d    050
13 00 00 15 00 24 00 00 00 00 00 00 00 00 00 00    060
00 00 00 00 03 00 00 00 00 28 00 00 00 00 61 00    070
00 7e 00 00 00 00 00 00 04 00 00 00 00 00 00 00    080
00 00 0a 00 1e 00 00 00 06 00 00 00 02 00 20 00    090
00 00 00 00 00 09 10 00 00 00 00 11 5e 00 00 00    0a0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    0b0
00 00 00 00 00 00 00 00 00 35 00 00 00 00 6e 47    0c0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    0d0
00 7e 00 10 00 00 00 00 00 00 00 09 00 00 00 00    0e0
00 06 00 03 00 00 00 00 00 00 00 0c 00 00 00 00    0f0
36 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    100
00 00 00 00 00 00 00 00 00 00 64 00 00 00 00 06    110
00 00 00 02 2c 00 00 00 00 00 00 00 00 00 00 18    120
00 00 12 00 00 00 00 00 58 00 00 00 00 00 00 00    130
00 00 0e 0f 00 00 00 00 00 00 00 00 1e 00 70 00    140
0d 00 00 32 00 00 00 00 00 00 00 00 00 00 00 00    150
00 00 04 00 00 00 00 03 00 00 00 00 00 00 00 15    160
00 00 00 00 00 00 00 00 00 00 05 00 00 22 00 00    170
00 00 2f 00 00 00 00 00 00 00 00 02 08 00 00 00    180
00 00 00 00 00 00 00 47 00 00 00 00 69 00 00 00    190
00 00 00 00 00 00 04 00 00 05 04 00 00 13 00 00    1a0
00 00 00 00 00 00 11 00 0a 00 00 00 00 00 00 00    1b0
00 00 00 00 02 00 00 00 00 07 05 0c 00 00 00 00    1c0
05 00 00 00 00 02 06 00 00 00 0a 00 00 00 00 00    1d0
00 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00    1e0
01 00 00 00 43 00 1f 00 00 00 00 00 00 00 0e 00    1f0
45 3f 00 00 00 00 00 00 00 00 45 00 00 00 00 07    200
04 33 27 00 00 00 00 00 00 0b 00 00 00 00 00 00    210
00 00 17 00 00 00 00 00 00 00 00 00 00 00 00 00    220
00 00 00 00 15 00 00 15 00 00 00 00 00 00 00 00    230
18 00 00 00 00 00 32 00 09 00 00 00 00 2a 00 2b    240
00 00 00 00 00 00 00 00 00 00 00 00 0c 00 00 0d    250
13 00 00 15 00 24 00 00 00 00 00 00 00 00 00 00    260
00 00 00 00 03 00 00 00 00 28 00 00 00 00 61 00    270
00 7e 00 00 00 00 00 00 04 00 00 00 00 00 00 00    280
00 00 0a 00 1e 00 00 00 06 00 00 00 02 00 20 00    290
00 00 00 00 00 09 10 00 00 00 00 11 5e 00 00 00    2a0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    2b0
00 00 00 00 00 00 00 00 00 35 00 00 00 00 6e 47    2c0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    2d0
00 7e 00 10 00 00 00 00 00 00 00 09 00 00 00 00    2e0
00 06 00 03 00 00 00 00 00 00 00 0c 00 00 00 00    2f0
36 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00    300
00 00 00 00 00 00 00 00 00 00 64 00 00 00 00 06    310
00 00 00 02 2c 00 00 00 00 00 00 00 00 00 00 18    320
00 00 12 00 00 00 00 00 58 00 00 00 00 00 00 00    330
00 00 0e 0f 00 00 00 00 00 00 00 00 1e 00 70 00    340
0d 00 00 32 00 00 00 00 00 00 00 00 00 00 00 00    350
00 00 04 00 00 00 00 03 00 00 00 00 00 00 00 15    360
00 00 00 00 00 00 00 00 00 00 05 00 00 22 00 00    370
00 00 2f 00 00 00 00 00 00 00 00 02 08 00 00 00    380
00 00 00 00 00 00 00 47 00 00 00 00 69 00 00 00    390
00 00 00 00 00 00 04 00 00 05 04 00 00 13 00 00    3a0
00 00 00 00 00 00 11 00 0a 00 00 00 00 00 00 00    3b0
00 00 00 00 02 00 00 00 00 07 05 0c 00 00 00 00    3c0
05 00 00 00 00 02 06 00 00 00 0a 00 00 00 00 00    3d0
00 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00    3e0
01 00 00 00 43 00 1f 00 00 00 00 00 00 00 0e 00    3f0

network layer:
  3742  -3273  -3233  -1450  -4509  -2301  -3467   2627    565  15290    725  -1574  -2780    513  -3608   1240    000

sqrclipped relu:
1a 14 13 04 26 0a 16 0d 00 7f 01 04 0e 00 18 02    000

clipped relu:
3a 00 00 00 00 00 00 29 08 7f 0b 00 00 08 00 13    000

network layer:
  1768  -5647  -3093  -2893   -315   2169 -12641  -4299  -7333  -4215  -6465   2080  -7375  -5664   2260  -1729    000
 -1877  -1811  -1877  -5630  -5232   -240  -1788  -7495    949  -1761   1320   2654  -3785  -2493  -5568  -4717    010

clipped relu:
1b 00 00 00 00 21 00 00 00 00 00 20 00 00 23 00    000
00 00 00 00 00 00 00 00 0e 00 14 29 00 00 00 00    010

network layer:
  -519    000

[ehenkes]

Idk if it helps, but Stockfish 16 removed the "classic, handcrafted evaluation" from source code, now 16.1 with small and big neural network.

--
Srdja

Thanks but I know that stockfish removed the classic evaluation.

I wonder if every evaluation that humans can understand is called HCE (hand craft evaluation) even if people did optimizations to have better weights in the evaluation.


I read that Weiss is the best engine with HCE that still play in TCEC and I do not like the fact that somebody hopes that it is going to go down.
I basically prefer engines that humans can understand what they do and not engines that people tell me that humans cannot understand what they do.

I am also sad about losing the HCE evaluation, because I use SF as a trainer and coach that delivers values like king safety, mobility, threats, space, pawn structure, PSQ score to the human player.

In SF 16.1 with eval you get a table with NN evaluation of the pieces (NN value with and without the piece), but nobody could explain these values.
Thus, I am interested in a SF 16.1 fork with the HCE added.

[Uri Blass]

Idk if it helps, but Stockfish 16 removed the "classic, handcrafted evaluation" from source code, now 16.1 with small and big neural network.

--
Srdja

Thanks but I know that stockfish removed the classic evaluation.

I wonder if every evaluation that humans can understand is called HCE (hand craft evaluation) even if people did optimizations to have better weights in the evaluation.


I read that Weiss is the best engine with HCE that still play in TCEC and I do not like the fact that somebody hopes that it is going to go down.
I basically prefer engines that humans can understand what they do and not engines that people tell me that humans cannot understand what they do.

I am also sad about losing the HCE evaluation, because I use SF as a trainer and coach that delivers values like king safety, mobility, threats, space, pawn structure, PSQ score to the human player.

In SF 16.1 with eval you get a table with NN evaluation of the pieces (NN value with and without the piece), but nobody could explain these values.
Thus, I am interested in a SF 16.1 fork with the HCE added.

The question is what is the advantage for SF16.1 fork because you can use HCE of older stockfish that had HCE and I do not know about HCE improvement for stockfish in the last years.

[Antihelion]

I am also sad about losing the HCE evaluation, because I use SF as a trainer and coach that delivers values like king safety, mobility, threats, space, pawn structure, PSQ score to the human player.

Couldn't you evaluate those positions yourself? IMO human players and NNUE are much better at evaluating those type of positional terms. There's a reason why neural networks are superior despite lower speed.

[Uri Blass]

I am also sad about losing the HCE evaluation, because I use SF as a trainer and coach that delivers values like king safety, mobility, threats, space, pawn structure, PSQ score to the human player.

Couldn't you evaluate those positions yourself? IMO human players and NNUE are much better at evaluating those type of positional terms. There's a reason why neural networks are superior despite lower speed.

At least human players who are not top chess players are not better at evaluating positional terms relative to HCE stockfish and what top chess players understand is not relevant for the average chess player.

[chesskobra]

You could install the last version of SF before HCE was removed. I think it is this branch: 6a8767a0d5d9502e6d4de1bef97468b5d6fab80a
I do keep a copy of it, with similar views. Also, if I remember, in the past it was possible to print a table of evaluation of a position (e.g., king safty, space advantage, ...) while running SF in terminal, but I didn't experiment with it enough, and don't remember how it was done. For this purpose, you may also consider using some older engines like crafty, phalanx, Toga, etc. whose evaluation functions may be easy to understand. In human terms they are quite strong engines.