Because it's not reliable I took a closer look, why? First I must notice, that I get 10% BETTER score, if I print output to file with > log.txt command than when printing to display (windows7)! With "stockfish.exe bench 256 2 20 default depth" I get with 2.2.2 1,86 Mb/s to display and 1,93 Mb/s to file. In benchmark.cpp I see it measures total time and nodes and calculates ratio. But I think it is better to count average NPS for these 16 positions. E.g. from log.txt
info depth 20 seldepth 26 score cp 36 nodes 10434518 nps 1760208
info depth 20 seldepth 32 score cp -181 nodes 26116595 nps 1935422
info depth 20 seldepth 44 score cp 113 nodes 3685658 nps 2953251
info depth 20 seldepth 30 score cp 72 nodes 15022894 nps 2159392
info depth 20 seldepth 32 score cp 202 nodes 3249327 nps 1964526
info depth 20 seldepth 24 score cp 0 nodes 4490634 nps 1881287
info depth 20 seldepth 29 score cp -56 nodes 11995418 nps 1982058
info depth 20 seldepth 30 score cp 254 nodes 22919343 nps 2086801
info depth 20 seldepth 31 score cp 52 nodes 17469128 nps 1832874
info depth 20 seldepth 36 score cp 113 nodes 10576686 nps 2158947
info depth 20 seldepth 30 score cp -145 nodes 15670760 nps 1816478
info depth 20 seldepth 29 score cp 64 nodes 77144230 nps 1757351
info depth 20 seldepth 32 score cp 210 nodes 15178871 nps 1926741
info depth 20 seldepth 31 score cp -20 nodes 26025658 nps 1849595
info depth 20 seldepth 28 score cp 121 nodes 2742549 nps 2283554
info depth 20 seldepth 39 score cp 141 nodes 6821552 nps 2196957
This gives real speed of 2,033 kn/s!
Jouni
Stockfish benchmark
Moderators: hgm, Rebel, chrisw
-
- Posts: 6442
- Joined: Tue Jan 09, 2007 12:31 am
- Location: PA USA
- Full name: Louis Zulli
Re: Stockfish benchmark
Let's eliminate non-determinism. Just use one thread, i.e, stick toJouni wrote:Because it's not reliable I took a closer look, why? First I must notice, that I get 10% BETTER score, if I print output to file with > log.txt command than when printing to display (windows7)! With "stockfish.exe bench 256 2 20 default depth" I get with 2.2.2 1,86 Mb/s to display and 1,93 Mb/s to file. In benchmark.cpp I see it measures total time and nodes and calculates ratio. But I think it is better to count average NPS for these 16 positions. E.g. from log.txt
info depth 20 seldepth 26 score cp 36 nodes 10434518 nps 1760208
info depth 20 seldepth 32 score cp -181 nodes 26116595 nps 1935422
info depth 20 seldepth 44 score cp 113 nodes 3685658 nps 2953251
info depth 20 seldepth 30 score cp 72 nodes 15022894 nps 2159392
info depth 20 seldepth 32 score cp 202 nodes 3249327 nps 1964526
info depth 20 seldepth 24 score cp 0 nodes 4490634 nps 1881287
info depth 20 seldepth 29 score cp -56 nodes 11995418 nps 1982058
info depth 20 seldepth 30 score cp 254 nodes 22919343 nps 2086801
info depth 20 seldepth 31 score cp 52 nodes 17469128 nps 1832874
info depth 20 seldepth 36 score cp 113 nodes 10576686 nps 2158947
info depth 20 seldepth 30 score cp -145 nodes 15670760 nps 1816478
info depth 20 seldepth 29 score cp 64 nodes 77144230 nps 1757351
info depth 20 seldepth 32 score cp 210 nodes 15178871 nps 1926741
info depth 20 seldepth 31 score cp -20 nodes 26025658 nps 1849595
info depth 20 seldepth 28 score cp 121 nodes 2742549 nps 2283554
info depth 20 seldepth 39 score cp 141 nodes 6821552 nps 2196957
This gives real speed of 2,033 kn/s!
Jouni
Code: Select all
stockfish.exe bench
Code: Select all
Nodes searched : 5447426
Do you really get significantly different values for this number based on whether or not you redirect the output to a file? I don't, using my own compile of Stockfish-2.2.2 on Mac OS X.
For example, without redirection I got 4379, 4382, 4375, 4388, 4378.
With redirection, I got 4381, 4382, 4382, 4378, 4380.
The sums are 21902 and 21903 respectively.
-
- Posts: 3282
- Joined: Wed Mar 08, 2006 8:15 pm
Re: Stockfish benchmark
Because there is 16 positions and 20 plies non-determinism is minimal. But here's bench with 1 cpu and 16 ply:
to display 2 tries:
===========================
Total time (ms) : 51043
Nodes searched : 48381913
Nodes/second : 947865
===========================
Total time (ms) : 51309
Nodes searched : 48381913
Nodes/second : 942951
to file 2 tries:
===========================
Total time (ms) : 48345
Nodes searched : 48381913
Nodes/second : 1000763
===========================
Total time (ms) : 48376
Nodes searched : 48381913
Nodes/second : 1000122
So is displaying is slower than filewriting
Jouni
to display 2 tries:
===========================
Total time (ms) : 51043
Nodes searched : 48381913
Nodes/second : 947865
===========================
Total time (ms) : 51309
Nodes searched : 48381913
Nodes/second : 942951
to file 2 tries:
===========================
Total time (ms) : 48345
Nodes searched : 48381913
Nodes/second : 1000763
===========================
Total time (ms) : 48376
Nodes searched : 48381913
Nodes/second : 1000122
So is displaying is slower than filewriting
Jouni
-
- Posts: 6442
- Joined: Tue Jan 09, 2007 12:31 am
- Location: PA USA
- Full name: Louis Zulli
Re: Stockfish benchmark
Here's what I get:
Without redirection to file:
With redirection to file:
I used ./stockfish-2.2.2 bench 256 1 16 default depth
Something related to timing seems wrong with your Windows binary. I guess you've been saying that for a while.
Without redirection to file:
Code: Select all
===========================
Total time (ms) : 40289
Nodes searched : 48381913
Nodes/second : 1200871
===========================
Total time (ms) : 40293
Nodes searched : 48381913
Nodes/second : 1200752
Code: Select all
===========================
Total time (ms) : 40454
Nodes searched : 48381913
Nodes/second : 1195973
===========================
Total time (ms) : 40434
Nodes searched : 48381913
Nodes/second : 1196565
Something related to timing seems wrong with your Windows binary. I guess you've been saying that for a while.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Stockfish benchmark
He HAS to have some sort of graphics/driver issue. Maybe the world's slowest graphics card or something, or a graphics library with some sort of internal debugging turned on...zullil wrote:Here's what I get:
Without redirection to file:
With redirection to file:Code: Select all
=========================== Total time (ms) : 40289 Nodes searched : 48381913 Nodes/second : 1200871 =========================== Total time (ms) : 40293 Nodes searched : 48381913 Nodes/second : 1200752
I used ./stockfish-2.2.2 bench 256 1 16 default depthCode: Select all
=========================== Total time (ms) : 40454 Nodes searched : 48381913 Nodes/second : 1195973 =========================== Total time (ms) : 40434 Nodes searched : 48381913 Nodes/second : 1196565
Something related to timing seems wrong with your Windows binary. I guess you've been saying that for a while.
Never seen anything like his numbers linux or windows.
-
- Posts: 2684
- Joined: Sat Jun 14, 2008 9:17 pm
Re: Stockfish benchmark
Yes can be, try this to remove the console delay (under Linux/mingw)Jouni wrote: So is displaying is slower than filewriting
Code: Select all
./stockfish bench > /dev/null