Hello Chess Friends,
I've just created a new Stockfish benchmark list based on TCEC's position
This new project/idea is born due to there was a question at Rybka Chess forum,
regarding how much faster is TCEC's hardware than SCCT...
So...there is no doubt that Martin's hardware is awesome!
But however, I think my hardwares are still not too old-dated...
Many Thanks to Arena Team, Stockfish Team, Martin Thoresen, Jean-Paul
Otherwise, without your work/ideas... the current list would not exist
Btw, I hope to see many new Stockfish benchmarks on your hardwares too!
For full list regarding Stockfish Benchmarks:
http://www.sedatcanbaz.com/chess/?page_id=864
Best Wishes,
Sedat
New: Stockfish Benchmarks are Acceptable!
Moderators: hgm, Rebel, chrisw
-
- Posts: 3018
- Joined: Thu Mar 09, 2006 11:58 am
- Location: Antalya/Turkey
-
- Posts: 6442
- Joined: Tue Jan 09, 2007 12:31 am
- Location: PA USA
- Full name: Louis Zulli
Re: New: Stockfish Benchmarks are Acceptable!
Seems like a rather complicated way to benchmark. I mean, why should I have to download Arena?
How about using Stockfish's standard built-in bench command?
Can't really tell what you're doing. Is Stockfish being asked to search just the position below until Qg4 appears?
[D]rnbq1rk1/pppnbppp/4p3/3pP1B1/3P3P/2N5/PPP2PP1/R2QKBNR w KQ - 0 7
How about using Stockfish's standard built-in bench command?
Can't really tell what you're doing. Is Stockfish being asked to search just the position below until Qg4 appears?
[D]rnbq1rk1/pppnbppp/4p3/3pP1B1/3P3P/2N5/PPP2PP1/R2QKBNR w KQ - 0 7
-
- Posts: 3018
- Joined: Thu Mar 09, 2006 11:58 am
- Location: Antalya/Turkey
Re: New: Stockfish Benchmarks are Acceptable!
I don't agree with you... I did not start creating benchmarks from yesterday.. since more than 10 years I am creator...and experience is already talking, understand ?zullil wrote:Seems like a rather complicated way to benchmark. I mean, why should I have to download Arena?
How about using Stockfish's standard built-in bench command?
Can't really tell what you're doing. Is Stockfish being asked to search just the position below until Qg4 appears?
[D]rnbq1rk1/pppnbppp/4p3/3pP1B1/3P3P/2N5/PPP2PP1/R2QKBNR w KQ - 0 7
So...actually is not complicated, if you have fast internet, to download it will take just a few seconds...
Otherwise... it will take more time to download Arena, to download exactly same Stockfish versions, to create position...to set time control, to set hashtables and so on... really this mesagge is more complicated...
I spent many efforts to replay you,,, in this time you could send me a benchmark
But however, no problem if you wish you can pıost your benchmark from here too, on this forum i mean...
Best,
Sedat
Last edited by Sedat Canbaz on Sat Feb 08, 2014 1:28 am, edited 1 time in total.
-
- Posts: 4565
- Joined: Sun Mar 12, 2006 2:40 am
- Full name:
Re: New: Stockfish Benchmarks are Acceptable!
Using the built in bench would make it possible for users of any operating system that Stockfish can run on to contribute. Then there is no need to download a special Stockfish and Arena, which I don't have. It does not make a big difference which version of Stockfish is used, as long as it does the benchmark with the same 30 positions of course. Okay, people can cheat but not many would I hope. With a screenshot you could still check if the benchnumber is correct so it is not super easy to fake the results. The normal bench only uses one thread though.
Eelco
Eelco
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
-
- Posts: 3018
- Joined: Thu Mar 09, 2006 11:58 am
- Location: Antalya/Turkey
Re: New: Stockfish Benchmarks are Acceptable!
Yes...this is my list, my rules... do you understand ?Eelco de Groot wrote:Using the built in bench would make it possible for users of any operating system that Stockfish can run on to contribute. Then there is no need to download a special Stockfish and Arena, which I don't have. It does not make a big difference which version of Stockfish is used, as long as it does the benchmark with the same 30 positions of course. Okay, people can cheat but not many would I hope. With a screenshot you could still check if the benchnumber is correct so it is not super easy to fake the results. The normal bench only uses one thread though.
Eelco
If you dont accept, sad, but no problem
Btw, no patience too see your Stockfish benchmark list
Best,
Sedat
-
- Posts: 719
- Joined: Thu Mar 09, 2006 1:21 am
- Location: Portland Oregon
Re: New: Stockfish Benchmarks are Acceptable!
Yes, understood but it is still a fair point to mention when something will limit contributions such as using an operating system dependent component. I like your idea anyway even if operating system constraints exclude me from potential contribution. Good luck with the list.Sedat Canbaz wrote: Yes...this is my list, my rules... do you understand ?
-
- Posts: 6442
- Joined: Tue Jan 09, 2007 12:31 am
- Location: PA USA
- Full name: Louis Zulli
Re: New: Stockfish Benchmarks are Acceptable!
Does Arena even run on linux or OS X? Without Wine? All I see in your download are .exe files. Is this Windows only? Do I have to install Wine just to use your method?Sedat Canbaz wrote:
I don't agree with you... I did not start creating benchmarks from yesterday.. since more than 10 years I am creator...and experience is already talking, understand ?
So...actually is not complicated, if you have fast internet, to download it will take just a few seconds...
Otherwise... it will take more time to download Arena, to download exactly same Stockfish versions, to create position...to set time control, to set hashtables and so on... really this mesagge is more complicated...
I spent many efforts to replay you,,, in this time you could send me a benchmark
But however, no problem if you wish you can pıost your benchmark from here too, on this forum i mean...
Best,
Sedat
Perhaps you could simply explain how the benchmark is done? I'm supposed to run one particular compile of one version of SF, with specific settings, on one position until the search does what?
Here's my output. What does it tell you?
Code: Select all
Stockfish 191113 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
Searching: rnbq1rk1/pppnbppp/4p3/3pP1B1/3P3P/2N5/PPP2PP1/R2QKBNR w KQ - 0 7
infinite: 0 ponder: 0 time: 0 increment: 0 moves to go: 0
1 +0.59 00:00 105 Nf3
2 +0.10 00:00 331 Nf3 Nc6
3 +0.08 00:00 602 Nf3 Nc6 a3
4 +0.20 00:00 984 Nf3 Nc6 g3 Bxg5 hxg5
5 +0.18 00:00 1616 Nf3 Nc6 g3 Re8 a3
6 +0.20 00:00 4659 Nf3 Nc6 Bd3 h6 Bxe7 Qxe7
7 +0.08 00:00 8050 Nf3 c5 Bxe7 Qxe7 Bb5 a6 Bxd7 Bxd7
8 +0.30 00:00 26983 Qh5 h6 Nf3 Nc6 O-O-O f6 Be3 fxe5 Nxe5 Ncxe5
9 +0.38 00:00 37599 Qh5 Nc6 O-O-O h6 Bd3 Bxg5+ hxg5 Qxg5+ Qxg5 hxg5 Nf3
10 -0.22 00:00 135485 Nf3 c5 Bb5 cxd4 Bxe7 Qxe7 Qxd4 Nc6 Bxc6 bxc6 O-O c5
11 -0.16 00:00 178752 Qh5 c5 Nf3 cxd4 Nxd4 g6 Bxe7 Qxe7 Qe2 Nc6 Nxc6 bxc6
O-O-O
12 +0.12 00:00 337612 Qg4 f5 Bxe7 Qxe7 Qg5 Qxg5 hxg5 c5 dxc5 Nxc5 Nf3 Nc6
13 -0.12 00:00 774739 Qg4 Re8 Bd3 c5 Nf3 cxd4 Qxd4 Nc5 Bxe7 Qxe7 O-O Nc6
Qf4 Bd7 Ng5 Nxd3 cxd3
14 +0.00 00:00 1069K Qg4 Re8 Bh6 Bf8 Bg5 Be7
15 +0.06 00:00 1779K Qg4 Kh8 Bd3 c5 Nf3 f5 exf6 Nxf6 Bxf6 Rxf6 dxc5 Nc6
b4 g6 O-O e5
16 -0.04 00:00 3022K Qg4 Kh8 Nf3 c5 Qh5 Nc6 Bd3 g6 Bxe7 Qxe7 Qh6 cxd4 Ng5
f5
17 -0.04 00:00 3601K Qg4 Kh8 Nf3 c5 Qh5 Nc6 Bd3 g6 Bxe7 Qxe7 Qh6 cxd4 Ng5
f5
18 -0.02 00:00 4757K Qg4 Kh8 Nf3 c5 dxc5 Nc6 Qh5 Nxc5 O-O-O Bd7 Bxe7 Qxe7
Kb1 h6 Be2 Rac8 g3 a5 a3 Ne4
19 +0.06 00:01 6452K Qg4 Kh8 Nf3 c5 dxc5 Nc6 Qh5 Nxc5 O-O-O Bd7 Bxe7 Qxe7
Bb5 Rac8 Kb1 h6 g3 Kg8 Bxc6 Bxc6 Nd4
20 +0.12 00:01 13089K Qg4 Kh8 Nf3 c5 dxc5 Nc6 Qh5 Nxc5 Be3 Kg8 O-O-O Bd7
Kb1 h6 Be2 Qb8 Bg5 Qd8 Bxe7 Nxe7 Nd4
21 +0.12 00:02 14332K Qg4 Kh8 Nf3 c5 dxc5 Nc6 Qh5 Nxc5 Be3 Kg8 O-O-O Bd7
Kb1 h6 Be2 Qb8 Bg5 Qd8 Bxe7 Nxe7 Nd4
22 +0.16 00:02 20684K Qg4 Kh8 Nf3 c5 dxc5 Nxc5 O-O-O Nc6 Kb1 a6 Qh5 Bd7
Be2 Rc8 Bxe7 Qxe7 Nd4 Nxd4 Rxd4 f6 f4 Be8
23 +0.08 00:04 32175K Qg4 Kh8 Nf3 c5 dxc5 Nxc5 O-O-O Nc6 Bxe7 Qxe7 h5 a6
h6 g6 Kb1 b5 Bd3 Bd7 Qh4 Qxh4 Rxh4 Rfb8
24 +0.02 00:07 66609K Qg4 Kh8 Nf3 c5 dxc5 Nxc5 O-O-O Nc6 Bxe7 Qxe7 h5 a6
h6 g6 Kb1 b5 Qf4 b4 Ne2 Bb7 Ng5 Nd7 Nd4 Ncxe5
25 +0.10 00:10 92605K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O a6 Kb1 b5 dxc5
Nxc5 Ne2 Bd7 Ned4 Nxd4 Bxe7 Qxe7 Nxd4 Ne4 h5 Qb4
Bxe4
26 +0.12 00:12 116541K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O a6 Kb1 b5 dxc5
Nxc5 Ne2 Ne4 Ned4 Nxd4 Bxe7 Qxe7 Nxd4 Qb4 Bxe4 fxe4
27 +0.06 00:15 152766K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O cxd4 Nb5 Nc5
Nfxd4 Nxd4 Nxd4 Ne4 Bxe7 Qxe7
28 +0.06 00:19 200729K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O cxd4 Nb5 Nc5
Nfxd4 Nxd4 Nxd4 Ne4 Bxe7
29 +0.18 00:26 289935K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O cxd4 Nb5 Nc5
Nfxd4 Nxd4 Nxd4 Ne4 Bxe7 Qxe7 Kb1 Bd7 h5 Rac8 Qe3
Qc5 c3 Qb6 Bxe4
30 +0.38 01:20 1005M Qg4 Re8 Bh6 Bf8 h5 f5 exf6 Nxf6 Qg3 Kh8 Be3 Bd6 f4
Nc6 O-O-O Ne7 h6 gxh6 Nf3 Nf5 Qe1 Rf8 Ne5 c5 Kb1
31 +0.59 02:13 1749M Qg4 f5 Bxe7 Qxe7 Qg5 Qe8 Nb5 h6 Qe3 Qd8 Qa3 a6 O-O-O
b6 c4 dxc4 Bxc4 Re8 Nc3 Nc6 Nf3 Bb7
32 +0.51 02:35 2038M Qg4 f5 Bxe7 Qxe7 Qg5 Qe8 Nb5 h6 Qe3 Qd8 Nf3 a6 Qa3
Nc6 O-O-O Rb8 Nc3 Qe7 Qxe7 Nxe7 Ne2 Kf7 Nf4 b5 Bd3
33 +0.53 04:05 3283M Qg4 Re8 Bh6 Bf8 h5 f5 exf6 Nxf6 Qg3 Kh8 Be3 Bd6 f4
Nc6 O-O-O Ne7 Qf3 Nf5 Bf2 Nh6 Kb1 Bd7 Be2 Rf8 Nh3 a6
34 +0.46 06:37 5641M Qg4 Re8 Bh6 Bf8 h5 f5 exf6 Qxf6 Bg5 Qf7 O-O-O e5 h6
g6 Qf3 c6 Qxf7+ Kxf7 Nf3 e4 Nd2 Bb4 Re1 Nb6 Be2 Bd7
f3 e3
35 +0.28 09:00 7846M Qg4 Re8 Bh6 Bf8 h5 f5 exf6 Qxf6 Bg5 Qf7 h6 g6 Qf3 c5
Qxf7+ Kxf7 Nf3 a6 dxc5 Bxc5 Be3 Nc6 O-O-O b5 Bxc5
Nxc5 Nxb5
Nodes: 7846182772
Nodes/second: 14517869
Best move: Qg4
Ponder move: Re8
-
- Posts: 3018
- Joined: Thu Mar 09, 2006 11:58 am
- Location: Antalya/Turkey
Re: New: Stockfish Benchmarks are Acceptable!
Thanks...but I never used Linux...that's why I have no any idea about Arena GUI runs on Linux or not...zullil wrote:Does Arena even run on linux or OS X? Without Wine? All I see in your download are .exe files. Is this Windows only? Do I have to install Wine just to use your method?Sedat Canbaz wrote:
I don't agree with you... I did not start creating benchmarks from yesterday.. since more than 10 years I am creator...and experience is already talking, understand ?
So...actually is not complicated, if you have fast internet, to download it will take just a few seconds...
Otherwise... it will take more time to download Arena, to download exactly same Stockfish versions, to create position...to set time control, to set hashtables and so on... really this mesagge is more complicated...
I spent many efforts to replay you,,, in this time you could send me a benchmark
But however, no problem if you wish you can pıost your benchmark from here too, on this forum i mean...
Best,
Sedat
Perhaps you could simply explain how the benchmark is done? I'm supposed to run one particular compile of one version of SF, with specific settings, on one position until the search does what?
Here's my output. What does it tell you?
Code: Select all
Stockfish 191113 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski Searching: rnbq1rk1/pppnbppp/4p3/3pP1B1/3P3P/2N5/PPP2PP1/R2QKBNR w KQ - 0 7 infinite: 0 ponder: 0 time: 0 increment: 0 moves to go: 0 1 +0.59 00:00 105 Nf3 2 +0.10 00:00 331 Nf3 Nc6 3 +0.08 00:00 602 Nf3 Nc6 a3 4 +0.20 00:00 984 Nf3 Nc6 g3 Bxg5 hxg5 5 +0.18 00:00 1616 Nf3 Nc6 g3 Re8 a3 6 +0.20 00:00 4659 Nf3 Nc6 Bd3 h6 Bxe7 Qxe7 7 +0.08 00:00 8050 Nf3 c5 Bxe7 Qxe7 Bb5 a6 Bxd7 Bxd7 8 +0.30 00:00 26983 Qh5 h6 Nf3 Nc6 O-O-O f6 Be3 fxe5 Nxe5 Ncxe5 9 +0.38 00:00 37599 Qh5 Nc6 O-O-O h6 Bd3 Bxg5+ hxg5 Qxg5+ Qxg5 hxg5 Nf3 10 -0.22 00:00 135485 Nf3 c5 Bb5 cxd4 Bxe7 Qxe7 Qxd4 Nc6 Bxc6 bxc6 O-O c5 11 -0.16 00:00 178752 Qh5 c5 Nf3 cxd4 Nxd4 g6 Bxe7 Qxe7 Qe2 Nc6 Nxc6 bxc6 O-O-O 12 +0.12 00:00 337612 Qg4 f5 Bxe7 Qxe7 Qg5 Qxg5 hxg5 c5 dxc5 Nxc5 Nf3 Nc6 13 -0.12 00:00 774739 Qg4 Re8 Bd3 c5 Nf3 cxd4 Qxd4 Nc5 Bxe7 Qxe7 O-O Nc6 Qf4 Bd7 Ng5 Nxd3 cxd3 14 +0.00 00:00 1069K Qg4 Re8 Bh6 Bf8 Bg5 Be7 15 +0.06 00:00 1779K Qg4 Kh8 Bd3 c5 Nf3 f5 exf6 Nxf6 Bxf6 Rxf6 dxc5 Nc6 b4 g6 O-O e5 16 -0.04 00:00 3022K Qg4 Kh8 Nf3 c5 Qh5 Nc6 Bd3 g6 Bxe7 Qxe7 Qh6 cxd4 Ng5 f5 17 -0.04 00:00 3601K Qg4 Kh8 Nf3 c5 Qh5 Nc6 Bd3 g6 Bxe7 Qxe7 Qh6 cxd4 Ng5 f5 18 -0.02 00:00 4757K Qg4 Kh8 Nf3 c5 dxc5 Nc6 Qh5 Nxc5 O-O-O Bd7 Bxe7 Qxe7 Kb1 h6 Be2 Rac8 g3 a5 a3 Ne4 19 +0.06 00:01 6452K Qg4 Kh8 Nf3 c5 dxc5 Nc6 Qh5 Nxc5 O-O-O Bd7 Bxe7 Qxe7 Bb5 Rac8 Kb1 h6 g3 Kg8 Bxc6 Bxc6 Nd4 20 +0.12 00:01 13089K Qg4 Kh8 Nf3 c5 dxc5 Nc6 Qh5 Nxc5 Be3 Kg8 O-O-O Bd7 Kb1 h6 Be2 Qb8 Bg5 Qd8 Bxe7 Nxe7 Nd4 21 +0.12 00:02 14332K Qg4 Kh8 Nf3 c5 dxc5 Nc6 Qh5 Nxc5 Be3 Kg8 O-O-O Bd7 Kb1 h6 Be2 Qb8 Bg5 Qd8 Bxe7 Nxe7 Nd4 22 +0.16 00:02 20684K Qg4 Kh8 Nf3 c5 dxc5 Nxc5 O-O-O Nc6 Kb1 a6 Qh5 Bd7 Be2 Rc8 Bxe7 Qxe7 Nd4 Nxd4 Rxd4 f6 f4 Be8 23 +0.08 00:04 32175K Qg4 Kh8 Nf3 c5 dxc5 Nxc5 O-O-O Nc6 Bxe7 Qxe7 h5 a6 h6 g6 Kb1 b5 Bd3 Bd7 Qh4 Qxh4 Rxh4 Rfb8 24 +0.02 00:07 66609K Qg4 Kh8 Nf3 c5 dxc5 Nxc5 O-O-O Nc6 Bxe7 Qxe7 h5 a6 h6 g6 Kb1 b5 Qf4 b4 Ne2 Bb7 Ng5 Nd7 Nd4 Ncxe5 25 +0.10 00:10 92605K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O a6 Kb1 b5 dxc5 Nxc5 Ne2 Bd7 Ned4 Nxd4 Bxe7 Qxe7 Nxd4 Ne4 h5 Qb4 Bxe4 26 +0.12 00:12 116541K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O a6 Kb1 b5 dxc5 Nxc5 Ne2 Ne4 Ned4 Nxd4 Bxe7 Qxe7 Nxd4 Qb4 Bxe4 fxe4 27 +0.06 00:15 152766K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O cxd4 Nb5 Nc5 Nfxd4 Nxd4 Nxd4 Ne4 Bxe7 Qxe7 28 +0.06 00:19 200729K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O cxd4 Nb5 Nc5 Nfxd4 Nxd4 Nxd4 Ne4 Bxe7 29 +0.18 00:26 289935K Qg4 Kh8 Nf3 c5 Bd3 f5 Qf4 Nc6 O-O-O cxd4 Nb5 Nc5 Nfxd4 Nxd4 Nxd4 Ne4 Bxe7 Qxe7 Kb1 Bd7 h5 Rac8 Qe3 Qc5 c3 Qb6 Bxe4 30 +0.38 01:20 1005M Qg4 Re8 Bh6 Bf8 h5 f5 exf6 Nxf6 Qg3 Kh8 Be3 Bd6 f4 Nc6 O-O-O Ne7 h6 gxh6 Nf3 Nf5 Qe1 Rf8 Ne5 c5 Kb1 31 +0.59 02:13 1749M Qg4 f5 Bxe7 Qxe7 Qg5 Qe8 Nb5 h6 Qe3 Qd8 Qa3 a6 O-O-O b6 c4 dxc4 Bxc4 Re8 Nc3 Nc6 Nf3 Bb7 32 +0.51 02:35 2038M Qg4 f5 Bxe7 Qxe7 Qg5 Qe8 Nb5 h6 Qe3 Qd8 Nf3 a6 Qa3 Nc6 O-O-O Rb8 Nc3 Qe7 Qxe7 Nxe7 Ne2 Kf7 Nf4 b5 Bd3 33 +0.53 04:05 3283M Qg4 Re8 Bh6 Bf8 h5 f5 exf6 Nxf6 Qg3 Kh8 Be3 Bd6 f4 Nc6 O-O-O Ne7 Qf3 Nf5 Bf2 Nh6 Kb1 Bd7 Be2 Rf8 Nh3 a6 34 +0.46 06:37 5641M Qg4 Re8 Bh6 Bf8 h5 f5 exf6 Qxf6 Bg5 Qf7 O-O-O e5 h6 g6 Qf3 c6 Qxf7+ Kxf7 Nf3 e4 Nd2 Bb4 Re1 Nb6 Be2 Bd7 f3 e3 35 +0.28 09:00 7846M Qg4 Re8 Bh6 Bf8 h5 f5 exf6 Qxf6 Bg5 Qf7 h6 g6 Qf3 c5 Qxf7+ Kxf7 Nf3 a6 dxc5 Bxc5 Be3 Nc6 O-O-O b5 Bxc5 Nxc5 Nxb5 Nodes: 7846182772 Nodes/second: 14517869 Best move: Qg4 Ponder move: Re8
Btw, looking at your current results:14517 Nodes/second are very good indeed
What is your hardware and how many cores, ponder off, HT ON or OFF ?
But however, I plan to publish the benchmark results especially which are done via Arena 2.0.1GUI (1024 Hashtable size, ponder off, 120min+30sec...) otherwise the results can lead to misunderstandings
Best,
Sedat
-
- Posts: 719
- Joined: Thu Mar 09, 2006 1:21 am
- Location: Portland Oregon
Re: New: Stockfish Benchmarks are Acceptable!
That is only about 10 times faster than an iPad.Sedat Canbaz wrote: Btw, looking at your current results:14517 Nodes/second are very good indeed
-
- Posts: 3018
- Joined: Thu Mar 09, 2006 11:58 am
- Location: Antalya/Turkey
Re: New: Stockfish Benchmarks are Acceptable!
Thank you dear RyanRyan Benitez wrote:Yes, understood but it is still a fair point to mention when something will limit contributions such as using an operating system dependent component. I like your idea anyway even if operating system constraints exclude me from potential contribution. Good luck with the list.Sedat Canbaz wrote: Yes...this is my list, my rules... do you understand ?