I have started re-running my SMP tests since the Intel compiler provides such a nice NPS improvement (multiple threads only, single thread seems to be the same as before).
Here's my first 20 core test runs (same 24 positions, each run 4 times). All results are geometric mean for a 24 position run.
run 1 run 2 run 3 run 4 avg
speedup 15.0 15.0 15.0 15.3 15.1
nps 16.3 16.4 16.3 16.3 16.3
Net results, NPS scaling improved from 14.7 or so to 16.3x. I know part of the bottleneck is 64GB of hash and TLB thrashing, so I do have plans to try mmap() and using the rather clumsy 1gb huge pages. But I have not done this as of yet, still using the automatic 2mb pages (transparent huge pages in current linux kernels). Speedup (geomean) went from 12.45 to 16.3 due to the intel issue plus a few further SMP refinements. BTW the 20 cpu search looked at about 5% more nodes (average) compared to the 1 cpu test, so overhead is pretty well managed for the moment.
The speedup has really captured my attention, because it is right at the theoretical max (15.1 with max of 16.3 based on NPS numbers). I am fixing to create another set of test positions just to be sure that these positions don't somehow happen to artificially inflate the speedup results. I am not sure exactly how I am going to choose this test set. I was leaning toward either (a) taking a set of positions from a long time control game played on ICC, or else (b) a random set of positions extracted from GM games (the way I extract starting positions for cluster testing).
A much larger set of positions would be better, statistically, but not so good practically as the tests would take forever to run. More on this later....
You should do a paper on your approach so that everyone can understand the 'why' part of the improvement better.
It seems like an awfully important part of modern computation, considering the way that CPU manufacturing is trending and with things like Larabee on the horizon.
We already have massive SMP with video cards, and I suspect that eventually CPU manufacturers will be forced to follow that lead.
Dann Corbit wrote:You should do a paper on your approach so that everyone can understand the 'why' part of the improvement better.
It seems like an awfully important part of modern computation, considering the way that CPU manufacturing is trending and with things like Larabee on the horizon.
We already have massive SMP with video cards, and I suspect that eventually CPU manufacturers will be forced to follow that lead.
I am actually working on writing this up. That's why the interest in doing all the repeated tests, statistical analysis, and now looking at the test positions to be sure they are not "too good". Still thinking about how to select a reasonable set of positions where I can search each for about 1 minute with 20 cores, which means no more than 20 minutes each with 1 core. 50 positions would be 1000 minutes, which would run in around 18 hours or so, which is doable. Only question is "which positions?" consecutive positions from a real game, random positions from random games? maybe one position from 50 divergent ECO openings? Be nice to have such a set of positions anyone could use for comparison. I suppose we could also look at this from "normal time controls" (i.e. 1 minute per move) and maybe a test for blitz time controls (5 secs over move) since the speedups will almost certainly be different.
Needs a bit of thought on my part. Suggestions always welcome...
Dann Corbit wrote:You should do a paper on your approach so that everyone can understand the 'why' part of the improvement better.
It seems like an awfully important part of modern computation, considering the way that CPU manufacturing is trending and with things like Larabee on the horizon.
We already have massive SMP with video cards, and I suspect that eventually CPU manufacturers will be forced to follow that lead.
BTW don't forget the intel Phi project. We have one such board here, 61 cores on a chip. But the processor is an in-order pentium-like CPU without all the out of order bells and whistles. 61 of 'em is still a significant computing resource however.
bob wrote:I am fixing to create another set of test positions just to be sure that these positions don't somehow happen to artificially inflate the speedup results. I am not sure exactly how I am going to choose this test set. I was leaning toward either (a) taking a set of positions from a long time control game played on ICC, or else (b) a random set of positions extracted from GM games (the way I extract starting positions for cluster testing).
A much larger set of positions would be better, statistically, but not so good practically as the tests would take forever to run. More on this later....
I have these positions extracted from TCEC S7, where side to move score >= 10cp better, and after a move its opponent also claimed its score >= 10 cp better, meaning both sides claimed to be good, and not only that next pair of move still continue the same claims, I am good, no I am good, so this is a 4-ply different view of the eval. I suspect these positions are difficult to evaluate, considering the TC and Threads used in TCEC S7. The positions in the following are the start of the 4 ply opposite scoring. You might be interested and have a look on these positions. I am only showing Stage 4 and 3. I can't find such positions in the super-final. After stage 4 is super-final. The c5 opcode is the eval in pawn unit, the move played by the engine in actual game is in sm opcode.
r4k1r/1bq1bppn/pp1pp3/4n1P1/P3P3/1NNBB2Q/1PP4P/3R1RK1 w - - fmvn 18; sm "d3e2"; c3 "Result 1-0"; c2 "Gull 3 - Houdini 4"; c1 "Round 3"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.18;
r5kr/1bq1bppn/pp1pp3/4n1P1/P3P3/1NN1B2Q/1PP1B2P/3R1RK1 w - - fmvn 19; sm "h3g3"; c3 "Result 1-0"; c2 "Gull 3 - Houdini 4"; c1 "Round 3"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.22;
r4nkr/1bq1bpp1/pp1pp3/4n1P1/P3P3/1NN1B1Q1/1PP1B2P/3R1RK1 w - - fmvn 20; sm "h2h4"; c3 "Result 1-0"; c2 "Gull 3 - Houdini 4"; c1 "Round 3"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.22;
4rnk1/1bq1bp1r/pp1pp1p1/4n1P1/P3P2P/1NN1B1Q1/1PP1B2R/3R2K1 w - - fmvn 23; sm "e3d4"; c3 "Result 1-0"; c2 "Gull 3 - Houdini 4"; c1 "Round 3"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.3;
5k2/pppRr1pp/5p2/8/8/2P3K1/PP1N4/8 w - - fmvn 29; sm "d7e7"; c3 "Result 0-1"; c2 "Gull 3 - Komodo 1331"; c1 "Round 5"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.3;
5krr/4qp2/3p1n1p/4pPn1/ppP1P1PB/3P4/BP2Q2R/6RK w - - fmvn 40; sm "h4f2"; c3 "Result 1/2-1/2"; c2 "Gull 3 - Houdini 4"; c1 "Round 9"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.17;
5krr/1q3p2/3p1n1p/4pPn1/ppP1P1P1/3P4/BP2QB1R/6RK w - - fmvn 41; sm "e2d1"; c3 "Result 1/2-1/2"; c2 "Gull 3 - Houdini 4"; c1 "Round 9"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.17;
6rr/1q3k2/3p1p1p/2n1pPn1/p1P1P1P1/1p1P2R1/1P1BQ1K1/1B5R w - - fmvn 69; sm "e2e1"; c3 "Result 1/2-1/2"; c2 "Gull 3 - Houdini 4"; c1 "Round 9"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.12;
7r/1q3kr1/3p1p1p/2n1pPn1/p1P1P1P1/1p1P2R1/1P1B2K1/1B2Q2R w - - fmvn 70; sm "g2f1"; c3 "Result 1/2-1/2"; c2 "Gull 3 - Houdini 4"; c1 "Round 9"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.14;
7r/1q2k1r1/3p1p1p/2n1pPn1/p1P1P1P1/1p1P2R1/1P1B4/1B2QK1R w - - fmvn 71; sm "h1h2"; c3 "Result 1/2-1/2"; c2 "Gull 3 - Houdini 4"; c1 "Round 9"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.13;
r6r/8/2kp1p1p/2q1pPnR/1QPnP1P1/Pp1P2R1/1B6/1B3K2 w - - fmvn 128; sm "b4c5"; c3 "Result 1/2-1/2"; c2 "Gull 3 - Houdini 4"; c1 "Round 9"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.17;
4k2r/6p1/r1pp3p/1pb1p3/p3p3/P1PP2nP/BP1B1PP1/R3R1K1 w k - fmvn 23; sm "d2e3"; c3 "Result 0-1"; c2 "Houdini 4 - Gull 3"; c1 "Round 12"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.12;
r1b1kb1r/1pqn1pp1/p2ppn1p/8/2BNP3/2N1BP2/PPPQ2PP/R3K2R w KQkq - fmvn 11; sm "c4e6"; c3 "Result 1/2-1/2"; c2 "Komodo 1331 - Gull 3"; c1 "Round 14"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.26;
r2qrbk1/1b1n1p1p/1np3p1/1p2p3/p3P3/PNNPB1PP/1PQ2PBK/R4R2 w - - fmvn 19; sm "b3c1"; c3 "Result 1-0"; c2 "Houdini 4 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.1;
r2qrbk1/3n1p1p/bnp3p1/1p2p3/p3P3/P1NPB1PP/1PQ2PBK/R1N2R2 w - - fmvn 20; sm "f2f4"; c3 "Result 1-0"; c2 "Houdini 4 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.14;
r2qrbk1/3n1p1p/bn4p1/1pp1p3/p3PP2/P1NPB1PP/1PQ3BK/R1N2R2 w - - fmvn 21; sm "f4f5"; c3 "Result 1-0"; c2 "Houdini 4 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.13;
r2qrbk1/3n1p1p/bn6/1pp1pPp1/p3P3/P1NPB1PP/1PQ3BK/R1N2R2 w - - fmvn 22; sm "c3d1"; c3 "Result 1-0"; c2 "Houdini 4 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.17;
1nrqrbk1/5p2/bn5p/1pp1pPp1/p3P3/P2PB1PP/1PQ2NBK/R1NR4 w - - fmvn 25; sm "h3h4"; c3 "Result 1-0"; c2 "Houdini 4 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.35;
1nrqrbk1/5p2/bn5p/1pp1pP2/p3P2p/P2PB1P1/1PQ2NBK/R1NR4 w - - fmvn 26; sm "c2e2"; c3 "Result 1-0"; c2 "Houdini 4 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.27;
2r2b1r/2kn4/bn1q1p1p/1pp1pP1B/p3P2K/P2PB3/NP1Q1N2/6RR w - - fmvn 38; sm "h5f7"; c3 "Result 1-0"; c2 "Houdini 4 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.47;
2q3k1/p2n1pp1/3Pb2p/p1r5/1r5P/3B1PB1/5QP1/3RR1K1 w - - fmvn 31; sm "g1h2"; c3 "Result 1/2-1/2"; c2 "Komodo 1331 - Gull 3"; c1 "Round 20"; c0 "Event TCEC Season 7 - Stage 4"; c5 0.2;
r1b2rk1/pp3pb1/1npp2p1/4pP2/2PnP1p1/2NP3P/PP2NBB1/2R2RK1 w - - fmvn 17; sm "h3g4"; c3 "Result 1/2-1/2"; c2 "Chiron 2 - Critter 1.6a"; c1 "Round 6"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
r1b2rk1/pp3p2/1npp2pb/4pP2/2PnP1P1/2NP4/PP2NBB1/2R2RK1 w - - fmvn 18; sm "c1b1"; c3 "Result 1/2-1/2"; c2 "Chiron 2 - Critter 1.6a"; c1 "Round 6"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.32;
5nk1/5p2/6pp/2NQ4/3b4/5KPP/8/5r2 w - - fmvn 54; sm "f3e2"; c3 "Result 1/2-1/2"; c2 "Junior 13.3 - Komodo 1324"; c1 "Round 8"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.18;
8/5p1k/4n1pp/5r2/8/6PP/1Q2K3/8 w - - fmvn 61; sm "b2b7"; c3 "Result 1/2-1/2"; c2 "Junior 13.3 - Komodo 1324"; c1 "Round 8"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.11;
8/1Q3p1k/6pp/5rn1/8/6PP/4K3/8 w - - fmvn 62; sm "b7g2"; c3 "Result 1/2-1/2"; c2 "Junior 13.3 - Komodo 1324"; c1 "Round 8"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.1;
2k3r1/2p3Pp/1pn2p2/p1b2N2/5B2/3R3P/5PPK/8 w - - fmvn 33; sm "d3g3"; c3 "Result 0-1"; c2 "Chiron 2 - Komodo 1324"; c1 "Round 11"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.29;
r2q1rk1/ppp1bppp/4p1b1/3PP3/5P2/2N3N1/Pn3BPP/R2Q1RK1 w - - fmvn 16; sm "d1d2"; c3 "Result 0-1"; c2 "Junior 13.3 - Stockfish 091114"; c1 "Round 13"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.19;
r2q1rk1/ppp1bppp/4p1b1/3PP3/2n2P2/2N3N1/P2Q1BPP/R4RK1 w - - fmvn 17; sm "d2e2"; c3 "Result 0-1"; c2 "Junior 13.3 - Stockfish 091114"; c1 "Round 13"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.21;
r2q1rk1/ppp1bppp/1n2p1b1/3PP3/5P2/2N3N1/P3QBPP/R4RK1 w - - fmvn 18; sm "a1d1"; c3 "Result 0-1"; c2 "Junior 13.3 - Stockfish 091114"; c1 "Round 13"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.2;
6k1/pp3p2/5p2/3q2pp/8/P1Br1NPP/1P2QPK1/8 w - g6 fmvn 34; sm "c3f6"; c3 "Result 1/2-1/2"; c2 "Chiron 2 - Protector 1.8b2"; c1 "Round 17"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.15;
r1b1r3/pp3pbk/2pp3p/q7/4PNn1/1PNQBpP1/P1P2P1P/1R3R1K w - - fmvn 19; sm "e3d4"; c3 "Result 1/2-1/2"; c2 "Critter 1.6a - Junior 13.3"; c1 "Round 17"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.3;
r1b1r3/pp3pbk/2pp3p/q3n3/3BPN2/1PNQ1pP1/P1P2P1P/1R3R1K w - - fmvn 20; sm "d3e3"; c3 "Result 1/2-1/2"; c2 "Critter 1.6a - Junior 13.3"; c1 "Round 17"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.26;
r1b1r3/pp3p1k/2pp1b1p/q3n3/3BPN2/1PN1QpP1/P1P2P1P/1R3R1K w - - fmvn 21; sm "b3b4"; c3 "Result 1/2-1/2"; c2 "Critter 1.6a - Junior 13.3"; c1 "Round 17"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.23;
r1bqr3/pp3p1k/2pp1b1p/4n3/1P1BPN2/2N1QpP1/P1P2P1P/1R3R1K w - - fmvn 22; sm "d4e5"; c3 "Result 1/2-1/2"; c2 "Critter 1.6a - Junior 13.3"; c1 "Round 17"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.17;
rnb1k2r/pp1p1ppp/1q1b1n2/2pP4/5B2/1PP2P2/P5PP/RN1QKBNR w KQkq - fmvn 10; sm "f4e3"; c3 "Result 0-1"; c2 "Junior 13.3 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.28;
rnb2rk1/pp1p1ppp/1q1b1n2/2pP4/8/1PP1BP2/P5PP/RN1QKBNR w KQ - fmvn 11; sm "b1a3"; c3 "Result 0-1"; c2 "Junior 13.3 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.24;
rnb1r1k1/pp1p1ppp/1q1b1n2/2pP4/8/NPP1BP2/P5PP/R2QKBNR w KQ - fmvn 12; sm "a3c4"; c3 "Result 0-1"; c2 "Junior 13.3 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.11;
rnb1r1k1/ppqp1ppp/3b1n2/2pP4/2N5/1PP1BP2/P5PP/R2QKBNR w KQ - fmvn 13; sm "e1f2"; c3 "Result 0-1"; c2 "Junior 13.3 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.31;
rnb1rbk1/ppqp1ppp/5n2/2pP4/2N5/1PP1BP2/P4KPP/R2Q1BNR w - - fmvn 14; sm "a2a4"; c3 "Result 0-1"; c2 "Junior 13.3 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.18;
r3rbk1/ppq2pp1/3p1n1p/2pPnb1P/P1N2B2/1PP2P2/4NKP1/R2Q1B1R w - - fmvn 19; sm "e2g3"; c3 "Result 0-1"; c2 "Junior 13.3 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.27;
r3rbk1/ppq2ppb/3p1n1p/2pPn2P/P1N2B2/1PP2PN1/5KP1/R2Q1B1R w - - fmvn 20; sm "c4e3"; c3 "Result 0-1"; c2 "Junior 13.3 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.27;
r3rbk1/ppq2ppb/3p1n1p/3Pn2P/P1p2B2/1PP1NPN1/5KP1/R2Q1B1R w - - fmvn 21; sm "b3c4"; c3 "Result 0-1"; c2 "Junior 13.3 - Gull 3"; c1 "Round 18"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.17;
8/5k2/3p1n1p/p1p1qP2/PrPpP2P/5BPK/2Q5/5R2 w - - fmvn 44; sm "h3g2"; c3 "Result 1/2-1/2"; c2 "Chiron 2 - Stockfish 091114"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.26;
8/3n1k2/3p3p/p1p1qP2/PrPpP2P/5BP1/2Q3K1/5R2 w - - fmvn 45; sm "f1f2"; c3 "Result 1/2-1/2"; c2 "Chiron 2 - Stockfish 091114"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.26;
r3q1k1/1p3pp1/p2b1n2/3Pr2p/1P2p1b1/P1N1P3/1B1QBPPP/2R2RK1 w - - fmvn 19; sm "e2c4"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.12;
r3q1k1/1p3pp1/p2b1n2/3Pr3/1PB1p1bp/P1N1P3/1B1Q1PPP/2R2RK1 w - - fmvn 20; sm "c3e2"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.33;
r3q1k1/1p3pp1/p2b1n2/3P1r2/1PB1p1bp/P3P3/1B1QNPPP/2R2RK1 w - - fmvn 21; sm "g1h1"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.37;
r3q1k1/1p3pp1/p2b1n2/3P1r2/1PB1p1b1/P3P2p/1B1QNPPP/2R2R1K w - - fmvn 22; sm "g2g3"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.37;
3rq1k1/1p3pp1/p2b1n2/3P1r2/1PB1p1b1/P3P1Pp/1B1QNP1P/2R2R1K w - - fmvn 23; sm "h1g1"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.25;
1b1rq1k1/1p3pp1/p4n2/3P1r2/1PB1p1b1/P3P1Pp/1B1QNP1P/2R2RK1 w - - fmvn 24; sm "b2a1"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.25;
3r2k1/1p1q1pp1/p2b3r/3P4/1P1Np1b1/PB2P1Pp/2RQ1P1P/4R1K1 w - - fmvn 32; sm "e1c1"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.46;
3r2k1/1p1q1pp1/p2b4/3P3r/1P1Np1b1/PB2P1Pp/2RQ1P1P/2R3K1 w - - fmvn 33; sm "c2c3"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.46;
3r2k1/1p1q1p2/p2b2p1/3P3r/1P1Np1b1/PBR1P1Pp/3Q1P1P/2R3K1 w - - fmvn 34; sm "c3c2"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.44;
3r2k1/1p2qp2/p2b2p1/3P3r/1P1Np1b1/PB2P1Pp/2RQ1P1P/2R3K1 w - - fmvn 35; sm "d2e1"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.42;
3r2k1/1p3p2/p2b1qp1/3P3r/1P1Np1b1/PB2P1Pp/2R2P1P/2R1Q1K1 w - - fmvn 36; sm "e1f1"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.42;
3r2k1/1p3p2/p2b1qp1/3Pr3/1P1Np1b1/PB2P1Pp/2R2P1P/2R2QK1 w - - fmvn 37; sm "c2d2"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.42;
3r2k1/1p1b1p2/p2b1qp1/3Pr3/1P1Np3/PB2P1Pp/3R1P1P/2R2QK1 w - - fmvn 38; sm "b3a2"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.44;
3r2k1/1p1brp2/p2b1qp1/3P4/1P1Np3/P3P1Pp/B2R1P1P/2R2QK1 w - - fmvn 39; sm "a2b1"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.39;
3r2k1/1p2rp2/p2b1qp1/3P4/1P1Np1b1/P3P1Pp/3R1P1P/1BR2QK1 w - - fmvn 40; sm "c1c3"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.4;
3rr1k1/1p3p2/p2b1qp1/3P4/1P1Np1b1/P1R1P1Pp/3R1P1P/1B3QK1 w - - fmvn 41; sm "c3c4"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.39;
4r1k1/1p1b1p2/p2b2p1/3Pq3/1PBNp3/P3P1Pp/3R1P1P/5QK1 w - - fmvn 45; sm "b4b5"; c3 "Result 1-0"; c2 "Houdini 4 - Junior 13.3"; c1 "Round 19"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.38;
r3r3/p2n1k2/1p1q1n1Q/2pp4/3P4/1P2PP2/PB2R1PP/2R3K1 w - - fmvn 24; sm "g2g3"; c3 "Result 0-1"; c2 "Stockfish 091114 - Junior 13.3"; c1 "Round 20"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.19;
r6r/p2n1k2/1p1q1n1Q/2pp4/3P4/1P2PPP1/PB2R2P/2R3K1 w - - fmvn 25; sm "h6g5"; c3 "Result 0-1"; c2 "Stockfish 091114 - Junior 13.3"; c1 "Round 20"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.24;
2kr3r/pbpnqp2/1p1ppn2/6pp/P1PP4/2PBP1B1/3N1PPP/R2Q1RK1 w - - fmvn 14; sm "h2h4"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Komodo 1324"; c1 "Round 21"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.17;
r4rk1/1bpqbppp/p1np1n2/1p2p3/P3P3/1B1PBN2/1PP1QPPP/RN3RK1 w - - fmvn 11; sm "h2h3"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.16;
r4rk1/1bpqbppp/p2p1n2/np2p3/P3P3/1B1PBN1P/1PP1QPP1/RN3RK1 w - - fmvn 12; sm "b3a2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.17;
r4rk1/1b1qbppp/p2p1n2/npp1p3/P3P3/3PBN1P/BPP1QPP1/RN3RK1 w - c6 fmvn 13; sm "e3g5"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.18;
r4rk1/1b1q1pp1/3p1b1p/p1p1p3/PpB1P3/3P1N1P/1PP1QPP1/R4RK1 w - - fmvn 18; sm "b2b3"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.1;
r4rk1/3q1pp1/2bp1b1p/p1p1p3/PpB1P3/1P1P1N1P/2P1QPP1/R4RK1 w - - fmvn 19; sm "f3h2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.12;
r4r2/3q1ppk/2bp1b1p/p1p1p3/PpB1P3/1P1P3P/2P1QPPN/R4RK1 w - - fmvn 20; sm "h2g4"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.12;
r4r2/3q3k/2bp2pp/p1p1ppb1/PpB1P3/1P1PN2P/2P1QPPK/R4R2 w - f6 fmvn 23; sm "e3d5"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.13;
4r3/2r4k/3p3b/p1pBp1p1/PpP1PqPp/1P1P1p1P/2Q2P2/R4RK1 w - - fmvn 105; sm "a1d1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
4r3/2r3k1/3p3b/p1pBp1p1/PpP1PqPp/1P1P1p1P/2Q2P2/3R1RK1 w - - fmvn 106; sm "d1b1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.28;
4r3/4r1k1/3p3b/p1pBp1p1/PpP1PqPp/1P1P1p1P/2Q2P2/1R3RK1 w - - fmvn 107; sm "f1e1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.24;
5r2/4r1k1/3p3b/p1pBp1p1/PpP1PqPp/1P1P1p1P/2Q2P2/1R2R1K1 w - - fmvn 108; sm "b1d1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
5r1k/4r3/3p3b/p1pBp1p1/PpP1PqPp/1P1P1p1P/2Q2P2/3RR1K1 w - - fmvn 109; sm "e1e3"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
7k/4r3/3p1r1b/p1pBp1p1/PpP1PqPp/1P1PRp1P/2Q2P2/3R2K1 w - - fmvn 110; sm "g1h1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.24;
8/4r1k1/3p1r1b/p1pBp1p1/PpP1PqPp/1P1PRp1P/2Q2P2/3R3K w - - fmvn 111; sm "c2d2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.26;
4r3/6k1/3p1r1b/p1pBp1p1/PpP1PqPp/1P1PRp1P/3Q1P2/3R3K w - - fmvn 112; sm "d2e1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.24;
4rk2/8/3p1r1b/p1pBp1p1/PpP1PqPp/1P1PRp1P/5P2/3RQ2K w - - fmvn 113; sm "d5c6"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.21;
3r1k2/8/2Bp1r1b/p1p1p1p1/PpP1PqPp/1P1PRp1P/5P2/3RQ2K w - - fmvn 114; sm "h1g1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
3r4/4k3/2Bp1r1b/p1p1p1p1/PpP1PqPp/1P1PRp1P/5P2/3RQ1K1 w - - fmvn 115; sm "c6d5"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
3r1r2/4k3/3p3b/p1pBp1p1/PpP1PqPp/1P1PRp1P/5P2/3RQ1K1 w - - fmvn 116; sm "d1d2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.26;
3r1r2/4k1b1/3p4/p1pBp1p1/PpP1PqPp/1P1PRp1P/3R1P2/4Q1K1 w - - fmvn 117; sm "e1d1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.27;
3r4/4k1b1/3p1r2/p1pBp1p1/PpP1PqPp/1P1PRp1P/3R1P2/3Q2K1 w - - fmvn 118; sm "d1c2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.21;
3r4/3k2b1/3p1r2/p1pBp1p1/PpP1PqPp/1P1PRp1P/2QR1P2/6K1 w - - fmvn 119; sm "d2d1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.27;
3r1b2/3k4/3p1r2/p1pBp1p1/PpP1PqPp/1P1PRp1P/2Q2P2/3R2K1 w - - fmvn 120; sm "d5g8"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.2;
3r1bB1/3k4/3p3r/p1p1p1p1/PpP1PqPp/1P1PRp1P/2Q2P2/3R2K1 w - - fmvn 121; sm "d1c1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.25;
3r2B1/3k2b1/3p3r/p1p1p1p1/PpP1PqPp/1P1PRp1P/2Q2P2/2R3K1 w - - fmvn 122; sm "c2d1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
5rB1/3k2b1/3p3r/p1p1p1p1/PpP1PqPp/1P1PRp1P/5P2/2RQ2K1 w - - fmvn 123; sm "g8d5"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.24;
5r2/3k2b1/3p1r2/p1pBp1p1/PpP1PqPp/1P1PRp1P/5P2/2RQ2K1 w - - fmvn 124; sm "g1h1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
2r5/3k2b1/3p1r2/p1pBp1p1/PpP1PqPp/1P1PRp1P/5P2/2RQ3K w - - fmvn 125; sm "c1c2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
2r5/3k4/3p1r1b/p1pBp1p1/PpP1PqPp/1P1PRp1P/2R2P2/3Q3K w - - fmvn 126; sm "d1g1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.23;
5r2/3k4/3p1r1b/p1pBp1p1/PpP1PqPp/1P1PRp1P/2R2P2/6QK w - - fmvn 127; sm "g1e1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.21;
5r2/3k2b1/3p1r2/p1pBp1p1/PpP1PqPp/1P1PRp1P/2R2P2/4Q2K w - - fmvn 128; sm "c2d2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.21;
5r2/4k1b1/3p1r2/p1pBp1p1/PpP1PqPp/1P1PRp1P/3R1P2/4Q2K w - - fmvn 129; sm "d2d1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.21;
5r2/4k3/3p1r1b/p1pBp1p1/PpP1PqPp/1P1PRp1P/5P2/3RQ2K w - - fmvn 130; sm "e1f1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.23;
5r2/3k4/3p1qrb/p1pBp1p1/PpP1P1Pp/1P1PRp1P/5P2/3R1K1Q w - - fmvn 136; sm "f1e1"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.21;
5r2/3k4/3p2rb/p1pBp1p1/PpP1PqPp/1P1PRp1P/5P2/3RK2Q w - - fmvn 137; sm "e1d2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.19;
5r2/3k2b1/3p2r1/p1pBp1p1/PpP1PqPp/1P1PRp1P/3K1P2/3R3Q w - - fmvn 138; sm "d2c2"; c3 "Result 1/2-1/2"; c2 "Houdini 4 - Gull 3"; c1 "Round 24"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.17;
2rr2k1/1q2bppp/p1bppn2/1p6/3BPP2/P1N2BQ1/1PP1R1PP/3R2K1 w - - fmvn 18; sm "h2h4"; c3 "Result 0-1"; c2 "Houdini 4 - Stockfish 091114"; c1 "Round 25"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.11;
2rr3k/1q2bppp/p1bppn2/1p6/3BPP1P/P1N2BQ1/1PP1R1P1/3R2K1 w - - fmvn 19; sm "b2b3"; c3 "Result 0-1"; c2 "Houdini 4 - Stockfish 091114"; c1 "Round 25"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.14;
2rr3k/1q2bppp/2bppn2/pp6/3BPP1P/PPN2BQ1/2P1R1P1/3R2K1 w - - fmvn 20; sm "b3b4"; c3 "Result 0-1"; c2 "Houdini 4 - Stockfish 091114"; c1 "Round 25"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.12;
r4rk1/ppb2pp1/1np2np1/q7/4P1P1/1Q2BP1P/PP2N1B1/3RK2R w K - fmvn 21; sm "e1f2"; c3 "Result 1/2-1/2"; c2 "Chiron 2 - Komodo 1324"; c1 "Round 25"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.22;
r2r2k1/ppb2pp1/1np2np1/q7/4P1P1/1Q2BP1P/PP2NKB1/3R3R w - - fmvn 22; sm "g4g5"; c3 "Result 1/2-1/2"; c2 "Chiron 2 - Komodo 1324"; c1 "Round 25"; c0 "Event TCEC Season 7 - Stage 3"; c5 0.19;
I have just started a 1 minute per position 20 core run to see what these look like. This might be just what I needed although I will probably want to cull the list a bit to control 1 cpu runtime.
First three positions Crafty agrees with best move, and more notably, there is pretty constant agreement from depth 17 to depth 27, which might mean these are pretty "stable". I'd prefer to see something more changeable which is the thing that wrecks SMP performance. Will see how the rest goes over the next 2 hours...
bob wrote:I have started re-running my SMP tests since the Intel compiler provides such a nice NPS improvement (multiple threads only, single thread seems to be the same as before).
Here's my first 20 core test runs (same 24 positions, each run 4 times). All results are geometric mean for a 24 position run.
run 1 run 2 run 3 run 4 avg
speedup 15.0 15.0 15.0 15.3 15.1
nps 16.3 16.4 16.3 16.3 16.3
Net results, NPS scaling improved from 14.7 or so to 16.3x. I know part of the bottleneck is 64GB of hash and TLB thrashing, so I do have plans to try mmap() and using the rather clumsy 1gb huge pages. But I have not done this as of yet, still using the automatic 2mb pages (transparent huge pages in current linux kernels). Speedup (geomean) went from 12.45 to 16.3 due to the intel issue plus a few further SMP refinements. BTW the 20 cpu search looked at about 5% more nodes (average) compared to the 1 cpu test, so overhead is pretty well managed for the moment.
The speedup has really captured my attention, because it is right at the theoretical max (15.1 with max of 16.3 based on NPS numbers). I am fixing to create another set of test positions just to be sure that these positions don't somehow happen to artificially inflate the speedup results. I am not sure exactly how I am going to choose this test set. I was leaning toward either (a) taking a set of positions from a long time control game played on ICC, or else (b) a random set of positions extracted from GM games (the way I extract starting positions for cluster testing).
A much larger set of positions would be better, statistically, but not so good practically as the tests would take forever to run. More on this later....
These are exceptionally high speedup numbers, surpassing even your DTS results. I have never seen such to 16 or 20 cores. I am tempted to think something is amiss.
bob wrote:I have started re-running my SMP tests since the Intel compiler provides such a nice NPS improvement (multiple threads only, single thread seems to be the same as before).
Here's my first 20 core test runs (same 24 positions, each run 4 times). All results are geometric mean for a 24 position run.
run 1 run 2 run 3 run 4 avg
speedup 15.0 15.0 15.0 15.3 15.1
nps 16.3 16.4 16.3 16.3 16.3
Net results, NPS scaling improved from 14.7 or so to 16.3x. I know part of the bottleneck is 64GB of hash and TLB thrashing, so I do have plans to try mmap() and using the rather clumsy 1gb huge pages. But I have not done this as of yet, still using the automatic 2mb pages (transparent huge pages in current linux kernels). Speedup (geomean) went from 12.45 to 16.3 due to the intel issue plus a few further SMP refinements. BTW the 20 cpu search looked at about 5% more nodes (average) compared to the 1 cpu test, so overhead is pretty well managed for the moment.
The speedup has really captured my attention, because it is right at the theoretical max (15.1 with max of 16.3 based on NPS numbers). I am fixing to create another set of test positions just to be sure that these positions don't somehow happen to artificially inflate the speedup results. I am not sure exactly how I am going to choose this test set. I was leaning toward either (a) taking a set of positions from a long time control game played on ICC, or else (b) a random set of positions extracted from GM games (the way I extract starting positions for cluster testing).
A much larger set of positions would be better, statistically, but not so good practically as the tests would take forever to run. More on this later....
These are exceptionally high speedup numbers, surpassing even your DTS results. I have never seen such to 16 or 20 cores. I am tempted to think something is amiss.
That is based on 4 runs of 24 positions. All I have so far is the 1 cpu run which didn't change at all, and these 4 20 cpu runs. Yes they are pretty high, but as to whether something is amiss or not I do not know. I have run literally thousands of positions comparing results with 1 to N cpus. there is always variability, but there have been no instances of bogus moves being chosen or bogus scores being chosen. And Crafty has been playing on ICC with this code for months. Only bug I have found in last 30 days is a statistical gathering bug counting the number of reductions that were done, overstepping an array bound. But that bug was in the search, period, not a SMP issue.
I am certainly looking at the code carefully and testing whatever I can. As of right now, it looks rock-solid to me. It might be that those 24 positions are very favorable to a search by Crafty. I have already run the set posted in this thread using 20 cores. I'm going to pick 60 of them and run to fixed depth with 1 and 20 cores to get a feel for whether the positions might be the issue. I'd rather run 1000 positions but that is simply not practical at these speeds...
