Laskos wrote: ↑Fri Sep 13, 2019 2:07 pm
Lc0 is very strong about positional openings. The best net to LTC and strong RTX hardware in games and performing the best in my opening test-suite is JHorthos 320x24b bignet J13B.2-136 (built upon late T40 nets). It practically understands in 1 million nodes search all human opening theory built in 100 years when only the positional things are considered. So, I decided to see what it thinks
about the starting position of chess in very long searches, well surpassing the 16GB capacity of my RAM, using 128GB SSD cache. And well surpassing the known opening knowledge (
presumably).
GPU: RTX 2070
The batch file can look like this:
Code: Select all
setoption name Backend value cudnn-fp16
setoption name MinibatchSize value 400
setoption name NNCacheSize value 10000000
setoption name WeightsFile value .\J13B.2-136
setoption name ScoreType value win_percentage
setoption name MultiPV value 20
setoption name VerboseMoveStats value true
go nodes 250000000 searchmoves d2d4 g1f3 c2c4
First, a fast 1 million search from the starting position. This is already enough to see practically all human 1-ply theory of the position (there is no tactics). The quantity to look at is "
Q", the performance from White POW here is (1+Q) / 2.
Code: Select all
info string g2g4 (378 ) N: 286 (+ 0) (P: 1.43%) (Q: -0.40008) (D: 0.282)
(U: 0.60112) (Q+U: 0.20104) (V: -0.4436)
info string f2f3 (346 ) N: 524 (+ 0) (P: 1.72%) (Q: -0.22862) (D: 0.364)
(U: 0.39472) (Q+U: 0.16610) (V: -0.2382)
info string g1h3 (161 ) N: 802 (+ 0) (P: 2.05%) (Q: -0.15754) (D: 0.414)
(U: 0.30835) (Q+U: 0.15081) (V: -0.1577)
info string h2h4 (403 ) N: 876 (+ 0) (P: 2.07%) (Q: -0.13819) (D: 0.408)
(U: 0.28454) (Q+U: 0.14635) (V: -0.1346)
info string b1a3 (34 ) N: 959 (+ 0) (P: 2.17%) (Q: -0.12870) (D: 0.410)
(U: 0.27319) (Q+U: 0.14449) (V: -0.1111)
info string b2b4 (234 ) N: 1080 (+ 0) (P: 2.29%) (Q: -0.11460) (D: 0.413)
(U: 0.25634) (Q+U: 0.14174) (V: -0.0917)
info string f2f4 (351 ) N: 1244 (+ 0) (P: 2.30%) (Q: -0.08744) (D: 0.407)
(U: 0.22316) (Q+U: 0.13572) (V: -0.1041)
info string a2a4 (207 ) N: 1693 (+ 0) (P: 2.57%) (Q: -0.05450) (D: 0.450)
(U: 0.18322) (Q+U: 0.12872) (V: -0.0337)
info string a2a3 (204 ) N: 2642 (+ 0) (P: 2.85%) (Q: -0.01052) (D: 0.450)
(U: 0.13013) (Q+U: 0.11961) (V: -0.0056)
info string d2d3 (288 ) N: 2737 (+ 0) (P: 2.77%) (Q: -0.00428) (D: 0.449)
(U: 0.12238) (Q+U: 0.11810) (V: -0.0129)
info string h2h3 (400 ) N: 2750 (+ 0) (P: 2.91%) (Q: -0.00897) (D: 0.464)
(U: 0.12797) (Q+U: 0.11900) (V: -0.0022)
info string b2b3 (230 ) N: 2991 (+ 0) (P: 2.95%) (Q: -0.00178) (D: 0.422)
(U: 0.11923) (Q+U: 0.11745) (V: -0.0061)
info string b1c3 (36 ) N: 4315 (+ 0) (P: 3.09%) (Q: 0.02509) (D: 0.437)
(U: 0.08652) (Q+U: 0.11161) (V: 0.0120)
info string c2c3 (259 ) N: 5604 (+ 0) (P: 3.62%) (Q: 0.03195) (D: 0.459)
(U: 0.07806) (Q+U: 0.11001) (V: 0.0444)
info string g2g3 (374 ) N: 10261 (+ 0) (P: 5.75%) (Q: 0.04036) (D: 0.464)
(U: 0.06775) (Q+U: 0.10811) (V: 0.0553)
info string e2e3 (317 ) N: 14263 (+ 0) (P: 5.98%) (Q: 0.05455) (D: 0.468)
(U: 0.05062) (Q+U: 0.10517) (V: 0.0571)
info string c2c4 (264 ) N: 28066 (+ 0) (P: 7.63%) (Q: 0.06911) (D: 0.451)
(U: 0.03283) (Q+U: 0.10194) (V: 0.0737)
info string g1f3 (159 ) N: 64502 (+ 0) (P: 10.01%) (Q: 0.08059) (D: 0.481)
(U: 0.01874) (Q+U: 0.09933) (V: 0.0848)
info string d2d4 (293 ) N: 439604 (+ 0) (P: 16.58%) (Q: 0.09024) (D: 0.490)
(U: 0.00456) (Q+U: 0.09480) (V: 0.0909)
info string e2e4 (322 ) N: 578066 (+850) (P: 19.27%) (Q: 0.09088) (D: 0.497)
(U: 0.00402) (Q+U: 0.09490) (V: 0.1079)
bestmove e2e4 ponder e7e5
So, the lead candidates are e2e4 with
54.54% performance and d2d4 with
54.51% performance, an insignificant difference. And it confirms the human opening theory.
But now the things become serious. Longish (many hours) 250+ million nodes search with this bignet. The difference widens dramatically, so much that d2d4 is hardly explored anymore:
Code: Select all
info string g2g4 (378 ) N: 9986 (+ 0) (P: 1.43%) (Q: -0.39817) (D: 0.275)
(U: 0.50416) (Q+U: 0.10599) (V: -.----)
info string f2f3 (346 ) N: 17137 (+ 0) (P: 1.72%) (Q: -0.24788) (D: 0.349)
(U: 0.35290) (Q+U: 0.10502) (V: -.----)
info string g1h3 (161 ) N: 26407 (+ 0) (P: 2.05%) (Q: -0.16915) (D: 0.403)
(U: 0.27364) (Q+U: 0.10449) (V: -.----)
info string h2h4 (403 ) N: 31379 (+ 0) (P: 2.07%) (Q: -0.12788) (D: 0.409)
(U: 0.23209) (Q+U: 0.10421) (V: -.----)
info string b1a3 (34 ) N: 31863 (+ 0) (P: 2.17%) (Q: -0.13594) (D: 0.390)
(U: 0.24021) (Q+U: 0.10428) (V: -.----)
info string b2b4 (234 ) N: 39780 (+ 0) (P: 2.29%) (Q: -0.09927) (D: 0.433)
(U: 0.20329) (Q+U: 0.10402) (V: -.----)
info string f2f4 (351 ) N: 41298 (+ 0) (P: 2.30%) (Q: -0.09236) (D: 0.413)
(U: 0.19634) (Q+U: 0.10398) (V: -.----)
info string a2a4 (207 ) N: 58013 (+ 0) (P: 2.57%) (Q: -0.05243) (D: 0.457)
(U: 0.15614) (Q+U: 0.10372) (V: -.----)
info string d2d3 (288 ) N: 83276 (+ 0) (P: 2.77%) (Q: -0.01397) (D: 0.455)
(U: 0.11743) (Q+U: 0.10347) (V: -.----)
info string a2a3 (204 ) N: 85754 (+ 0) (P: 2.85%) (Q: -0.01359) (D: 0.447)
(U: 0.11705) (Q+U: 0.10346) (V: -.----)
info string h2h3 (400 ) N: 97278 (+ 0) (P: 2.91%) (Q: -0.00223) (D: 0.454)
(U: 0.10562) (Q+U: 0.10339) (V: -.----)
info string b2b3 (230 ) N: 98455 (+ 0) (P: 2.95%) (Q: -0.00236) (D: 0.442)
(U: 0.10575) (Q+U: 0.10339) (V: -.----)
info string b1c3 (36 ) N: 122110 (+ 0) (P: 3.09%) (Q: 0.01402) (D: 0.437)
(U: 0.08925) (Q+U: 0.10328) (V: -.----)
info string c2c3 (259 ) N: 201218 (+ 0) (P: 3.62%) (Q: 0.03965) (D: 0.463)
(U: 0.06346) (Q+U: 0.10310) (V: -.----)
info string g2g3 (374 ) N: 348785 (+ 0) (P: 5.75%) (Q: 0.04488) (D: 0.467)
(U: 0.05818) (Q+U: 0.10305) (V: -.----)
info string e2e3 (317 ) N: 413542 (+ 0) (P: 5.98%) (Q: 0.05205) (D: 0.475)
(U: 0.05095) (Q+U: 0.10300) (V: -.----)
info string c2c4 (264 ) N: 688398 (+ 0) (P: 7.63%) (Q: 0.06384) (D: 0.464)
(U: 0.03907) (Q+U: 0.10291) (V: -.----)
info string g1f3 (159 ) N: 1576963 (+ 0) (P: 10.01%) (Q: 0.08036) (D: 0.500)
(U: 0.02238) (Q+U: 0.10274) (V: -.----)
info string d2d4 (293 ) N: 5674011 (+ 0) (P: 16.58%) (Q: 0.08522) (D: 0.510)
(U: 0.01030) (Q+U: 0.09552) (V: -.----)
info string e2e4 (322 ) N: 248014778 (+121) (P: 19.27%) (Q: 0.10216) (D: 0.51
8) (U: 0.00027) (Q+U: 0.10243) (V: -.----)
bestmove e2e4 ponder e7e5
e2e4 performs at an
increasing 55.11%, while d2d4 performs at a
decreasing 54.26%. Observe that d2d4 is not that well explored, and things get even worse for d2d4 when it is further explored. I restricted the moves to be explored to d2d4 g1f3 and c2c4, the following candidates after e2e4. The most explored, as expected, was d2d4:
About 130 million nodes (hours of search again):
Code: Select all
info string c2c4 (264 ) N: 1444102 (+ 0) (P: 7.63%) (Q: 0.05985) (D: 0.477)
(U: 0.01254) (Q+U: 0.07239) (V: -.----)
info string g1f3 (159 ) N: 6657797 (+ 0) (P: 10.01%) (Q: 0.06870) (D: 0.520)
(U: 0.00357) (Q+U: 0.07227) (V: -.----)
info string d2d4 (293 ) N: 124278559 (+117) (P: 16.58%) (Q: 0.07200) (D: 0.48
8) (U: 0.00032) (Q+U: 0.07231) (V: -.----)
bestmove d2d4 ponder g8f6
The performance of d2d4
decreases further to
53.60% (while that previous of e2e4 increased to
55.11% in the previous very long search). To observe that all these values are VERY slowly moving, there are no any jumps like those with regular AB engines. So,
e4 performs about 40% better than d4, taking the draw as the baseline. All other opening moves are worse than both. The difference IS significant.
To have a glimpse of what Leela sees ahead, a speculative one following the PV:
info depth 36 seldepth 99 time 31466907 nodes 257660432 score cp 5510 hashfull 1
000 nps 8188 tbhits 0 multipv 1 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4
e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 b1c3 f8e7 h2h3 f5h4 f3h4 e7h4 f1d1 d8e8 g2g4
h7h5 f2f3 c8e6 c3e2 e6d5 g1g2 f7f6 e2f4 h5g4 h3g4 a8d8 c1e3 e8f7 e5e6 d5e6 f4e6
f7e6 f3f4 b7b6 g2f3 c6c5 a2a4 h8e8 a4a5 g7g6 c2c4 d8d6 d1d6 e6d6 e3d2 e8d8 d2c3
d6e6 a1h1 g6g5 f4f5 e6e7 f3e2 b6a5 b2b3 d8b8 h1h3 e7f7 c3a5 b8e8 h3e3 h4g3 a5c3
e8e3 e2e3 c7c6 e3d3 f7e7 d3c2 g3f2 c2d3 f2g3 c3d2
After 8 moves, it's still in main theory:
C67
Spanish Game, Berlin Defense, l'Hermet Variation, Berlin Wall Defense
[d]r1bk1b1r/ppp2ppp/2p5/4Pn2/8/5N2/PPP2PPP/RNB2RK1 w - - 0 9
It is played by many top players today. Its performance among 2700+ players from my database is:
Players average: 2761
Performance: 2815
White Win: 22.1%
Draw: 66.2%
Black Win: 11.7%
========================
Overall performance: 55.2%
Although the draw rate is very high (66.2% compared to 52.4% from the starting position), White Win/Loss ratio is among the highest (almost 2) of the opening repertoire. According to Lc0, Black cannot defend better against e2e4.
info depth 28 seldepth 75 time 38173609 nodes 132380459 score cp 5359 hashfull 1
000 nps 3467 tbhits 0 multipv 1 pv d2d4 g8f6 c2c4 e7e6 g2g3 f8b4 c1d2 b4e7 g1f3
e8g8 f1g2 d7d5 e1g1 c7c6 d1c2 b8d7 d2f4 b7b6 f1d1 c8a6 f3e5 a8c8 b1c3 a6c4 e5c4
d5c4 e2e4 b6b5 a2a4 a7a6 a4b5 a6b5 d4d5 c6d5 e4d5 e6e5 f4g5 e7c5 g2h3 h7h6 h3d7
h6g5 d7b5 c5d4 b5c6 f6e8 c3b5 e8d6 b5d4 e5d4 b2b3 c4b3 c2b3 c8b8 b3d3 b8b4 a1a4
b4a4 c6a4 d8f6 d3d4 f6d4 d1d4 f8b8 d4g4 f7f6 a4d7 g8f7 g4a4 f6f5 a4a6 f7e7 d7e6
It is soon out of general theory (Catalan Opening, General), but after 6 moves it's still played by many top players today, and it seems not that rare among top players (maybe a new trend?)
[d]rnbq1rk1/ppp1bppp/4pn2/3p4/2PP4/5NP1/PP1BPPBP/RN1QK2R w KQ d6 0 7
Its performance among 2700+ players from my database is:
Players average: 2752
Performance: 2775
White Win: 22.2%
Draw: 60.4%
Black Win: 17.4%
========================
Overall performance: 52.4%
The draw rate is somewhat lower here (still pretty high), but the White performance is not that good, too many Black wins. According to Lc0, White cannot do better with d2d4.
============================
Is it all due to some simple or stupid peculiarities of Lc0, this net, some tactics or something like that?