Something goes wrong with lc0 since yesterday?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Something goes wrong with lc0 since yesterday?

Post by Laskos »

yanquis1972 wrote: Wed Jul 04, 2018 11:20 pm selfplay elo on the testserver progressing well & as of 10025 is backed up with results against A/B engines per https://docs.google.com/spreadsheets/d/ ... 1519374103
2700+ CCRL

mainserver: selfplay elo regressed but 460, 461 & 464 all did well in matches at the previous link & here https://docs.google.com/spreadsheets/d/ ... edit#gid=0, at the same time 390 crushed 464 (+30 elo) in selfplay

3100+(??) CCRL
I am not sure. In my gauntlets ID10028 performs only 50-60 Elo points better than ID10017 against AB engines (but only 200 games each gauntlet), while their self-games Elo is erratic, and shows a total of 270 Elo points progress. My CCRL 40/4' rating of ID10028 would be 2600+ on GTX 1060 GPU. I don't like something in their setup, do they use gating for promoting and progression graph?
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: Something goes wrong with lc0 since yesterday?

Post by yanquis1972 »

i'm running 10k29 against rybka 1.0 & it's clearly 'strong' but not in the same league. i was thinking 2600 (-250 elo by ccrl 40/4) seems possible, but definitely not the 2800 i was rather optimistically hoping for.

fwiw after same number of games the original net was at 1700 elo (sourcing from the same site), & i don't think they've dropped the learning rate on the testserver yet either -- maybe if the next net doesn't show a significant increase? so there's definitely that.
Werewolf
Posts: 1796
Joined: Thu Sep 18, 2008 10:24 pm

Re: Something goes wrong with lc0 since yesterday?

Post by Werewolf »

yanquis1972 wrote: Thu Jul 05, 2018 4:22 am i'm running 10k29 against rybka 1.0 & it's clearly 'strong' but not in the same league. i was thinking 2600 (-250 elo by ccrl 40/4) seems possible, but definitely not the 2800 i was rather optimistically hoping for.

fwiw after same number of games the original net was at 1700 elo (sourcing from the same site), & i don't think they've dropped the learning rate on the testserver yet either -- maybe if the next net doesn't show a significant increase? so there's definitely that.
My own results from yesterday also point to about 2600 elo.

I see this morning it has climbed another 90 elo in self play, so as long as it keeps going up...
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: Something goes wrong with lc0 since yesterday?

Post by yanquis1972 »

leela was inches close to SF6, but i'm near the end of a match with 467 against it (possibly fatal flaw: at a new TC, 20/1' instead of 40/1' and...are you freaking kidding me?
[pgn] [Event "Lc0 id467 vs SF6-1cpu 5-manTB VAOv2 20/1'"] [Site "PC"] [Date "2018.07.05"] [Round "38"] [White "Lc0"] [Black "Stockfish-6-bmi2"] [Result "1-0"] [BlackElo "3314"] [ECO "C63"] [Opening "Spanish"] [Time "03:59:44"] [Variation "Schliemann, 4.Nc3 fxe4 5.Nxe4 Nf6 6.Qe2"] [WhiteElo "3295"] [TimeControl "20/60:20/60:20/60"] [Termination "adjudication"] [PlyCount "92"] [WhiteType "program"] [BlackType "program"] 1. e4 e5 2. Nf3 Nc6 3. Bb5 f5 4. Nc3 (Nc3 fxe4 Nxe4 Nf6 Qe2 d5 Ng3 Bd6 Nxe5 O-O Bxc6 bxc6 O-O Re8 f4 Ng4 d4 Qh4 h3 Qxg3 hxg4 Bxe5 dxe5) {+0.43/2 4} fxe4 (fxe4 Nxe4 Nf6 Nxf6 Qxf6 O-O Be7 Qe1 O-O d3 d6 Bg5 Qf7 Bxc6 bxc6 Qe4 Bd7 Bxe7 Qxe7 Rae1) {-0.41/19 3} 5. Nxe4 (Nxe4 Nf6 Qe2 d5 Ng3 Bd6 Nxe5 O-O Bxc6 bxc6 O-O Re8 d4 c5 c3 cxd4 cxd4 c5 Bg5 cxd4 f4 Qb6 Qf3 d3 Kh1) {+0.36/2 2} Nf6 (Nf6 Nxf6 Qxf6 Qe2 Be7 Bxc6 bxc6 d4 Qe6 Qxe5 d6 Qxe6 Bxe6 Bg5 Rb8 b3 Bxg5 Nxg5 Bf5 O-O Kd7 Rfe1 Bxc2 Ne6) {-0.30/20 2} 6. Qe2 (Qe2 d5 Ng3 Bd6 Nxe5 O-O Bxc6 bxc6 O-O Re8 d4 c5 Bg5 cxd4 f4 Rb8 Qf3 Rxb2 Nh5 Be7 Rae1 Rxc2 f5 Rc3 Qd1) {+0.27/2 4} Qe7 (Qe7 d3 Nd8 Nxf6 gxf6 d4 e4 Nd2 f5 Qh5 Nf7 Bc4 Rg8 Qxh7 Rxg2 Qxf5 c6 Bxf7 Qxf7 Qe5 Qe7 Nxe4 d5 Nd6 Kd8 Qxe7 Bxe7 Bf4 Bxd6 Bxd6 Bf5) {-0.43/19 2} 7. O-O (O-O d5 Ng3 e4 Nd4 Bd7 Bxc6 bxc6 d3 c5 Ndf5 Qe6 Bg5 O-O-O dxe4 Re8 f3 h5 Rae1 Nh7 Bd2 g6 Nh4 dxe4) {+0.47/2 3} d5 (d5 Nc3 e4 Nd4 Bd7 Bxc6 bxc6 d3 c5 Ndb5 Bxb5 Nxb5 c6 Nc3 exd3 Qxd3 c4 Qf3 Qf7 Re1 Be7 Bf4 O-O b3 Bb4 bxc4 dxc4 h3) {-0.59/21 5} 8. Ng3 (Ng3 e4 Nd4 Qc5 c3 Bd6 d3 O-O dxe4 Nxd4 cxd4 Qxd4 Be3 Qe5 f4 Qe7 e5 Ng4 Rf3 Bb4 a3 Nxe3 axb4 Nf5 Qd2 c6 Bd3 Nd4) {+0.37/2 2} e4 (e4 Nd4 Bg4 f3 exf3 Qxe7 Kxe7 Nxc6 bxc6 Bxc6 Rb8 b3 Kf7 Bb2 Bc5 d4 Bd6 gxf3 Bh3 Rfe1 Rb6 Ba4 h6 Bc3 Rbb8) {-0.41/19 2} 9. Nd4 (Nd4 Qc5 c3 Bd6 d3 O-O Bxc6 bxc6 dxe4 dxe4 Be3 Qd5 c4 Qe5 Nxc6 Qe8 Nxa7 Bg4 Qc2 h5 Nb5 h4 Nxd6 cxd6) {+0.38/2 1} Qc5 (Qc5 c3 Bd6 d3 O-O dxe4 Bxg3 hxg3 Nxd4 cxd4 Qxd4 exd5 Bg4 Qd3 Qxd3 Bxd3 Nxd5 Re1 Rae8 Bd2 Bh5 Rac1 c6 f3 Bg6 Bc4 Bf7 Bd3) {-0.52/22 7} 10. c3 (c3 Bd6 d3 O-O Bxc6 bxc6 dxe4 Qb6 Re1 Ba6 Qc2 Rae8 Be3 c5 Ndf5 dxe4 b4 Bd3 Qb3 Kh8 bxc5 Bxc5 Bxc5) {+0.37/2 2} Bd6 (Bd6 d3 O-O dxe4 Nxd4 cxd4 Qxd4 Rd1 Qe5 exd5 a6 Qxe5 Bxe5 Bc4 Kh8 Bg5 Bd7 Rab1 Rad8 f3 h6 Be3 Rfe8 Bd4 Bxg3 hxg3 Bf5 Rbc1 Nxd5) {-0.34/22 3} 11. d3 (d3 O-O Bxc6 bxc6 dxe4 Qb6 Re1 Ba6 Qc2 Rae8 Be3 c5 Ndf5 dxe4 b4 Bd3 Qb3 Kh8 Nxd6 cxd6 bxc5 dxc5) {+0.38/2 1} O-O (O-O dxe4 Nxd4 cxd4 Qxd4 Rd1 Qb6 Be3 Bg4 f3 Bc5 Bxc5 Qxc5 Kh1 c6 e5 Qxb5 Qxb5 cxb5 exf6 Be6 Nh5 g6 Nf4 Rxf6 Nxd5 Bxd5 Rxd5 a6 Rd7 Re8) {-0.22/23 3} 12. dxe4 (dxe4 Nxd4 cxd4 Qxd4 Be3 Qe5 f4 Qe7 e5 Ng4 Rf3 Bb4 Bd4 c5 a3 Ba5 Bc3 Bb6 h3 Nh6 Raf1 c4 Kh1 Bg4 hxg4 Qh4) {+0.62/2 6} Nxd4 (Nxd4 cxd4 Qxd4 Rd1 Qb6 Be3 Bg4 f3 Bc5 Bxc5 Qxc5 Kh1 c6 e5 Qxb5 Qxb5 cxb5 exf6 Be6 fxg7 Kxg7 Rd4 Kh6 Re1 Rfe8 Rh4 Kg6 Ne2 Bf7 Kg1 h6 Kf2 Rac8 Nd4 Rxe1 Kxe1) {-0.25/22 2} 13. cxd4 (cxd4 Qxd4 Be3 Qe5 f4 Qe7 e5 Ng4 Rf3 Bb4 Bd4 c5 a3 Ba5 Bc3 Bb6 h3 Nh6 Rd1 Be6 Kh2 Rad8 Bd2 c4 Bb4 Bc5 Bxc5 Qxc5) {+0.56/2 0} Qxd4 (Qxd4 Rd1 Qb6 Be3 Bg4 f3 Bc5 Bxc5 Qxc5 Kh1 c6 e5 Nh5 Nxh5 Bxh5 Bd3 Rae8 Rac1 Qe7 Re1 Qe6 Qc2 Bg6 Bxg6 Qxg6 Qxg6 hxg6 e6 Re7) {-0.19/22 2} 14. Be3 (Be3 Qe5 f4 Qe7 e5 Ng4 Rf3 Bb4 Bd4 c5 a3 Ba5 Bc3 Bb6 h3 Nh6 Rd1 Be6 f5 Nxf5 Nxf5 Bxf5 Rxd5 Be6 Rd6 Rxf3 Qxf3 c4 Kh2 Rf8 Qe4 Qf7 Bd4 Bc7) {+0.59/2 1} Qe5 (Qe5 f4 Qe7 e5 Ng4 exd6 Qxe3 Qxe3 Nxe3 d7 Nxf1 Rxf1 c6 dxc8=Q Raxc8 Bd3 c5 Ne2 Rce8 b3 g6 Kf2 b6 Rc1 Re7 Rd1 d4 g3 Re3 Bc4 Kg7 Rc1) {-0.31/23 2} 15. f4 (f4 Qe7 e5 Ng4 Rf3 Bb4 Bd4 c5 a3 Ba5 Bc3 Bb6 h3 Nh6 Rd1 Be6 f5 Nxf5 Nxf5 Bxf5 Rxd5 Be6 Rd6 Rxf3 Qxf3 c4 Kh2 Rf8 Qe4 Qf7 Bd4 Bc7 Qxb7) {+0.56/2 1} Qe7 (Qe7 e5 Ng4 exd6 Qxe3 Qxe3 Nxe3 d7 Nxf1 Rxf1 Bxd7 Bxd7 Kf7 f5 c6 Be6 Ke7 Ne2 Kd6 g4 Rae8 Kg2 g6 Nf4 gxf5 Bxf5 h6 Kf2 a6 Kg2 c5) {-0.44/24 4} 16. e5 (e5 Ng4 Rf3 Bb4 Bd4 c5 a3 Ba5 Bc3 Bb6 h3 Nh6 Rd1 Be6 f5 Nxf5 Nxf5 Bxf5 Rxd5 Be6 Rd6 Rxf3 Qxf3 c4 Kh2 Rf8 Qe4 Qf7 Bd4 Bc7 Qxb7) {+0.58/2 1} Ng4 (Ng4 exd6 Qxe3 Qxe3 Nxe3 d7 Nxf1 Rxf1 a6 dxc8=Q Raxc8 Bd7 Rcd8 Be6 Kh8 f5 g6 Ne2 d4 Nc1 d3 Rd1 gxf5 Nb3 Kg7 Nc5 d2 Nxb7 Rde8 Nc5 Kf6 Rxd2) {-0.41/25 3} 17. Rf3 (Rf3 Bb4 Bd4 c5 a3 Ba5 Bc3 Bb6 h3 Nh6 Rd1 Be6 f5 Nxf5 Nxf5 Bxf5 Rxd5 Be6 Rd6 Rxf3 Qxf3 c4 Kh2 Rf8 Qe4 Qf7 Bd4 Bc7 Qxb7) {+0.68/2 2} Bb4 (Bb4 a3 Ba5 Bd4 c5 b4 cxd4 bxa5 Ne3 h3 g5 fxg5 Rxf3 Qxf3 Qxg5 Qf6 Qxg3 Qd8 Kg7 Qf6 Kg8) {0.00/22 2} 18. Bd4 (Bd4 c5 a3 Ba5 Bc3 Bb6 h3 Nh6 Rd1 Be6 f5 Nxf5 Nxf5 Bxf5 Rxd5 Be6 Rd6 Rxf3 Qxf3 c4 Kh2 Rf8 Qe4 Qf7 Bd4 Bc7 Qxb7) {+0.70/2 2} c5 (c5 a3) {-0.12/23 5} 19. a3 (a3 cxd4 axb4 g5 fxg5 Rxf3 Qxf3 Be6 Qf4 Rf8 Qxd4 Qxg5 Be2 Ne3 Rxa7 h5 Bxh5 Nc2 Qd3 Qc1 Nf1 Ne3 Qg6 Kh8 Qh6 Kg8 Qxe6 Kh8 Qh6) {+0.70/2 4} Ba5 (Ba5 Bc3 Bb6 h3 Nh6 Rd1 Be6 Be1 Rad8 Bf2 Nf5 Nxf5 Bxf5 Bd3 Bxd3 Qxd3 g5 fxg5 Rxf3 Qxf3 Qxe5 Qf6 Qxf6 gxf6) {+0.12/20 1} 20. Bc3 (Bc3 Bb6 Bc4 dxc4 f5 Nxe5 Re3 Bc7 Bxe5 Bxe5 Rxe5 Qh4 Rd1 b6 Re8 Bb7 Re7 Bc6 Rd6 Rae8 Rde6 Rxe7 Rxe7) {+0.94/2 5} Bb6 (Bb6 h3 Nh6 Rd1 a6 Ba4 c4 Kf1 Be6 Bc2 g6 Bb4 Bc5 Bxc5 Qxc5 Qf2 Qxf2 Rxf2 Rad8 Ne2 Nf5 Bxf5 Bxf5 Nd4 Be4 g4) {-0.13/20 3} 21. Bc4 (Bc4 dxc4 f5 Nxe5 Re3 Bc7 Bxe5 Bxe5 Rxe5 Qh4 Rd1 b6 Re8 Bb7 Re7 Bc6 Rd6 Rae8 Rde6 Rxe7 Rxe7) {+1.13/2 2} dxc4 (dxc4 f5 Rf7 h3 Nxe5 Re3 Bd7 Re1 Qf8 Bxe5 c3 Rxc3 Ba5 Ne4 Bxc3 bxc3 Bxf5 Nd6 Rd8 Nxf7 Qxf7 Rd1 Rxd1 Qxd1 Qd7 Qxd7) {+0.38/19 2} 22. f5 (f5 Qh4 h3 Nh6 e6 Bc7 f6 Bxg3 Raf1 gxf6 e7 Bf5 Rxf5 Nxf5 exf8=N Rxf8 Rxf5 Be5 Bxe5 fxe5 Rxe5 Qd4 Kh1 Kh8 Re7 b5) {+1.14/2 1} Rf7 (Rf7 Re1 Bc7 Qxc4 Nxe5 Bxe5 Bxe5 Rfe3 Be6 Rxe5 Bxc4 Rxe7 b6 Re8 Rxe8 Rxe8 Rf8 Re7 Rd8 h3 Rd2 Ne4 Re2 Re8 Kf7 Nd6 Kf6 Ne4 Kf7) {0.00/21 3} 23. e6 (e6 Rf8 Qxc4 Nf6 Ne4 Ng4 Ng3 Nf6 Ne4 Ng4 h3 Nh6 Bxg7 Qxg7 f6 Qg6 Rg3 Qxg3 Nxg3) {+1.22/2 4} Rf8 (Rf8 Nh5 Bc7 Nxg7 Nf6 Bxf6 Rxf6 Nh5 Rf8 Qxc4 Bxe6 fxe6 Rxf3 Qg4 Kh8 Qxf3 Be5 Qd5 Bxb2 Rd1 Bd4 Rxd4 cxd4 Qxd4 Kg8 Nf6 Kf8 Nd5 Qxe6 Qh8 Kf7 Qxh7 Kf8) {-0.15/21 2} 24. Qxc4 (Qxc4 Nf6 Ne4 Ng4 Ng5 Nf6 Rd1 Bd8 Bxf6 Rxf6 Rxd8 Qxd8 e7 Kh8 exd8=Q Rf8 Qxf8) {+1.41/2 2} Nf6 (Nf6 Ne4 Bc7 Rd1 b5 Nxf6 gxf6 Qxb5 Kh8 Rf2 Rb8 Qc4 Rb6 Rfd2 Rd6 Rxd6 Bxd6 Qh4 Bc7 h3 Qg7 Qe4 Rg8 Qc6 Qe7 b4) {+0.43/19 2} 25. Ne4 (Ne4 Ng4 Ng5 Nf6 Rd1 h6 Ne4 Ne8 Rdf1 Bc7 Qe2 Bxe6 fxe6 Rxf3 Qxf3 Qxe6 Nxc5 Qg6 Qd5 Kh7) {+1.52/2 2} Bc7 (Bc7 Rd1 b6 Nxf6 gxf6 Rh3 a5 Rh6 Be5 Bxe5 fxe5 Rh5 Ra7 Qb3 Kh8 Qxb6 Rb7 Qxa5 Rxb2 Qc3 Re2 Re1 Rxe1 Qxe1 Qg7 Qg3 Qxg3) {+0.29/20 3} 26. Nxf6+ (Nxf6 gxf6 Qe4 Kh8 Rd1 a5 Rh3 Ra6 Rh6 Rd6 Re1 b5 Qh4 Bd8 Re3 b4) {+1.43/2 4} gxf6 (gxf6 Rd1 a5 Rh3 b5 Qxb5 Kh8 Rd2 Ra7 Qa4 Qg7 Qc4 Bb6 Qb5 Rb7 Qc6 Bc7 Rd7 Bxd7 Qxb7 Be8 Qd5 Bb5 Qxc5) {+0.13/19 2} 27. a4 (a4 Kh8 Rd1 Qg7 Rdd3 b5 axb5 a6 Qxc5 axb5 e7 Rg8 g3 Ra1 Kf2 b4 Bxf6 Qxf6) {+1.44/2 5} b6 (b6 Rd3 a6 Rad1 Ra7 b4 a5 bxa5 bxa5 Rd5 Rb7 Kh1 Bb6 Qb5 Qg7 Bxa5 Bxa5 Qxa5 Rb2 R5d2 Rxd2 Rxd2 Qe7 Qc3 Rd8 Rxd8 Qxd8) {+0.71/20 4} 28. Rd1 (Rd1 Be5 Bxe5 fxe5 a5 bxa5 Qe4 Bb7 Qg4 Qg7 Qxg7 Kxg7 Rd7 Kf6 Rxb7 e4 Rf4 Rab8 Rxh7) {+1.31/2 3} a6 (a6 Rfd3 Ra7 b4 b5 Qg4 Qg7 Qxg7 Kxg7 bxc5 bxa4 e7 Re8 Rd8 Bxd8 exd8=Q Rxd8 Rxd8 Rc7 Bd4 a3 Rd6 a2 Rxf6 Rd7 Rg6 Kf8 Rg4) {+0.24/22 5} 29. Qh4 (Qh4 Kh8 a5 b5 b4 Qg7 bxc5 b4 Qxb4 Rb8 Qh4 Rb5 c6 Rc5 Bb4 Rg8 Rf2 Rxc6) {+1.49/2 6} b5 (b5 Rfd3 Ra7 Rd5 Rb7 axb5 axb5 Bd2 c4 Be3 Bb6 Bxb6 Rxb6 Qg4 Kh8 Qd4 Bxe6 fxe6 Rxe6 Rf5 Rg8 Rf2 Rg7 g3 Re1 Rxe1 Qxe1 Kg2) {+0.72/20 3} 30. b4 (b4 bxa4 bxc5 Kh8 c6 Rb8 Qxa4 Rb5 Kh1 Rd8 Re1 Be5 Qh4 Bxc3) {+1.30/2 4} c4 (c4 Rd5 Bb8 Bd4 Ba7 Bc5 Kh8 Qd4 Bxc5 bxc5 Rg8 c6 Rb8 axb5 Rxb5 g3 Rb3 Rf2 c3 c7 Rb1 Kg2) {+0.95/19 4} 31. a5 (a5 Kh8 Re3 Ra7 Re4 Bd8 Rg4 Qe8 h3 Rg7 Rxg7 Kxg7 Qg5 Kh8) {+1.93/2 3} Bb8 (Bb8 Kh1 Ra7 Re3 Bb7 Qg4 Kh8 Rd7 Bxg2 Qxg2 Rxd7 exd7 Qxd7 Re6 Ba7 Qd2 Qxd2 Bxd2 Bd4 Rd6 Bb2 Rxa6 c3 Be3 c2 Re6 c1=Q Bxc1) {+0.83/21 2} 32. Rd5 (Rd5 Ra7 Bd4 Bxe6 Qg4 Qg7 Qxg7 Rxg7 fxe6 Be5 Bxe5 fxe5 Re3 Rc7 Rdxe5 c3 Re1 c2 Rc1 Rd8) {+1.57/2 5} Ra7 (Ra7 Re3 Kh8 Bd4 Rc7 Rd7 Bxd7 exd7 Qxe3 Bxe3 Rxd7 Bd4 Ba7 Bb6 Rdf7 Qe4 Bxb6 axb6 Rb8 Qd4 Kg7 Qg4 Kf8 Qd4 Kg8 Kf2 Rfb7 Qxf6 Rxb6 Qg5 Kf7 Qh5 Kf6 Qxh7) {+0.36/20 1} 33. Kf2 (Kf2 Bd6 Qd4 Bb8 Qh4 Qg7 Rd4 Kh8 Rg4 Qe7 Rg6 Be5 Bxe5 fxe5) {+1.65/2 5} Rb7 (Rb7 Kg1 Kh8 Rf1 Ba7 Kh1 Rc7 Rfd1 Kg8 Qg3 Qg7 Rd8 Qxg3 hxg3 Rxd8 Rxd8 Kg7 e7 Rxe7 Rxc8 Bf2 Rc6 Re3 Bxf6 Kf7 Bh4 Bxg3 Bxg3 Rxg3 Rxa6 Rb3 g4 Rxb4 g5 Ra4 Ra7 Kf8 Rxh7) {+0.19/21 3} 34. Kf1 (Kf1 Kh8 g4 Kg8 Rf2 Be5 Bxe5 fxe5 Qxe7 Rxe7 Rxe5 Rd8 g5 c3 Rc2) {+1.37/2 3} Kh8 (Kh8 Re3 Ra7 Bd4 Rc7 Rd7 Rxd7 exd7 Qxd7 Bxf6 Rxf6 Qxf6 Kg8 Qg5 Kf8 Qf6 Kg8) {0.00/23 1} 35. Re3 (Re3 Bd6 Qh6 Bc7 Re4 Qg7 Qxg7 Kxg7 Rg4 Kh8 e7 Re8 Bxf6) {+1.63/2 4} Ra7 (Ra7 Bd4 Rc7 Rd7 Rxd7 exd7 Qxd7 Bxf6 Rxf6 Qxf6 Kg8 Qg5 Kf8 Qf6 Kg8) {0.00/25 1} 36. Re4 (Re4 Rc7 g4 Rc6 g5 Be5 Rexe5 fxe5 Bxe5 Kg8 Bf6 Qxb4 g6 Qb1 Kf2 Qc2 Kg3 Qb3 Kf4 Rc7) {+2.22/2 3} Rb7 (Rb7 Rg4 Bxe6 Re4 Be5 Rdxe5 Qd6 Bd4 fxe5 Bxe5 Qxe5 Rxe5 Bxf5 Ke1 Rbf7 Re7 Rxe7 Qxe7 Kg8 Qd6 Re8 Kd2 Bc8 Qd5 Kg7 Qg5 Kh8 Qh5 Re7 Qc5 Re8 Qc6 Rd8 Kc3 Kg8 Qe4 Rd3 Kc2) {-1.94/22 2} 37. Rg4 (Rg4 Bxe6 Re4 Qg7 Rxe6 Rbf7 Bd2 Qg8 Bc3 Qg7 g3 Kg8 Rxa6 Be5 Bxe5 c3) {+2.98/2 2} Bxe6 (Bxe6 Re4 Be5 Rdxe5 Qd6 Bd4 fxe5 Bxe5 Qxe5 Rxe5 Bxf5 Ke1 Rbf7 Re7 Kg8 Qg3 Bg6 Rxf7 Rxf7 Qd6 c3 Qxa6 Re7 Kf2 Rf7 Ke3 Re7 Kf4 Rf7 Kg5 c2 Qc8 Rf8 Qc6 Rf1) {-1.81/24 2} 38. Re4 (Re4 Qg7 Rxe6 Rbf7 Bd2 Qg8 Bh6 Rd8 Qd4 Rxd5 Qxd5 Bxh2 Rxa6 Bb8 Ra8 c3) {+3.00/2 0} Be5 (Be5 Rdxe5 Qd6 Bd4 fxe5 Bxe5 Qxe5 Rxe5 Bxf5 Ke1 Bg6 Qd4 Kg8 Qd5 Rbf7 Re6 Kh8 Kd2 Rf2 Ke3 Ra2 Qe5 Kg8 Rf6 Ra3 Kf4 Rxf6 Qxf6 c3 Qxa6 c2 Qc8 Kf7 Qc7 Kg8) {-2.09/24 2} 39. Rdxe5 (Rdxe5 Qd6 Bd4 fxe5 Bxe5 Qxe5 Rxe5 Bxf5 Kg1 Rbf7 Qd4 Kg8 h4 h6 Rc5 Bd3 Rc6) {+2.84/2 1} Qd6 (Qd6 Bd4 fxe5 Bxe5 Qxe5 Rxe5 Bxf5 Ke2 Rbf7 Re7 Bd3 Ke3 Bf5 Kd2 Rd8 Ke1 Rdf8 Qd4 Kg8 Rxf7 Rxf7 Qe5 Rf8 Qd5 Kh8 Kd2 Bc8 Qe5 Kg8 Kc3 Kf7 Qh5 Kg7 Qg5 Kf7 Kd2 Rg8 Qd5 Kg7 Qe5 Kg6 Qe4 Kf6 Kc3) {-2.31/24 3} 40. Kg1 (Kg1 Bd5 Re2 Kg8 Re8 Rg7 Rxf8 Qxf8 Bxf6 Rf7 Qg5 Rg7 Bxg7 Qxg7 Re8 Kf7) {+2.94/2 1} Bd5 (Bd5 Rxd5 Qxd5 Bxf6 Rg7 Bxg7 Kxg7 Qg3 Kf6 Re6 Kf7 h3 Rc8 Rxa6 Qd4 Kh2 Qd7 Qe3 Qxf5 Ra7 Kg8 Qd4 Qg5 Qe4 Qg6 Qd5 Kh8 Qd4 Kg8 Rb7 c3 Rc7 Rxc7 Qd8 Kf7 Qxc7 Kg8 Qxc3) {-2.66/23 4} 41. Re2 (Re2 Kg8 Re8 Rbf7 Qh6 Bb7 Rd2 Bd5 Rxf8 Qxf8 Qxf8 Kxf8 Rxd5 Ke7 Kf2) {+3.22/2 3} Bg8 (Bg8 Rd2 Qxd2 Bxd2 fxe5 Bc3 Rxf5 Qe4 Rbf7 Bxe5 Rxe5 Qxe5 Rg7 Qf6 c3 Qxa6 Bc4 Qc8 Rg8 Qh3 c2 Qc3 Rg7 Qxc2 Rf7 g3 Kg8 Qe4 Rf6 Qb7 Rf7 Qb8 Rf8 Qe5 Rf1 Kg2 Rf7 a6 h6) {-4.45/26 13} 42. Re8 (Re8 Rbf7 h3 Qd1 Re1 Qd3 Rxf8 Qxc3 Rxf7 Bxf7 Rd1 Kg7 Qg4 Kf8 Qd4 Qxb4 Qxf6 Qe7 Rd8 Qxd8 Qxd8) {+5.76/2 3} Rbf7 (Rbf7 R2e6 Qd1 Re1 Qd6 R8e6 Qd3 Bxf6 Rxf6 Rxf6 Qd8 Rh6 Qxh4 Rxh4 Kg7 g4 Rd8 Rh3 Rd4 g5 Kf8 Ree3 Rf4 Re5 Rg4 Kf2 Rxg5 Rh6 c3 Rf6 Kg7) {-2.98/25 2} 43. h3 (h3 Qd1 Re1 Qd3 Bd4 Qxf5 Rxf8 Rxf8 Rf1 Qg6 Rxf6 Rxf6 Bxf6 Qg7 Bxg7 Kxg7 Qd4 Kf7 Qb6 c3) {+6.84/2 2} Qd3 (Qd3 Rxf8 Qxc3 Rxf7 Bxf7 Re7 Kg7 Qf4 h5 Kh2 h4 Qc7 Qg3 Qxg3 hxg3 Kxg3 Kg8 Re2 Bd5 Kf4 Kf8 g4 Bg8 Rd2 Ke8 h4 Ke7 h5 Bf7 h6 Bg8 Rh2 Bh7 Re2 Kd7 Re3 Bg8 g5 fxg5 Kxg5 Kd6 Kf4 Kd7) {-4.58/25 3} 44. Rxf8 (Rxf8 Qxc3 Rxf7 Bxf7 Kh2 h5 Re7 Kg8 Qg3 Qxg3 Kxg3 c3 Re3 c2 Rc3 Bb3 Kh4 Ba4 Kxh5 Kf7 g4 Ke7 Kg6) {+8.69/2 2} Qxc3 (Qxc3 Rxf7 Bxf7 Re7 Kg7 Qf4 h5 Kh2 h4 Qc7 Qg3 Qxg3 hxg3 Kxg3 Kf8 Re2 Be8 Re6 c3 Re2 Bf7 Rc2 Ke7 Rxc3 Kd6 Kf4 Bc4 g4 Kd5 Re3 Kd4 Re6) {-5.01/26 2} 45. Rxf7 (Rxf7 Bxf7 Kh2 h5 Re7 Kg8 Qg3 Qxg3 Kxg3 c3 Re3 c2 Rc3 Bb3 Kh4 Ba4 Kxh5 Kf7 g4 Ke7 Kg6 Kd6) {+9.23/2 2} Bxf7 (Bxf7 Re7 Kg7 Qf4 h5 Kh2 h4 Qc7 Qg3 Qxg3 hxg3 Kxg3 Kf8 Re2 Bd5 Kf4 Bf7 g4 Bg8 Rd2 Ke7 h4 Bf7 h5 Kf8 h6 c3 Rc2 Bc4 Rxc3 Kg8 g5 fxg5 Kxg5 Kh7) {-5.37/26 2} 46. Kh2 (Kh2 h5 Re7 Kg8 Qg3 Qxg3 Kxg3 c3 Re3 c2 Rc3 Bb3 Kh4 Ba4 Kxh5 Kf7 g4 Ke7 Kg6 Kd6) {+9.61/2 1} h5 (h5 Re7 Kg7 Qxh5 Qe5 Rxe5 Bxh5 Re6 c3 Rc6 Bf7 Rxc3 Bd5 Kg3 Bc4 Kf4 Kg8 g4 Kg7 h4 Kf8 h5 Kf7 g5 fxg5 Kxg5 Kg8 Re3 Kf8 h6 Kf7 Rg3 Kg8 Kf4 Kh8 Rg6 Kh7 Rxa6) {-6.70/25 3 Arena Adjudication} 1-0[/pgn]

it lost both sides of the evans gambit, but going into the last 2 games it's 18W-2L-18D. games here https://lichess.org/study/b9FB7XH2
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Something goes wrong with lc0 since yesterday?

Post by Laskos »

Werewolf wrote: Thu Jul 05, 2018 9:53 am
yanquis1972 wrote: Thu Jul 05, 2018 4:22 am i'm running 10k29 against rybka 1.0 & it's clearly 'strong' but not in the same league. i was thinking 2600 (-250 elo by ccrl 40/4) seems possible, but definitely not the 2800 i was rather optimistically hoping for.

fwiw after same number of games the original net was at 1700 elo (sourcing from the same site), & i don't think they've dropped the learning rate on the testserver yet either -- maybe if the next net doesn't show a significant increase? so there's definitely that.
My own results from yesterday also point to about 2600 elo.

I see this morning it has climbed another 90 elo in self play, so as long as it keeps going up...
Their self-plays by now are almost completely useless. If I am not doing something very wrong, or my time control is too short, I am getting for ID10030 almost identical result to ID10028 in 200 games gauntlets against AB engine. Still only some 50-60 Elo points improvement compared to ID10017, while their self-play ratings indicate some 330 Elo points progress. And still, in 40/4' CCRl conditions, about a 2600+ engine on GTX 1060 GPU. I am a bit worried, something odd is going on, which apparently did not happen with earlier fast runs using 6x64 nets.
crem
Posts: 177
Joined: Wed May 23, 2018 9:29 pm

Re: Something goes wrong with lc0 since yesterday?

Post by crem »

Laskos wrote: Thu Jul 05, 2018 12:16 pm Their self-plays by now are almost completely useless. If I am not doing something very wrong, or my time control is too short, I am getting for ID10030 almost identical result to ID10028 in 200 games gauntlets against AB engine. Still only some 50-60 Elo points improvement compared to ID10017, while their self-play ratings indicate some 330 Elo points progress. And still, in 40/4' CCRl conditions, about a 2600+ engine on GTX 1060 GPU. I am a bit worried, something odd is going on, which apparently did not happen with earlier fast runs using 6x64 nets.
I've just started testrun 10017 vs 10030 and it seems that at least in self play it is consistent.
I also worry that something is wrong (e.g. there are blind spots and Lc0 misses some moves), but other devs believe that it's doing fine (e.g. that 800 nodes per move play is poor predictor for longer TC play).

Also there was a bug in selfplay matches that makes them color-imbalanced (on one side or another).
That's fixed but new client version is not yet built.

Also selfplay games in matches are run with default settings:
--cpuct=1.2
--fpu-reduction=0.0
--policy-softmax-temp=1.0
Rather then CLOP-optimized settings.. It may happen that old optimized settings are not good for new nets.
jp
Posts: 1470
Joined: Mon Apr 23, 2018 7:54 am

Re: Something goes wrong with lc0 since yesterday?

Post by jp »

Laskos wrote: Wed Jul 04, 2018 11:45 pm I don't like something in their setup, do they use gating for promoting and progression graph?
Kai, what do you mean by "gating for promoting and progression graph"?
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Something goes wrong with lc0 since yesterday?

Post by Laskos »

crem wrote: Thu Jul 05, 2018 12:47 pm
Laskos wrote: Thu Jul 05, 2018 12:16 pm Their self-plays by now are almost completely useless. If I am not doing something very wrong, or my time control is too short, I am getting for ID10030 almost identical result to ID10028 in 200 games gauntlets against AB engine. Still only some 50-60 Elo points improvement compared to ID10017, while their self-play ratings indicate some 330 Elo points progress. And still, in 40/4' CCRl conditions, about a 2600+ engine on GTX 1060 GPU. I am a bit worried, something odd is going on, which apparently did not happen with earlier fast runs using 6x64 nets.
I've just started testrun 10017 vs 10030 and it seems that at least in self play it is consistent.
I also worry that something is wrong (e.g. there are blind spots and Lc0 misses some moves), but other devs believe that it's doing fine (e.g. that 800 nodes per move play is poor predictor for longer TC play).

Also there was a bug in selfplay matches that makes them color-imbalanced (on one side or another).
That's fixed but new client version is not yet built.

Also selfplay games in matches are run with default settings:
--cpuct=1.2
--fpu-reduction=0.0
--policy-softmax-temp=1.0
Rather then CLOP-optimized settings.. It may happen that old optimized settings are not good for new nets.
Interesting, thanks. It's entirely possible that this 20b net needs another optimal settings. Also, fixed nodes=800 matches in self-games with default settings are a bit crude a measure of progress, although it probably correlates to some degree with playing at fixed time against AB engines.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Something goes wrong with lc0 since yesterday?

Post by Laskos »

jp wrote: Thu Jul 05, 2018 1:27 pm
Laskos wrote: Wed Jul 04, 2018 11:45 pm I don't like something in their setup, do they use gating for promoting and progression graph?
Kai, what do you mean by "gating for promoting and progression graph"?
From the paper, I remember AlphaGoZero used some performance criteria in matches to promote (maybe 55% in self-games), this is gating, they promote if they perform. AlphaZero used no such criteria, they automatically promoted (no gating). Each way has advantages and drawbacks, I am not sure what they are doing in lc0 testserver framework.
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: Something goes wrong with lc0 since yesterday?

Post by yanquis1972 »

the notes from this site https://docs.google.com/spreadsheets/d/ ... 1519374103 (CCRL 40/4 2700 elo for 10025) say default settings for jun 4 lco.exe are used, fyi. (no idea what those are)