Need Lc0 help

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Need Lc0 help

Post by Dann Corbit »

zullil wrote: Wed Jun 26, 2019 6:06 pm Nodes = 1, without search.

Code: Select all

pos	320x24.T13-swa-273
------------------------------------------
01 	e5   (55.3)
02	g3   (56.9)
03	Bd3  (63.6)
04	Nf6  (47.8)
05	g3   (54.9)
06 	c4   (54.9)
07	e4   (57.8)
08	d5   (47.9)
09	h4   (62.9)
10	d5   (64.6)
11	Nc3  (62.0)
12	exd5 (57.5)
13	Nc3  (57.0)
14	e4   (56.3)
15	dxc5 (66.7)
16	d5   (44.2)
17	d5   (47.1)
18	Nf3  (51.0)
Network 320x24.T13-swa-273000 was posted in the Lc0 group at discord.com
I assume that for each of your 18 positions, he's given you best move and evaluation (= 100 * probability of win??) from two different nets.
I removed one net to simplify the list.
If e5 is the best move then why is probability of win less than g3, Bd3, e4, h4, d5, Nc3, e4 (again?) and dxc5? Wouldn't dxc5 be best?
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Need Lc0 help

Post by zullil »

Dann Corbit wrote: Wed Jun 26, 2019 8:05 pm
zullil wrote: Wed Jun 26, 2019 6:06 pm I assume that for each of your 18 positions, he's given you best move and evaluation (= 100 * probability of win??) from two different nets.
No.

For each of the moves in the list, the computer evaluation and the actual outcomes from real games disagree.
When the move played by the computer is performed, the outcomes are not as good as when the best move (as a functionality of the probability of winning) is played.

This is actually a very rare occurrence when the depths are greater than 36 plies.

For twenty years now, I have been refining my database by looking for positions with the above metric, and then searching deeper.
Almost invariably, the correct move gets found. This particular batch had already survived many cycles.
I think we may be arriving at a point where my technique no longer works as reliably as it used to.

I think it is still a good idea, especially when the searches are shallow (less than 36 ply). But a high percentage of the "obviously wrong" positions have already been corrected, so of those that remain, maybe they are not really obviously wrong.
I think some wires are crossed. :wink: What you quoted above was my explaining to you what Max's numbers meant. He later confirmed that my explanation was correct.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Need Lc0 help

Post by zullil »

Dann Corbit wrote: Wed Jun 26, 2019 8:10 pm
zullil wrote: Wed Jun 26, 2019 6:06 pm Nodes = 1, without search.

Code: Select all

pos	320x24.T13-swa-273
------------------------------------------
01 	e5   (55.3)
02	g3   (56.9)
03	Bd3  (63.6)
04	Nf6  (47.8)
05	g3   (54.9)
06 	c4   (54.9)
07	e4   (57.8)
08	d5   (47.9)
09	h4   (62.9)
10	d5   (64.6)
11	Nc3  (62.0)
12	exd5 (57.5)
13	Nc3  (57.0)
14	e4   (56.3)
15	dxc5 (66.7)
16	d5   (44.2)
17	d5   (47.1)
18	Nf3  (51.0)
Network 320x24.T13-swa-273000 was posted in the Lc0 group at discord.com
I assume that for each of your 18 positions, he's given you best move and evaluation (= 100 * probability of win??) from two different nets.
I removed one net to simplify the list.
If e5 is the best move then why is probability of win less than g3, Bd3, e4, h4, d5, Nc3, e4 (again?) and dxc5? Wouldn't dxc5 be best?
Dann,

He analyzed all 18 of your positions. That's the 1-18.!
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Need Lc0 help (summary)

Post by Dann Corbit »

Code: Select all

Here, I suppose that the stats are probably wrong.
rnbqkb1r/pppp1ppp/4pn2/8/2P5/2N5/PP1PPPPP/R1BQKBNR w KQkq - bm Nf3; pv Nf3; c0 "Engine Statistical";
rnbqkb1r/pppp1ppp/4pn2/8/2P5/2N5/PP1PPPPP/R1BQKBNR w KQkq - bm e4; pv e4 d5; c0 "Engine LC0";
rnbqkb1r/pppp1ppp/4pn2/8/2P5/2N5/PP1PPPPP/R1BQKBNR w KQkq - bm e4; pv e4 d5; c0 "Engine Stockfish";

Not sure what to think about this one:
rnbqkb1r/pppp1ppp/4pn2/8/2P5/5N2/PP1PPPPP/RNBQKB1R w KQkq - bm Nc3; pv Nc3 d5; c0 "Engine Stockfish";
rnbqkb1r/pppp1ppp/4pn2/8/2P5/5N2/PP1PPPPP/RNBQKB1R w KQkq - bm d4; pv d4 d5; c0 "Engine LC0";
rnbqkb1r/pppp1ppp/4pn2/8/2P5/5N2/PP1PPPPP/RNBQKB1R w KQkq - bm g3; pv g3; c0 "Engine Statistical";

Here, I suppose that the stats are probably wrong:
rnbqkb1r/pppppppp/5n2/8/3P4/2P5/PP2PPPP/RNBQKBNR b KQkq - bm c5; pv c5 d5; c0 "Engine LC0";
rnbqkb1r/pppppppp/5n2/8/3P4/2P5/PP2PPPP/RNBQKBNR b KQkq - bm c5; pv c5 e3; c0 "Engine Stockfish";
rnbqkb1r/pppppppp/5n2/8/3P4/2P5/PP2PPPP/RNBQKBNR b KQkq - bm g6; pv g6; c0 "Engine Statistical";

Here, I suppose that Stockfish probably wrong:
rnbqkb1r/pppppppp/5n2/8/8/5N2/PPPPPPPP/RNBQKB1R w KQkq - bm b3; pv b3 c5; c0 "Engine Stockfish";
rnbqkb1r/pppppppp/5n2/8/8/5N2/PPPPPPPP/RNBQKB1R w KQkq - bm c4; pv c4 e6; c0 "Engine LC0";
rnbqkb1r/pppppppp/5n2/8/8/5N2/PPPPPPPP/RNBQKB1R w KQkq - bm c4; pv c4; c0 "Engine Statistical";

Not sure what to think about this one:
rnbqkbnr/1pp1pppp/p7/3p4/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - bm Bg5; pv Bg5; c0 "Engine Statistical";
rnbqkbnr/1pp1pppp/p7/3p4/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - bm c4; pv c4 dxc4; c0 "Engine Stockfish";
rnbqkbnr/1pp1pppp/p7/3p4/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - bm g3; pv g3 c6; c0 "Engine LC0";

Here, I suppose that the stats are probably wrong:
rnbqkbnr/1ppp1ppp/p3p3/8/3PP3/8/PPP2PPP/RNBQKBNR w KQkq - bm Bd3; pv Bd3 d5; c0 "Engine LC0";
rnbqkbnr/1ppp1ppp/p3p3/8/3PP3/8/PPP2PPP/RNBQKBNR w KQkq - bm Bd3; pv Bd3 d5; c0 "Engine Stockfish";
rnbqkbnr/1ppp1ppp/p3p3/8/3PP3/8/PPP2PPP/RNBQKBNR w KQkq - bm Nf3; pv Nf3; c0 "Engine Statistical";

Here, I suppose that Stockfish probably wrong:
rnbqkbnr/pppp1ppp/8/4p3/2P5/3P4/PP2PPPP/RNBQKBNR b KQkq - bm Bb4+; pv Bb4+ Bd2; c0 "Engine Stockfish";
rnbqkbnr/pppp1ppp/8/4p3/2P5/3P4/PP2PPPP/RNBQKBNR b KQkq - bm Nf6; pv Nf6 Nf3; c0 "Engine LC0";
rnbqkbnr/pppp1ppp/8/4p3/2P5/3P4/PP2PPPP/RNBQKBNR b KQkq - bm Nf6; pv Nf6; c0 "Engine Statistical";

Here, I suppose that the stats are probably wrong:
rnbqkbnr/ppppp2p/6p1/5p2/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - bm Nc3; pv Nc3 Nf6; c0 "Engine LC0";
rnbqkbnr/ppppp2p/6p1/5p2/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - bm Nc3; pv Nc3 Nf6; c0 "Engine Stockfish";
rnbqkbnr/ppppp2p/6p1/5p2/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - bm g3; pv g3; c0 "Engine Statistical";

Here, I suppose that the stats are probably wrong:
rnbqkbnr/pppppppp/8/8/P7/8/1PPPPPPP/RNBQKBNR b KQkq - bm Nf6; pv Nf6 d4; c0 "Engine LC0";
rnbqkbnr/pppppppp/8/8/P7/8/1PPPPPPP/RNBQKBNR b KQkq - bm Nf6; pv Nf6; c0 "Engine Stockfish";
rnbqkbnr/pppppppp/8/8/P7/8/1PPPPPPP/RNBQKBNR b KQkq - bm e5; pv e5; c0 "Engine Statistical";
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Need Lc0 help

Post by zullil »

Dann Corbit wrote: Wed Jun 26, 2019 5:56 pm
zullil wrote: Wed Jun 26, 2019 11:47 am
I'm still wondering about what you mean by "get it wrong" and "find the answer." After 1. a4, probably every move (except 1...b5, perhaps) likely draws with perfect play. I don't understand what you hope to determine from any number of human games that opened 1. a4. Probably every one of those games that was decided was decided by multiple inaccuracies or outright errors.

It sounds like you believe that 1. a4 e5 loses with perfect play. I guess I simply doubt this.
No, I have a statistical lead for black of one pawn after 1.a4 e5, based off of w/l/d statistics
BTW, here's Cfish-dev after an overnight run:

-0.16 1... Nf6 2. d4 e6 3. e3 b6 4. a5 bxa5 5. Bd2 Nc6 6. Nf3 Rb8 7. Bd3 Rxb2 8. O-O d5 9. c4 dxc4 10. Bxc4 Rb8 11. Bxa5 Nxa5 12. Rxa5 Bb4 13. Rxa7 O-O 14. Ne5 c5 15. dxc5 Qxd1 16. Rxd1 Bxc5 17. Ra5 Bb6 18. Rb5 Ba6 19. Rb4 Bc7 20. Rxb8 Rxb8 21. Bxa6 Bxe5 22. Nd2 Nd5 23. Rc1 Bc3 24. Ne4 Bb4 25. Rc8+ Rxc8 26. Bxc8 Nc3 27. Nxc3 Bxc3 (depth 51, 8:43:45)

What depth did you reach with your Stockfish search?
47 plies
At depth 55, Cfish has switched to 1...d5. I'm stopping this now, since I don't think has much meaning. Here's the PV:

-0.21 1... d5 2. d4 c5 3. e3 Nc6 4. Nf3 Nf6 5. c3 e6 6. Nbd2 b6 7. Bb5 Bb7 8. a5 a6 9. Bxc6+ Bxc6 10. axb6 Qxb6 11. b3 Be7 12. Ba3 Nd7 13. c4 O-O 14. cxd5 Bxd5 15. e4 Bb7 16. O-O Qd8 17. dxc5 Bxc5 18. Re1 Bxa3 19. Rxa3 Nc5 20. Qe2 Nd3 21. Rb1 Rc8 22. Ra4 e5 23. Qe3 f5 24. exf5 Rxf5 25. Rc4 Qd5 26. Rd1 Nf4 27. Qe4 Qxe4 28. Nxe4 Rxc4 29. Rd8+ Kf7 30. Nd6+ Ke6 31. Nxc4 (depth 55, 21:55:05)
giovanni
Posts: 142
Joined: Wed Jul 08, 2015 12:30 pm

Re: Need Lc0 help

Post by giovanni »

zullil wrote: Thu Jun 27, 2019 12:07 am
Dann Corbit wrote: Wed Jun 26, 2019 5:56 pm
zullil wrote: Wed Jun 26, 2019 11:47 am
I'm still wondering about what you mean by "get it wrong" and "find the answer." After 1. a4, probably every move (except 1...b5, perhaps) likely draws with perfect play. I don't understand what you hope to determine from any number of human games that opened 1. a4. Probably every one of those games that was decided was decided by multiple inaccuracies or outright errors.

It sounds like you believe that 1. a4 e5 loses with perfect play. I guess I simply doubt this.
No, I have a statistical lead for black of one pawn after 1.a4 e5, based off of w/l/d statistics
BTW, here's Cfish-dev after an overnight run:

-0.16 1... Nf6 2. d4 e6 3. e3 b6 4. a5 bxa5 5. Bd2 Nc6 6. Nf3 Rb8 7. Bd3 Rxb2 8. O-O d5 9. c4 dxc4 10. Bxc4 Rb8 11. Bxa5 Nxa5 12. Rxa5 Bb4 13. Rxa7 O-O 14. Ne5 c5 15. dxc5 Qxd1 16. Rxd1 Bxc5 17. Ra5 Bb6 18. Rb5 Ba6 19. Rb4 Bc7 20. Rxb8 Rxb8 21. Bxa6 Bxe5 22. Nd2 Nd5 23. Rc1 Bc3 24. Ne4 Bb4 25. Rc8+ Rxc8 26. Bxc8 Nc3 27. Nxc3 Bxc3 (depth 51, 8:43:45)

What depth did you reach with your Stockfish search?
47 plies
At depth 55, Cfish has switched to 1...d5. I'm stopping this now, since I don't think has much meaning. Here's the PV:

-0.21 1... d5 2. d4 c5 3. e3 Nc6 4. Nf3 Nf6 5. c3 e6 6. Nbd2 b6 7. Bb5 Bb7 8. a5 a6 9. Bxc6+ Bxc6 10. axb6 Qxb6 11. b3 Be7 12. Ba3 Nd7 13. c4 O-O 14. cxd5 Bxd5 15. e4 Bb7 16. O-O Qd8 17. dxc5 Bxc5 18. Re1 Bxa3 19. Rxa3 Nc5 20. Qe2 Nd3 21. Rb1 Rc8 22. Ra4 e5 23. Qe3 f5 24. exf5 Rxf5 25. Rc4 Qd5 26. Rd1 Nf4 27. Qe4 Qxe4 28. Nxe4 Rxc4 29. Rd8+ Kf7 30. Nd6+ Ke6 31. Nxc4 (depth 55, 21:55:05)
Well, if after a 55 ply search the outcome doesn't match the OTB w/l/d statistics, chances are that humans rather the computers don't understand the position.
On the other hand, Dann you seem to be pretty convinced that this position could be an exception. Why don't you try to drive the engine search starting from 1)..e5? I mean, if you know of a reasonable plan for black starting from this move, you could prime lots of beta-cutoffs during the analysis, ultimately pushing the search beyond the horizon. You could do this in more sessions by saving the hash table or using tools like IDEA. If you don't know af any plan to apply, may be you could get some inspiration from the games whose statistics you are referring to.
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Need Lc0 help

Post by Daniel Shawul »

Here is the complete analysis result

Code: Select all

position fen rnbqkbnr/pppppppp/8/8/P7/8/1PPPPPPP/RNBQKBNR b KQkq - 0 1
bestmove g8f6 ponder d2d4
position fen rnbqkbnr/1pp1pppp/p7/3p4/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - 0 1
bestmove g2g3 ponder c7c6
position fen rnbqkbnr/1ppp1ppp/p3p3/8/3PP3/8/PPP2PPP/RNBQKBNR w KQkq - 0 1
bestmove f1d3 ponder d7d5
position fen rnbqkbnr/pppp1ppp/8/4p3/2P5/3P4/PP2PPPP/RNBQKBNR b KQkq - 0 1
bestmove g8f6 ponder g1f3
position fen rnbqkb1r/pppp1ppp/4pn2/8/2P5/5N2/PP1PPPPP/RNBQKB1R w KQkq - 0 1
bestmove d2d4 ponder d7d5
position fen rnbqkb1r/pppppppp/5n2/8/8/5N2/PPPPPPPP/RNBQKB1R w KQkq - 0 1
bestmove c2c4 ponder e7e6
position fen rnbqkb1r/pppp1ppp/4pn2/8/2P5/2N5/PP1PPPPP/R1BQKBNR w KQkq - 0 1
bestmove e2e4 ponder d7d5
position fen rnbqkb1r/pppppppp/5n2/8/3P4/2P5/PP2PPPP/RNBQKBNR b KQkq - 0 1
bestmove c7c5 ponder d4d5
position fen rnbqkbnr/ppppp2p/6p1/5p2/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - 0 1
bestmove b1c3 ponder g8f6
position fen rnbqkbnr/pp1p1ppp/4p3/2p5/3PP3/8/PPP2PPP/RNBQKBNR w KQkq - 0 1
bestmove d4d5 ponder g8f6
position fen rnbqkb1r/pp1ppppp/5n2/2pP4/8/8/PPP1PPPP/RNBQKBNR w KQkq - 0 1
bestmove c2c4 ponder g7g6
position fen rnbqkbnr/pp2pppp/2p5/3p4/3PP3/8/PPP2PPP/RNBQKBNR w KQkq - 0 1
bestmove e4e5 ponder c8f5
position fen rnbqkbnr/ppp2ppp/4p3/3p4/3PP3/8/PPP2PPP/RNBQKBNR w KQkq - 0 1
bestmove b1c3 ponder g8f6
position fen rnbqkbnr/pppp1ppp/4p3/8/3P4/8/PPP1PPPP/RNBQKBNR w KQkq - 0 1
bestmove b1c3 ponder g8f6
position fen rnbqkbnr/pp1ppp1p/6p1/2p5/3PP3/8/PPP2PPP/RNBQKBNR w KQkq - 0 1
bestmove d4c5 ponder f8g7
position fen rnbqkbnr/pp1ppppp/8/2p5/3P4/4P3/PPP2PPP/RNBQKBNR b KQkq - 0 1
bestmove d7d5 ponder c2c4
position fen rnbqkbnr/pp1ppppp/8/2p5/8/2N2N2/PPPPPPPP/R1BQKB1R b KQkq - 0 1
bestmove d7d5 ponder e2e4
position fen rnbqkbnr/pp1p1ppp/4p3/2p5/4P3/1P6/P1PP1PPP/RNBQKBNR w KQkq - 0 1
bestmove c1b2 ponder d7d5
Log file attached.

Daniel
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Need Lc0 help

Post by Ovyron »

giovanni wrote: Thu Jun 27, 2019 12:40 pm Why don't you try to drive the engine search starting from 1)..e5? I mean, if you know of a reasonable plan for black starting from this move, you could prime lots of beta-cutoffs during the analysis, ultimately pushing the search beyond the horizon. You could do this in more sessions by saving the hash table or using tools like IDEA. If you don't know af any plan to apply, may be you could get some inspiration from the games whose statistics you are referring to.
Yes. The true score of a position doesn't come from w/l/d statistics, because w/l/d statistics are built from suboptimal moves. The true score of a position comes from a tree where the main moves are played until "the end" (a leaf node), in a critical line where deviation from it produces a worse score (for either side, so that a side can't improve their play over this critical line), where you reach Depth 55 at the leaf node (or produce w/l/d statistics from this position), not at the root! And this produces a score, and this score is backsolved to the root.

In practice, over time the critical line keeps changing, and the most important lines get refuted. So that in this position, I'd expect that at some point 1...e5 and 1...d5 would have the same "true score". I think Dann Corbit could test the value of his approach by playing a game against a strong chess opponent, who could probably play into a position where Dann's "true score" of the leaf node is great for him, but in reality Dann is in a lost position. At least, if we played a game, I'm sure I'd not have problems holding the white side of 1.a4 e5 against Dann.
Your beliefs create your reality, so be careful what you wish for.