Leela and the curse of not finishing a won match cleanly.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
OliverBr
Posts: 334
Joined: Tue Dec 18, 2007 8:38 pm
Location: Munich, Germany
Full name: Dr. Oliver Brausch
Contact:

Leela and the curse of not finishing a won match cleanly.

Post by OliverBr » Mon Jun 08, 2020 10:02 pm

This game is another example, how mightily Leela plays the openings and mid-game and then... The first 30 moves are about a strategic dream for every chess player.
But then... instead of finished black with 31. Qc3+ she goes otherwise and trolls 180+ moves until checkmate.
So I decided to take action and let Stockfish 11 itself play it from Move 31:



No, surprise, Black is checkmate in move 52. Forced.
Chess Engine OliThink: http://brausch.org/home/chess

Dann Corbit
Posts: 11218
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Leela and the curse of not finishing a won match.

Post by Dann Corbit » Mon Jun 08, 2020 10:10 pm

There are currently some solutions which play LC0 to start the game and automatically finish with Stockfish at some pre-arranged breaking point. Kai was very interested in that and did some experiments.

I think a better solution is an engine that fully utilizes both AB and NN. An interesting irony is that you can run them both at full throttle on a machine, because SF uses no GPU and LC0 only uses 3 threads with 2 x RTX 2080 Super GPUs. So on a machine with 64 threads, I often have SF running with 60 threads and LC0 running with 3 threads and both GPUs. This gives me one thread left for email, web browsing, etc.

The real solution will come with the next generation of infinity fabric. According to my understanding, it will give the GPU and CPU transparent access to the same memory.

Now, you can really cook with gas.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
Ovyron
Posts: 4346
Joined: Tue Jul 03, 2007 2:30 am

Re: Leela and the curse of not finishing a won match cleanly.

Post by Ovyron » Tue Jun 09, 2020 1:00 am

Solution to what? I only see two scenarios:

1. Winning games earlier doesn't matter. If it did, instead of 1 point awarded for a win, we'd have 1 for winning in 100 moves, 1.1 for winning on 90 moves, 1.2 for winning in 80 moves and so on. On this chess variant Leela would be learning to mate earlier to score more points.

2. The user doesn't want to waste resources with games that long when they could be playing more games.

For the 2., then:

a) So just adjudicate wins at certain point for Leela games :roll: - if you know it's going to win then just call it a win, and you don't have to waste resources if the game has been decided.

b) If you can't adjudicate the games because you're playing someone online, then who is wasting your resources is that person that refuses to resign and wastes their own resources and your resources instead of surrendering. But solving it comes from person to person, if the game is resigned then we don't need to worry about how long the game would have taken.

For a), then:

i - If you don't like adjudication because you're not certain about the game's outcome and you have to have to let it till the end and suffer the long wins, then the games where Leela doesn't win the won position ARE the actual problem, now how long she takes to win.

ii - If shorter wins are "cleaner" than longer ones, then not having adjudication is "dirty", but who's to blame about that? (hint: it's a 3 letter word that starts with y and ends with u with o in the middle.)
Your beliefs create your reality, so be careful what you wish for.

Robert Pope
Posts: 522
Joined: Sat Mar 25, 2006 7:27 pm

Re: Leela and the curse of not finishing a won match cleanly.

Post by Robert Pope » Tue Jun 09, 2020 1:50 am

What about
3. By playing an extra 100 pointless moves, it will occasionally wander into positions where an AB searcher can find a drawing continuation, turning a won game into a draw.

carldaman
Posts: 1952
Joined: Sat Jun 02, 2012 12:13 am

Re: Leela and the curse of not finishing a won match cleanly.

Post by carldaman » Tue Jun 09, 2020 1:59 am

Could there be an algorithmic solution to the unnecessarily drawn-out wins (or wins that become draws)?

For example, whenever Leela thinks it's winning big, why not temporarily 'pretend' the 50-move rule is really a 25-move rule, forcing the engine to look for faster wins, and if none are seen, then gradually and recursively lengthen the cutoff till eventually you default back to the 50-move rule, barring any faster wins?

Would this create havoc elsewhere? Unwanted or unexpected consequences?

Houdini had a feature like this, but best used for analysis, since Houdini 2 had frequent trouble making fast progress when it had the advantage. If you tried it in games, it could backfire and Houdini could walk into sudden losses where it was assuming it was safe because of the modified rule. But here, we're assuming Leela is winning big in the endgame.

Dann Corbit
Posts: 11218
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Leela and the curse of not finishing a won match cleanly.

Post by Dann Corbit » Tue Jun 09, 2020 7:44 pm

The problem is caused by LC0 not searching deeply enough.
Searching less deeply won't fix that.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

MikeGL
Posts: 943
Joined: Thu Sep 01, 2011 12:49 pm

Re: Leela and the curse of not finishing a won match cleanly.

Post by MikeGL » Tue Jun 09, 2020 11:57 pm

I agree, adjudication is the best route to stop the game earlier avoid reaching checkmate and avoid wasting cpu/gpu, and electric resources..
On the above specific case, stop Lc0 at move 26 after engine eval of both engines is over the roof.
On human eye, resignation or end of game is between move 36 to 39 where black is obviously hopeless.
Ovyron wrote:
Tue Jun 09, 2020 1:00 am
...
...

2. The user doesn't want to waste resources with games that long when they could be playing more games.

For the 2., then:

a) So just adjudicate wins at certain point for Leela games :roll: - if you know it's going to win then just call it a win, and you don't have to waste resources if the game has been decided.
...
...
...
I told my wife that a husband is like a fine wine; he gets better with age. The next day, she locked me in the cellar.

Werewolf
Posts: 1273
Joined: Thu Sep 18, 2008 8:24 pm

Re: Leela and the curse of not finishing a won match cleanly.

Post by Werewolf » Wed Jun 10, 2020 9:08 am

Slightly off-topic, but it is related.

Could this be used for Lc0

https://www.nvidia.com/en-gb/data-center/dgx-a100/

How many cards does Lc0 scale up to?

Dann Corbit
Posts: 11218
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Leela and the curse of not finishing a won match cleanly.

Post by Dann Corbit » Wed Jun 10, 2020 7:52 pm

I don't think we know that max cards is for LC0.
I heard 2. Then I saw three running well.
Then I heard 3, and we see 4 in TCEC, with them running well.
I suspect it is a function of the OS, the cards, and LC0's cooperation algorithms.
Maybe someday we will have a better idea.
I wish it was not such a pain to set up.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

IanO
Posts: 481
Joined: Wed Mar 08, 2006 8:45 pm
Location: Portland, OR
Contact:

Re: Leela and the curse of not finishing a won match cleanly.

Post by IanO » Wed Jun 10, 2020 10:28 pm

Another possibility is to use stricter endgame counts which subsume the 50-move rule when training.

As an example, consider the Thai chess variant, Makruk. It has a host of different counts, which depend on the amount of material left.* Does anyone know if there is a project to apply the LC0 framework to Makruk?

* These counts only come into effect when the last pawn is traded off or promoted, so a NN might instead learn to preserve the last pawns as a general endgame technique.

Post Reply