Is the 320x24b larger net the strongest around for RTX GPU?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

corres
Posts: 3657
Joined: Wed Nov 18, 2015 11:41 am
Location: hungary

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by corres »

corres wrote: Wed Jul 24, 2019 1:11 pm I think during an infinite analysis and one Leela`s game the enhancement of Leela`s memory usage is a natural consequence of the usage of MCTS-like search.
But the lack of reset between two consecutive game is a real bug.
From this bug Leela may yield benefit: During the next game she can use the results of searches of earlier games.
Note
This bug may influence the results of line of games between Leela and other (AB) engines.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by zullil »

corres wrote: Wed Jul 24, 2019 1:40 pm
corres wrote: Wed Jul 24, 2019 1:11 pm I think during an infinite analysis and one Leela`s game the enhancement of Leela`s memory usage is a natural consequence of the usage of MCTS-like search.
But the lack of reset between two consecutive game is a real bug.
From this bug Leela may yield benefit: During the next game she can use the results of searches of earlier games.
Note
This bug may influence the results of line of games between Leela and other (AB) engines.
No one seems to confirm that this "bug" exists. Are you using a GUI? Is the GUI issuing a 'ucinewgame' after each game?
corres
Posts: 3657
Joined: Wed Nov 18, 2015 11:41 am
Location: hungary

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by corres »

zullil wrote: Wed Jul 24, 2019 3:29 pm
corres wrote: Wed Jul 24, 2019 1:40 pm
corres wrote: Wed Jul 24, 2019 1:11 pm I think during an infinite analysis and one Leela`s game the enhancement of Leela`s memory usage is a natural consequence of the usage of MCTS-like search.
But the lack of reset between two consecutive game is a real bug.
From this bug Leela may yield benefit: During the next game she can use the results of searches of earlier games.
Note
This bug may influence the results of line of games between Leela and other (AB) engines.
No one seems to confirm that this "bug" exists. Are you using a GUI? Is the GUI issuing a 'ucinewgame' after each game?
Instead of superfluous debate you do make some experiment.
As I noted I use Fritz 11, Fritz 14 and Fritz 16 GUI.
I check the memory usage of Leela with hwinfo 5.5 and windows task manager.
In general "Ucinewgame" does not work in the case of Stockfish and other AB engines either.
Moreover you could use "Ucinewgame" manually only.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by zullil »

corres wrote: Wed Jul 24, 2019 4:09 pm
zullil wrote: Wed Jul 24, 2019 3:29 pm
corres wrote: Wed Jul 24, 2019 1:40 pm
corres wrote: Wed Jul 24, 2019 1:11 pm I think during an infinite analysis and one Leela`s game the enhancement of Leela`s memory usage is a natural consequence of the usage of MCTS-like search.
But the lack of reset between two consecutive game is a real bug.
From this bug Leela may yield benefit: During the next game she can use the results of searches of earlier games.
Note
This bug may influence the results of line of games between Leela and other (AB) engines.
No one seems to confirm that this "bug" exists. Are you using a GUI? Is the GUI issuing a 'ucinewgame' after each game?
Instead of superfluous debate you do make some experiment.
As I noted I use Fritz 11, Fritz 14 and Fritz 16 GUI.
I check the memory usage of Leela with hwinfo 5.5 and windows task manager.
In general "Ucinewgame" does not work in the case of Stockfish and other AB engines either.
Moreover you could use "Ucinewgame" manually only.
Sorry, I don't use Windows or any of those GUIs. But many other Leela users probably do, so if this bug exists at all it should be simple for someone to reproduce.
brianr
Posts: 536
Joined: Thu Mar 09, 2006 3:01 pm

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by brianr »

There is a clear difference in match results with and without the restart=on option in cutechess-cli.
Below are two identical Leelas.

Score of A vs B: 31 - 18 - 51 [0.565] 100
Elo difference: 45.42 +/- 47.89

Dead even within one game with restart=on
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by Laskos »

brianr wrote: Wed Jul 24, 2019 4:26 pm There is a clear difference in match results with and without the restart=on option in cutechess-cli.
Below are two identical Leelas.

Score of A vs B: 31 - 18 - 51 [0.565] 100
Elo difference: 45.42 +/- 47.89

Dead even within one game with restart=on
Could you elaborate bit?
A with 31 wins is the restart=off and B with 18 wins is the restart=on same Leela, right?

What openings do you use? I would expect that by using sufficiently different openings, saving the Hash between the games would help minimally. I saw the same effect as yours in Komodo.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by zullil »

brianr wrote: Wed Jul 24, 2019 4:26 pm There is a clear difference in match results with and without the restart=on option in cutechess-cli.
Below are two identical Leelas.

Score of A vs B: 31 - 18 - 51 [0.565] 100
Elo difference: 45.42 +/- 47.89

Dead even within one game with restart=on
Interesting, but not entirely clear to me. In the 31-18-51 match, Leela B was forced to restart each game but Leela A was not? And then a dead-even match when both engines restarted each game?
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by zullil »

corres wrote: Wed Jul 24, 2019 4:09 pm
zullil wrote: Wed Jul 24, 2019 3:29 pm
corres wrote: Wed Jul 24, 2019 1:40 pm
corres wrote: Wed Jul 24, 2019 1:11 pm I think during an infinite analysis and one Leela`s game the enhancement of Leela`s memory usage is a natural consequence of the usage of MCTS-like search.
But the lack of reset between two consecutive game is a real bug.
From this bug Leela may yield benefit: During the next game she can use the results of searches of earlier games.
Note
This bug may influence the results of line of games between Leela and other (AB) engines.
No one seems to confirm that this "bug" exists. Are you using a GUI? Is the GUI issuing a 'ucinewgame' after each game?
Instead of superfluous debate you do make some experiment.
As I noted I use Fritz 11, Fritz 14 and Fritz 16 GUI.
I check the memory usage of Leela with hwinfo 5.5 and windows task manager.
In general "Ucinewgame" does not work in the case of Stockfish and other AB engines either.
Moreover you could use "Ucinewgame" manually only.
It seems strange to me that a GUI would not send 'ucinewgame' to an engine at the start of a new game. So I assume that Fritz does this. And if it does, then Lc0 does the following in response:

Code: Select all

void EngineController::NewGame() {
  // In case anything relies upon defaulting to default position and just calls
  // newgame and goes straight into go.
  move_start_time_ = std::chrono::steady_clock::now();
  SharedLock lock(busy_mutex_);
  cache_.Clear();
  search_.reset();
  tree_.reset();
  time_spared_ms_ = 0;
  current_position_.reset();
  UpdateFromUciOptions();
}
So if there is a bug, it seems like it's a GUI bug.
User avatar
pohl4711
Posts: 2439
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by pohl4711 »

zullil wrote: Wed Jul 24, 2019 5:11 pm
It seems strange to me that a GUI would not send 'ucinewgame' to an engine at the start of a new game. So I assume that Fritz does this. And if it does, then Lc0 does the following in response:
The problem is, that the FritzGUI does it NOT. It is a known weakness since Fritz 6, that no "ucinewgame"-command is sent to the engines, when a new game starts in an engine match. And it was never fixed by ChessBase, because they believe, it is "not necessary"... in an engine-tournament, the engines are reloaded for every game, so there it is not a problem.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Is the 320x24b larger net the strongest around for RTX GPU?

Post by zullil »

pohl4711 wrote: Wed Jul 24, 2019 5:56 pm
zullil wrote: Wed Jul 24, 2019 5:11 pm
It seems strange to me that a GUI would not send 'ucinewgame' to an engine at the start of a new game. So I assume that Fritz does this. And if it does, then Lc0 does the following in response:
The problem is, that the FritzGUI does it NOT. It is a known weakness since Fritz 6, that no "ucinewgame"-command is sent to the engines, when a new game starts in an engine match. And it was never fixed by ChessBase, because they believe, it is "not necessary"... in an engine-tournament, the engines are reloaded for every game, so there it is not a problem.
Oh, thanks for that information. What does "reloaded" mean? Does it mean that each engine is terminated and then the engine binaries are restarted?