Hi,
It looks like my program plays way worse when it is not restarted. Most likely this is due to a tt change I did a couple of weeks ago.
I would like to test this of course. Usually I use cutechess-cli for testing (because it is command line so I can easily start it on all nodes of the test cluster) but using strace I saw that it restarts every program after an iteration which gives me a clean transposition table every time.
So I wonder: does anyone know a way around this? I've got 4 versions to test.
cutechess-cli: not restarting an engine because of tt
Moderators: hgm, Rebel, chrisw
-
- Posts: 1753
- Joined: Tue Apr 19, 2016 6:08 am
- Location: U.S.A
- Full name: Andrew Grant
Re: cutechess-cli: not restarting an engine because of tt
I reset History, Pawn Table, and Trans Table between searches. I leave the killer moves in tact, as well as counter moves.
If you actually notice a big difference due to not restarting, I imagine you have a bug of sorts.
Are you using a different set of zorbist keys between games?
If you actually notice a big difference due to not restarting, I imagine you have a bug of sorts.
Are you using a different set of zorbist keys between games?
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
-
- Posts: 1753
- Joined: Tue Apr 19, 2016 6:08 am
- Location: U.S.A
- Full name: Andrew Grant
Re: cutechess-cli: not restarting an engine because of tt
I'm sure if you try zeroing out all of your tables between games you will find no losses.
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
-
- Posts: 2487
- Joined: Tue Aug 30, 2016 8:19 pm
- Full name: Rasmus Althoff
Re: cutechess-cli: not restarting an engine because of tt
I just reset TT, history and PV if it's a new game or a different position.flok wrote:It looks like my program plays way worse when it is not restarted.
That's a bit tricky with UCI because you have to work around a stateless protocol while maintaining state, but still possible:
- after every move the engine makes, store the position hash and the ply counter.
- if you get a "go", first check that the ply counter is exactly one more than it was after the engine made its last move.
- if so, take back the last ply. Check if the position hash is identical to what it was after the engine made its last move.
- if you have a match, you can keep hash tables, history and PV for move sorting. Otherwise, reset everything.
Then redo the ply you undid and launch the search.
Re: cutechess-cli: not restarting an engine because of tt
This is my replacement scheme:
"age" is increased between moves and notvalid is set when the program is started (and the tt initialized)
Code: Select all
int useSubIndex = -1, worstDepth = 99999;
for(int i=0; i<N_TE_PER_HASH_GROUP; i++)
{
if (e[i].hash == hash) {
useSubIndex = i;
break;
}
if (e[i].age != age || e[i].flags == NOTVALID || e[i].depth < worstDepth) {
worstDepth = e[i].depth;
useSubIndex = i;
}
}
// here the bin with index "useSubIndex" is replaced
Re: cutechess-cli: not restarting an engine because of tt
[/quote]Ras wrote:I just reset TT, history and PV if it's a new game or a different position.flok wrote:It looks like my program plays way worse when it is not restarted.
That should be easy to test. Will do so!
-
- Posts: 3232
- Joined: Mon May 31, 2010 1:29 pm
- Full name: lucasart
Re: cutechess-cli: not restarting an engine because of tt
Cutechess CLI does NOT restart engines, if you use it correctly...flok wrote:Usually I use cutechess-cli for testing (because it is command line so I can easily start it on all nodes of the test cluster) but using strace I saw that it restarts every program after an iteration which gives me a clean transposition table every time.
It just sends "ucinewgame" between games. If you want to clear HT between games, that's where you should do it: upon receipt of "ucinewgame".
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
Re: cutechess-cli: not restarting an engine because of tt
lucasart wrote:Cutechess CLI does NOT restart engines, if you use it correctly...flok wrote:Usually I use cutechess-cli for testing (because it is command line so I can easily start it on all nodes of the test cluster) but using strace I saw that it restarts every program after an iteration which gives me a clean transposition table every time.
It just sends "ucinewgame" between games. If you want to clear HT between games, that's where you should do it: upon receipt of "ucinewgame".
Can you show me a correct example?
I usually do this:
Code: Select all
cutechess-cli \
-engine cmd=/usr/local/bin/dorpsgek proto=xboard name=dorpsgek \
-engine cmd=./E_trunk.sh proto=uci name=trunk \
-engine cmd=./E_trunk-age.sh proto=uci name=trunk-age \
-engine cmd=./E_4171.sh proto=uci name=4171 \
-engine cmd=./E_4173.sh proto=uci name=4173 \
-engine cmd=./E_4182.sh proto=uci name=4182 \
-engine cmd=./E_4183.sh proto=uci name=4183 \
-concurrency $CONC \
-each dir=$DIR tc=40/10+0.25 -rounds 500 -recover -repeat -tournament gauntlet -pgnout out/$HOST.pgn -site "$HOST" | tee log/$HOST.log
-
- Posts: 27790
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: cutechess-cli: not restarting an engine because of tt
This would only work when the engine plays itself, right? Othersise you would have to take back two ply. And it doesn't work at all when the engine ponders.Ras wrote:I just reset TT, history and PV if it's a new game or a different position.
That's a bit tricky with UCI because you have to work around a stateless protocol while maintaining state, but still possible:
- after every move the engine makes, store the position hash and the ply counter.
- if you get a "go", first check that the ply counter is exactly one more than it was after the engine made its last move.
- if so, take back the last ply. Check if the position hash is identical to what it was after the engine made its last move.
- if you have a match, you can keep hash tables, history and PV for move sorting. Otherwise, reset everything.
Then redo the ply you undid and launch the search.
Why would you want to clear the TT on a new game anyway? Empty entries are not more helpful than entires from another game. As long as you are still in the same variant, that is. Aging would make the entries from the previous search be considered empty pretty quickly anyway.
A simpler method would be to remember the last two position-move commands, and see how much they have in common. If the sum of the length of the non-common parts is below a certain limit (like 20 characters), you can assume the same game. That would prevent clearing of tables on a ponder miss, or a takeback.