Losing on time

[Greg Strong]

I'm having problems with my engine, Quadrox, losing on time and I can't imagine why. Actually, I had been using Arena for my testing and I didn't have the problem. I've now switched to cutechess-cli and I'm having lots of problems... And it's not super-fast time controls, either. 30 sec + 1 sec increment.

At the top of the search functions I am incrementing the node count and checking time every 1024 nodes. I could check more frequently, but I can't believe that's necessary. The absolute max time to use is set at time remaining / 4 if there's an increment and time remaining / 8 otherwise. After unmaking a move in the move loop, if there's been a timeout, I return immediately (without updating PV or anything.) The way I'm doing it is very similar to glaurung/stockfish (at least as far as I can tell.)

Any ideas?

Thanks, and happy new year!!!

[Sven]

I'm having problems with my engine, Quadrox, losing on time and I can't imagine why. Actually, I had been using Arena for my testing and I didn't have the problem. I've now switched to cutechess-cli and I'm having lots of problems... And it's not super-fast time controls, either. 30 sec + 1 sec increment.

At the top of the search functions I am incrementing the node count and checking time every 1024 nodes. I could check more frequently, but I can't believe that's necessary. The absolute max time to use is set at time remaining / 4 if there's an increment and time remaining / 8 otherwise. After unmaking a move in the move loop, if there's been a timeout, I return immediately (without updating PV or anything.) The way I'm doing it is very similar to glaurung/stockfish (at least as far as I can tell.)

Any ideas?

Thanks, and happy new year!!!

Hi Greg,

happy new year to you!
I would print all time control related values (total elapsed/remaining time for game, elapsed/remaining time for current move, ...) at various points, e.g.:
- prior to starting the search (after allocating time for this search),
- perhaps at the end of each iteration,
- when detecting the timeout,
- when delivering the best move to the opponent.
This way you should be able to find out whether it is your time control implementation that causes the time losses, or any external influence. If significant portions of thinking time are not used for searching but "somewhere else" then you need to find out where and why that happens. But sometimes it is the TC algorithm itself that is responsible.

Have you also checked whether other time control styles, like 40/4 (i.e. no increment), are affected as well? If not then it might be a problem specific for handling time increment, either by your engine or by the testing environment.

Does Quadrox use the WB or the UCI protocol?

Sven

[Greg Strong]

Thanks, Sven, those are good ideas. I haven't tested other types of time controls, although that sounds obvious now

Quadrox closely resembles Stockfish, but uses Winboard protocol (because it will be a multi-variant engine...)

[lucasart]

At the top of the search functions I am incrementing the node count and checking time every 1024 nodes. I could check more frequently, but I can't believe that's necessary. The absolute max time to use is set at time remaining / 4 if there's an increment and time remaining / 8 otherwise. After unmaking a move in the move loop, if there's been a timeout, I return immediately (without updating PV or anything.) The way I'm doing it is very similar to glaurung/stockfish (at least as far as I can tell.)

What do you mean by "return immediately"? If you do like Glaurung/Stockfish, that means you use a global variable "bool AbortSearch", and test it to gradually exit the various recursion levels. This is actually difficult and prone to errors. The problem of this solution is that there are SO MANY ways the search can recurse again. To be sure you would have to put a line like

Code: Select all

if (AbortSearch) return 0

in every search function (if you have several), and after _every_ call to the search() or the qsearch(). I don't like this solution, and instead I would recommend the "big ugly cross function goto" (ie. "longjmp()" in C or "throw" in C++). It's much easier to get it right (at least if your search is single threaded).

[bob]

I'm having problems with my engine, Quadrox, losing on time and I can't imagine why. Actually, I had been using Arena for my testing and I didn't have the problem. I've now switched to cutechess-cli and I'm having lots of problems... And it's not super-fast time controls, either. 30 sec + 1 sec increment.

At the top of the search functions I am incrementing the node count and checking time every 1024 nodes. I could check more frequently, but I can't believe that's necessary. The absolute max time to use is set at time remaining / 4 if there's an increment and time remaining / 8 otherwise. After unmaking a move in the move loop, if there's been a timeout, I return immediately (without updating PV or anything.) The way I'm doing it is very similar to glaurung/stockfish (at least as far as I can tell.)

Any ideas?

Thanks, and happy new year!!!

Do you produce a log file that shows the target search time at the start of the search and the actual time used at the end?

If you look through that log, you almost certainly overstep every single target by some small amount. How large is that amount? If it is more than a second, something is up.

Do you do anything odd like "I am going to finish this iteration no matter what." or do you always have some limit (Crafty prints two times at the start of a search, the target, and the absolute max time limit (which is 5x the original target). I can look at a log and see exactly what time decisions were made and why (fail low, use more time, changing mind too often, use more time, locked on a single move, use less time, etc...)

A good log file will help find such issues instantly.

[Greg Strong]

Thanks, everyone! With some detailed logging I was able to track it down. No wonder I couldn't find it - I was looking in totally the wrong place.

Turns out it was related to the switch from Arena to CuteChess. CuteChess doesn't send the "white" and "black" commands to xboard engines and I was relying on that (which isn't correct for protocol version 2 engines like Quadrox.) So, sometimes it thought the time remaining was actually the opponent's time remaining.

So now I'm not losing on time anymore. I'm still getting way fewer wins than I was under Arena, though, so I wonder what else is wrong... I'll look at the debug outputs of the programs and see if they send other commands differently...

Cheers,
Greg

[bob]

Thanks, everyone! With some detailed logging I was able to track it down. No wonder I couldn't find it - I was looking in totally the wrong place.

Turns out it was related to the switch from Arena to CuteChess. CuteChess doesn't send the "white" and "black" commands to xboard engines and I was relying on that (which isn't correct for protocol version 2 engines like Quadrox.) So, sometimes it thought the time remaining was actually the opponent's time remaining.

So now I'm not losing on time anymore. I'm still getting way fewer wins than I was under Arena, though, so I wonder what else is wrong... I'll look at the debug outputs of the programs and see if they send other commands differently...

Cheers,
Greg

Surely you mean "time" and "otim" which are the commands it sends to tell you how much time you (time) and opponent (otim) have left on clock???

[mcostalba]

in every search function (if you have several), and after _every_ call to the search() or the qsearch(). I don't like this solution, and instead I would recommend the "big ugly cross function goto" (ie. "longjmp()" in C or "throw" in C++). It's much easier to get it right (at least if your search is single threaded).

In SMP this I think does not work. I have tried in the past to use exceptions and they worked in single thread, but not in SMP.

I have also tried to catch exceptions just upon returning from split node, handle the split point housekeeping and re-throw: it seemed to work, but on real test games it crashed often.

Perhaps it is possible to make it right but is really non trivial and the speed gain is really low. Currently stop flag is checked on entering a search function and once each moves loop cycle: this ensures fast returns (although not instant ones) and safe housekeeping under any case.

[Greg Strong]

Thanks, everyone! With some detailed logging I was able to track it down. No wonder I couldn't find it - I was looking in totally the wrong place.

Turns out it was related to the switch from Arena to CuteChess. CuteChess doesn't send the "white" and "black" commands to xboard engines and I was relying on that (which isn't correct for protocol version 2 engines like Quadrox.) So, sometimes it thought the time remaining was actually the opponent's time remaining.

So now I'm not losing on time anymore. I'm still getting way fewer wins than I was under Arena, though, so I wonder what else is wrong... I'll look at the debug outputs of the programs and see if they send other commands differently...

Cheers,
Greg

Surely you mean "time" and "otim" which are the commands it sends to tell you how much time you (time) and opponent (otim) have left on clock???

The "white" and "black" commands tell the engine what side it is playing so that when it receives the "time" and "otim" commands it knows which side is which. Those are obsolete now, so I guess whenever the engine is asked to think it should use "time" for that side...

[hgm]

The 'white' and 'black' commands primarily tell the engine which side currently has the move. They should be considered as part of the 'edit' command for setting up position (which just alters the board state, but leaves the side to move unchanged). To prevent that this would set the engine thinking, it was given the side effect to set the engine always playing the side that is not to move. This was probably a bad idea, (inhherited with all those other bad ideas from GNU Chess.) As the FEN in 'setboard' does specify a color, and can specify either one, so to prevent triggering an engine search it was given the side effect to swicth to force mode.

There never should be any need to send 'white' or 'black' to an engine that supports 'setboard'.

XBoard uses a hideous kludge to avoid using 'black' after setting up a position with 'edit' when before that white had the move (because a new game was started with 'new'): it sends a dummy move a2a3 to the engine in force mode to give black the move before it uses 'edit'. This way ''black needs never be used, and was declared deprecated.

In general variants won't have a piece on a2 that can legally move to a3, however (Xiangqi, Shogi, Makruk...). And the 'setboard' using FEN isn't so hot when pieces can no longer be indicated by single letters, and FEN becomes awkward. So for variant engines I don't consider 'black' deprecated at all, and 'edit' is a very convenient solution for variants where no FEN format can be defined.

It is ill-adviced to use commands for any purpose other than which the specs say they should perform, however.