UCI extension: nodestime

[mcostalba]

What about pondering?

Pondering is not affected by this change. Indeed when pondering, you are free to use all the time you want as long as the opponent has not moved.

Sorry, that breaks the test and you are RIGHT back to where you started, unpredictable results. Think about what is happening. Wildly varying ponder time is just as bad as the wildly varying search time which you are concerned about. Your node fix doesn't change the ponder noise whatsoever.

Hence ponder=off or it won't work. Fixed nodes is also bad for several reasons as it eliminates time allocation which is an important part of a chess engine. The only real solution is dedicated hardware to get rid of the time jitter.

I understand ponder makes no sense in this context, but not because it is not possible, just because it is useless: here this scheme is effective in self-testing and possibly just tuning patches, so ponder has no reason to be. Ponder testing makes sense only to test time management patches, and in this case you really _don't_ want to use this mode.

Regarding fixed nodes games, please note that this is _not_ fixed nodes games, it is something very different, namely the engine is free to use only a fraction of available nodes, and how much of available nodes it uses is calculated with exactly the same time management formulas used in standard case. Eventually please read the other my post on this topic, so to better get this idea.

[Desperado]

Therefore you can produce stronger / weaker oppenents artifically somehow.

This of course has always been possible, by using time-odds games. The NIT approach was only conceived as a work-around for highly-variable machine loads. And it makes much faster TC possible than ay timed approach, as the latter have resolutions of 1 millisec (in UCI, 10 msec in WB protocol), which you can only realize by unusual timer calls, as the standard calls are based on clocks that tick every 16 msec. But that is mainly a consequence of delegating the measurement to the engine, which could have been done for a true time measurement as well.

Well, a true time measurement will never be that accurate like the NIT approach which allows absolute determenistic behaviour and precision on a single node.
This context is the difference, not that handicaps can be done with true time measurement too. It is a different control with very similar properties compared with true time measurement.

So, no need to say that it would not matter to play matches on a machine from the nineties or a super modern computer which would lead to the same results.
Based on this, it allows to have a user platform where each user can provide test games which you can merge finally. Having 1000 users only who would provide games in that way,
you can easily play 500000 games and integrate them into one database. Therefore it would be possible to measure 1 Elo improvements in very short time for example.
You can never do something like that with true time measurement and different hardware.

Just some more thoughts from my side.

[Daniel Mehrmann]

(1) How would the engine installer even know which GUIs are on the system?

(2) What happens if a new GUI is installed after the engine is installed?

That would be no problem, because the conf file would be part of the engine package. The conf file would be created always - With or without a installed gui. It's a common way for a lot of programs.
The gui package only create a (empty) conf.d directory if it doesn't exist.

So there would be a conf.d directory somewhere on the system in a known location (it would be created if it wasn't there) and the engine installer drops a configuration file in there...

Congratulations. You have rephrazed HGM's proposal (clumsily though).

I don't think so. Basicly it's completly different concept. However, i'm just tired and hgm is a nigtmare and he doesn't have to learn. Sadly. Oay, i'm out.

[Desperado]

I introduced the "NIT" ( Node Is Time ) option in my engine a long time ago now with a simple translation from time to nodes.

I think I miss something here: without help of the GUI how you can keep track of lost time?

For instance the engine can take 1 second or 2 second to compute the same number of nodes, for instance because in the second case is slowed down by some other background process. The GUI should account for the same elapsed time in both cases (because nodes searched are the same), but this is possible only if the GUI knows that nodes, and not elapsed time, is the reference value to look for. And this is possible only if engine tells the GUI that is operating in NIT mode.

So I miss how your scheme can work transparently without the GUI knowing you are in NIT mode.

well, it works like this:

1. True time behaviour

The GUI sends a time contingent. For a bullet game it starts with 60000ms.
Now the time manager ( of the engine ) will compute a base time for the next move like 1500ms.

So, nothing new so far. At this point the interaction of engine and GUI works like known for true time measurement.

2. Conversion from "True Time" into "Virtual Time"

But the engine does not care about the unit (ms). You can convert the unit into apples and oranges or simply nodes.
If you have 1500 ms you can convert it into 1500 * factor == nodes / move ( eg: 1500 * 100 = 150000 n/m )
The result would be, that the engine stops the search after 150000 nodes.

3. Speedup Trick

Well, normally your engine would compute 3 million nodes in the 1500 ms, but you already stopped after 150000 nodes.
Your virtual time runs faster than the true time caused by the factor. ( in this case the speedup would be 20 )
Ok, that means that the game may end with a rest time on the clock of 50000ms.
Finally the engine plays much faster intentionally than it is required, so you will avoid any timeloss.
The only thing which must be considered is that "time" * "factor" == nodes/Move < nps (noticeable smaller than nps)

4. The GUI / any other game playing interface

The GUI simply does not care if engines play faster, but if the engine runs out of time although of a speedup,
well, then the GUI will do what it always is doing. It reports a time loss.

That is all. Hope i could explain it somehow.
In that way you get a very similar true time measurement behaviour with full control of the "virtual time", repeatable ( deterministic if you like) games and the feature i really like, that you can produce identical results on different machines. ( the duration of producing the games will be different, but not the results )

[hgm]

Just some more thoughts from my side.

All true. You don't have to convince me that this idea has a lot of useful applications. That is exactly why I added it to WinBoard, 6 years ago.

[Desperado]

Just some more thoughts from my side.

All true. You don't have to convince me that this idea has a lot of useful applications. That is exactly why I added it to WinBoard, 6 years ago.

[bob]

Sure, this is not recommended with ponder on. Just as playing time-odds games is meaningless with ponder on. That doesn't mean that it is useless to have this. Just that you should be careful to use it properly. The GUI could always disable / refuse to use the feature when ponder is on.

I'd bet that would prevent dozens of questions.

[bob]

What about pondering?

Pondering is not affected by this change. Indeed when pondering, you are free to use all the time you want as long as the opponent has not moved.

Sorry, that breaks the test and you are RIGHT back to where you started, unpredictable results. Think about what is happening. Wildly varying ponder time is just as bad as the wildly varying search time which you are concerned about. Your node fix doesn't change the ponder noise whatsoever.

Hence ponder=off or it won't work. Fixed nodes is also bad for several reasons as it eliminates time allocation which is an important part of a chess engine. The only real solution is dedicated hardware to get rid of the time jitter.

I understand ponder makes no sense in this context, but not because it is not possible, just because it is useless: here this scheme is effective in self-testing and possibly just tuning patches, so ponder has no reason to be. Ponder testing makes sense only to test time management patches, and in this case you really _don't_ want to use this mode.

Regarding fixed nodes games, please note that this is _not_ fixed nodes games, it is something very different, namely the engine is free to use only a fraction of available nodes, and how much of available nodes it uses is calculated with exactly the same time management formulas used in standard case. Eventually please read the other my post on this topic, so to better get this idea.

It really is the same thing however. You are limiting the search based on a number of nodes. Which changes over the course of the game, sometimes by a factor of 3x or more. So it is an odd artificial limit that like all artificial limits other than time, can distort the results.

If your goal is to compare say stockfish vs XYZ, before and after a change, that might work perfectly well, since all you care about is if the new version is better or not. But the actual results (which side wins more games) might not be exactly representative of anything. And it certainly would not be a good way to run any sort of tournament.

Only problem with these kinds of ideas is that they can work pretty well in a narrow set of circumstances, yet they will inevitably be used in a much wider range of circumstances and then we see the "WOW XXX beats YYY convincingly in my tests, how is everybody else screwing their tests up since they are getting different results?"

[mcostalba]

3. Speedup Trick

Well, normally your engine would compute 3 million nodes in the 1500 ms, but you already stopped after 150000 nodes.
Your virtual time runs faster than the true time caused by the factor. ( in this case the speedup would be 20 )
Ok, that means that the game may end with a rest time on the clock of 50000ms.
Finally the engine plays much faster intentionally than it is required, so you will avoid any timeloss.

Ok, now I understand, you keep the GUI out of this picture playing much faster than the available time.

But this alters the available time progression during the game. For instance in normal case you start with, say, 10 seconds available time and after 20 moves you may remain with 5 seconds, after another 20 moves with 2 seconds.

Instead in your case you start with 10 seconds an after 40 moves you still have 9 seconds on the clock, because engines played much faster.

So your speed up trick yields to a game played with (almost) fixed number of nodes per move and you lose the usual decrease of time during the match.

[mcostalba]

Only problem with these kinds of ideas is that they can work pretty well in a narrow set of circumstances, yet they will inevitably be used in a much wider range of circumstances and then we see the "WOW XXX beats YYY convincingly in my tests, how is everybody else screwing their tests up since they are getting different results?"

It is very clear that you _can't_ use this mode to play a casual match between casual players.

Engines must have same nps speed across all the game for results to be reliable and this limits to self-play and even not a general patch, but a patch that does not alters speed, so mainly a parameter tuning patch.