Standard candles

[hgm]

OK, I see you already wrote that. So recognizing mate-in-1 is an enormous advantage (as could be expected).

[sje]

Another Tale of Two Players

Player Zero will play a randomly selected capture/promotion move if one is available; otherwise, it will play a randomly selected move.

Player One will play a randomly selected move.

What happens in a one million game match between these players? With alternating colors and from the viewpoint of Player Zero, the win/lose/draw/total counts are:

Code: Select all

[173,417/2,649/823,934/1,000,000]

[hgm]

This is an amazing result, in particular that the number of draws goes up so much. Do you have a breakdown of that number per draw cause? Is this because the number of stalemates goes up so much when the capturing side gets an overwhelming advantage?

Perhaps a better strategy would be to avoid capturing of Knights, but capture anything else as soon as you can. Leaving him the Knights would not get him stalemated so easily, while they represent little danger for yourself.

[sje]

Yet another Tale of Two Players

Player Zero will play a randomly selected checkmating move if one is available; otherwise, it will play a randomly selected move.

Player One will play a randomly selected capture/promotion move if one is available; otherwise, it will play a randomly selected move.

What happens in a one million game match between these players? With alternating colors and from the viewpoint of Player Zero, the win/lose/draw/total counts are:

Code: Select all

W/L/D/T: [41,362/167,011/791,627/1,000,000]

[sje]

Do you have a breakdown of that number per draw cause? Is this because the number of stalemates goes up so much when the capturing side gets an overwhelming advantage?

Sorry, no breakdown is available with the current configuration.

Before going too much further with player construction and match playing, I'd like to first see someone else work on this to confirm the general results I've computed.

[Don]

I would like to add something to this. Most of the rating lists are calibrated differently. Ipon uses the commercial Shredder 12 set to 2800 ELO and I don't know what other lists use.

But if we took SF 4 and freeze it in time - make it the standard and provide a level where you can actually set the nodes per second it could be "blessed" by the computer chess community as the reference standard and assigned a fixed rating of 3000 - for example. (On IPON it is at 3015 at the moment.)

Being able to set the nodes per second (a feature I am going to provide in Komodo from now on) allow is to be used as an ABSOLUTE hardware independent standard.

In the future computing platforms will evolve and Stockfish might not even run on the computers of tomorrow. But if the source code is preserved effort can be made to keep it up to date. If compilers evolve significantly or C++ features even become deprecated and go away the source code can be updated accordingly.

I think we should do this. Stockfish is the ideal candidate now as it is open source and represents something close the best we have. We probably do not want to start with something that is several hundred ELO below state the art when we have a good choice.

I agree 100% with hardware independence issue - but it can still have a normal time control if the nodes per second is strictly controlled.

In fact a really good reference program would be some open source program such as Stockfish with the only modification being that you can SET the nodes per second - AND/OR have a NIT level - nodes is time.

The advantage of being able to set the nodes per second is that you can run on ANY hardware and play matches with humans and get a REAL rating if you choose to. You could play a human time control - the only requirement being that the machine is fast enough to support the nodes per second you choose.

Of course for a static level you can support the fixed nodes level with no other change.

[sje]

Being able to set the nodes per second (a feature I am going to provide in Komodo from now on) allow is to be used as an ABSOLUTE hardware independent standard.

A nodes per second metric is not invariant as it is dependent on the processing speed of the host platform.

A "fixed node count limit" or "fixed iteration limit" might be suitable.

Even then there could be problems. Has the program been shown to have the same move ordering regardless of the endian sense of its host? Can it even run on a non Intel/AMD system? Do all the operations produce the same result regardless of 32 bit vs 64 bit word sizes? Is the search single-threaded so that its result is independent of thread scheduling on multiple core hosts? If the program uses assembly language intrinsic calls for bit scanning, what happens if those intrinsic routines are changed or are not always available?

I have seen all of the above and I have successfully resolved all such cases. This is not hard to do, but it does require care.

[Don]

Being able to set the nodes per second (a feature I am going to provide in Komodo from now on) allow is to be used as an ABSOLUTE hardware independent standard.

A nodes per second metric is not invariant as it is dependent on the processing speed of the host platform.

I have already implemented it. I can now tell Komodo to do 1 thousand nodes per second and it will do so. Of course it cannot go faster than the hardware permits but it can go slower.

It's easy to do with hardly any overhead - since this is a technical forum I'm not going to explain it unless someone asks but it's not complicated.

A "fixed node count limit" or "fixed iteration limit" might be suitable.

Even then there could be problems. Has the program been shown to have the same move ordering regardless of the endian sense of its host? Can it even run on a non Intel/AMD system? Do all the operations produce the same result regardless of 32 bit vs 64 bit word sizes? Is the search single-threaded so that its result is independent of thread scheduling on multiple core hosts? If the program uses assembly language intrinsic calls for bit scanning, what happens if those intrinsic routines are changed or are not always available?

I have seen all of the above and I have successfully resolved all such cases. This is not hard to do, but it does require care.

[Don]

Being able to set the nodes per second (a feature I am going to provide in Komodo from now on) allow is to be used as an ABSOLUTE hardware independent standard.

A nodes per second metric is not invariant as it is dependent on the processing speed of the host platform.

I have already implemented it. I can now tell Komodo to do 1 thousand nodes per second and it will do so. Of course it cannot go faster than the hardware permits but it can go slower.

It's easy to do with hardly any overhead - since this is a technical forum I'm not going to explain it unless someone asks but it's not complicated.

And the point is that could run Komodo at 10k nodes per second on ANY machine and expect to get exactly the same level of play.

A "fixed node count limit" or "fixed iteration limit" might be suitable.

Even then there could be problems. Has the program been shown to have the same move ordering regardless of the endian sense of its host? Can it even run on a non Intel/AMD system? Do all the operations produce the same result regardless of 32 bit vs 64 bit word sizes? Is the search single-threaded so that its result is independent of thread scheduling on multiple core hosts? If the program uses assembly language intrinsic calls for bit scanning, what happens if those intrinsic routines are changed or are not always available?

I have seen all of the above and I have successfully resolved all such cases. This is not hard to do, but it does require care.

[Michel]

I have already implemented it. I can now tell Komodo to do 1 thousand nodes per second and it will do so. Of course it cannot go faster than the hardware permits but it can go slower.

The xboard protocol has the nps option for this. Perhaps the corresponding UCI option should be standardized as well. Perhaps UCI_nps since it is similar to UCI_elo?

I guess a cheap UCI_elo implementation could just translate it into UCI_nps.