Null-moves and zugzwang

[bob]

Second, I just commented the code out in Glaurung and Stockfish (I currently use both but am going to just use stockfish in the future if I can find a replacement for Glaurung that doesn't have time loss issues and such.)

Please set UCI option "LSN filtering" to false if you don't want Stockfish to lose on time when in big disadvantage and few seconds remain.

In both cases, there was very little change, although I could see perhaps +1 to +2 Elo by removing the test, but it would take more games to have a lot of confidence in that number.

I am still doing the test now, after around 500 games at 1'+0" the version WITH zugzwang verification is ahead of about 6 ELO points.


BTW thanks a lot for testing with Stockfish. If you have any problem with SF setup (as the tendency to lose on time) please tell us, I will do my best to fix it. In this case is easy because is an intended behaviour

Your statement above doesn't make much sense to me. 500 games and +6 elo do not belong in the same sentence.

[bob]

But here's the question. Are you willing to give up a couple of Elo in real games, to get this effect in test positions?

It depends. I don't care any more than you do about solving test positions fast, but I do consider it a serious issue if my program is never able to solve a simple test position, even if given an infinite amount of time.

My goal is max elo. Nothing more, nothing less.

I'm a little bit more nuanced. I wouldn't want to sacrifice 50 Elo points in order to detect all zugzwangs, but I wouldn't hesitate to sacrifice 5 Elo points. It's easy to compensate by doing some improvement elsewhere in the program.

If I were worried about test positions, I would have left the various extensions in (mate threat, one legal move, passed pawn pushes) as they clearly made Crafty faster in the WAC-type positions.

But these are fundamentally different: Extensions are about solving some test positions a little faster, and not about solving positions which wouldn't otherwise have been solvable in any finite amount of time.

I don't agree. 99% of the people that include extensions in their search are doing so because they believe it plays _stronger_, not that it finds problem solutions faster. Unfortunately, they are testing it incorrectly.

But in real games they hurt. Some not much, others quite a bit. Anything that doesn't help goes unless there is some reason to keep the code in place. If it hurts, it goes, period.

I mostly agree, but with some reservations. "If it hurts" right now doesn't mean quite the same as "if it hurts" in some future descendant of the current program. This is an important point, which I'll try to explain in more detail some day when I'm not quite as tired as right now.

[Uri Blass]

Second, I just commented the code out in Glaurung and Stockfish (I currently use both but am going to just use stockfish in the future if I can find a replacement for Glaurung that doesn't have time loss issues and such.)

Please set UCI option "LSN filtering" to false if you don't want Stockfish to lose on time when in big disadvantage and few seconds remain.

In both cases, there was very little change, although I could see perhaps +1 to +2 Elo by removing the test, but it would take more games to have a lot of confidence in that number.

I am still doing the test now, after around 500 games at 1'+0" the version WITH zugzwang verification is ahead of about 6 ELO points.


BTW thanks a lot for testing with Stockfish. If you have any problem with SF setup (as the tendency to lose on time) please tell us, I will do my best to fix it. In this case is easy because is an intended behaviour

Your statement above doesn't make much sense to me. 500 games and +6 elo do not belong in the same sentence.

I do not see the problem with reporting results.

After 500 games the version with zugzwang verification performed slightly better.
It does not mean that the version with zugzwang verification is better.

Uri

[Dann Corbit]

But here's the question. Are you willing to give up a couple of Elo in real games, to get this effect in test positions?

It depends. I don't care any more than you do about solving test positions fast, but I do consider it a serious issue if my program is never able to solve a simple test position, even if given an infinite amount of time.

My goal is max elo. Nothing more, nothing less.

I'm a little bit more nuanced. I wouldn't want to sacrifice 50 Elo points in order to detect all zugzwangs, but I wouldn't hesitate to sacrifice 5 Elo points. It's easy to compensate by doing some improvement elsewhere in the program.

If I were worried about test positions, I would have left the various extensions in (mate threat, one legal move, passed pawn pushes) as they clearly made Crafty faster in the WAC-type positions.

But these are fundamentally different: Extensions are about solving some test positions a little faster, and not about solving positions which wouldn't otherwise have been solvable in any finite amount of time.

I don't agree. 99% of the people that include extensions in their search are doing so because they believe it plays _stronger_, not that it finds problem solutions faster. Unfortunately, they are testing it incorrectly.

But in real games they hurt. Some not much, others quite a bit. Anything that doesn't help goes unless there is some reason to keep the code in place. If it hurts, it goes, period.

I mostly agree, but with some reservations. "If it hurts" right now doesn't mean quite the same as "if it hurts" in some future descendant of the current program. This is an important point, which I'll try to explain in more detail some day when I'm not quite as tired as right now.

There are two goals of a chess program that are cross-purposes.
1. Analysis. Give me the best possible sequence of moves from here. Take as much time as you like.
2. Game play. Give me the move that maximizes my winning chances given the time limitations.

Optimal analysis does not necessarily make for optimal game play.

Perhaps the ultimate engine has two modes:
1. Analysis mode (which uses less extensions and so it never misses a best move)
2. Game play mode (which extends like a madman to squeeze out the tiniest little Elo fraction)

There may be other modes that have value (correspondence chess mode, win-finder mode, etc.)

All of these ideas have some value if properly implemented for the chess player.

[bob]

Second, I just commented the code out in Glaurung and Stockfish (I currently use both but am going to just use stockfish in the future if I can find a replacement for Glaurung that doesn't have time loss issues and such.)

Please set UCI option "LSN filtering" to false if you don't want Stockfish to lose on time when in big disadvantage and few seconds remain.

In both cases, there was very little change, although I could see perhaps +1 to +2 Elo by removing the test, but it would take more games to have a lot of confidence in that number.

I am still doing the test now, after around 500 games at 1'+0" the version WITH zugzwang verification is ahead of about 6 ELO points.


BTW thanks a lot for testing with Stockfish. If you have any problem with SF setup (as the tendency to lose on time) please tell us, I will do my best to fix it. In this case is easy because is an intended behaviour

Your statement above doesn't make much sense to me. 500 games and +6 elo do not belong in the same sentence.

I do not see the problem with reporting results.

After 500 games the version with zugzwang verification performed slightly better.
It does not mean that the version with zugzwang verification is better.

Uri

It does not make any sense when the _actual_ numbers look like this:

X +/-32
X+6 +/-32

Comparing them is pointless when the error bar is that large. What is your confidence that X+6 is better after only 500 games? Mine is almost zero. Which is why I don't report such numbers.

[Karlo Bala]

Based on testing I have done, and am right now repeating, you can throw this idea out. It does not help one bit.

That depends on what you try to achieve: I am sure you are right that it does not help one bit from the perspective of practical playing strength. It does help in the sense that many simple positions which are never solved without zugzwang verification are solved reasonably fast with zugzwang verification. This is the whole point. I don't like having a program which is unable to find simple wins even when given an infinite amount of time. Given infinite time, a chess program should be able to play perfectly from any position, with no other code changes than increasing a few arrays.

Zugzwang verification is very cheap, both in terms of code complexity and playing strength. I wouldn't want to remove it even if it turned out to cost 5--10 Elo points, and in practice I believe the cost is much smaller. If it does turn out to be more expensive than I think, I would increase the depth reduction and/or the depth limit for zugzwang verification rather than removing it altogether.

I found that NULL move work better if I try verification first (as condition to try NULL move at all) because verification search is cheaper then NULL move. Also it is useful as IID.

[Dann Corbit]

Based on testing I have done, and am right now repeating, you can throw this idea out. It does not help one bit.

That depends on what you try to achieve: I am sure you are right that it does not help one bit from the perspective of practical playing strength. It does help in the sense that many simple positions which are never solved without zugzwang verification are solved reasonably fast with zugzwang verification. This is the whole point. I don't like having a program which is unable to find simple wins even when given an infinite amount of time. Given infinite time, a chess program should be able to play perfectly from any position, with no other code changes than increasing a few arrays.

Zugzwang verification is very cheap, both in terms of code complexity and playing strength. I wouldn't want to remove it even if it turned out to cost 5--10 Elo points, and in practice I believe the cost is much smaller. If it does turn out to be more expensive than I think, I would increase the depth reduction and/or the depth limit for zugzwang verification rather than removing it altogether.

I found that NULL move work better if I try verification first (as condition to try NULL move at all) because verification search is cheaper then NULL move. Also it is useful as IID.

Doesn't the verification search give you the same data as the null move search? IOW, if you perform a verification search, do you even need to perform the null move search or could you just use the verification search return value instead?

[Karlo Bala]

Based on testing I have done, and am right now repeating, you can throw this idea out. It does not help one bit.

That depends on what you try to achieve: I am sure you are right that it does not help one bit from the perspective of practical playing strength. It does help in the sense that many simple positions which are never solved without zugzwang verification are solved reasonably fast with zugzwang verification. This is the whole point. I don't like having a program which is unable to find simple wins even when given an infinite amount of time. Given infinite time, a chess program should be able to play perfectly from any position, with no other code changes than increasing a few arrays.

Zugzwang verification is very cheap, both in terms of code complexity and playing strength. I wouldn't want to remove it even if it turned out to cost 5--10 Elo points, and in practice I believe the cost is much smaller. If it does turn out to be more expensive than I think, I would increase the depth reduction and/or the depth limit for zugzwang verification rather than removing it altogether.

I found that NULL move work better if I try verification first (as condition to try NULL move at all) because verification search is cheaper then NULL move. Also it is useful as IID.

Doesn't the verification search give you the same data as the null move search? IOW, if you perform a verification search, do you even need to perform the null move search or could you just use the verification search return value instead?

No, I think. First I try verification, which is basically IID with depth-6. If score is >= beta then and only then I make NULL move and try NM search with depth-4.

[Karlo Bala]

Based on testing I have done, and am right now repeating, you can throw this idea out. It does not help one bit.

That depends on what you try to achieve: I am sure you are right that it does not help one bit from the perspective of practical playing strength. It does help in the sense that many simple positions which are never solved without zugzwang verification are solved reasonably fast with zugzwang verification. This is the whole point. I don't like having a program which is unable to find simple wins even when given an infinite amount of time. Given infinite time, a chess program should be able to play perfectly from any position, with no other code changes than increasing a few arrays.

Zugzwang verification is very cheap, both in terms of code complexity and playing strength. I wouldn't want to remove it even if it turned out to cost 5--10 Elo points, and in practice I believe the cost is much smaller. If it does turn out to be more expensive than I think, I would increase the depth reduction and/or the depth limit for zugzwang verification rather than removing it altogether.

I found that NULL move work better if I try verification first (as condition to try NULL move at all) because verification search is cheaper then NULL move. Also it is useful as IID.

Doesn't the verification search give you the same data as the null move search? IOW, if you perform a verification search, do you even need to perform the null move search or could you just use the verification search return value instead?

No, I think. First I try verification, which is basically IID with depth-6. If score is >= beta then and only then I make NULL move and try NM search with depth-4.

It works "almost" like double NULL move, but without funny tricks. And, it dos not have problem with some depth like double NULL move. About drawbacks of double NULL move you can find in Christophe Theron's post somewhere in CCC archives.

[Dann Corbit]

Based on testing I have done, and am right now repeating, you can throw this idea out. It does not help one bit.

That depends on what you try to achieve: I am sure you are right that it does not help one bit from the perspective of practical playing strength. It does help in the sense that many simple positions which are never solved without zugzwang verification are solved reasonably fast with zugzwang verification. This is the whole point. I don't like having a program which is unable to find simple wins even when given an infinite amount of time. Given infinite time, a chess program should be able to play perfectly from any position, with no other code changes than increasing a few arrays.

Zugzwang verification is very cheap, both in terms of code complexity and playing strength. I wouldn't want to remove it even if it turned out to cost 5--10 Elo points, and in practice I believe the cost is much smaller. If it does turn out to be more expensive than I think, I would increase the depth reduction and/or the depth limit for zugzwang verification rather than removing it altogether.

I found that NULL move work better if I try verification first (as condition to try NULL move at all) because verification search is cheaper then NULL move. Also it is useful as IID.

Doesn't the verification search give you the same data as the null move search? IOW, if you perform a verification search, do you even need to perform the null move search or could you just use the verification search return value instead?

No, I think. First I try verification, which is basically IID with depth-6. If score is >= beta then and only then I make NULL move and try NM search with depth-4.

It works "almost" like double NULL move, but without funny tricks. And, it dos not have problem with some depth like double NULL move. About drawbacks of double NULL move you can find in Christophe Theron's post somewhere in CCC archives.

Do you do null move for shallow searches also?