Pondering? Yes. Ponder move? Maybe not.

[Daniel Shawul]

Is the result of your experiment available on your website ?

I switched to pondering strategy (I) but the behavior of the engine during ponder fails is very sad and noticeable with huge differences in ponder score and normal score. I might need to switch to ponder on the position when such score fluctuations are detected.

[Bill Rogers]

H.G.
I don't mean to be insulting maybe it is just a misunderstanding but you last sentence seems to be illogical. What I mean is if you are correct 60% of the time then how can you be wrong 90% of the time. Did you mean you could be wrong 40% of the time?
Bill

[plattyaj]

H.G.
I don't mean to be insulting maybe it is just a misunderstanding but you last sentence seems to be illogical. What I mean is if you are correct 60% of the time then how can you be wrong 90% of the time. Did you mean you could be wrong 40% of the time?
Bill

He didn't say it was "wrong" 90% of the time but that it "wasted" 90% of the available time.

The point being (and I have not verified or otherwise looked at this data myself) that if the ponder move is mostly predicted when the opponent is moving quickly, you have hardly had any time to ponder. But if it's mostly missed when the opponent has been thinking for a long time you have wasted all that time.

Andy.

[hgm]

I said: you can predict 60% of the ponder moves correctly, and still waste 90% (or more) of the time thinking on the remaining 40%, which you predicted incorrectly.

Suppose out of 50 opponent moves you predict 30 correctly, but the opponent thinks only 1 sec on each of them. So you spend 30 sec of useful pondering. Now on the other 20 opponent moves you picked a wrong one, and the opponent was thinking 15 sec on each of them. That would make 300 sec of pondering on a wrong move. Thus 300 out of 330 sec, or 90.9% would have been wasted by pondering the wrong move.

That was my point: it is not the _number_ of correctly predicted moves that is relevant, but the ponder _time_ spent on correctly predicted vs incorrectly predicted moves.

Note that many programs have an "easy-move" feature, where they move very quickly on a move that is very obvious (e.g. recapturing a Queen that can only be recaptured in one way). The extreme case is that of only a single legal move. Most programs move instantly in that case(and guess what, the opponent predicted the move correctly and started pondering on it for exactly 0 sec).

The longer an opponent thinks, the more likely it is that he switches move. Usually the move switch is the reason for the long think. But the ponder move was determined at comparatively shallow depth, so you are much more likely to ponder the move from before the switch, which will then be the wrong one. Thus it seems logical to expect a correlation: good predictions-short thinking, wrong predictions-long thinking.

[Harald]

An idea:

Let's assume that a chess program, capable of pondering, stores nearly all of its search results in a traditional transposition table. The usual ponder technique is to take the second move of the predicted variation, execute that move, and start an indefinitely lasting ponder search assuming that the opponent will make the predicted move. If the opponent makes the move, then everything's fine, the ponder search is converted into a direct search, and no work was wasted. But if a different move was made by the opponent, then a new direct search has to be started and much work was done for nothing.

How about NOT making the predicted move prior to a ponder search? Instead, just start a search with the opponent on the move. When the opponent actually makes a move, the ponder search is killed, but all of its goodness remains in the transposition table. That data is used on the direct search and there should be a good amount of valuable data there regardless of what move the opponent actually made.

For more discussion see these threads:

http://www.talkchess.com/forum/viewtopi ... 78&t=25494

http://www.talkchess.com/forum/viewtopi ... 34&t=24870

Harald

[bob]

Logically, pondering on the projected move is a benefit if you can guess the right move at least 50% of the time. Statistically, it appears that most engines achieve at least that hit rate for ponder against all peers:
...

This subject was indeed discussed before, and the conclusion then was that the ponder-hit rate was a meaningless statistic: an engine can have 60% ponder hits, and pondering the move can still be inferior, as the ponder hits occur when the the opponent moves fast or immediate, and the misses when he thinks long. So predicting 60% of the ponder moves correct can still mean that 90% of your pondering time is wasted, pondering the wrong move...

Whereas with the ponder the position aproach, you either give up a ply or don't save any time...

I'll take ponder the move any time. From testing both plus other alternatives not being discussed.

[bob]

I said: you can predict 60% of the ponder moves correctly, and still waste 90% (or more) of the time thinking on the remaining 40%, which you predicted incorrectly.

Suppose out of 50 opponent moves you predict 30 correctly, but the opponent thinks only 1 sec on each of them. So you spend 30 sec of useful pondering. Now on the other 20 opponent moves you picked a wrong one, and the opponent was thinking 15 sec on each of them. That would make 300 sec of pondering on a wrong move. Thus 300 out of 330 sec, or 90.9% would have been wasted by pondering the wrong move.

But how would "ponder the position rather than the move" get any benefit here? So it is a wash if your opponent moves quickly, which is intuitively obvious anyway. The only case where pondering helps is when your opponent takes a signficant amount of time to search, by definition. And there, ponder the move offers more.

That was my point: it is not the _number_ of correctly predicted moves that is relevant, but the ponder _time_ spent on correctly predicted vs incorrectly predicted moves.

Note that many programs have an "easy-move" feature, where they move very quickly on a move that is very obvious (e.g. recapturing a Queen that can only be recaptured in one way). The extreme case is that of only a single legal move. Most programs move instantly in that case(and guess what, the opponent predicted the move correctly and started pondering on it for exactly 0 sec).

The longer an opponent thinks, the more likely it is that he switches move. Usually the move switch is the reason for the long think. But the ponder move was determined at comparatively shallow depth, so you are much more likely to ponder the move from before the switch, which will then be the wrong one. Thus it seems logical to expect a correlation: good predictions-short thinking, wrong predictions-long thinking.

[krazyken]

Perhaps you can collect some data based on the ponder hit time. If (total ponder hit time)/(total ponder time) is > 50% then yes there is still nothing to be gained by trying other things.

[Edmund]

Maybe it would be interesting to test on a correlation between certain ratios and the ponder hit ratio.

For example if the search suggests that the node prediction is weak (not just from one ply to another, which would suggest a tactical plan, but rather a general node prediction uncertainty) it may be more likely that our current pv is pretty close in value to some other moves. Thus a ponder on position might be more beneficial here.

[bob]

Perhaps you can collect some data based on the ponder hit time. If (total ponder hit time)/(total ponder time) is > 50% then yes there is still nothing to be gained by trying other things.

I simply played tens of thousands of ponder-on games using several strategies that were proposed and a couple I wanted to try (for example, after pondering for the target time, and the opponent is still thinking, do a search to find the "second best" move (I used a search 1/10th the normal target, from the opponent's perspective and used this as the second best move since I excluded the original ponder move from searching completely this time around).

I found nothing that produced results as good as the simple ponder the best move approach. Some things were close (previous idea). Some were not as close. The "ponder the position" (something Crafty can do trivially since it already does this if it has no predicted move, so that it can find something to ponder) was at the bottom of the heap. I don't remember the precise results, but the best-to-worst Elo span was something like 50, where ponder the move was +50 better than the worst case idea.

As far as your 50% threshold above, we first need to pick a more accurate number to beat, since I don't believe any other pondering scheme can even reach 50%....