Uri Blass wrote:I can add that another reason for not being deterministic for programs is that search is effected by result of previous searches and movei does not suffer from that problem.
I think that at level of 2700-2750 CCRL rating this is a mistake to spend time on things like that because it will make debugging error later harder and it is probably better to try to find bigger improvements by other means.
Uri
So you want to use what we call the "goose effect"??? That is, every time your program "wakes up" it is a brand new day, no day before, no tomorrow, just search for now and give no thought to what will happen tomorrow?
I don't play chess like that. I don't think it even remotely reasonable that a computer plays chess like that...
Uri Blass wrote:I can add that another reason for not being deterministic for programs is that search is effected by result of previous searches and movei does not suffer from that problem.
I think that at level of 2700-2750 CCRL rating this is a mistake to spend time on things like that because it will make debugging error later harder and it is probably better to try to find bigger improvements by other means.
Uri
So you want to use what we call the "goose effect"??? That is, every time your program "wakes up" it is a brand new day, no day before, no tomorrow, just search for now and give no thought to what will happen tomorrow?
I don't play chess like that. I don't think it even remotely reasonable that a computer plays chess like that...
I think it's the "UCI effect"
Good point. Or the old chessbase winboard adapter effect...
both suck like nothing has ever sucked before...
--beavis and butthead
Uri Blass wrote:Note that Movei also complete the iteration in almost all cases in ponder off games(in ponder on games it may play immediatly in part of the cases but otherwise it will also try to finish the iteration).
It seems that Bob is basing his opinion based on the past:
He says:
"complete iteration searches is a primitive idea that almost everyone stopped using in the late 70's and early 80's."
I can say that most programmers never programmed chess engines in the late 70's or early 80's and many do not care about the small improvement that they may get by not finishing the iteration.
Uri
If you do an instant reply termination, you have the _same_ problem in ponder=on matches. So what exactly is your point??? It doesn't matter where the variability comes from. The fact that it is there at all is the main point here. And doing what you do will cause two types of variability. The first is time usage changes since you won't move until your opponent moves and how he uses time is beyond your control and probably varies as well. The second is whatever data your search alters while it runs (at least the transposition table) so that if your opponent takes longer on this move than the last time we played this game, your hash table has different stuff in it, history counters are changed. Killer moves are changed. Etc. And all of those add up to the potential for a different move or score from this run even if you do use the same amount of time or search to the same depth.
Again, we drift far afield where things are still as I said they are...
I practically clear the hash after every search(or to be more correct increase generation number so the program does not use positions with different generation number)
I also reset the killers at the beginning of every search.
I use today only knowledge and number of move in the order of moves
for pruning but if I use history I will reset the history table after every search.
different searches are clearly undependent regardless to the question if it is ponder off or ponder on games.
Uri
First, not using old hash entries is a horrible idea. If you don't trust the values, get rid of the table. I _never_ clear the entries. Because I trust them for as long as they stay around. Just as much as I trust my memory of positions in the fried-liver attack and would not do a "brain purge" and have to re-learn all of that stuff again.
Why would you eliminate _any_ data that can help your move ordering and save you time or give you better results in the same time frame?
Your last statement is 100% wrong. Two successive positions in a real game of chess are _not_ independent in any shape, form or fashion. They are part of a "stream" of positions that are related based on piece/pawn placement and previous moves played in the game. If this were true as you stated it, we would not even worry about the 50 move rule or repetitions.
Further, If I complete a 16 ply search and start pondering, I don't start over, I start at depth=15, since I assume you will play the second move, leaving me 14 plies of stuff completed, and I see no reason to re-search those trees again when I already have the result...
That kind of information loss I don't understand...
I did not explain myself well in the last statement.
I meant to say that for my program different searches are independent.
Uri Blass wrote:I can add that another reason for not being deterministic for programs is that search is effected by result of previous searches and movei does not suffer from that problem.
I think that at level of 2700-2750 CCRL rating this is a mistake to spend time on things like that because it will make debugging error later harder and it is probably better to try to find bigger improvements by other means.
Uri
So you want to use what we call the "goose effect"??? That is, every time your program "wakes up" it is a brand new day, no day before, no tomorrow, just search for now and give no thought to what will happen tomorrow?
I don't play chess like that. I don't think it even remotely reasonable that a computer plays chess like that...
I also do not play chess like that but my engine does.
There are things that are more important for me than playing strength
and it is being able to reproduce the analysis later.
I am sure that it is possible to earn something by learning from previous searches(I guess something like 20 elo) but when the gap between movei and rybka is near 300 elo I consider these 20 elo as unimportant.
Gerd Isenberg wrote:I mean in each game of the match the time from move to move fluctuates randomly as mentioned. How does it affect the variance or the occurrence of your mentioned "gang x" events?
That depends on the engines in question. In engines that play practically deterministically, so that such events with constant move timing would not occur at al, varying the move timing, and varying it differently and independently in each game, could cause induce variability so that in the end the events could occur once in a million times.
If the engines had already enough intrinsic variability to cause such events to occur once in a million times, they would still occur uonce in a million time. Randomly changing elements of a sequence that is already random is basically pointless. It remains a random sequence.
I don't agree. This kind of change is not independent events. If you use more time on move N, you have less time for the next N moves and that will induce more variability. Time usage variability can cause other side-effects as well. You can change history values or hash entries when you suddenly get to search one ply deeper than the last run because you used less time on some of the preceeding moves. And that affects the size/shape of the tree being searched now which can further influence your time usage. The result is a cascading sequence of events that lead to a bubble that bursts at some point and changes the game completely when you choose a different move...
I don't see what you don't agree on, as you seem to say exactly the same thing as I did, with a more verbose circumscription of "induce variability". So probably you did not read...
bob wrote:First, not using old hash entries is a horrible idea.
I think that Uri convincingly argued that it actually is a good idea, and why.
All your arguments can be condensed in the single phrase "it makes the engine weaker". As Uri pointed out that he does not care about this minute drop in strength (~10 Elo?), and considers it more important to have absolute reproducibility of the search (not to confuse with determinism, which depends on how you terminate the search), this is not to the point. If the reproducibility helps him to catch a 5-Elo bug every 2 month that he otherwise could not have hunted down, after one year he will have gained 30 Elo by investing this 10. So he sacrifices a fixed amount of strength for steepening the development curve. A very sensible choice. If this works for him, it would be absolutely stupid to use the old hash entries...
Gerd Isenberg wrote:I mean in each game of the match the time from move to move fluctuates randomly as mentioned. How does it affect the variance or the occurrence of your mentioned "gang x" events?
That depends on the engines in question. In engines that play practically deterministically, so that such events with constant move timing would not occur at al, varying the move timing, and varying it differently and independently in each game, could cause induce variability so that in the end the events could occur once in a million times.
If the engines had already enough intrinsic variability to cause such events to occur once in a million times, they would still occur uonce in a million time. Randomly changing elements of a sequence that is already random is basically pointless. It remains a random sequence.
I don't agree. This kind of change is not independent events. If you use more time on move N, you have less time for the next N moves and that will induce more variability. Time usage variability can cause other side-effects as well. You can change history values or hash entries when you suddenly get to search one ply deeper than the last run because you used less time on some of the preceeding moves. And that affects the size/shape of the tree being searched now which can further influence your time usage. The result is a cascading sequence of events that lead to a bubble that bursts at some point and changes the game completely when you choose a different move...
I don't see what you don't agree on, as you seem to say exactly the same thing as I did, with a more verbose circumscription of "induce variability". So probably you did not read...
I don't agree that varying the time multiple times will have a tiny effect on the match. If I am reading your "once in a million" comment correctly. It is a bit ambiguous and I may well be interpreting it wrongly. But to clarify, I believe that _any_ timing change starts an avalanche that eventually causes a different move to be played. Whether it be at the point of the timing change, or later due to the other cascading effects I mentioned.
Uri Blass wrote:Note that Movei also complete the iteration in almost all cases in ponder off games(in ponder on games it may play immediatly in part of the cases but otherwise it will also try to finish the iteration).
It seems that Bob is basing his opinion based on the past:
He says:
"complete iteration searches is a primitive idea that almost everyone stopped using in the late 70's and early 80's."
I can say that most programmers never programmed chess engines in the late 70's or early 80's and many do not care about the small improvement that they may get by not finishing the iteration.
Uri
If you do an instant reply termination, you have the _same_ problem in ponder=on matches. So what exactly is your point??? It doesn't matter where the variability comes from. The fact that it is there at all is the main point here. And doing what you do will cause two types of variability. The first is time usage changes since you won't move until your opponent moves and how he uses time is beyond your control and probably varies as well. The second is whatever data your search alters while it runs (at least the transposition table) so that if your opponent takes longer on this move than the last time we played this game, your hash table has different stuff in it, history counters are changed. Killer moves are changed. Etc. And all of those add up to the potential for a different move or score from this run even if you do use the same amount of time or search to the same depth.
Again, we drift far afield where things are still as I said they are...
I practically clear the hash after every search(or to be more correct increase generation number so the program does not use positions with different generation number)
I also reset the killers at the beginning of every search.
I use today only knowledge and number of move in the order of moves
for pruning but if I use history I will reset the history table after every search.
different searches are clearly undependent regardless to the question if it is ponder off or ponder on games.
Uri
First, not using old hash entries is a horrible idea. If you don't trust the values, get rid of the table. I _never_ clear the entries. Because I trust them for as long as they stay around. Just as much as I trust my memory of positions in the fried-liver attack and would not do a "brain purge" and have to re-learn all of that stuff again.
Why would you eliminate _any_ data that can help your move ordering and save you time or give you better results in the same time frame?
Your last statement is 100% wrong. Two successive positions in a real game of chess are _not_ independent in any shape, form or fashion. They are part of a "stream" of positions that are related based on piece/pawn placement and previous moves played in the game. If this were true as you stated it, we would not even worry about the 50 move rule or repetitions.
Further, If I complete a 16 ply search and start pondering, I don't start over, I start at depth=15, since I assume you will play the second move, leaving me 14 plies of stuff completed, and I see no reason to re-search those trees again when I already have the result...
That kind of information loss I don't understand...
I did not explain myself well in the last statement.
I meant to say that for my program different searches are independent.
Uri
But why should they be? wouldn't this introduce a bizarre form of randomness? For example, you do check extensions in one search, but not in the next, as a gross case...
I'm looking for consistency across a series of moves so that one search doesn't see something shallow, and the next sees something deep, then back to shallow again, which causes the eval to bounce all over and you have to hope you pick the right move(s).
bob wrote:First, not using old hash entries is a horrible idea.
I think that Uri convincingly argued that it actually is a good idea, and why.
All your arguments can be condensed in the single phrase "it makes the engine weaker". As Uri pointed out that he does not care about this minute drop in strength (~10 Elo?), and considers it more important to have absolute reproducibility of the search (not to confuse with determinism, which depends on how you terminate the search), this is not to the point. If the reproducibility helps him to catch a 5-Elo bug every 2 month that he otherwise could not have hunted down, after one year he will have gained 30 Elo by investing this 10. So he sacrifices a fixed amount of strength for steepening the development curve. A very sensible choice. If this works for him, it would be absolutely stupid to use the old hash entries...
You overlook _one_ important detail. Lose X elo due to clearing the hash, lose Y elo to do a deterministic timing algorithm. Lose Z elo by clearing killer moves. Before long you have lost a _bunch_. A 10 elo improvement is not exactly something one can throw out if he hopes to make progress...
Uri Blass wrote:I can add that another reason for not being deterministic for programs is that search is effected by result of previous searches and movei does not suffer from that problem.
I think that at level of 2700-2750 CCRL rating this is a mistake to spend time on things like that because it will make debugging error later harder and it is probably better to try to find bigger improvements by other means.
Uri
So you want to use what we call the "goose effect"??? That is, every time your program "wakes up" it is a brand new day, no day before, no tomorrow, just search for now and give no thought to what will happen tomorrow?
I don't play chess like that. I don't think it even remotely reasonable that a computer plays chess like that...
I also do not play chess like that but my engine does.
There are things that are more important for me than playing strength
and it is being able to reproduce the analysis later.
I am sure that it is possible to earn something by learning from previous searches(I guess something like 20 elo) but when the gap between movei and rybka is near 300 elo I consider these 20 elo as unimportant.
Uri
In a real game you have no hope of reproducing the same game anyway, because your opponent will vary his time per move with pondering, and no matter how good you are, you will never enter his predicted move at _exactly_ the same node on the re-test...
So I miss the point since games are non-deterministic by nature unless you don't ponder, don't parallel search, etc...
