Wall and Regression

[smatovic]

What do you do, if you hit repeatedly a wall and intended optimizations end up
in regression?

Time for a break?

--
Srdja

[Henk]

First you must have proof a solution exists. Then find it. If you cannot find it you searched in the wrong direction. Sampling may help and you need to know the search space. Maybe finding a solution can be too expensive. Other things to do that are more worthwhile.

[Henk]

Or you search space/scope/domain was too limited/ill defined or your optimizing method need to be improved.

[cdani]

What do you do, if you hit repeatedly a wall and intended optimizations end up
in regression?

Time for a break?

--
Srdja

If your engine is 3250 this will not happen, as is too weak and you can find many ways to improve it. The problem of a 3250 engine is the number of games needed to find 1-3 elo win.

If is less than 3250 but put more than 2900-3000, you can find a lot of improvements, but you need a reliable testing method.

If you hitted a wall, maybe we can help with ideas.

[Henk]

What do you do, if you hit repeatedly a wall and intended optimizations end up
in regression?

Time for a break?

--
Srdja

If your engine is 3250 this will not happen, as is too weak and you can find many ways to improve it. The problem of a 3250 engine is the number of games needed to find 1-3 elo win.

If is less than 3250 but put more than 2900-3000, you can find a lot of improvements, but you need a reliable testing method.

If you hitted a wall, maybe we can help with ideas.

This is what usually happens. They ask you to see your source code and steal all your good ideas and then they come up with bad ideas. After long discussion they say yes maybe you are right and sorry we could not help you. All because engines are unique.

Of course if you only have bad code/ideas you have nothing to lose except time.

[smatovic]

The problem of a 3250 engine is the number of games needed to find 1-3 elo win.

Hehe, i am still far away from that.

Regarding the wall,
i found the problem in another part of program logic,
i did not see the causal connection between the two parts.

Regarding the regression,
like Henk said,
maybe i have to redefine the optimum i am looking for....

--
Srdja

[smatovic]

First you must have proof a solution exists. Then find it. If you cannot find it you searched in the wrong direction. Sampling may help and you need to know the search space. Maybe finding a solution can be too expensive. Other things to do that are more worthwhile.

Thanks,
maybe i have to redefine the optimum i am looking for....

--
Srdja

[cdani]

Regarding the wall,
i found the problem in another part of program logic,
i did not see the causal connection between the two parts.

Yes, is very important to have all very clear and bug free.
Other than known to work ideas that sure will be the ones that will boost more your engine strenght, understanding the causes of bad played positions is the easiest way to find improvements. I see very few games of Andscacs, because each maybe 10 I see, I find 2 or 3 ideas to try, and those tries are very slow to test.

[Michael Sherwin]

What do you do, if you hit repeatedly a wall and intended optimizations end up
in regression?

Time for a break?

--
Srdja

If your engine is 3250 this will not happen, as is too weak and you can find many ways to improve it. The problem of a 3250 engine is the number of games needed to find 1-3 elo win.

If is less than 3250 but put more than 2900-3000, you can find a lot of improvements, but you need a reliable testing method.

If you hitted a wall, maybe we can help with ideas.

This is what usually happens. They ask you to see your source code and steal all your good ideas and then they come up with bad ideas. After long discussion they say yes maybe you are right and sorry we could not help you. All because engines are unique.

Of course if you only have bad code/ideas you have nothing to lose except time.

It is more insidious than that. When everyone of the top engine authors with accumulated accolades that were haranguing Vas (Rybka) and saying how worthless he was for only taking and not giving I researched Vas's post. Vas shared every idea he was putting into Rybka. He even said ahead of time that he was basing his engine on mostly Fruit and other open source engines. He did not simply add to Fruit and try to pass it off as his own. He did a complete rewrite of Fruit into bitboard and made it better. The fact that some data may have been used directly in his first release that they crucified him over only means he should have waited awhile and improved on that data also before releasing his engine. His final result did not contain much of Fruit except that Fruit was a sound base for his endeavor. And Vas was demonized for it. And yet all the top engines of today owe something to his work and too Fruit. There are telltale signs in Fruit that Fruit started off as TSCP. TSCP is a dumbed down version of Faile for teaching purposes. And Faile is a mailbox version of an early Crafty. The irony of it!

[jdart]

Happens to me all the time. Sometimes you just have to put whatever you were trying back on the TODO list and take something else to try.

Lately I have been trying to improve the scoring terms that are concerned with king position, especially the transition from middlegame to endgame. This is not so much tuning as trying to change how they are scored, adding or removing eval terms, etc. Everything so far has regressed. So I still have the variant code in a couple of branches in case I want to go back to it, but right now that path does not look promising.

--Jon