Wall and Regression

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
smatovic
Posts: 1837
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Wall and Regression

Post by smatovic » Thu Aug 31, 2017 9:51 am

What do you do, if you hit repeatedly a wall and intended optimizations end up
in regression?

Time for a break?

--
Srdja

Henk
Posts: 6996
Joined: Mon May 27, 2013 8:31 am

Re: Wall and Regression

Post by Henk » Thu Aug 31, 2017 10:03 am

First you must have proof a solution exists. Then find it. If you cannot find it you searched in the wrong direction. Sampling may help and you need to know the search space. Maybe finding a solution can be too expensive. Other things to do that are more worthwhile.

Henk
Posts: 6996
Joined: Mon May 27, 2013 8:31 am

Re: Wall and Regression

Post by Henk » Thu Aug 31, 2017 10:44 am

Or you search space/scope/domain was too limited/ill defined or your optimizing method need to be improved.

User avatar
cdani
Posts: 2200
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: Wall and Regression

Post by cdani » Thu Aug 31, 2017 2:43 pm

smatovic wrote:What do you do, if you hit repeatedly a wall and intended optimizations end up
in regression?

Time for a break?

--
Srdja
If your engine is 3250 this will not happen, as is too weak and you can find many ways to improve it. The problem of a 3250 engine is the number of games needed to find 1-3 elo win.

If is less than 3250 but put more than 2900-3000, you can find a lot of improvements, but you need a reliable testing method.

If you hitted a wall, maybe we can help with ideas.

Henk
Posts: 6996
Joined: Mon May 27, 2013 8:31 am

Re: Wall and Regression

Post by Henk » Thu Aug 31, 2017 3:23 pm

cdani wrote:
smatovic wrote:What do you do, if you hit repeatedly a wall and intended optimizations end up
in regression?

Time for a break?

--
Srdja
If your engine is 3250 this will not happen, as is too weak and you can find many ways to improve it. The problem of a 3250 engine is the number of games needed to find 1-3 elo win.

If is less than 3250 but put more than 2900-3000, you can find a lot of improvements, but you need a reliable testing method.

If you hitted a wall, maybe we can help with ideas.
This is what usually happens. They ask you to see your source code and steal all your good ideas and then they come up with bad ideas. After long discussion they say yes maybe you are right and sorry we could not help you. All because engines are unique.

Of course if you only have bad code/ideas you have nothing to lose except time.

smatovic
Posts: 1837
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: Wall and Regression

Post by smatovic » Thu Aug 31, 2017 4:04 pm

The problem of a 3250 engine is the number of games needed to find 1-3 elo win.
Hehe, i am still far away from that.

Regarding the wall,
i found the problem in another part of program logic,
i did not see the causal connection between the two parts.

Regarding the regression,
like Henk said,
maybe i have to redefine the optimum i am looking for....

--
Srdja

smatovic
Posts: 1837
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: Wall and Regression

Post by smatovic » Thu Aug 31, 2017 4:07 pm

First you must have proof a solution exists. Then find it. If you cannot find it you searched in the wrong direction. Sampling may help and you need to know the search space. Maybe finding a solution can be too expensive. Other things to do that are more worthwhile.
Thanks,
maybe i have to redefine the optimum i am looking for....

--
Srdja

User avatar
cdani
Posts: 2200
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: Wall and Regression

Post by cdani » Thu Aug 31, 2017 4:18 pm

smatovic wrote: Regarding the wall,
i found the problem in another part of program logic,
i did not see the causal connection between the two parts.
Yes, is very important to have all very clear and bug free.
Other than known to work ideas that sure will be the ones that will boost more your engine strenght, understanding the causes of bad played positions is the easiest way to find improvements. I see very few games of Andscacs, because each maybe 10 I see, I find 2 or 3 ideas to try, and those tries are very slow to test.

Michael Sherwin
Posts: 3196
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Wall and Regression

Post by Michael Sherwin » Thu Aug 31, 2017 4:19 pm

Henk wrote:
cdani wrote:
smatovic wrote:What do you do, if you hit repeatedly a wall and intended optimizations end up
in regression?

Time for a break?

--
Srdja
If your engine is 3250 this will not happen, as is too weak and you can find many ways to improve it. The problem of a 3250 engine is the number of games needed to find 1-3 elo win.

If is less than 3250 but put more than 2900-3000, you can find a lot of improvements, but you need a reliable testing method.

If you hitted a wall, maybe we can help with ideas.
This is what usually happens. They ask you to see your source code and steal all your good ideas and then they come up with bad ideas. After long discussion they say yes maybe you are right and sorry we could not help you. All because engines are unique.

Of course if you only have bad code/ideas you have nothing to lose except time.
It is more insidious than that. When everyone of the top engine authors with accumulated accolades that were haranguing Vas (Rybka) and saying how worthless he was for only taking and not giving I researched Vas's post. Vas shared every idea he was putting into Rybka. He even said ahead of time that he was basing his engine on mostly Fruit and other open source engines. He did not simply add to Fruit and try to pass it off as his own. He did a complete rewrite of Fruit into bitboard and made it better. The fact that some data may have been used directly in his first release that they crucified him over only means he should have waited awhile and improved on that data also before releasing his engine. His final result did not contain much of Fruit except that Fruit was a sound base for his endeavor. And Vas was demonized for it. And yet all the top engines of today owe something to his work and too Fruit. There are telltale signs in Fruit that Fruit started off as TSCP. TSCP is a dumbed down version of Faile for teaching purposes. And Faile is a mailbox version of an early Crafty. The irony of it!
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through

jdart
Posts: 4162
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Wall and Regression

Post by jdart » Thu Aug 31, 2017 4:38 pm

Happens to me all the time. Sometimes you just have to put whatever you were trying back on the TODO list and take something else to try.

Lately I have been trying to improve the scoring terms that are concerned with king position, especially the transition from middlegame to endgame. This is not so much tuning as trying to change how they are scored, adding or removing eval terms, etc. Everything so far has regressed. So I still have the variant code in a couple of branches in case I want to go back to it, but right now that path does not look promising.

--Jon

Post Reply