Improve the search or the evaluation?

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
JBNielsen
Posts: 250
Joined: Thu Jul 07, 2011 8:31 pm
Location: Denmark
Contact:

Improve the search or the evaluation?

Post by JBNielsen » Sat Aug 31, 2013 7:23 pm

Has anyone tried this:

You have program A with a rating 2200.
You compare it with program B with a rating 2500.

You want to improve program A.
Should you improve the search or the evaluation? Or both?

You make two new programs:
Program C with the A-search and the B-evaluation
Program D with the B-search and the A-evaluation

By comparing these versions you will get more knowledge about what to do.

I am aware it depends on the structures of the two programs how difficult this is.

Uri Blass
Posts: 8378
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Improve the search or the evaluation?

Post by Uri Blass » Sat Aug 31, 2013 8:40 pm

I think that improving the search is more important certainly if you are at the level of 2200.

It is possible to get rating of 2500 only by search with a very simple evaluation and even at the level of stockfish it seems that most of the improvement is search improvements and not evaluation improvements.

Ferdy
Posts: 3874
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Improve the search or the evaluation?

Post by Ferdy » Sun Sep 01, 2013 2:27 am

You want to improve program A.
Should you improve the search or the evaluation? Or both?
I get a higher gain when touching both areas.

User avatar
hgm
Posts: 23220
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Improve the search or the evaluation?

Post by hgm » Sun Sep 01, 2013 6:59 am

I think it all depends how poor your search is versus your evaluation. Top engines have extremely simple evaluations, but they have to be very well tuned. It is very easy to spoil the engine completely by a gross mistuning of one of the eval terms.

Uri Blass
Posts: 8378
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Improve the search or the evaluation?

Post by Uri Blass » Sun Sep 01, 2013 7:12 am

hgm wrote:I think it all depends how poor your search is versus your evaluation. Top engines have extremely simple evaluations, but they have to be very well tuned. It is very easy to spoil the engine completely by a gross mistuning of one of the eval terms.
I remember that I tested strelka when I change it's evaluation to only piece square table evaluation(average between opening evaluation and
endgame evaluation based on stage of the game) and it had no problem to beat joker with this simple stupid evaluation and the piece square table was not close to optimal piece square table because I did not tune it to compensate for the missing knowledge in the evaluation and inspite of it
I remember 70-80% result in 1+1 time control games against Joker1.1.14.

Joker is not a top engine but has a CCRL rating of 2293 and if a very simple stupid evaluation is better than it then it means that probably if you have CCRL rating of only 2200 the main advantage that you can get is by search improvement(I assume that most people do some basic things except piece square table like mobility and pawn structure and king safety).

I do not claim that evaluation is not important except piece square table and I believe that mobility pawn structure and king safety can give 200-300 elo but search improvements relative to Joker probably can give more than 500 elo improvement.

User avatar
hgm
Posts: 23220
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Improve the search or the evaluation?

Post by hgm » Sun Sep 01, 2013 7:21 am

Well Joker does not have much more than PST. It has some Pawn structure, but that is probably grossly mistuned.

User avatar
hgm
Posts: 23220
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Improve the search or the evaluation?

Post by hgm » Sun Sep 01, 2013 7:22 am

Well Joker does not have much more than PST. It has some Pawn structure, but that is probably grossly mistuned. Not having an eval term could be better than having the term with wrong value.

tpetzke
Posts: 686
Joined: Thu Mar 03, 2011 3:57 pm
Location: Germany
Contact:

Re: Improve the search or the evaluation?

Post by tpetzke » Sun Sep 01, 2013 8:56 am

If you can improve both then improve both of course.

I personally value improvements gained by better eval higher than improvements gained by a better search. I think eval gains are more permanent and scale better.

If hardware gets faster and faster the value of an additional ply of search might get smaller over time.
Thomas...

=======
http://macechess.blogspot.com - iCE Chess Engine

Uri Blass
Posts: 8378
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Improve the search or the evaluation?

Post by Uri Blass » Sun Sep 01, 2013 9:01 am

Piece square table was also not tuned with the strelka version that I used that for example did not use material imbalance table.

I agree that not having an eval term can be better than having the term with the wrong value but I assume that you did some tests to show that the knowledge that you have about pawn structure is productive for Joker(you do not need a lot of CPU time for it).

In other words I believe that strelka could beat joker convincgly inspite of inferior evaluation so I guess that there is at least 300 elo advantage for Strelka1.8 relative to Joker.

I also do not think that strelka1.8's search is close to optimal search
and the results of the tests that I see show search improvement even in significantly stronger programs like stockfish.

Uri Blass
Posts: 8378
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Improve the search or the evaluation?

Post by Uri Blass » Sun Sep 01, 2013 9:03 am

tpetzke wrote:If you can improve both then improve both of course.

I personally value improvements gained by better eval higher than improvements gained by a better search. I think eval gains are more permanent and scale better.

If hardware gets faster and faster the value of an additional ply of search might get smaller over time.
I do not agree and I think that it is dependent on the search change.

I expect an improvement in the order of moves to scale well.

Post Reply