Improve the search or the evaluation?

[JBNielsen]

Has anyone tried this:

You have program A with a rating 2200.
You compare it with program B with a rating 2500.

You want to improve program A.
Should you improve the search or the evaluation? Or both?

You make two new programs:
Program C with the A-search and the B-evaluation
Program D with the B-search and the A-evaluation

By comparing these versions you will get more knowledge about what to do.

I am aware it depends on the structures of the two programs how difficult this is.

[Uri Blass]

I think that improving the search is more important certainly if you are at the level of 2200.

It is possible to get rating of 2500 only by search with a very simple evaluation and even at the level of stockfish it seems that most of the improvement is search improvements and not evaluation improvements.

[Ferdy]

You want to improve program A.
Should you improve the search or the evaluation? Or both?

I get a higher gain when touching both areas.

[hgm]

I think it all depends how poor your search is versus your evaluation. Top engines have extremely simple evaluations, but they have to be very well tuned. It is very easy to spoil the engine completely by a gross mistuning of one of the eval terms.

[Uri Blass]

I think it all depends how poor your search is versus your evaluation. Top engines have extremely simple evaluations, but they have to be very well tuned. It is very easy to spoil the engine completely by a gross mistuning of one of the eval terms.

I remember that I tested strelka when I change it's evaluation to only piece square table evaluation(average between opening evaluation and
endgame evaluation based on stage of the game) and it had no problem to beat joker with this simple stupid evaluation and the piece square table was not close to optimal piece square table because I did not tune it to compensate for the missing knowledge in the evaluation and inspite of it
I remember 70-80% result in 1+1 time control games against Joker1.1.14.

Joker is not a top engine but has a CCRL rating of 2293 and if a very simple stupid evaluation is better than it then it means that probably if you have CCRL rating of only 2200 the main advantage that you can get is by search improvement(I assume that most people do some basic things except piece square table like mobility and pawn structure and king safety).

I do not claim that evaluation is not important except piece square table and I believe that mobility pawn structure and king safety can give 200-300 elo but search improvements relative to Joker probably can give more than 500 elo improvement.

[hgm]

Well Joker does not have much more than PST. It has some Pawn structure, but that is probably grossly mistuned.

[hgm]

Well Joker does not have much more than PST. It has some Pawn structure, but that is probably grossly mistuned. Not having an eval term could be better than having the term with wrong value.

[tpetzke]

If you can improve both then improve both of course.

I personally value improvements gained by better eval higher than improvements gained by a better search. I think eval gains are more permanent and scale better.

If hardware gets faster and faster the value of an additional ply of search might get smaller over time.

[Uri Blass]

Piece square table was also not tuned with the strelka version that I used that for example did not use material imbalance table.

I agree that not having an eval term can be better than having the term with the wrong value but I assume that you did some tests to show that the knowledge that you have about pawn structure is productive for Joker(you do not need a lot of CPU time for it).

In other words I believe that strelka could beat joker convincgly inspite of inferior evaluation so I guess that there is at least 300 elo advantage for Strelka1.8 relative to Joker.

I also do not think that strelka1.8's search is close to optimal search
and the results of the tests that I see show search improvement even in significantly stronger programs like stockfish.

[Uri Blass]

If you can improve both then improve both of course.

I personally value improvements gained by better eval higher than improvements gained by a better search. I think eval gains are more permanent and scale better.

If hardware gets faster and faster the value of an additional ply of search might get smaller over time.

I do not agree and I think that it is dependent on the search change.

I expect an improvement in the order of moves to scale well.