investigating why stockfish is strong idea

[ymatioun]

I did some testing on SF4, in a way similar to what you described: i started with full SF4, and then disabled one search feature at a time (cumulatively), running it against my program and recording incremental impact of each feature. Here are the results:

LMR=100
Sing. Ext=38
prob cut=28
step 7- futility pruning=56
step 6 - razoring=14
IID=-7
step 13a=27
step 13b=55
step 13c=39

That is, disabling LMR reduces ELO by 100. Then disabling singular extensions reduces it by another 38, and so on. These steps together have an impact of 357 ELO, and i guess they explain most of SF search strength.

Youri.

[lucasart]

This should be done by the dev team.
The testing system is very good to evaluate the Elo perf : http://tests.stockfishchess.org/tests

Certainly not!

If someone is willing to spend his time doing the programming, that's his problem. But wasting endless resources in fishtest only to satisfy Uri's curiosity, no thanks!

[Vinvin]

This should be done by the dev team.
The testing system is very good to evaluate the Elo perf : http://tests.stockfishchess.org/tests

Certainly not!

If someone is willing to spend his time doing the programming, that's his problem. But wasting endless resources in fishtest only to satisfy Uri's curiosity, no thanks!

It's a choice : you can use 100% of your resources to improve SF or you can use a part to make some research (produce some papers, rise the level of computer science, ...).

[syzygy]

This should be done by the dev team.
The testing system is very good to evaluate the Elo perf : http://tests.stockfishchess.org/tests

Certainly not!

If someone is willing to spend his time doing the programming, that's his problem. But wasting endless resources in fishtest only to satisfy Uri's curiosity, no thanks!

It's a choice : you can use 100% of your resources to improve SF or you can use a part to make some research (produce some papers, rise the level of computer science, ...).

I don't think there is a choice. Resources are provided to fishtest by volunteers, at their cost, in order to enable the SF developers to improve SF.

It is always an option to run tests locally, as Youri has done. It may take longer, so what.

[Uri Blass]

This should be done by the dev team.
The testing system is very good to evaluate the Elo perf : http://tests.stockfishchess.org/tests

Certainly not!

If someone is willing to spend his time doing the programming, that's his problem. But wasting endless resources in fishtest only to satisfy Uri's curiosity, no thanks!

It's a choice : you can use 100% of your resources to improve SF or you can use a part to make some research (produce some papers, rise the level of computer science, ...).

I don't think there is a choice. Resources are provided to fishtest by volunteers, at their cost, in order to enable the SF developers to improve SF.

It is always an option to run tests locally, as Youri has done. It may take longer, so what.

There is clearly a choice and the choice is by the leader of the project.

volunteers who give computer time simply decide to support the leader.
If the leader decides about a different choice then the result may be different number of volunteers(I do not know if more volunteers or less volunteers).

Of course the leader is free to decide what to do and I blame nobody in this post and only wanted to corrected the claim that there is no choice.

[mcostalba]

Of course the leader is free to decide what to do and I blame nobody in this post and only wanted to corrected the claim that there is no choice.

You tests are useless because take nowhere, it's pure academic.

People not involved in SF development in any way (even not donating CPU time) have no saying on our choices, even less to tell us what "we should do".

[Uri Blass]

Of course the leader is free to decide what to do and I blame nobody in this post and only wanted to corrected the claim that there is no choice.

You tests are useless because take nowhere, it's pure academic.

People not involved in SF development in any way (even not donating CPU time) have no saying on our choices, even less to tell us what "we should do".

It is possible that with different choice different people may donate CPU time.
I do not tell the stockfish team what to do and only explain that the claim that there is no different choice is not correct.

My posts are not in order to convince the stockfish team to do something different.

I do not think that they will cause a change but if they cause a change there is a bigger chance that they will cause another person to create his own team when volunteers are going to donate computer time for testing why stockfish is strong and not for improving stockfish.

[mcostalba]

I do not tell the stockfish team what to do

I was not referring to you.

[lucasart]

My posts are not in order to convince the stockfish team to do something different.

Then I wonder what the purpose of your posts are. It's always the same. You make suggestions like "we" should do this, "we" should do that. It would be interesting to study the effect of this and that. Yet, when it comes to writing the code, you never do it, unless it is completely trivial. And when it comes to spending CPU resources, you never give your own CPU time. This kind of attitude is not welcome in open source projects.

Academic masturbation leads nowhere. All the theoretical stuff in computer chess has been discovered 20+ years ago. The answer to your initial question is that there is no magic recipe in SF. As Pavel Koziol explained, it's all about fine tuning and synergy between things. There's no fundamental difference between SF and an "average amateur engine", apart from the absence (or rareness) of bugs, and much better fine tuning. They all have the same fundamentals in (alpha beta, null move, qsearch, iid, pvs, hash table, etc.)

[Michel]

All the theoretical stuff in computer chess has been discovered 20+ years ago.

There are still plenty of things that are not understood from a theoretical point of view.

(1) It there a theoretical explanation why minimax search works so well in Chess? The research into game tree pathologies appears rather unsatisfying to me.

(2) Why is difference in engine strength roughly additive? In other words why is there something like the elo model?

(3) Are there convincing examples of non-transitivity in Chess? Of course you can easily make up pathological examples but this is not what I mean.

(4) Why does self play magnify elo differences? In other words how does the "mind reading effect" _really_ work?

I could go on for a while...