Stockfish direction and improvements

[Venom]

Hi everyone,

I have been following Stockfish code since about version 1.4 and I guess that this question would most likely be appropriate for Tord and Marco, but I would be interested in hearing from anyone who has been working with Stockfish in depth...

Overall I have been quite impressed with the amount of refactoring and the shift to templates for compile-time optimization (among other things) that have been taking place from version to version. I haven't really been following any changes to search or eval but I am curious as to the overall approach to determining areas of improvement, what you all are focusing on, any testing methodologies, profiling, etc you all are using to keep up this rate of improvement?

Specifically, I'm hoping to gain some technical insight into how you all are approaching improving search and eval and any other areas that may be running at sub-optimal levels.

Of course I would also have to say thank you to Tord (for publishing Glaurung to begin with), Marco, and all who have contributed to Stockfish for pushing the envelope and keeping the code open so that everyone can benefit. I truly admire that, and I do look forward to your response

[mcostalba]

Thanks for your kind words. Well there have been many (around 100) functionality changes apart from optimizations and cleanups.

The biggest booster has been evaluation tuning because it has a leverage effect also on other changes: if your evaluation is well tuned then you can also improve pruning or, for instance, a change in move ordering will give you more ELO then in case eval is not well tuned.

In SF 1.6 also the search is changed a lot. Anyhow the only _secret_ is to devote 90% of time and resources to testing becasue we have seen many, really many, apparently good ideas to crash against the testing wall, so also if something seems obviously good we test anyway with real games...and this takes soooo long

[lkaufman]

What time limit or limits do you do most of your testing at?

[Hart]

And since we are on the subject, could you please explain to me how the option "Randomness" works?

[Tord Romstad]

Of course I would also have to say thank you to Tord (for publishing Glaurung to begin with), Marco, and all who have contributed to Stockfish for pushing the envelope and keeping the code open so that everyone can benefit. I truly admire that, and I do look forward to your response

Thanks for your kind words -- but don't forget Joona, whose contributions to the last few versions have been just as important as Marco's, and far more important than mine.

[Tord Romstad]

What time limit or limits do you do most of your testing at?

Almost all our tests are at 1+0 (1000 games at 1+0 when testing whether some new feature improves the strength) or even faster levels (thousands of games at super-fast time controls when auto-tuning parameters). Ideally we would of course prefer to test at slower time controls, but because of severely limited hardware resources and the huge number of things we want to test, this just isn't possible.

[Tord Romstad]

And since we are on the subject, could you please explain to me how the option "Randomness" works?

Sure, that's very simple. When "Randomness" is greater than zero, a small random number is added to each entry of all piece square tables at the beginning of every new game. The idea is to make it a more interesting and less predictable opponent for human players. Of course, the parameter should be left at the default value of 0 for optimal strength.

[lkaufman]

That's pretty much what we do with Doch, but I often wonder whether the optimum values for positional features, even for material, might be very different if we could test at tournament levels.

[Michel]

Almost all our tests are at 1+0 (1000 games at 1+0 when testing whether some new feature improves the strength)

Sorry for my ignorance but do you mean one second or one minute games?

I assume you mean one second but I just wanted to be sure.

[zamar]

Almost all our tests are at 1+0 (1000 games at 1+0 when testing whether some new feature improves the strength)

Sorry for my ignorance but do you mean one second or one minute games?

I assume you mean one second but I just wanted to be sure.

He meant one minute / game, no increment.