How do you organize a self-play?

[jaroslav.tavgen]

I want to improve my engine by making it play itself. How should I organize it? What should be the time limit or searching-depth limit? How can I evaluate if a stronger version of engine won or was it just a pure luck?

Thank you!

[syzygy]

I want to improve my engine by making it play itself. How should I organize it? What should be the time limit or searching-depth limit? How can I evaluate if a stronger version of engine won or was it just a pure luck?

The more games the better. Let two versions play each other at very fast time controls (seconds per full game, not minutes) and play as many games in parallel as your computer has cores (or even threads).

Do not limit search depth. Limit total time for the game.

You can use cutechess-cli to run the games and keep track of the results.

Then learn about statistics to gain some understanding of when a result is significant or not.

[flok]

I want to improve my engine by making it play itself. How should I organize it? What should be the time limit or searching-depth limit? How can I evaluate if a stronger version of engine won or was it just a pure luck?

The more games the better. Let two versions play each other at very fast time controls (seconds per full game, not minutes) and play as many games in parallel as your computer has cores (or even threads).

Do not limit search depth. Limit total time for the game.

You can use cutechess-cli to run the games and keep track of the results.

Then learn about statistics to gain some understanding of when a result is significant or not.

...or use 'sprt' in fastchess and let fastchess tell you what the result is

[syzygy]

...or use 'sprt' in fastchess and let fastchess tell you what the result is

I did not know fastchess. It looks interesting!
https://github.com/Disservin/fastchess

[benvining]

Here's an example of integrating fastchess SPRT into a CMake workflow: https://github.com/benthevining/BenBot/ ... eLists.txt

I create 2 custom targets, sprt_set_baseline and sprt. sprt_set_baseline builds the engine and copies the binary to a eng_baseline file, sprt always tests the latest build against the last baseline. So then the workflow is, when I'm at a stable point, run sprt_set_baseline, then make some changes and run sprt, and that should tell you if your changes have helped or hurt.