Recently I am doing an experiment with Senpai v1 engine by replacing its original evaluation function to the evaluation function of Stockfish, just curious about how much ELO gain I can get.
I use the 40 moves in 5 minutes time control, and play 400 games in each experiment. The original Senpai outperforms the baseline engine for around 80 ELO. Say the baseline engine’s ELO is 0, then Senpai is around 80. However, when I switch to Stockfish evaluation, the engine strength surprisingly dropped 400 ELO, which is a very confusing result.
After debugging for a while, I realized that the problem comes from the "delta pruning" with "futility-pruning" margin. The main reason is that, Stockfish evaluation use a range of (-31000, 31000), while original Senpai use (-9000, 9000) which is relatively smaller than Stockfish evaluation. In the futility-pruning, Senpai use a margin (val in the code):
Code: Select all
val = eval(sl) + depth * 40 + 50;
Code: Select all
if (use_fp &&
move::is_tactical(mv) &&
!move::is_check(mv, bd) &&
val + move::see_max(mv) <= alpha) { // delta pruning
continue;
}
In summary, with time control 40moves/300secs:
Baseline engine: 0 ELO
Senpai_searcher + Senpai_eval: +80 ELO +/- 15
Senpai_searcher + Stock_eval + original_futility: -400 ELO +/- 15
Senpai_searcher + Stock_eval + no_futility: +160 ELO +/- 15
Senpai_searcher + Stock_eval + tuned_futility: +170 ELO +/- 15
BTW, one more found during this experiment is that, the g++/clang++ compiling option "-fomit-frame-pointer" gives me some mysterious runtime errors (it makes the program even produce different behaviors under different OS). We'd better void using this compiling optimization option. This also confused me for a long time