How much ELO should I expect to gain from killer moves?
Posted: Fri Jul 16, 2021 3:14 am
I'm currently in the process of rebuilding Blunder from the ground up, as I wasn't happy with the code structure and it was riddled with too many arcane bugs.
After a week or so of work, from my testing against MinimalChess 2.0 and Shallow Blue 1.1.0, my engine is now around ~1500, with alpha-beta pruning, quiescence search, and piece-square tables. And my plan going forward is to add one feature at a time, test the hell out if to make sure it works, and then add another feature.
So with this established, I added killer move ordering into Blunder and started testing it against the previous version of itself without it. As a preliminary, I ran 50 games with a 1 min time format, and the newer version actually seemed to perform slightly worse (the outputs from cutechess):
Of course, this is only 50 games, but I do have some questions. What kind of ELO gain should I be expecting with killer moves added? (I know every engine is different, I'm just more so asking generally). And depending on the minuteness of the ELO gain, how many games would I need to run to see a result?
What I find odd about the killer moves being added as well is that, from my testing, it speed-up the engine by a very sizeable amount by pruning huge chunks of the game tree, which allowed it to search a bit deeper than my previous version. This seemed great to me and it should increase the ELO, no?
Also, with regards to testing, I obviously need more tests, so I'm going to be using a sub-30-second time control. In this kind of control, all the engines I'm testing with are searching to around a depth of 4-5 each move, occasionally 6-10 very, very late in the endgame. In slower time controls, my engine usually searches to around a depth of 6-8 for the middle game and 10-15 towards the endgame. Would these lower search depths affect my results?
After a week or so of work, from my testing against MinimalChess 2.0 and Shallow Blue 1.1.0, my engine is now around ~1500, with alpha-beta pruning, quiescence search, and piece-square tables. And my plan going forward is to add one feature at a time, test the hell out if to make sure it works, and then add another feature.
So with this established, I added killer move ordering into Blunder and started testing it against the previous version of itself without it. As a preliminary, I ran 50 games with a 1 min time format, and the newer version actually seemed to perform slightly worse (the outputs from cutechess):
Code: Select all
Score of Blunder 1.1.0 vs Blunder 1.0.0 : 5 - 9 - 36 [0.460]
... Blunder 1.0.0 playing White: 4 - 3 - 18 [0.520] 25
... Blunder 1.0.0 playing Black: 1 - 6 - 18 [0.400] 25
... White vs Black: 10 - 4 - 36 [0.560] 50
Elo difference: -27.9 +/- 51.0, LOS: 14.3 %, DrawRatio: 72.0 %
50 of 50 games finished.
What I find odd about the killer moves being added as well is that, from my testing, it speed-up the engine by a very sizeable amount by pruning huge chunks of the game tree, which allowed it to search a bit deeper than my previous version. This seemed great to me and it should increase the ELO, no?
Also, with regards to testing, I obviously need more tests, so I'm going to be using a sub-30-second time control. In this kind of control, all the engines I'm testing with are searching to around a depth of 4-5 each move, occasionally 6-10 very, very late in the endgame. In slower time controls, my engine usually searches to around a depth of 6-8 for the middle game and 10-15 towards the endgame. Would these lower search depths affect my results?