Self testing and high draw ratio
Posted: Mon Aug 19, 2019 8:05 pm
As Minic become smarter, the draw ratio between the last release and the current dev is growing high.
I often see 65% of draws using 2 moves opening book.
Hopefully the draw ratio versus other engines is still correct (~20%) so that a little tourney gives reliable and quite quick results.
But before going to a tourney I always like to run a little head-to-head self test.
Because a little tourney takes at least 2 days, including 3 or 4 versions of Minic and 6 to 10 other engines, running 7 concurrent games at TC40/20sec).
SPRT seems to not help much here, but I may not be using it right ...
Let's look at today example (still running...):
Can I use SPRT to conclude quickly here that the patch is good or not ?
A connected question might be, what are the odds to see a +25 +/-20 (looks good to me) become a 15 +/-18 (look less good to me ...) that's what I saw today ...
I often see 65% of draws using 2 moves opening book.
Hopefully the draw ratio versus other engines is still correct (~20%) so that a little tourney gives reliable and quite quick results.
But before going to a tourney I always like to run a little head-to-head self test.
Because a little tourney takes at least 2 days, including 3 or 4 versions of Minic and 6 to 10 other engines, running 7 concurrent games at TC40/20sec).
SPRT seems to not help much here, but I may not be using it right ...
Let's look at today example (still running...):
Code: Select all
Score of minic_0.86 vs minic_dev: 75 - 96 - 316 [0.478]
Elo difference: -15.0 +/- 18.3, LOS: 5.4 %, DrawRatio: 64.9 %
A connected question might be, what are the odds to see a +25 +/-20 (looks good to me) become a 15 +/-18 (look less good to me ...) that's what I saw today ...