Opening book for self test purpose

[xr_a_y]

For Minic self test purpose I often use the inner "book_big.bin", this is a 400000 lines book I get somewhere during Weini development but I must admit I don't remember where ...

I also use the AH-12moves, AH-48_Basic_Positions or AH-150, or the silver-suite-large and sometimes books from the drawkiller2.1 suite.

What I am worried about is that I often get very different result depending on the book use. I guess this is due do book length and middle game knowledge ...

I was thus wondering what book you are using to test your engine versus a previous version ?

[konsolas]

I use 2moves_LT_1000.epd

[Joost Buijs]

For Nightmare I use the drawkillers too, Salc v5 and Hert500. Self-tests are not very reliable, sometimes I see a differences of more than 70 Elo between 2 versions of my engine depending upon what kind of opening-book I use. Playing a gauntlet against a variety of different opponents usually gives me a more reliable result.

[xr_a_y]

Yes I know but I "only" have a 8 cores machine and validate a change using a 40/20sec TC and 6 opponents correctly already take 2 or 3 days...

[Joost Buijs]

Atm I only have one 8 core and one 10 core, TC 60 moves in 30 sec. with 0.1 sec increment. The 8 core is used for testing and the 10 core for development. I need better hardware, something like an Intel 36 core (2 x 18) would be nice. AMD TR would be cheaper though.

[Ferdy]

I was thus wondering what book you are using to test your engine versus a previous version ?

I have a big pgn file consisting mostly of shared pgn opening suite. So there are 2-ply, 3-ply, 4-ply and more here. It is used by Cutechess randomly during tests. From time to time, I also run specific tests from popular openings such as ruy lopez, slav, nimzo-indian and others.

[xr_a_y]

As i'd like my engine to behave well during the opening even without book, I guess I should use a 2moves opening.
But on the other hand, using a long book (12 moves) can also lead to position that the engine dislike and it is good to work on them also...

[Joost Buijs]

Indeed, you need a book with as much diversity as possible, containing both strategical and tactical lines. Otherwise you will always get skewed results. A 2-mover won't give you enough diversity.

[xr_a_y]

Ok, so i'll made a mixed book ! thanks