TalkChess.com

jdart wrote:Personally I think opening books are important. The shallow fixed book competitions have their place and they do provide a uniform platform for testing. But sometimes even good engines quickly go into known sub-optimal lines. Opening theory is very deep in some areas now, especially with correspondence play that is engine-aided. Search during a game does not easily replace this accumulated knowledge, especially in the early opening stages where the end points of known lines are far away.

Some development teams (Hiarcs in particular) have put a huge amount of effort into book development including finding strong novelties. However, I know some engine authors don't care about books and don't want to spend any effort there.

--Jon

   # PLAYER                : RATING    POINTS  PLAYED    (%)
   1 K8 Oops.bin           : 3066.3    1221.0    2000   61.0%
   2 K8 RpC.bin            : 3033.2    1111.5    2000   55.6%
   3 K8 Komodo.bin         : 3003.9    1013.0    2000   50.6%
   4 K8 SF book.bin        : 2992.9     976.0    2000   48.8%
   5 K8 Performance.bin    : 2977.5     924.0    2000   46.2%
   6 K8 No Book            : 2926.2     754.5    2000   37.7% 

   # PLAYER                        : RATING    POINTS  PLAYED    (%)
   1 Rising Star.ctg               : 3387.7     676.5    1140   59.3%
   2 Book X.bin                    : 3359.0     287.5     540   53.2%
   3 PlaychessNightmare.ctg        : 3355.0     284.0     540   52.6%
   4 Anaconda.ctg                  : 3337.6     268.5     540   49.7%
   5 Invisible Stars.ctg           : 3336.7     580.0    1140   50.9%
   6 Hiarcs 14l.ctg                : 3333.7     265.0     540   49.1%
   7 King Asad Vipre.ctg           : 3322.0     650.0    1200   54.2%
   8 Sicilian Avenged.ctg          : 3320.2     646.5    1200   53.9%
   9 IPmanbook.ctg                 : 3301.4     276.5     600   46.1%
  10 Stormworm 5.0.bin             : 3281.9     312.5     600   52.1%
  11 Rodent.bin                    : 3279.6     254.5     600   42.4%
  12 My Friends 10.1.bin           : 3264.6     204.5     540   37.9%
   .............

   .............
  78 Monks 1.4.ctg                 : 2745.7     516.5    1050   49.2%
  79 RedKep.ctg                    : 2738.0     249.5     450   55.4%
  80 AMALIA frANCINE.ctg           : 2737.9     194.0     420   46.2%
  81 The Sniper 6.ctg              : 2729.7     503.0    1020   49.3%
  82 Amalia Power.ctg              : 2724.4     309.0     600   51.5%
  83 Doom Re-Born.ctg              : 2719.3     181.5     420   43.2%
  84 Boom Lite 3.ctg               : 2678.9     199.5     450   44.3%
  85 Doom 3 revisited.ctg          : 2673.6     195.0     450   43.3%
  86 Doom FX.ctg                   : 2670.9     253.0     600   42.2%
  87 Rock 7.ctg                    : 2662.7     244.5     600   40.8%

Milos wrote: Also there is plenty of good books available so even +/- 50 Elo range on books is improbable.
Your assumptions are simply grossly exaggerated.

   # PLAYER                : RATING    POINTS  PLAYED    (%)
   1 K8 MyFriends.bin      : 3057.6    1192.5    2000   59.6%
   2 K8 RpC.bin            : 3033.2    1111.5    2000   55.6%
   3 K8 Komodo.bin         : 3005.1    1017.0    2000   50.9%
   4 K8 SF Book.bin        : 2992.8     975.5    2000   48.8%
   5 K8 Performance.bin    : 2984.8     948.5    2000   47.4%
   6 K8 No Book            : 2926.5     755.0    2000   37.8%

   # PLAYER                : RATING    POINTS  PLAYED    (%)
   1 K8 Oops.bin           : 3066.3    1221.0    2000   61.0%
   2 K8 RpC.bin            : 3033.2    1111.5    2000   55.6%
   3 K8 Komodo.bin         : 3003.9    1013.0    2000   50.6%
   4 K8 SF Book.bin        : 2992.9     976.0    2000   48.8%
   5 K8 Performance.bin    : 2977.5     924.0    2000   46.2%
   6 K8 No Book            : 2926.2     754.5    2000   37.7%

   # PLAYER              : RATING    POINTS  PLAYED    (%)
   1 K8 MyFriends.bin    : 3022.5    1127.5    2000   56.4%
   2 K8 Oops.bin         : 2977.5     872.5    2000   43.6%

Laskos wrote:

Milos wrote: Also there is plenty of good books available so even +/- 50 Elo range on books is improbable.
Your assumptions are simply grossly exaggerated.

You seem to know little about opening books. One cannot just pick a "good book" and think it will hold against anything. With books it often happens that A>B>C>A, and I will exemplify by some tests here:

Suppose one has a new book MyFriends.bin, performing as in the following test:

Code: Select all

   # PLAYER                : RATING    POINTS  PLAYED    (%)
   1 K8 MyFriends.bin      : 3057.6    1192.5    2000   59.6%
   2 K8 RpC.bin            : 3033.2    1111.5    2000   55.6%
   3 K8 Komodo.bin         : 3005.1    1017.0    2000   50.9%
   4 K8 SF Book.bin        : 2992.8     975.5    2000   48.8%
   5 K8 Performance.bin    : 2984.8     948.5    2000   47.4%
   6 K8 No Book            : 2926.5     755.0    2000   37.8%

And another new book, Oops.bin, against the same opponents:

Code: Select all

   # PLAYER                : RATING    POINTS  PLAYED    (%)
   1 K8 Oops.bin           : 3066.3    1221.0    2000   61.0%
   2 K8 RpC.bin            : 3033.2    1111.5    2000   55.6%
   3 K8 Komodo.bin         : 3003.9    1013.0    2000   50.6%
   4 K8 SF Book.bin        : 2992.9     976.0    2000   48.8%
   5 K8 Performance.bin    : 2977.5     924.0    2000   46.2%
   6 K8 No Book            : 2926.2     754.5    2000   37.7%

You will conclude that Oops.bin is better by 9 Elo points, and pick that book. But MyFriends.bin was specifically tuned against Oops.bin, and the direct match-up is the following:

Code: Select all

   # PLAYER              : RATING    POINTS  PLAYED    (%)
   1 K8 MyFriends.bin    : 3022.5    1127.5    2000   56.4%
   2 K8 Oops.bin         : 2977.5     872.5    2000   43.6%

So you picked a book which loses to another book by 45 Elo points, but performs better against other opponents.

Hence a variability of at least 50 points is a minimum for books, despite your bragging, as the books are tricky.

Adam Hair wrote:I can not believe that Book X is ranked that high among the publicly available engines. I created it by playing thousands of games against other books and by adding Playchess and Infinity won games to it. I did not add any lines to it by hand.

Anyway, after the 2 months I spent creating Book X, I agree with Kai. There is a lot of variability associated with books. Transitivity is not a given property.

   # PLAYER                : RATING    POINTS  PLAYED    (%)
   1 K8 MyFriends.bin      : 3047.8     574.5     997   57.6%
   2 K8 Oops.bin           : 3034.0     556.0    1003   55.4%
   3 K8 Book X.bin         : 3031.7     549.0     997   55.1%
   4 K8 RpC.bin            : 3012.8     522.0    1003   52.0%
   5 K8 Performance.bin    : 2992.1     485.5     997   48.7%
   6 K8 SF Book.bin        : 2987.1     477.5     997   47.9%
   7 K8 Komodo.bin         : 2977.9     465.5    1003   46.4%
   8 K8 No Book            : 2916.7     367.0     997   36.8%

3 K8 Book X.bin             :   31  1750 (+544,=838,-368), 55.0 %

K8 Komodo.bin                 : 250 (+ 96,=103,- 51), 59.0 %
K8 SF Book.bin                : 250 (+ 69,=124,- 57), 52.4 %
K8 Performance.bin            : 250 (+ 69,=124,- 57), 52.4 %
K8 RpC.bin                    : 250 (+ 82,=108,- 60), 54.4 %
K8 No Book                    : 250 (+110,=109,- 31), 65.8 %
K8 MyFriends.bin              : 250 (+ 53,=144,- 53), 50.0 %
K8 Oops.bin                   : 250 (+ 65,=126,- 59), 51.2 %

   # PLAYER              : RATING    POINTS  PLAYED    (%)
   1 K8 Book X.bin       : 3012.5    1071.0    2000   53.5%
   2 K8 MyFriends.bin    : 2987.5     929.0    2000   46.5%

   # PLAYER           : RATING    POINTS  PLAYED    (%)
   1 K8 Book X.bin    : 3006.8    1038.5    2000   51.9%
   2 K8 Oops.bin      : 2993.2     961.5    2000   48.1%

APassionForCriminalJustic wrote:
Kai, what is your opinion surrounding both the 1337 ChessPRO series and the KING ASAD books? I use the 1337 commercial books.

Laskos wrote:

APassionForCriminalJustic wrote:
Kai, what is your opinion surrounding both the 1337 ChessPRO series and the KING ASAD books? I use the 1337 commercial books.

I am no expert, and it depends on version used. AFAIK King Asad is larger and tends to be better with white. In BooksWar, the older version of 1337 performs worse than the public (and older) version of King Asad. That's all I can suggest, my bet would be on an updated King Asad.

Laskos wrote:

Laskos wrote:

APassionForCriminalJustic wrote:
Kai, what is your opinion surrounding both the 1337 ChessPRO series and the KING ASAD books? I use the 1337 commercial books.

I am no expert, and it depends on version used. AFAIK King Asad is larger and tends to be better with white. In BooksWar, the older version of 1337 performs worse than the public (and older) version of King Asad. That's all I can suggest, my bet would be on an updated King Asad.

After a long while (several years) I tested some good CTG books, I got some taste after seeing polyglot ones. Yes, using the latest Stockfish, King Asad Vipre from January clobbers at 1'+1'' both ChessPro 1337 from November (300 games, +39 -9, the rest equal, high draw rate) and the BooksWar champion Rising Star from December (100 games, +36 -2, the rest equal, lower draw rate). I suspect that the later King Asad Vipre (January) is tuned against the two strong, but earlier books. It's important to be updated with these books

Laskos wrote:

michiguel wrote:

Laskos wrote:

michiguel wrote:
I have the feeling that using books may increase white advantage. Anyway, simulations with 30 elo of white advantage are below.
Tie breaks are not applied. Those cases are the ones listed as "shared". When there was only one winner is listed as "outright".
I can try to implement some SB rules.

Miguel

Code: Select all

===========

Color when 1 plays white against 2

Total engines = 11
Total games = 55
Total rounds = 11
Total boards = 5
Total cycles = 1000000
draw rate (equal strength) = 64.0%
White advantage = 30.0
rating[0]=3200
rating[1]=3100
rating[2]=3050
rating[3]=3000
rating[4]=2950
rating[5]=2900
rating[6]=2700
rating[7]=2700
rating[8]=2200
rating[9]=2200
rating[10]=2200

won    = 622003
shared = 177919
loss   = 200078
total  = 1000000
won outright % = 62.2  <===============
won shared   % = 17.8

========

Reversed colors

Total engines = 11
Total games = 55
Total rounds = 11
Total boards = 5
Total cycles = 1000000
draw rate &#40;equal strength&#41; = 64.0%
White advantage = 30.0
rating&#91;0&#93;=3200
rating&#91;1&#93;=3100
rating&#91;2&#93;=3050
rating&#91;3&#93;=3000
rating&#91;4&#93;=2950
rating&#91;5&#93;=2900
rating&#91;6&#93;=2700
rating&#91;7&#93;=2700
rating&#91;8&#93;=2200
rating&#91;9&#93;=2200
rating&#91;10&#93;=2200

won    = 586411
shared = 186268
loss   = 227321
total  = 1000000
won outright % = 58.6 <===============
won shared   % = 18.6

I used drawelo of 200, which gives lower draw rate than your 64% for equal opponents. 64% seems a bit high, it's valid only for several top engines, the rest have lower draw rates. Can you use 56%? Probably my drawelo of 200 is a bit too low, though.

I used 64% just to compare with Peter's. Just to make it clear, this is between equal opponents. Between opponents of different strength scales down automatically.

One of the limitations of Ordo (and any other rating software for that matter) is that the draw rate (between equal opponents) is assumed constant throughout the rating spectrum, and this is not true. So, an average needs to be used (the model will be improved when I work with the wilo model).

Here are several combinations (reversed colors, #2 plays white against #1)

draw rate (equal strength) = 56.0%
White advantage = 30.0
won = 564497
shared = 179396
loss = 256107
total = 1000000
won outright % = 56.4
won shared % = 17.9

draw rate (equal strength) = 50.0%
White advantage = 30.0
won = 548826
shared = 173783
loss = 277391
total = 1000000
won outright % = 54.9
won shared % = 17.4

draw rate (equal strength) = 40.0%
White advantage = 30.0
won = 524417
shared = 164144
loss = 311439
total = 1000000
won outright % = 52.4
won shared % = 16.4

Thanks, that is very close to my results (52% draw rate for drawelo 200 and equal opponents). In fact I used a hacked in 5 minutes simulator for TCEC, where the drawelo_var was a variable function of Elos of opponents as drawelo*Elo/3200, with SF and Komodo Elo at 3200, and drawelo of 260. This is a gross approximation to take into account the strength of engines. When hacking the sim, I set drawelo 200 for WCCC, but inadvertently kept the dependence on Elos.

Then, in case of ties, I used random assignment of places, say 2 engines are tied, one engine has 50% of a win in that run, 50% the other. Didn't use colors.