creating an opening book

[jdart]

no, it does not have that opening

Well, your game is interesting too, and the Anti-Meran Gambit is a good example of an opening you don't want to go into without book knowledge (and I don't mean a 15-ply search).

--Jon

[flok]

I looked at one and it looks like this:

Code: Select all

mysql> select * from entries where hash=-9221985131600632708;
+----------------------------------------------------------------+----------------------+-------+------+
| fen                                                            | hash                 | plies | eval |
+----------------------------------------------------------------+----------------------+-------+------+
| rnbqkbnr/1ppppp1p/8/p5p1/1P1P4/6P1/P1P1PP1P/RNBQKBNR b KQkq -  | -9221985131600632708 |    15 |   23 |
| rnbqkbnr/1ppppp1p/8/p5p1/1P1P4/6P1/P1P1PP1P/RNBQKBNR b KQkq b3 | -9221985131600632708 |    16 |   29 |
| rnbqkbnr/1ppppp1p/8/p5p1/1P1P4/6P1/P1P1PP1P/RNBQKBNR b KQkq d3 | -9221985131600632708 |    16 |   23 |
+----------------------------------------------------------------+----------------------+-------+------+
3 rows in set (2.23 sec)

So the problem obviously is that you are not hashing in en passant rights.

Oh indeed! That's surprising. I've been using that code for ages. Well, that explains a few things.

It seems likely that you are also not hashing in castling rights. That will give even more collisions.

Well for pawn, rook and king I have a bit that says "first move". So my 64x12 table is in my situation 64x32: 3 bit for piece-type, 1 bit for first move and 1 for color.

Thanks!

[sje]

For some tasks, Symbolic uses a compressed FEN format: 8 bytes for the FEN scalar set and 32 bytes for the board:

Code: Select all

class CpFSS
{
public:
  void Reset(void);

  void LoadFromFSS(const FSS& fss);
  void StoreToFSS(FSS& fss) const;

  Color GetGood(void) const {return (Color) good;}
  Color GetEvil(void) const {return (Color) evil;}
  Cabs  GetCabs(void) const {return (Cabs)  cabs;}
  Sq    GetEpSq(void) const {return (Sq)    epsq;}
  ui    GetHmvc(void) const {return (ui)    hmvc;}
  ui    GetFmvn(void) const {return (ui)    fmvn;}

  void PutGood(const Color color) {good = (si8)  color;}
  void PutEvil(const Color color) {evil = (si8)  color;}
  void PutCabs(const Cabs value)  {cabs = (Cabs) value;}
  void PutEpSq(const Sq sq)       {epsq = (Sq)   sq;}
  void PutHmvc(const ui value)    {hmvc = (ui16) value;}
  void PutFmvn(const ui value)    {fmvn = (ui16) value;}

private:
  si8  good; // Active color
  si8  evil; // Passive color
  ui8  cabs; // Castling availability bits
  si8  epsq; // En passant target square (may be nil)
  ui16 hmvc; // Half move counter
  ui16 fmvn; // Full move number
};

#define CpBoardLen (SqLen / 2)

class CpBoard
{
public:
  void Reset(void);

  void LoadFromBoard(const Board& board);
  void StoreToBoard(Board& board) const;

  Man GetMan(const Sq sq) const;
  void PutMan(const Sq sq, const Man man);

private:
  void ResetSq(const Sq sq) {PutMan(sq, ManVacant);}

  ui8 bvec[CpBoardLen];
};

class CpFEN
{
public:
  void Reset(void);

private:
  CpFSS   cpfss;
  CpBoard cpboard;
}

Accessing the compacted board:

Code: Select all

Man CpBoard::GetMan(const Sq sq) const
{
  return (Man) ((bvec[sq / 2] >> ((sq % 2) * 4)) & 0x0f);
}

void CpBoard::PutMan(const Sq sq, const Man man)
{
  bvec&#91;sq / 2&#93; &= ~&#40;MX&#40;4&#41; << (&#40;sq % 2&#41; * 4&#41;);
  bvec&#91;sq / 2&#93; |= ((&#40;ui8&#41; man&#41; << (&#40;sq % 2&#41; * 4&#41;);
&#125;

With a slight increase in complexity, there are substantial gains in encode/decode speed and space efficiency. Also, each entry has the same 40 byte length, so locating the Nth record in a file is simple and fast.

----

From the initial array, perft(7) = 3,195,901,860. The number of unique positions here is unique(7) = 96,400,068. To store the unique(7) compact FEN set requires about 3.6 GiB.

[stegemma]

[...]
Well for pawn, rook and king I have a bit that says "first move". So my 64x12 table is in my situation 64x32: 3 bit for piece-type, 1 bit for first move and 1 for color.

Thanks!

In normal chess, how do you use the first-move bit for pawn? Maybe saving a bit today is not very important, but you can still use 4 bits for all the information, instead of 5, just adding a piece type "never moved rook" and "never moved king", so to have 8 piece types (3 bit) plus 1 bit for color. One bit less means two pieces in a byte, instead a byte+1/4 of a byte...

[Dann Corbit]

no, it does not have that opening

Well, your game is interesting too, and the Anti-Meran Gambit is a good example of an opening you don't want to go into without book knowledge (and I don't mean a 15-ply search).

--Jon

I am going to have to delve into your opening position more deeply.
My analysis only goes to 38 plies, but I have Qc8 as best:

[d]3qr1k1/pp1bppb1/3p1npB/2r1n2p/3NP2P/1BN2P2/PPPQ2P1/1K1R3R b - - acd 38; acs 1891; bm Qc8; cce -72; ce -29; pm Bh8 {94} Qa5 {54} b5 {37} Nc4 {33} Bxh6 {8} Qb6 {7} Qc7 {3} Qc8 {1}; pv Qc8 Bxg7 Kxg7 Nd5 Nxd5 Bxd5 Nc4 Bxc4 Rxc4 Ne2 Be6 b3 Rc5 Nf4 Rh8 Nd3 Rc7 Rdg1 b6 Nf4 Rc5 c4 f6 Nd3 Rc7 g3 Kf7 Nf4 Rc5 Qb2 Bd7 Rc1 Be6 Rh2 Kg7 Qd2 Bd7 Rhh1 Be6 Rhg1 Kf7; white_wins 123; black_wins 85; draws 25;

[jdart]

I have Qc8 as best

That appears to be a novelty, and possibly a good one.

In a more recent Karjakin-Carlsen game, Carlsen played .. a5, but that could have been partly to avoid his opponent's preparation.

--Jon

[flok]

What I'm wondering: does it make sense for an openingbook to fill it with a quiescence search? Because the number of nodes explodes a bit with that

[Dann Corbit]

I have Qc8 as best

That appears to be a novelty, and possibly a good one.

In a more recent Karjakin-Carlsen game, Carlsen played .. a5, but that could have been partly to avoid his opponent's preparation.

--Jon

I have one game with that move choice, but it departed from the pv pretty quickly:

Code: Select all

[Event "RT.2006.0.00001"]
[Site "LSS"]
[Date "2006.04.15"]
[Round "53.264"]
[White "Schuster, Peter"]
[Black "Guzman, Carlos M"]
[Result "1-0"]
[ECO "B78q"]
[Variation "Sicilian: Dragon, Yugoslav, Main Line, 12.h4 h5 13.Bg5 Rc5 14.Kb1"]
[Source "Chess Mail Ltd"]

1.e4 c5 2.Nf3 d6 3.d4 cxd4 4.Nxd4 Nf6 5.Nc3 g6 6.Be3 Bg7 7.f3 O-O 8.Qd2 Nc6 
9.Bc4 Bd7 10.h4 Rc8 11.Bb3 h5 12.O-O-O Ne5 13.Bg5 Rc5 14.Kb1 Re8 15.Bh6 Qc8 
{
suggested pv was:
Bxg7 Kxg7 [b]Nd5 Nxd5 Bxd5[/b] ...
Game actually continued as:
Bxg7 Kxg7[b] Rhe1 b5 f4[/b] ...
}
16.Bxg7 Kxg7 17.Rhe1 b5 18.f4 Neg4 19.Nf3 Qc7 20.Ng5 b4 21.Ne2 Nf2 22.Bxf7 N6xe4 
23.Qxb4 a5 24.Qd4+ e5 25.fxe5 Rexe5 26.Bb3 a4 27.Nxe4 axb3 28.Nxc5 bxc2+ 
29.Kxc2 Bf5+ 30.Kc1 Nxd1 31.Rxd1 Kh7 32.Nc3 Rxc5 33.g3 d5 34.Qe3 Be4 35.Rd4 Rc6 
36.Kd2 Rf6 37.Nxe4 dxe4 38.Rxe4 Qa5+ 39.Qc3 Qd5+ 40.Qd4 Qa5+ 41.Kc2 Qc7+ 
42.Qc4 Qxg3 43.a4 1-0

[Dann Corbit]

What I'm wondering: does it make sense for an openingbook to fill it with a quiescence search? Because the number of nodes explodes a bit with that

The more statistical and analysis information you can pack into your book the better. You should also update it with live information.

If you have a position for which you know a given move is very good from a 24 hour Stockfish search, but your program loses 20 blitz games in a row using that move, you should update your database with that information.

There are also positions which are great for blitz games (Dutch Stonewall) which are not nearly so dazzling at correspondence time control.

If you have the evaluation from a dozen different strong engines at long time control, save that.

If you have the statistics from 40,000 blitz games, store that.

If you have 20 correspondence chess games, store that.

And if your engine spots a novelty, store that.

Your book will be limited by the energy you put into it.
And you book will be as great as the effort you put into it.

Same as everything else in life.

[syzygy]

It seems likely that you are also not hashing in castling rights. That will give even more collisions.

Well for pawn, rook and king I have a bit that says "first move". So my 64x12 table is in my situation 64x32: 3 bit for piece-type, 1 bit for first move and 1 for color.

In that case it seem you're assigning different hash keys to identical positions.

For example, once the king has moved there is no difference anymore between a rook that has never moved and a rook that has (and then returned to its initial position).