ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Database storage methods
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Edmund Moshammer



Joined: 03 Dec 2007
Posts: 598
Location: St. Gallen, Switzerland
PostPost subject: Re: Database storage methods    Posted: Fri Mar 09, 2012 12:51 am Reply to topic Reply with quote
JuLieN wrote:
bob wrote:

Couple of points.

First, you need the full 8 bits for the index. There are positions with > 128 moves, which means 7 bits is not enough. The record is something like 230 legal moves.


I disagree with that: the huge majority of positions will get a movelist inferior to 127 moves. So 7 bits is enough for this vast majority of situations. And handling the rare situations when you need more than 7 bits is easy: just use 255 (and 127) as an escape code: the next byte will contain the (nb_of_moves - 127) remaining moves. And if this number is, again, 255, then repeat. And if you code exactly for the 127th move then it would get coded 255 (or 127) and 0.

Ten years ago, I programmed a compression program à la winzip, and I used escape codes a lot: they are very handy Smile

That's what my compressor looked like, btw ^^

[...]

bob wrote:

Second, for chess engines, I don't think anyone stores a book in the way you describe, nor have they for many years. Problem is that transpositions are difficult to detect. Most just store a huge file of zobrist hash signatures. If you make a move in any position, and can find the zobrist signature in the book file, you know that was a book move.


Bob, did you read this thread, or do you now jump over all of Ed's post? Very Happy

Rebel wrote:
JuLieN wrote:
Code:

(e4(e5(Nf3(Nc6(Bb5 a6)Bc4)Nf6)f4)(c5 Nf3)(d6 d4)Nc6)e6)(d4(Nf6 c4(g6 Nc3)e6(Nc3 Bb4)Nf3)(d5(c4(c6 Nf3)(e6 Nc3)dxc4)Nf3)f5) etc.

Funny, mine is the same, still is, but added transpositions to it later Wink


Anyway, I was just answering David's request, and this answer obviously helped him, so no need to jump on me. ^^


I agree with Bob that it is imperfect to limit the number of moves to 127. I also think the solution with escape-codes is not optimal here. I would suggest the following solution:
Generate all the moves, apply some static evaluation (SEE and piecesquare-values) and order the moves according to the evaluation. Eventually use huffman coding before you store the moves in the database.

Furthermore I disagree with Bob regarding the "Most just store a huge file of zobrist hash signatures.". When talking about engines here I suppose you are describing opening books and not databases in general. The problem with this method here is that it invites unwanted transpositions.
e.g. after 1. e4 e5 2. Nf3 a6 3. Bb5 the opening book would suggest the transposition into the well known line with Nc6.
The most streight forward countermesure is to not only calculate hash values of positions but also add the move into the equation.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Subject Author Date/Time
Database storage methods david nash Thu Mar 08, 2012 10:56 am
      Re: Database storage methods Ed Schroder Thu Mar 08, 2012 11:59 am
            Re: Database storage methods Julien MARCEL Thu Mar 08, 2012 12:33 pm
                  Re: Database storage methods david nash Thu Mar 08, 2012 12:43 pm
                  Re: Database storage methods Martin Sedlak Thu Mar 08, 2012 1:00 pm
                        Re: Database storage methods Julien MARCEL Thu Mar 08, 2012 1:13 pm
                              Re: Database storage methods david nash Thu Mar 08, 2012 1:32 pm
                              Re: Database storage methods Martin Sedlak Thu Mar 08, 2012 1:45 pm
                                    Re: Database storage methods Julien MARCEL Thu Mar 08, 2012 1:51 pm
                        Re: Database storage methods Ed Schroder Thu Mar 08, 2012 3:42 pm
                  Re: Database storage methods Ed Schroder Thu Mar 08, 2012 3:46 pm
                  Re: Database storage methods Robert Hyatt Thu Mar 08, 2012 10:54 pm
                        Re: Database storage methods Julien MARCEL Thu Mar 08, 2012 11:53 pm
                              Re: Database storage methods Robert Hyatt Fri Mar 09, 2012 12:18 am
                              Re: Database storage methods Edmund Moshammer Fri Mar 09, 2012 12:51 am
                                    Re: Database storage methods H.G.Muller Fri Mar 09, 2012 9:29 am
                                          Re: Database storage methods Don Dailey Sat Mar 10, 2012 10:25 pm
                                                Re: Database storage methods Edmund Moshammer Sun Mar 11, 2012 12:27 am
                                                      Re: Database storage methods Don Dailey Sun Mar 11, 2012 5:25 am
                                    Re: Database storage methods Don Dailey Sat Mar 10, 2012 10:44 pm
                                    Re: Database storage methods Ronald de Man Sat Mar 10, 2012 11:45 pm
            Re: Database storage methods david nash Thu Mar 08, 2012 12:40 pm
      Re: Database storage methods H.G.Muller Thu Mar 08, 2012 4:36 pm
            Re: Database storage methods david nash Thu Mar 08, 2012 5:03 pm
                  Re: Database storage methods H.G.Muller Thu Mar 08, 2012 5:28 pm
                        Re: Database storage methods david nash Thu Mar 08, 2012 8:48 pm
      Re: Database storage methods Harald Lüßen Thu Mar 08, 2012 8:45 pm
            Re: Database storage methods david nash Thu Mar 08, 2012 8:51 pm
      Re: Database storage methods Nguyen Pham Thu Mar 08, 2012 10:39 pm
      Re: Database storage methods Vincent Diepeveen Sun Mar 11, 2012 6:41 am
            Re: Database storage methods david nash Sun Mar 11, 2012 4:49 pm
                  Re: Database storage methods Vincent Diepeveen Sun Mar 11, 2012 4:58 pm
                        Re: Database storage methods david nash Sun Mar 11, 2012 5:51 pm
      Re: Database storage methods Vincent Diepeveen Sun Mar 11, 2012 7:03 am
            Re: Database storage methods Vincent Diepeveen Sun Mar 11, 2012 7:21 am
            Re: Database storage methods Edmund Moshammer Sun Mar 11, 2012 9:21 am
                  Re: Database storage methods Vincent Diepeveen Sun Mar 11, 2012 3:41 pm
                        Re: Database storage methods david nash Sun Mar 11, 2012 5:08 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads