ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

how to measure frequency of hash collisions.
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Joshua Shriver



Joined: 08 Mar 2006
Posts: 836
Location: Toledo, OH, USA
PostPost subject: Re: how to measure frequency of hash collisions.    Posted: Thu Jun 21, 2012 1:51 am Reply to topic Reply with quote
syzygy wrote:
jshriver wrote:
My first couple attempts had 8-10% collisions, even with 64bit keys.

For how many positions in total?


Forget, its' been a while but it was in the trillions.

A lot of my interest and work goes into retaining chess knowledge and I work with over 100gigs of PGN data, plus maybe another 4-5gigs in home crafted data. So this has been an interesting obstacle for me.

Need the data to be small for storage and lookups, but when working with large volumes traditional hash algorithms break down.

When I tried with smaller amounts say less than a gig I had no problems. Which is ok for opening books and such though, or transposition tables which are even smaller.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Subject Author Date/Time
how to measure frequency of hash collisions. Daniel Shawul Sat Jun 16, 2012 4:45 pm
      Re: how to measure frequency of hash collisions. Marcel van Kervinck Sat Jun 16, 2012 5:25 pm
            Some tests: Attn Don Daniel Shawul Sat Jun 16, 2012 6:25 pm
                  Re: Some tests: Attn Don Daniel Shawul Sat Jun 16, 2012 7:15 pm
      Re: how to measure frequency of hash collisions. Don Dailey Sat Jun 16, 2012 9:17 pm
            Re: how to measure frequency of hash collisions. Daniel Shawul Sat Jun 16, 2012 10:05 pm
                  Re: how to measure frequency of hash collisions. Don Dailey Sat Jun 16, 2012 10:23 pm
                        Re: how to measure frequency of hash collisions. Don Dailey Sat Jun 16, 2012 10:29 pm
                        Re: how to measure frequency of hash collisions. Daniel Shawul Sat Jun 16, 2012 11:06 pm
                              Re: how to measure frequency of hash collisions. Don Dailey Sat Jun 16, 2012 11:39 pm
                                    Re: how to measure frequency of hash collisions. Kevin Hearn Sat Jun 16, 2012 11:50 pm
                                          Re: how to measure frequency of hash collisions. Don Dailey Sun Jun 17, 2012 12:08 am
                                          Re: how to measure frequency of hash collisions. Daniel Shawul Sun Jun 17, 2012 12:40 am
                                    Re: how to measure frequency of hash collisions. Daniel Shawul Sat Jun 16, 2012 11:54 pm
                                          Re: how to measure frequency of hash collisions. Don Dailey Sun Jun 17, 2012 12:23 am
                                    Re: how to measure frequency of hash collisions. Marcel van Kervinck Sun Jun 17, 2012 7:55 am
                  Re: how to measure frequency of hash collisions. Don Dailey Sun Jun 17, 2012 1:26 am
      Re: how to measure frequency of hash collisions. Ed Schroder Sat Jun 16, 2012 9:36 pm
            Re: how to measure frequency of hash collisions. Daniel Shawul Sat Jun 16, 2012 10:01 pm
      Re: how to measure frequency of hash collisions. H.G.Muller Sat Jun 16, 2012 10:12 pm
            Re: how to measure frequency of hash collisions. Daniel Shawul Sat Jun 16, 2012 10:31 pm
            Re: how to measure frequency of hash collisions. Daniel Shawul Sun Jun 17, 2012 2:00 am
                  Re: how to measure frequency of hash collisions. Rémi Coulom Sun Jun 17, 2012 8:30 am
                        Re: how to measure frequency of hash collisions. Daniel Shawul Sun Jun 17, 2012 10:49 am
                        Re: how to measure frequency of hash collisions. Daniel Shawul Sun Jun 17, 2012 2:58 pm
                              Re: how to measure frequency of hash collisions. Ronald de Man Sun Jun 17, 2012 3:41 pm
                                    Re: how to measure frequency of hash collisions. Daniel Shawul Sun Jun 17, 2012 5:58 pm
                                          Re: how to measure frequency of hash collisions. Ronald de Man Sun Jun 17, 2012 9:06 pm
                                                Re: how to measure frequency of hash collisions. Daniel Shawul Sun Jun 17, 2012 9:57 pm
      Re: how to measure frequency of hash collisions. Lucas Braesch Sun Jun 17, 2012 3:02 am
            Re: how to measure frequency of hash collisions. Ronald de Man Sun Jun 17, 2012 3:08 pm
            Re: how to measure frequency of hash collisions. Daniel Shawul Sun Jun 17, 2012 6:12 pm
                  Re: how to measure frequency of hash collisions. Ronald de Man Sun Jun 17, 2012 9:14 pm
                        Re: how to measure frequency of hash collisions. Daniel Shawul Sun Jun 17, 2012 10:01 pm
      Re: how to measure frequency of hash collisions. Joshua Shriver Wed Jun 20, 2012 8:45 pm
            Re: how to measure frequency of hash collisions. Ronald de Man Wed Jun 20, 2012 10:44 pm
                  Re: how to measure frequency of hash collisions. Joshua Shriver Thu Jun 21, 2012 1:51 am
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads