Komodo 9.1 Release

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mjlef
Posts: 1428
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Komodo 9.1 Release

Post by mjlef » Wed Jun 24, 2015 3:47 pm

ernst wrote:
mjlef wrote:
RJN wrote:Does anyone else see hash size for Komodo 9.1 being 1-2GB smaller when looking at what is actually being reported with task manager?

With 4GB hash, I see 3GB reported as used.
With 8GB hash, I see 6GB reported as used.

(I did not try other sizes yet)

Komodo 9.1 Win7 64 bit
That is normal. I redesigned the hash system to fit in about 75% of the space it took up before. When you tell the computer to use say 8 GB of memory, Komodo will allocate the multiple of 2 for the biggest hash table it can fit in that space. Due to the way hashing works, each increase in size is a doubling. So anyway, 6 GB is the biggest hash it can allocate in an 8 GB chunk of memory. The next step up would be 12 GB, and so on. Doing this let us fit in more hashed positions in a given amount of memory. Basically you are getting what was 8 GB of memory now compacted to only take up 6 GB.
I wonder if you achieved this by reducing the size of hash entries by 1/3. Doesn't that increase the chance of hash collisions?
The smaller you make the "key" part of the hash entry, the greater the risk of "collisions" which are entries that might falsely match the position. Some programs use very tiny keys, and depend on using another part of the key as a index to the position. I have seen some program use 16 bits for this key, and experiments using even less. Surprisingly, even the tiny keys do not seem to cause many collisions, and even if they collide, few of these collisions are bad enough to change the root score in a game result changing way.

Bob Hyatt and Anthony Cozzie wrote a very good paper on this:

https://cis.uab.edu/hyatt/collisions.html

I know Stockfish reduced there hash key size a while ago, and there is probably some good data about this on the stockfish forum.

In the future, we might have to rethink things as machines get faster and faster.

lkaufman
Posts: 3741
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Cannot upgrade without paying another fee

Post by lkaufman » Wed Jun 24, 2015 3:51 pm

cma6 wrote:LK:
Thank you for your continued rapid development of Komodo.
Unfortunately, although I bought version 9.0, there is no online way to upgrade to 9.1 without paying another $99. I hope you can iron out the glitches in the online upgrade process.

On happier note, let me recommend LK's "Sabotage the Gruenfeld", which I have just gone through page by page and line by line. It is the first GM opening book to fully use computer evals with GM insight to create part of your opening repertoire.
You should be able to purchase version 9.1 for $47.98 reflecting the 20% discount for K9 buyers. Others have done so. Let me know if you can't do this.
Komodo rules!

User avatar
AdminX
Posts: 5185
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: Cannot upgrade without paying another fee

Post by AdminX » Wed Jun 24, 2015 4:12 pm

cma6 wrote:Ted,
Thanks for your rapid response, greatly appreciated.
I have emailed Mark but no response there.
:shock: :oops: I should have been clearer, I meant the book. Glad to see Larry responded in any event.
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

Dann Corbit
Posts: 10187
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Cannot upgrade without paying another fee

Post by Dann Corbit » Wed Jun 24, 2015 7:02 pm

cma6 wrote:LK:
Thank you for your continued rapid development of Komodo.
Unfortunately, although I bought version 9.0, there is no online way to upgrade to 9.1 without paying another $99. I hope you can iron out the glitches in the online upgrade process.

On happier note, let me recommend LK's "Sabotage the Gruenfeld", which I have just gone through page by page and line by line. It is the first GM opening book to fully use computer evals with GM insight to create part of your opening repertoire.
People who always want the bleeding edge version should get the "give me every bleeding edge update" plan.

christian
Posts: 9
Joined: Thu May 30, 2013 5:54 pm

Re: Cannot upgrade without paying another fee

Post by christian » Wed Jun 24, 2015 8:42 pm

I would like to upgrade from 9.2 to subscriber for this Money because I have bought 9.2 some days before 9.1 was there.

christian
Posts: 9
Joined: Thu May 30, 2013 5:54 pm

Re: Cannot upgrade without paying another fee

Post by christian » Wed Jun 24, 2015 11:08 pm

I would like to upgrade from 9.02 to subscriber for this Money because I have bought 9.02 some days before 9.1 was there.

now it is correct

bob
Posts: 20639
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Komodo 9.1 Release

Post by bob » Fri Jun 26, 2015 3:02 pm

mjlef wrote:
ernst wrote:
mjlef wrote:
RJN wrote:Does anyone else see hash size for Komodo 9.1 being 1-2GB smaller when looking at what is actually being reported with task manager?

With 4GB hash, I see 3GB reported as used.
With 8GB hash, I see 6GB reported as used.

(I did not try other sizes yet)

Komodo 9.1 Win7 64 bit
That is normal. I redesigned the hash system to fit in about 75% of the space it took up before. When you tell the computer to use say 8 GB of memory, Komodo will allocate the multiple of 2 for the biggest hash table it can fit in that space. Due to the way hashing works, each increase in size is a doubling. So anyway, 6 GB is the biggest hash it can allocate in an 8 GB chunk of memory. The next step up would be 12 GB, and so on. Doing this let us fit in more hashed positions in a given amount of memory. Basically you are getting what was 8 GB of memory now compacted to only take up 6 GB.
I wonder if you achieved this by reducing the size of hash entries by 1/3. Doesn't that increase the chance of hash collisions?
The smaller you make the "key" part of the hash entry, the greater the risk of "collisions" which are entries that might falsely match the position. Some programs use very tiny keys, and depend on using another part of the key as a index to the position. I have seen some program use 16 bits for this key, and experiments using even less. Surprisingly, even the tiny keys do not seem to cause many collisions, and even if they collide, few of these collisions are bad enough to change the root score in a game result changing way.

Bob Hyatt and Anthony Cozzie wrote a very good paper on this:

https://cis.uab.edu/hyatt/collisions.html

I know Stockfish reduced there hash key size a while ago, and there is probably some good data about this on the stockfish forum.

In the future, we might have to rethink things as machines get faster and faster.
Remember that with a 64 bit hash signature, if you store 16, you are not really using a 16 bit signature, you are using something significantly larger since you use the other part of the signature to address the table. In the current version of Crafty, I have moved to an 8 byte hash entry. 23 bits for the upper part of the signature. But when I run with a 32gb hash table, that turns into 4 billion entries (2^32). Which means I use the rightmost 32 bits of the entry to address the table, and store the leftmost 23 bits. That only leaves 9 bits unaccounted for out of the original 64. An advantage of this is that you can use a hash bucket size of 8, since that is still exactly one cache block (if you align things correctly of course) which gives you a better chance to replace something useless rather than something good.

I have tested this to exhaustion with zero problems. Even when searching at 100M nodes per second, giving billions of nodes even for pretty fast games.

Post Reply