Komodo 9 vs Komodo 10 memory difference

[Paul Bedrey]

I noticed that Komodo 10 runs in Arena with a significant memory difference than Komodo 9. Arena reports 4096 for Komodo 9 and all other engines but only 3181 for Komodo 10. I double checked it with Windows Task manager and it agrees with Arena. Is this normal? Can I change it? Under UCI options I have set a common hash size to 4096. When loading out Komodo 10 it reports hash table is now set at 4096 but memory is reported as 3181 mb. Up till now I thought memory and hash were the same. Are they?

[Modern Times]

The Komodo documentation explains this.

[Paul Bedrey]

Komodo 10 documentation explains how to calculate hash size. My question is why Komodo 10 ignores the 4096 number I set in the UCI option I set in Arena. All other engines including Komodo 9 use my settings. I've read your and other test results and there is no question that Komodo 10 is at least 50 points stronger than Komodo 9 yet I have not had that experience and was wondering why.

[Modern Times]

The explanation is in the documentation....

[mjlef]

Komodo does things differently than other engines. One change since early Komodo 9 versions is I made the size of a hash entry smaller. The smaller the entry the faster the access. Most engines hash entries are power of 2. Like 8 or 16 bytes. Komodo uses 12 bytes, so hash allocation uses he formula 12 * 2^n. So 12, 24, 48, 96, 192... are the hash suzes used. Some GUIs only allow powers of 2. So Komodo will use the largest size that fits in the GUIs hash request that fits the formula above. Think of it as getting 25% more hash in the same space as old Komodo.a nd it is about 2% faster.

Mark

[Karol Majewski]

Komodo does things differently than other engines. One change since early Komodo 9 versions is I made the size of a hash entry smaller. The smaller the entry the faster the access. Most engines hash entries are power of 2. Like 8 or 16 bytes. Komodo uses 12 bytes, so hash allocation uses he formula 12 * 2^n. So 12, 24, 48, 96, 192... are the hash suzes used. Some GUIs only allow powers of 2. So Komodo will use the largest size that fits in the GUIs hash request that fits the formula above. Think of it as getting 25% more hash in the same space as old Komodo.a nd it is about 2% faster.

Mark

Mark, shouldn't you then set default Hash value to 96 or 192 instead of 128?

[Paul Bedrey]

OK thanks Mark as long as I know that I don't have a crippled engine. I noticed that Komodo 10 was using roughly 80% of whatever hash I set. I verified that Komodo 9.42 is doing the same too. I run head to to head matches between Komodo and Stockfish using 2 machines entering moves by hand. It's time consuming since I'm using 30 minute 3 sec time control with ponder on. I've stopped the match twice so far because I just can't believe the lopsided result. I'm using the Silver testsuite so maybe it'll even out eventually. Since each engine plays both sides I never had a result this strange before. I even broke out a 3rd quad playing no ponder games all other conditions the same and it is showing a very similar result. In reported head to head matches Stockfish usually is scoring 52% but I'm getting an unbelievable 62-63% difference. I've played hundreds of games with Komodo 9 the past year and never had that happen so I was surprised by this Komodo 10 result. I had assumed by using a testsuite rather than a book I was lowering statistical randomness. I guess I was wrong.

[hgm]

Think of it as getting 25% more hash in the same space as old Komodo.a nd it is about 2% faster.

Actually in the described case it is more like you get the same hash in 25% smaller memory. This still of course benefits from the 2% speed increase, but leaving memory unused is of no benefit.

You would of course benefit enormously when users set all engine's hash size to 12*2^n, as that would reduce opponents that need powers of 2 for hash size to use 8*2^n, i.e. 33% less memory than Komodo. (And they would likely have larger entries too.)

Can't you use all the memory the user allows you? It is not that more expensive to derive an index in an arbitrary range from the hash key. Just an extra multiply and shift.

[zullil]

hash allocation uses he formula 12 * 2^n. So 12, 24, 48, 96, 192... are the hash suzes used.
Mark

Gee, I thought it was 3 * 2^n.

[hgm]

Can it be set to 3 GB?

Probably. But not to 3 bytes. But I guess the semantics of the UCI Hash option would prevent that anyway.