UCI Hash Usage Rules

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
MOBMAT
Posts: 74
Joined: Sat Feb 04, 2017 10:57 pm
Location: USA

UCI Hash Usage Rules

Post by MOBMAT » Sun Jul 01, 2018 10:23 pm

The "hash" option in UCI is used to set the size of the program's hash table. Most programs make use of hash tables.Some programs also have additional hashes/caches for eval, pawns, TBs, etc.

I recall reading somewhere (sorry, I can't provide a link), where it was stated that the hash size specified by UCI is suppose to represent the upper limit of memory use. So, if one adds up the sizes of all the hashes, it shouldn't exceed the value specified by the hash value.
  • Is this an accurate interpretation of the "rule"?
    Do programs (read yours) really adhere to this?
    Is it actually enforced by any any GUI/Tournament software?
    What is the common sense approach to this making it fair?
    Should I just use the size for my main hash size and do whatever I want with the other hashes/caches?
Vince S
Author of MOBMAT

"Reductions, extensions, and pruning, oh my!"

AndrewGrant
Posts: 472
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: UCI Hash Usage Rules

Post by AndrewGrant » Sun Jul 01, 2018 11:07 pm

As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.

As for "Upper Bound", I think this is nonsense. Once you start capping the top memory usage, you have to start thinking about all the tiny "tables" that are in use. For example, I have Counter Move Tables, Killer Move Tables, History Tables, Counter Move History Tables, Pawn Tables (Static size), ...

Ras
Posts: 1140
Joined: Tue Aug 30, 2016 6:19 pm
Contact:

Re: UCI Hash Usage Rules

Post by Ras » Sun Jul 01, 2018 11:43 pm

MOBMAT wrote:
Sun Jul 01, 2018 10:23 pm
So, if one adds up the sizes of all the hashes, it shouldn't exceed the value specified by the hash value.
Correct. But usually, the main hash tables are the only important memory consumer. Static pawn hash tables are typically below 100 kB, which is small in comparison to the 1 MB resolution that you can set via UCI. More doesn't make sense for the pawn hash table because the hit rate doesn't really increase beyond roughly 10k entries. Other static caches are only 1-2kB or so, not worth considering.

Of course, if you had other hashing buffers of several MB, then you would need to take them into account.
Rasmus Althoff
https://www.ct800.net

User avatar
hgm
Posts: 23610
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: UCI Hash Usage Rules

Post by hgm » Mon Jul 02, 2018 7:24 am

AndrewGrant wrote:
Sun Jul 01, 2018 11:07 pm
As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.
This is wrong. The UCI specs clearly mention 'hash tables', plural. Insignificant tables don't have to be counted, but when an engine starts to use significant memory for other purposes (such as keeping bitbases in memory), it should come from the value indicated by the Hash option.

User avatar
Matthias Gemuh
Posts: 3238
Joined: Thu Mar 09, 2006 8:10 am
Contact:

Re: UCI Hash Usage Rules

Post by Matthias Gemuh » Mon Jul 02, 2018 9:47 am

hgm wrote:
Mon Jul 02, 2018 7:24 am
AndrewGrant wrote:
Sun Jul 01, 2018 11:07 pm
As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.
This is wrong. The UCI specs clearly mention 'hash tables', plural. Insignificant tables don't have to be counted, but when an engine starts to use significant memory for other purposes (such as keeping bitbases in memory), it should come from the value indicated by the Hash option.
ChessGUI has a parameter "Reduce Mem" that reduces Hash value of engines that steal too much extra Hash.
My engine was quite strong till I added knowledge to it.
http://www.chess.hylogic.de

elcabesa
Posts: 806
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: UCI Hash Usage Rules

Post by elcabesa » Mon Jul 02, 2018 9:57 am

To make an example Vajolet has a footprint in memory of 3.1MB plus the hash table. In the 3.1MB are counted all variables, pawn hash table, magic move initialization tables, search history tables, quiet moves stat tables and so on. I don't know what it's considered fair, but I think that if you have an additional table of 64 MB somewhere... It should be counted and it's not trascurable.
Stockfish seems to have a footprint of 8 MB on my machine for example

DustyMonkey
Posts: 46
Joined: Wed Feb 19, 2014 9:11 pm

Re: UCI Hash Usage Rules

Post by DustyMonkey » Fri Jul 06, 2018 4:23 am

It is up to the user to decide the value of every parameter. UCI is not defined with "fairness" built in because thats wholly subjective. A tournament organizer would be responsible for "fairness" and that extends beyond just the hash value.

User avatar
hgm
Posts: 23610
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: UCI Hash Usage Rules

Post by hgm » Sat Jul 07, 2018 3:15 am

The point is that the parameters should have a fixed meaning for the user to sensibly set them. If an engine would use 640MB hash when the user specifies 64MB, it is simply not compliant. The idea of these standard options is that the GUI can apply them to all engines, having the same effect in each case.

DustyMonkey
Posts: 46
Joined: Wed Feb 19, 2014 9:11 pm

Re: UCI Hash Usage Rules

Post by DustyMonkey » Sat Jul 07, 2018 11:42 am

Most engines use 1 thread more than is specified by the user. This thread is merely for the UI in hopefully all of these engines and is considered by many to be inconsequential.

Haven't seen that debate.

Some engines ponder on the ponder move specifically, while others ponder on all the possible opponent moves.

Haven't seen that debate either.

Shouldn't be seeing this one. UCI is not a chess engine virtual machine. You guys are going to really hate it as engines with network evaluators grow in number. These will easily have hundreds of megabytes of built-in feature vectors.

User avatar
hgm
Posts: 23610
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: UCI Hash Usage Rules

Post by hgm » Sun Jul 08, 2018 1:19 am

' Threads' is not a UCI standard option, and engines that define it specifically define it as the number of search threads. So it is weird that you think anyone should object to that they use that number of search threads.

UCI specs allow explicitly to do whatever you want in ponder time. So again, you seem to have no point. To be non-compliant w.r.t. pondering, the engine would have to use significant CPU while pondering is off. An engine that would just run a task to flush the caches as quickly as possible while the opponent is thinking, to slow him down in a no-ponder game, would spark a debate, don't you think? The excuse "what are you complaining about, I am not pondering" is not likely to be accepted.

Post Reply