Page 1 of 7

UCI Hash Usage Rules

Posted: Mon Jul 02, 2018 12:23 am
by MOBMAT
The "hash" option in UCI is used to set the size of the program's hash table. Most programs make use of hash tables.Some programs also have additional hashes/caches for eval, pawns, TBs, etc.

I recall reading somewhere (sorry, I can't provide a link), where it was stated that the hash size specified by UCI is suppose to represent the upper limit of memory use. So, if one adds up the sizes of all the hashes, it shouldn't exceed the value specified by the hash value.
  • Is this an accurate interpretation of the "rule"?
    Do programs (read yours) really adhere to this?
    Is it actually enforced by any any GUI/Tournament software?
    What is the common sense approach to this making it fair?
    Should I just use the size for my main hash size and do whatever I want with the other hashes/caches?

Re: UCI Hash Usage Rules

Posted: Mon Jul 02, 2018 1:07 am
by AndrewGrant
As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.

As for "Upper Bound", I think this is nonsense. Once you start capping the top memory usage, you have to start thinking about all the tiny "tables" that are in use. For example, I have Counter Move Tables, Killer Move Tables, History Tables, Counter Move History Tables, Pawn Tables (Static size), ...

Re: UCI Hash Usage Rules

Posted: Mon Jul 02, 2018 1:43 am
by Ras
MOBMAT wrote: Mon Jul 02, 2018 12:23 amSo, if one adds up the sizes of all the hashes, it shouldn't exceed the value specified by the hash value.
Correct. But usually, the main hash tables are the only important memory consumer. Static pawn hash tables are typically below 100 kB, which is small in comparison to the 1 MB resolution that you can set via UCI. More doesn't make sense for the pawn hash table because the hit rate doesn't really increase beyond roughly 10k entries. Other static caches are only 1-2kB or so, not worth considering.

Of course, if you had other hashing buffers of several MB, then you would need to take them into account.

Re: UCI Hash Usage Rules

Posted: Mon Jul 02, 2018 9:24 am
by hgm
AndrewGrant wrote: Mon Jul 02, 2018 1:07 am As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.
This is wrong. The UCI specs clearly mention 'hash tables', plural. Insignificant tables don't have to be counted, but when an engine starts to use significant memory for other purposes (such as keeping bitbases in memory), it should come from the value indicated by the Hash option.

Re: UCI Hash Usage Rules

Posted: Mon Jul 02, 2018 11:47 am
by Matthias Gemuh
hgm wrote: Mon Jul 02, 2018 9:24 am
AndrewGrant wrote: Mon Jul 02, 2018 1:07 am As far as I know, the main Hash UCI option is meant to only talk about the primary Transposition Table.

Any other large tables, sized dynamically, I would list seperate options for. Some engines have an Eval Cache.
This is wrong. The UCI specs clearly mention 'hash tables', plural. Insignificant tables don't have to be counted, but when an engine starts to use significant memory for other purposes (such as keeping bitbases in memory), it should come from the value indicated by the Hash option.
ChessGUI has a parameter "Reduce Mem" that reduces Hash value of engines that steal too much extra Hash.

Re: UCI Hash Usage Rules

Posted: Mon Jul 02, 2018 11:57 am
by elcabesa
To make an example Vajolet has a footprint in memory of 3.1MB plus the hash table. In the 3.1MB are counted all variables, pawn hash table, magic move initialization tables, search history tables, quiet moves stat tables and so on. I don't know what it's considered fair, but I think that if you have an additional table of 64 MB somewhere... It should be counted and it's not trascurable.
Stockfish seems to have a footprint of 8 MB on my machine for example

Re: UCI Hash Usage Rules

Posted: Fri Jul 06, 2018 6:23 am
by DustyMonkey
It is up to the user to decide the value of every parameter. UCI is not defined with "fairness" built in because thats wholly subjective. A tournament organizer would be responsible for "fairness" and that extends beyond just the hash value.

Re: UCI Hash Usage Rules

Posted: Sat Jul 07, 2018 5:15 am
by hgm
The point is that the parameters should have a fixed meaning for the user to sensibly set them. If an engine would use 640MB hash when the user specifies 64MB, it is simply not compliant. The idea of these standard options is that the GUI can apply them to all engines, having the same effect in each case.

Re: UCI Hash Usage Rules

Posted: Sat Jul 07, 2018 1:42 pm
by DustyMonkey
Most engines use 1 thread more than is specified by the user. This thread is merely for the UI in hopefully all of these engines and is considered by many to be inconsequential.

Haven't seen that debate.

Some engines ponder on the ponder move specifically, while others ponder on all the possible opponent moves.

Haven't seen that debate either.

Shouldn't be seeing this one. UCI is not a chess engine virtual machine. You guys are going to really hate it as engines with network evaluators grow in number. These will easily have hundreds of megabytes of built-in feature vectors.

Re: UCI Hash Usage Rules

Posted: Sun Jul 08, 2018 3:19 am
by hgm
' Threads' is not a UCI standard option, and engines that define it specifically define it as the number of search threads. So it is weird that you think anyone should object to that they use that number of search threads.

UCI specs allow explicitly to do whatever you want in ponder time. So again, you seem to have no point. To be non-compliant w.r.t. pondering, the engine would have to use significant CPU while pondering is off. An engine that would just run a task to flush the caches as quickly as possible while the opponent is thinking, to slow him down in a no-ponder game, would spark a debate, don't you think? The excuse "what are you complaining about, I am not pondering" is not likely to be accepted.