Starting with Hash Tables.

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
Luis Babboni
Posts: 464
Joined: Sat Feb 28, 2015 4:37 pm
Location: Argentina

Re: Starting with Hash Tables.

Post by Luis Babboni »

Sven Schüle wrote:
Luis Babboni wrote:Sorry if is a too stupid question:

At this moment I think in make my TT as an array like:

Dim TT(key, piece, move, score, typescore, turn, remainingdepth, age)

Where:
key have 4 bytes
score have 2 bytes and all others 1 byte.
This would not be "the TT" but only the structure of one entry of that table, so I suggest that you do not call it "TT" but "TTEntry" or similar. The whole TT would itself be an array of many "TTEntry" items (e.g. 256 MB / 12 byte = about 22 million entries - you might also want to arrange the size of an entry to be 16 bytes so that both the total TT size and the number of TT entries are a power of 2).
I´m still not sure haow to do it, I think I need to ask some questions in FreeBasic forum first.
Having 16 bytes instead of 12 will be easier than use just 12 I think! :D
User avatar
Luis Babboni
Posts: 464
Joined: Sat Feb 28, 2015 4:37 pm
Location: Argentina

Re: Starting with Hash Tables.

Post by Luis Babboni »

Sven Schüle wrote:
Luis Babboni wrote:Sorry if is a too stupid question:

At this moment I think in make my TT as an array like:

Dim TT(key, piece, move, score, typescore, turn, remainingdepth, age)

Where:
key have 4 bytes
score have 2 bytes and all others 1 byte.
...

I would also like to know
a) how you would use "piece" and "turn" in this context, and
b) how you would manage to represent a move in 1 byte?

...
I think I will need a byte for "turn" cause the actual way of coding a position that I use for 3erd repetirion rule, do not includes turn*. I do not know if that´s Zobrist´s numbers way include it or not.
*: not means that for 3rd repetition I do not take in account the turn, just compared previous positions jumping 2 steps each time.

Sorry, the pair "piece"/"move" is my "move". Example: move is piece 1 (pawn) move 11 (capturing to left and promoting queen).
.... I think I could use just one byte for move anyway:
I have:
Pawn: 18 types of moves.
Knight: 8 types of moves.
Bishop: 28 types of moves.
Rook: 28 types of moves.
Queen:56 types of moves.
King: 10 types of moves.
TOTAL: 148 types of moves.
Last edited by Luis Babboni on Sun Nov 13, 2016 8:42 pm, edited 4 times in total.
User avatar
Luis Babboni
Posts: 464
Joined: Sat Feb 28, 2015 4:37 pm
Location: Argentina

Re: Starting with Hash Tables.

Post by Luis Babboni »

Sven Schüle wrote: ...
Luis Babboni wrote:I use many other arrays in my code too.
I suggest to consider using user-defined data types (keyword TYPE in FreeBasic) where appropriate, instead of arrays. In fact it would be appropriate for many concepts in a chess engine like TTEntry, Move, Board, and some others. It would improve the program's readability and would simplify changing the code as well as debugging it.
I was just persuaded to do this, but need a complete rewrite of the code, so is scheduled for Soberango 1.xx.x :oops:
User avatar
Luis Babboni
Posts: 464
Joined: Sat Feb 28, 2015 4:37 pm
Location: Argentina

Re: Starting with Hash Tables.

Post by Luis Babboni »

I still do not understand how the ruler could distingish if I´m using a TT or another kind of table to allow me to use for examepl just 256MB only for TT and more memory for others tables.

I think there is something I missing here. :(
Ras
Posts: 2487
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: Starting with Hash Tables.

Post by Ras »

I don't think the Windows ressource monitor is a suitable tool for measuring hash table usage.

Maybe zeroing the hash tables before a move would be better, and after the move, counting how many entries are non-zero. Dividing that number by the total number of entries gives the usage percentage.
User avatar
Luis Babboni
Posts: 464
Joined: Sat Feb 28, 2015 4:37 pm
Location: Argentina

Re: Starting with Hash Tables.

Post by Luis Babboni »

Sorry for noob question:

1MB = 1,024 x 1,024 bytes = 1,048,576 bytes
256MB = 256 x 1024 x 1024 bytes = 268,435,456 bytes > 256,000,000 bytes.

I´m right?
Ras
Posts: 2487
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: Starting with Hash Tables.

Post by Ras »

Luis Babboni wrote:1MB = 1,024 x 1,024 bytes = 1,048,576 bytes
256MB = 256 x 1024 x 1024 bytes = 268,435,456 bytes > 256,000,000 bytes.
Correct. The only point where they use 1MB = 1,000,000 bytes is hard disks. The reason is that the manufacturers can display bigger numbers that way.
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Starting with Hash Tables.

Post by hgm »

Luis Babboni wrote:I still do not understand how the ruler could distingish if I´m using a TT or another kind of table to allow me to use for examepl just 256MB only for TT and more memory for others tables.

I think there is something I missing here. :(
Well, the protocol commands specify total memory that you are allowed to use for all tables together. It is sometimes assumed that all tables of significant size will be hash tables.

Unfortunately many engines cheat, and when the option controlling their tablesize is set to 128MB, would not hesitate to use 128MB just for their TT, and then another 600MB on top of it for, say, bitbases. If testers allow this kind of cheating is upto them.
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Starting with Hash Tables.

Post by Sven »

Luis Babboni wrote:
Sven Schüle wrote:
Luis Babboni wrote:Sorry if is a too stupid question:

At this moment I think in make my TT as an array like:

Dim TT(key, piece, move, score, typescore, turn, remainingdepth, age)

Where:
key have 4 bytes
score have 2 bytes and all others 1 byte.
This would not be "the TT" but only the structure of one entry of that table, so I suggest that you do not call it "TT" but "TTEntry" or similar. The whole TT would itself be an array of many "TTEntry" items (e.g. 256 MB / 12 byte = about 22 million entries - you might also want to arrange the size of an entry to be 16 bytes so that both the total TT size and the number of TT entries are a power of 2).
I´m still not sure haow to do it, I think I need to ask some questions in FreeBasic forum first.
Maybe you could read this: http://www.freebasic.net/wiki/wikka.php?wakka=KeyPgType
Example:

Code: Select all

TYPE Move
    pieceIndex AS UBYTE
    moveIndex AS UBYTE
END TYPE

TYPE TTEntry
    key AS ULONGINT   ' using the whole 8 byte key
    move AS Move
    score AS SHORT
    scoreType AS UBYTE   ' could as well be an enum but that might take more than 1 byte
    remainingDepth AS UBYTE
    age AS UBYTE
    paddingBytes AS UBYTE(1)
END TYPE

DIM tt(0 TO 16777215) AS TTEntry
...
DIM ttIndex = hashKey MOD (2 ^ 24)   ' take the lower bits of the hash key as table index, 2 ^ 24 = 16777216
DIM ttEntry AS TTEntry
ttEntry = tt(ttIndex)
ttEntry.key = hashKey
ttEntry.move = ...
...
Luis Babboni wrote:Having 16 bytes instead of 12 will be easier than use just 12 I think! :D
Yes, and as usual, you can optimize it later.
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Starting with Hash Tables.

Post by Sven »

Ras wrote:
Luis Babboni wrote:1MB = 1,024 x 1,024 bytes = 1,048,576 bytes
256MB = 256 x 1024 x 1024 bytes = 268,435,456 bytes > 256,000,000 bytes.
Correct. The only point where they use 1MB = 1,000,000 bytes is hard disks. The reason is that the manufacturers can display bigger numbers that way.
I was quite shocked when I learned some years ago that they had changed the "official" definition of 1 MB that everyone had learned at school from 1,048,576 to 1,000,000 bytes while introducing the new "MiB" term. For me 1 kB, MB, GB is still counting as 2 ^ 10, 2 ^ 20, 2 ^ 30 bytes. But if my harddisk has 500 GB then today it really has 500,000,000,000 bytes, unfortunately, which is only about 466 "good old GB" :(