Zahak, a GoLang based chess engine

[amanjpro]

Nice work. I played 4 games against it with leela2200 (lc0 running a 16x2 T10 distilled net on cpu) on lichess. All tied up at 2 wins a piece.

Oh, thank you. I think leela2200 won the mini match 3-2 though

[dkappe]

Nice work. I played 4 games against it with leela2200 (lc0 running a 16x2 T10 distilled net on cpu) on lichess. All tied up at 2 wins a piece.

Oh, thank you. I think leela2200 won the mini match 3-2 though

It appears that white won every game.

[Gabor Szots]

As for the memory allocation, I am not seeing it on Mac, Linux and also Windows (on VritualBox though). Here is a Windows screenshot:

https://drive.google.com/file/d/1J2_6GG ... sp=sharing

I set hash to 32 (MB), and the allocation is around 45 (for the entire program)

I tried some hash settings under Cute Chess.

When set to 16 MB, the allocated memory is 65.
When set to 32 MB, the allocated memory is 129 MB.
64 --> 286
128 --> more than 600 but then gradually decreases down to about 400.
256 --> about 1400, then gradually decreases down to about 800.

So it seems that for lower values the allocated memory increase is about 4 times the hash size increase. For greater values the allocated memory changes with time (it went down a bit with each move I made).

[Guenther]

As for the memory allocation, I am not seeing it on Mac, Linux and also Windows (on VritualBox though). Here is a Windows screenshot:

https://drive.google.com/file/d/1J2_6GG ... sp=sharing

I set hash to 32 (MB), and the allocation is around 45 (for the entire program)

I tried some hash settings under Cute Chess.

When set to 16 MB, the allocated memory is 65.
When set to 32 MB, the allocated memory is 129 MB.
64 --> 286
128 --> more than 600 but then gradually decreases down to about 400.
256 --> about 1400, then gradually decreases down to about 800.

So it seems that for lower values the allocated memory increase is about 4 times the hash size increase. For greater values the allocated memory changes with time (it went down a bit with each move I made).

I can confirm that behaviour under Win7-64 from cmd w/o any GUI.

Code: Select all

uci
id name Zahak
id author Amanj
option name Ponder type check default false
option name Hash type spin default 10 min 1 max 8000
option name Book type check default false
uciok
ucinewgame
isready
readyok
setoption name Hash value 320
position startpos
go infinite
...

Zahak loaded around 400MB here after setting it to 320 (that's completely normal), but after I fired it up with go infinite from the startpostion it went ub to around 1.6GB and while thinking, the memory was reduced by about 4-5MB per second for a long time.
The reducing later became slower and slower and halted at around 1GB.

[mvanthoor]

As for the memory allocation, I am not seeing it on Mac, Linux and also Windows (on VritualBox though). Here is a Windows screenshot:

https://drive.google.com/file/d/1J2_6GG ... sp=sharing

I set hash to 32 (MB), and the allocation is around 45 (for the entire program)

I tried some hash settings under Cute Chess.

When set to 16 MB, the allocated memory is 65.
When set to 32 MB, the allocated memory is 129 MB.
64 --> 286
128 --> more than 600 but then gradually decreases down to about 400.
256 --> about 1400, then gradually decreases down to about 800.

So it seems that for lower values the allocated memory increase is about 4 times the hash size increase. For greater values the allocated memory changes with time (it went down a bit with each move I made).

Maybe it uses a vector without giving it a capacity or specific number of elements to start with. In Rust, if you create a vector and then push elements into it, the vector needs to resize if it's not big enough. It'll then resize to about 1.5x its current size. For example:

You could initialize a vector with a capacity of say, 8 elements and make sure you don't ever put more than 256 elements in it. Then the vector will stay at 8 elements all the time. You could also initialize it without a capacity and leave it empty. After that, when you put elements in it, it will go something like this in many programming languages:

0 -> empty
1 -> reserved memory for 2 elements.
2 -> 2
3 -> 4
4 -> 4
5 -> 6
6 - > 8
7 -> 8
8 -> 8
9 -> 16
10 -> 16
...

Maybe that's what you're seeing.

[amanjpro]

As for the memory allocation, I am not seeing it on Mac, Linux and also Windows (on VritualBox though). Here is a Windows screenshot:

https://drive.google.com/file/d/1J2_6GG ... sp=sharing

I set hash to 32 (MB), and the allocation is around 45 (for the entire program)

I tried some hash settings under Cute Chess.

When set to 16 MB, the allocated memory is 65.
When set to 32 MB, the allocated memory is 129 MB.
64 --> 286
128 --> more than 600 but then gradually decreases down to about 400.
256 --> about 1400, then gradually decreases down to about 800.

So it seems that for lower values the allocated memory increase is about 4 times the hash size increase. For greater values the allocated memory changes with time (it went down a bit with each move I made).

Maybe it uses a vector without giving it a capacity or specific number of elements to start with. In Rust, if you create a vector and then push elements into it, the vector needs to resize if it's not big enough. It'll then resize to about 1.5x its current size. For example:

You could initialize a vector with a capacity of say, 8 elements and make sure you don't ever put more than 256 elements in it. Then the vector will stay at 8 elements all the time. You could also initialize it without a capacity and leave it empty. After that, when you put elements in it, it will go something like this in many programming languages:

0 -> empty
1 -> reserved memory for 2 elements.
2 -> 2
3 -> 4
4 -> 4
5 -> 6
6 - > 8
7 -> 8
8 -> 8
9 -> 16
10 -> 16
...

Maybe that's what you're seeing.

That is the thing, I already initialize it with a fixed size:
https://github.com/amanjpro/zahak/blob/ ... go#L84-L85

And I never append to it (I only insert in already allocated indices): https://github.com/amanjpro/zahak/blob/ ... go#L43-L69

I use modulo to make sure I stay within the bounds: https://github.com/amanjpro/zahak/blob/ ... go#L39-L41

I am using a garbage collected language, I cannot really control when memory is freed when searching. So, the increase during search is probably normal. But why does it stay high even after search, I might have some memory leak, need to check, which is not exactly an easy task

[amanjpro]

I might have an idea where the memory consumption is coming from, the way I keep PV is too memory intense (I create almost one array per node to keep track of PV (each with the size of 100), I need to think about a better way to do that)

[dkappe]

I am using a garbage collected language, I cannot really control when memory is freed when searching. So, the increase during search is probably normal. But why does it stay high even after search, I might have some memory leak, need to check, which is not exactly an easy task

Ideally you would allocate objects on the stack, but that’s not possible with the TT. In Julia, which is also a GC language (non-compacting to boot), I’ve had to use an arena memory allocation approach for the AB/TT and the MCTS node tree. Otherwise it slows down appreciably over the course of a game.

[amanjpro]

Oh snap, found the issue. I had to call runtime.GC() manually for it to remove (old versions of the transposition table). I believe I have fixed it. I might do a new release soon with the fix. Go's GC is lazy, it doesn't run if it is not needed apparently. I ran the search for 14 plies deep (from startpos), and memory barely touched early 400MB with tt set to 256 (which I believe is normal, given all the stack I allocate)

[Archimedes]

There was also a problem on Android. Version 0.1.0 works for DroidFish, but versions 0.2.0 and 0.2.1 didn't. The newest version works again with DroidFish.