Egbb dll neural network support

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Daniel Shawul
Posts: 3470
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Egbb dll neural network support

Post by Daniel Shawul » Tue May 29, 2018 10:01 pm

Hello,

I have just added neural network evaluation support via egbb dlls. This is not about training a neural network on endgame tablebases
(although nothing will prevent us from doing that), but of providing evaluation for any position using a neural network using the existing
interface of dll/so of Scorpio egbbs. I am posting to get feedback on the interface and approach before releasing. The way it works is similar to how
egbbs were probed using new functions

Code: Select all

typedef int (CDECL *PPROBE_NN) (int player, int* piece,int* square);
typedef void (CDECL *PLOAD_NN) (char* path);
These functions are loaded at the time egbbs are loaded.

The egbbdll are compiled with a tensorflow backend that will also allow you to run the NN evaluation on a GPU or CPU without you engine code
knowing anything about it. The work required to get your engine using neural networks is very minimal which is why I like this approach. The programmer can focus on experimenting with search approaches (alpha-beta, MCTS, hybrid etc) without worrying about the NN aspect of it.

I have a working implementation now. I have generated four 4 networks (1x32, 3x64, 6x64, 12x128 modified resnets) on a set of 2 million epd positions.
My alpha-beta and MCTS search work with it well.

Thoughts:
a) Do we need to have the policy values as well or is the eval network enough ? I only use the value network in my engine but
some people may want to have both for an MCTS implementation. It seems redundant to have both value + policy, but it seems
it is more efficient in an actor-critic setting though i don't fully understand it.

b) One can train a network of his own using PGN (both policy+eval) and labeled EPD files (value network only) from any set of games.
eg. Leela's 15 million games.. The format is a google's protobuf (.pb files) and this can be shared. One can experiment with any kind of
neural network and the interface remains unchanged since the network file will be part of the .pb files.

c) We can train neural networks on specific endgames, waiting for 7-men syzygy to finsih :)

d) The tensorflow interface is currently working only on linux (don't know how to compile it on windows). The libtensorflow_cc library is not
statically linked with egbbdll.so, which means you can download and use a libtensorflow_cc that works on the GPU, optimized for specific
cpu architecture etc. I could ofcourse link tensorflow statically and ship only egbbdll.so but the latter gives more options for the user but
probably more headaches too.

e) We will totally rely on tensorflow backend. The good: automatic support for cuDNN and optimized subroutines such as winograd transform
on 3x3 kernels, The bad: No OpenCL support yet.

... and probably many other things

regards,
Daniel

brianr
Posts: 271
Joined: Thu Mar 09, 2006 2:01 pm

Re: Egbb dll neural network support

Post by brianr » Wed May 30, 2018 4:35 pm

Daniel Shawul wrote:
Tue May 29, 2018 10:01 pm

I have a working implementation now. I have generated four 4 networks (1x32, 3x64, 6x64, 12x128 modified resnets) on a set of 2 million epd positions.
This looks quite interesting. I have been, ahem, tinkering with NNs for a while. These are just informal type comments based on my stumbling around.

First, 2MM epd positions seems like a very small number for chess training. I think Alpha Zero used 4,096 as the batch size; 1,024 with Leela Zero Chess. They are described as "mini-batches", and seem to be used more to count mini-batch "steps", and are not necessarily the gpu batch sizes. With 1,024 epd positions (samples) per batch, 2MM is less than 2K steps. Most training runs are for around 50,000 steps. However, only one pass thru all of the steps is generally done, or 1 epoch. I think I saw 20 epochs in your code. I gather that the idea with chess is that many more samples are better than many epochs with fewer samples.

Next, the smallest NNs that I have tried and seen any minimal chess training is 10x128. 7x256 is quite a bit better. I have not trained any larger NNs long enough yet.

And, although I have not taken an in depth look at the code as yet, the size of the value head (in resnet.py) should be large enough to encompass the granularity of the scores that you want.

Naturally, these comments relate to the NN training, and not to their use in your framework. I mention them thinking that the framework might not be an effective thing to test unless the NNs used are "pretty good" to begin with. Lastly, I am not a "zero" purist, and think using endgame tablebases are a good idea with NNs. I don't think of egtbs as chess strategy, and having them would seem to shorten the NN training time as it does not have to learn about those positions.

Thank you for sharing all of your work for so many years.

Daniel Shawul
Posts: 3470
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: Egbb dll neural network support

Post by Daniel Shawul » Wed May 30, 2018 9:58 pm

brianr wrote:
Wed May 30, 2018 4:35 pm
Daniel Shawul wrote:
Tue May 29, 2018 10:01 pm

I have a working implementation now. I have generated four 4 networks (1x32, 3x64, 6x64, 12x128 modified resnets) on a set of 2 million epd positions.
This looks quite interesting. I have been, ahem, tinkering with NNs for a while. These are just informal type comments based on my stumbling around.

First, 2MM epd positions seems like a very small number for chess training. I think Alpha Zero used 4,096 as the batch size; 1,024 with Leela Zero Chess. They are described as "mini-batches", and seem to be used more to count mini-batch "steps", and are not necessarily the gpu batch sizes. With 1,024 epd positions (samples) per batch, 2MM is less than 2K steps. Most training runs are for around 50,000 steps. However, only one pass thru all of the steps is generally done, or 1 epoch. I think I saw 20 epochs in your code. I gather that the idea with chess is that many more samples are better than many epochs with fewer samples.
Thanks for the feedback Brian! Those 2 million positions were what I used for tuning standard eval of my engine, but you are right that it is too small for training any decent sized resnet. It is hard to find labeled epd positions so I think I am going to focus on training from PGN files instead. As you also pointed out a couple of passes should be enough once you have a lot of data to train your network with.
Next, the smallest NNs that I have tried and seen any minimal chess training is 10x128. 7x256 is quite a bit better. I have not trained any larger NNs long enough yet.
I did a 1x32 just to see by how much it slow it down (about 400x). The tensorflow overhead already makes it like 50-100x slower even without evaluating a neural network anyway.
And, although I have not taken an in depth look at the code as yet, the size of the value head (in resnet.py) should be large enough to encompass the granularity of the scores that you want.
Note that what I am doing is not exactly a resnet in that it has some auxillary inputs, currently material differences for Q,R,B,N. This is to kind of "shortcut" determination of piece values and also not to waste a whole 8x8 plane on a scalar feature like material differences, or side to move.
My 12 input planes are attack maps instead of piece placements so that it would have an easier time figuring out mobility, king safety etc.

I also use symmetry such that neural network evaluates positions only from white to move, and also horizontal symmetry as well (i don't consdier castling). Without that the NN was giving me different results for mirrored positions and i think it is better to remove the symmetry during training. Although not worth the effort, pawnless postions have 8-fold symmetry just like in the tbs.
Naturally, these comments relate to the NN training, and not to their use in your framework. I mention them thinking that the framework might not be an effective thing to test unless the NNs used are "pretty good" to begin with. Lastly, I am not a "zero" purist, and think using endgame tablebases are a good idea with NNs. I don't think of egtbs as chess strategy, and having them would seem to shorten the NN training time as it does not have to learn about those positions.
Neither am I a "zero" purists as you can tell from my NN design choices.
Thank you for sharing all of your work for so many years.
Thanks for the feedbacks!

brianr
Posts: 271
Joined: Thu Mar 09, 2006 2:01 pm

Re: Egbb dll neural network support

Post by brianr » Thu May 31, 2018 2:12 am

I use pgn files as input since I don't have the resources myself for millions of self-play games. Once I understand things better and can get reproducible results, then I want to see how important the prior move "history" inputs are.

After the initial slow-down for TF, the size of the NN might not matter quite as much if you are running on a fairly robust GPU. Changing the batch size is yet another parameter to try.

The value head size I mentioned is for the output tensor shape just before applying the tanh. There should be room for many features.

I think most use white-to-move and simplify the input quite a lot. Of course, castle status, ep, and rep (3-fold and 50) are always problematic (to normalize or not, one full plane or just scalars, etc). That said, generally, the number of inputs is a relatively small factor relative to the size of the NN. To minimize CPU RAM I have tried ints for large sample shuffle spaces and convert to floats within the NN. I think Leela Zero chess primarily uses bits to make things even smaller. Right now, I am opting for raw pgn input over more pre-processing for flexibility.

Daniel Shawul
Posts: 3470
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: Egbb dll neural network support

Post by Daniel Shawul » Thu May 31, 2018 2:41 pm

brianr wrote:
Thu May 31, 2018 2:12 am
I use pgn files as input since I don't have the resources myself for millions of self-play games. Once I understand things better and can get reproducible results, then I want to see how important the prior move "history" inputs are.

After the initial slow-down for TF, the size of the NN might not matter quite as much if you are running on a fairly robust GPU. Changing the batch size is yet another parameter to try.

The value head size I mentioned is for the output tensor shape just before applying the tanh. There should be room for many features.

I think most use white-to-move and simplify the input quite a lot. Of course, castle status, ep, and rep (3-fold and 50) are always problematic (to normalize or not, one full plane or just scalars, etc). That said, generally, the number of inputs is a relatively small factor relative to the size of the NN. To minimize CPU RAM I have tried ints for large sample shuffle spaces and convert to floats within the NN. I think Leela Zero chess primarily uses bits to make things even smaller. Right now, I am opting for raw pgn input over more pre-processing for flexibility.
The reason why they have history planes is probably mostly for the sake of Go where move patterns are common. Direct policy gradient reinforcement learning works well in that domain and history planes are quite important (policy network alone is over 3000 elo there). I don't think they bring much benefit for chess but as you said additional input planes do not affect size as they go away after the first convolution.

I agree about the output tensor shape being small, will expand it to 128 or 256.

Albert Silver
Posts: 2758
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: Egbb dll neural network support

Post by Albert Silver » Fri Jun 01, 2018 4:36 pm

Daniel Shawul wrote:
Tue May 29, 2018 10:01 pm
Hello,

I have just added neural network evaluation support via egbb dlls. This is not about training a neural network on endgame tablebases
(although nothing will prevent us from doing that), but of providing evaluation for any position using a neural network using the existing
interface of dll/so of Scorpio egbbs. I am posting to get feedback on the interface and approach before releasing. The way it works is similar to how
egbbs were probed using new functions

Code: Select all

typedef int (CDECL *PPROBE_NN) (int player, int* piece,int* square);
typedef void (CDECL *PLOAD_NN) (char* path);
These functions are loaded at the time egbbs are loaded.

The egbbdll are compiled with a tensorflow backend that will also allow you to run the NN evaluation on a GPU or CPU without you engine code
knowing anything about it. The work required to get your engine using neural networks is very minimal which is why I like this approach. The programmer can focus on experimenting with search approaches (alpha-beta, MCTS, hybrid etc) without worrying about the NN aspect of it.

I have a working implementation now. I have generated four 4 networks (1x32, 3x64, 6x64, 12x128 modified resnets) on a set of 2 million epd positions.
My alpha-beta and MCTS search work with it well.

Thoughts:
a) Do we need to have the policy values as well or is the eval network enough ? I only use the value network in my engine but
some people may want to have both for an MCTS implementation. It seems redundant to have both value + policy, but it seems
it is more efficient in an actor-critic setting though i don't fully understand it.

b) One can train a network of his own using PGN (both policy+eval) and labeled EPD files (value network only) from any set of games.
eg. Leela's 15 million games.. The format is a google's protobuf (.pb files) and this can be shared. One can experiment with any kind of
neural network and the interface remains unchanged since the network file will be part of the .pb files.

c) We can train neural networks on specific endgames, waiting for 7-men syzygy to finsih :)

d) The tensorflow interface is currently working only on linux (don't know how to compile it on windows). The libtensorflow_cc library is not
statically linked with egbbdll.so, which means you can download and use a libtensorflow_cc that works on the GPU, optimized for specific
cpu architecture etc. I could ofcourse link tensorflow statically and ship only egbbdll.so but the latter gives more options for the user but
probably more headaches too.

e) We will totally rely on tensorflow backend. The good: automatic support for cuDNN and optimized subroutines such as winograd transform
on 3x3 kernels, The bad: No OpenCL support yet.

... and probably many other things

regards,
Daniel
I think this is quite brilliant. I would ask, if at all possible, two things:

1) Easy-ish to use tool to build NN from material (games, etc)
2) A final NN format that is either compatible with LCZero (meaning would be a standard that was a two-way street), or a tool that could convert this (I suspect impossible). If your format is somehow better for some reason, then provide instructions so LCZero devs could add support.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

Daniel Shawul
Posts: 3470
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: Egbb dll neural network support

Post by Daniel Shawul » Sat Jun 02, 2018 8:46 am

Albert Silver wrote:
Fri Jun 01, 2018 4:36 pm

I think this is quite brilliant. I would ask, if at all possible, two things:

1) Easy-ish to use tool to build NN from material (games, etc)
2) A final NN format that is either compatible with LCZero (meaning would be a standard that was a two-way street), or a tool that could convert this (I suspect impossible). If your format is somehow better for some reason, then provide instructions so LCZero devs could add support.
1) Currently you can build a NN from set of games or EPD files. It is a 400 python code that takes advantage of the python-chess library.

2) I think so too -- though note that you can also take just the 15 million or so games and train your network.
It would be great to have the lc0 backend (not lczero) in egbbso as it supports lots of other backends. Currently I only have the tensorflow backend (just one backend among many) implemented in egbbdll. If someone from lc0 group is willing to work with me on this this should be done quickly.

I really hate the dependency hell with deep learning libraries like lczero's and now totally understand why GCP decided to go with hand-written openCL kernels.

Post Reply