A question to MCTS + NN experts

[maksimKorzh]

Hi, guys!

I've written for about 5 alpha-beta engines with material + positional score evaluation in past years (1200 - 1400 average ELO, pretty basic).
Now I'd like to implement my own model that would be learning by playing itself, adjusting weights along the way. I know there already exist
some implementations and you need just to connect them to your move generator and start with training NN, not implementing, but my
interest is to gain knowledge in machine learning, hence I'd like to write everything myself. I have no idea where to start, I can't even clearly
find out which part is regression problem(probably predicting eval score?) and which is classification(if any?), I also have zero experience in
any but supervised learning tasks. Could anybody please kindly help me to find some resources to start with? (I've already read CPW posts about deep learning and MCTS, but that didn't satisfy me)

Thanks in advance!

[maksimKorzh]

For start I'd like to know what's the most basic way of converting a chess position(array based) into an input dataset for the NN. They call this board representation in some papers but that sounds confusing because I used to understand "board representation" term as "array based vs bitboard". So let's simplify the task as follows:

How to predict position's evaluation bearing in mind only material?

[brianr]

I am far from an expert, but here are some suggestions.
You mentioned SL NN experience, but perhaps start with something simpler than chess.
This repo I found to be very helpful (there is another one for chess after understanding Connect-4; the overall process is the same)
https://github.com/Zeta36/connect4-alpha-zero

For chess representation, look at Giraffe code.
This is a non-zero implementation in that a lot of chess-specific info is encoded.

Finally, the Leela (Lc0) wiki pages and code have the most info, but the project is still moving very fast.
https://github.com/LeelaChessZero/lc0/wiki

PS: Self-play or RL learning will require millions of games which may not be practical to generate on your own.

[Daniel Shawul]

For start I'd like to know what's the most basic way of converting a chess position(array based) into an input dataset for the NN. They call this board representation in some papers but that sounds confusing because I used to understand "board representation" term as "array based vs bitboard". So let's simplify the task as follows:

How to predict position's evaluation bearing in mind only material?

The inputs for A0 style networks are planes (arrays) of floats of size 64 or (8x8). So you encode the positon as planes for each of the 12 pieces separately so that it knows what piece is at each square. A0/Lc0 actually use more input planes to encode the previous 8 positions as well but i suggest you start with the simpler one that encodes jus the current postion.

So you have "float input_planes[12][8][8]" and you put a 1.0 where the pieces are. These stacked planes are passed through convolution->bathnormalization->relu etc.

See this poster to get the gist of what A0 is doing.

If you don't want to follow A0 network structure (not use convolutions for automatic feature extraction), you could dump the input planes and encode features you want. For example, 10 input neurons for the piece counts etc.. giraffe did that kind of thing.

[maksimKorzh]

I am far from an expert, but here are some suggestions.
You mentioned SL NN experience, but perhaps start with something simpler than chess.
This repo I found to be very helpful (there is another one for chess after understanding Connect-4; the overall process is the same)
https://github.com/Zeta36/connect4-alpha-zero

For chess representation, look at Giraffe code.
This is a non-zero implementation in that a lot of chess-specific info is encoded.

Finally, the Leela (Lc0) wiki pages and code have the most info, but the project is still moving very fast.
https://github.com/LeelaChessZero/lc0/wiki

PS: Self-play or RL learning will require millions of games which may not be practical to generate on your own.

Thank you, brianr, it looks like what I am looking for right now! Thanks!

[maksimKorzh]

For start I'd like to know what's the most basic way of converting a chess position(array based) into an input dataset for the NN. They call this board representation in some papers but that sounds confusing because I used to understand "board representation" term as "array based vs bitboard". So let's simplify the task as follows:

How to predict position's evaluation bearing in mind only material?

The inputs for A0 style networks are planes (arrays) of floats of size 64 or (8x8). So you encode the positon as planes for each of the 12 pieces separately so that it knows what piece is at each square. A0/Lc0 actually use more input planes to encode the previous 8 positions as well but i suggest you start with the simpler one that encodes jus the current postion.

So you have "float input_planes[12][8][8]" and you put a 1.0 where the pieces are. These stacked planes are passed through convolution->bathnormalization->relu etc.

See this poster to get the gist of what A0 is doing.

If you don't want to follow A0 network structure (not use convolutions for automatic feature extraction), you could dump the input planes and encode features you want. For example, 10 input neurons for the piece counts etc.. giraffe did that kind of thing.

Thanks for your reply, Daniel, dumping input planes in giraffe's manner is the point I'd like to start at. Just to make things a bit more clear - I consider "float input_planes[12][8][8]" to be what is called "dataset", so it's the source for the further evaluation predictions, is that correct? This 3-dimensional array also looks like the piece-list, well at list in terms of "for each piece [12] get corresponding coordinates [8] x [8]" - but I'm just wondering what's the need to use 2-dimensional array for to represent board squares, why not just [64]? Or is it nonsense from the input dataset perspective like being unavailable to extract features correctly?

P.S. Sorry if my questions/assumptions are stupid - I'm complete noob in ML for now)

[brianr]

The net architectures used with chess are very similar to those used in general image detection and classification, which work off of 2D grids of pixels (with additional color channels in more dimensions), so you can roughly think of the piece type 2D bitmaps like a piece list and each type of piece as an image color. For chess the input is often always set to white being on move and the actual position is "flipped" if black is on move, and flipped back accordingly. Additional chess info like castle rights and such are encoded differently. Read about CNNs and image training to see that the neighbors of each pixel are assembled and scanned (filters), so it is sort of like looking at what squares are next to a piece. Here is one place to start:
http://cs231n.github.io/convolutional-networks/

[maksimKorzh]

The net architectures used with chess are very similar to those used in general image detection and classification, which work off of 2D grids of pixels (with additional color channels in more dimensions), so you can roughly think of the piece type 2D bitmaps like a piece list and each type of piece as an image color. For chess the input is often always set to white being on move and the actual position is "flipped" if black is on move, and flipped back accordingly. Additional chess info like castle rights and such are encoded differently. Read about CNNs and image training to see that the neighbors of each pixel are assembled and scanned (filters), so it is sort of like looking at what squares are next to a piece. Here is one place to start:
http://cs231n.github.io/convolutional-networks/

Great tip! Thanks!

[thomasahle]

I wrote a simple MCTS + Machine Learning engine: herehttps://github.com/thomasahle/fastchess
The MCTS part is only a page or two of python code, and the others are not much more, so you should quickly be able to get the gist.
I'm not currently training it from self play, but one might as well do that.