designing neural networks

[Gerd Isenberg]

I am interested in applying neural networks e.g. to scale some eval terms, but I am a totally newbie in that field. What would be a good didactical, eval related example to play with, to get first experience?

Might this be an practical example?

I like to scale some eval terms to a final score - more or less depending on gamestate {early, late}X{opening, middlegame, endgame} and closeness of the pawn structure and number of pawn versus piece ratio. I can feed in total material, material-balance, pawn-piece ratio and some "heuristic" numbers, somehow representing a pawn-structure measure of the close-/openness (and/or ability to open) of the center and both wings (e.g. based on number of pawns, number of open and halfopen (per side) files, number of closed files with rams, and number of closed or halfopen files with levers and lever options, etc).

Code: Select all

                  input  hidden    output layer
  side to move       []\   
  white material sum []\\=X[~]\
  black material sum []\\\X[~]\\
  closeness-center   []\\\X[~]\\\
  ...                []>==X[~]-===>[]  score
  pawn/piece ratio   []///X[~]///
  eval-term 1        []///X[~]//
  ....               []//=X[~]/
  eval-term N        []/

What are the techniques to design such stuff? The topology of the net, the number of neurons, the initial weights, the characteristic curves and the backpropagation algorithm to adjust the weights after some training? I guess to design the hidden layer depends on how the input neurons interact, and which are independently. Is it about set of equations and desired characteristic curves in the first place?

Thanks for any help and advice!

Cheers,
Gerd

[Dan Andersson]

It sounds like you are trying to do a function approximation. Radial basis function networks are a common choice. It's a three level network with fast training methods.

MvH Dan Andersson

[berkantakin]

ı have seen some usage of neural networks for computer chess but I have some doubt about this.
1) neural nets are very slow. You have to compute complicated time consuming functions.
2) If you want to use back propogation, training phase gets very important. You have to pay special attention (so you must be exprienced in Back Propagation).

I have my own back probagation class. It is available at
http://www.berkantakin.com/index_files/professional.htm
http://www.berkantakin.com/SoftwareDeve ... l_nets.rar

[Harald]

I am interested in applying neural networks e.g. to scale some eval terms, but I am a totally newbie in that field. What would be a good didactical, eval related example to play with, to get first experience?

This may have nothing to do with neural nets, but here are my thoughts.

I would try to build a set of neurons which each is responsible for a special
situation or purpose. The number of neurons could be twice the number of
inputs or you can use as much as you want to calculate later for each evaluation
when the neural net is ready to use.

The neural net is something like the function
outval = sum( outweight[n] * f[n](inval[n] * inweight[n]) );
with 0 <= n < N neurons and 0 <= i < I inputs.

You have to compute this for every evaluation. Therefore I would
use ints instead of floats. The functions f[n] could be made of tables like
int f[N][256];
or it could be interpolated with j = 4 to 16 linear parts
y = m[n][j] * x + b[n][j];
This way the neurons can simulate linear or sigmoid functions or
a function with a peak somewhere.
The invalues or inweights must be transformed and scaled to 0..255.

You could even decide to only use k inputs with maximum inweight
for calculations in each neuron (0 <= k < K < 8). The vector of
k input numbers could be stored in each neuron. This would
speed up the calculation with the trained neural net.

The start configuration of all parameters is chosen with random values.

Training could be done with positions from a PGN database or with
real games. But you need another trusted evaluation function as a teacher.
After each evaluation with the neural net and the teacher eval
you compare the output and change something.

The one neuron with the best output can be given a reward. Whatever that is.
May be a little correction of its outweight in the right direction
and a small random change of the parameters inside the neuron.

All other neurons should learn to shut up. That is they are given a learning
motivation depending on the difference of their output from 0.
This motivation can modify the outweights or all the parameters inside the
neuron. Most of this with random changes.

The random change rate could decrease with learning time.

I do not know whether this builds a stable system of neurons
or just leads to chaos.

Harald

[smcracraft]

At least for neural networks in position evaluation, they
are too slow for computer chess. Now if you want to use
them for other methods not called-upon within the search
itself, go for it.

Sebastian Thrun of Stanford found this out when he tried
to design a neural-network-based program to beat even
a weak standard chess program a majority of the time.
He couldn't do it.

The research has pretty much been shutdown worldwide
as a result.

Get the paper here:

http://robots.stanford.edu/papers/thrun ... chess.html

Note: Thrun is the Director of the Stanford A.I. Lab at
Stanford University (after founder John McCarthy who
is in frail health.)

--Stuart

cf. McCarthy http://en.wikipedia.org/wiki/John_McCar ... scientist)

cf. Thrun
http://en.wikipedia.org/wiki/Sebastian_Thrun

[CRoberson]

Hi Gerd,

As has been stated, ANN's are slow. After training, the speed can
be improved. If you are talking about learning in a non-realtime
scenario then you can do it. After learning is achieved, one would
use the ANN (artificial neural network) to create an equation that
is fast and approximates the results of an ANN.

As for your specific questions:

• The initial weights - random numbers
  The hidden layer - this layer is where the knowledge aquistion comes in.
  Not possible to know apriori the number of neurons here.
  There is a proof that shows only one hidden layer is needed
  to learn anything but experience shows that 2 hidden layers
  learns faster.
  The input layer - always add an extra neuron known as the bias.
  Also, learning can be improved if binary inputs are made into
  two neurons. For instance, side-to-move could be one neuron
  with values 0 or 1 or it could be two neurons that could be 0 1 or
  1 0. Their values are mutually exclusive.
  Training - there are several options here and options within those.
  You can have it learning from experience - typically TD learning.
  You can learn from data - big data set of inputs and fixed outputs.
  you'll need a training set and a test set. The elements of the
  test set should not intersect the training set.
  If using a test set ensure that you shuffle the input/data pairs before
  running the next epoch. Otherwise, the ANN will learn the data
  presentation ordering will be important when it may not be.

[Gerd Isenberg]

Thanks for all your replies.

I want to make the whole application issue a bit more concrete. For instance Glaurung (and Fruit) interpolate explicite middlegame versus endgame scores by a fixed point number 0...1.0 == 128, representing the gamephase like a normalized sum of total material of both sides.

Code: Select all

score = (middlegameValue*gamephase + endgameValue*(128 - gamephase)) / 128

I have an "improvement" of this interpolation in mind, eventually using some (simple) neural network approach - since it might be a nice didactial application to start with. To somehow "train" the "smoothness" of middlegame-endgame transitions. Dan's mentioned radial basis function networks seems to point in this direction, I guess. To make the function a more or less sigmoid function with scalable "bending" between the extremes straight and rectanglar curve - plus a scalable middle-endgame - turning point M.

Code: Select all

score m-opening/middlegame, e-endgame

 ^
 |  ( bending == 0 )
 e                                  m
 |   e                           m
 |       e                   m
 +           e           m
 |               e . m
 |               m | e
 +           m     |     e
 |       m         |         e
 |   m             |             e
 m-----------+-----|-----+-----------e---> gamephase (0...128 => endgame...middlegame
                <- M -> &#40;middle-endgame - turning point&#41;

 ^
 |   ( bending == 0.5 )
 e                                   m
 |         e               m
 |             e       m
 +               e   m
 |                e.m
 |                m|e
 +               m | e
 |             m   |   e
 |         m       |       e
 m-----------+-----|-----+-----------e---> gamephase &#40;0...128 => endgame...middlegame
                <- M -> &#40;middle-endgame - turning point&#41;

 ^
 |   ( bending == 1.0 )
 eeeeeeeeeeeeeeeeee mmmmmmmmmmmmmmmm
 |                 |
 |                 |
 +                 |
 |                 .
 |                 |
 +                 |
 |                 |
 |                 |                 
 mmmmmmmmmmmmmmmmmm|eeeeeeeeeeeeeeee---> gamephase &#40;0...128 => endgame...middlegame
                <- M -> &#40;middle-endgame - turning point&#41;

I wish the gamephase not only depending by material, but also by the pawn-structure (and may be by some piece/pawn-ratio, existence of queens, and inbalance terms). The idea is in closed positions the endgame-phase may start earlier with a bit more material on the board than in open positions.

Terms of interest are King-Safety (Middlegame related), King-Activity (Endgame related) and pawn evaluation for passers and candidates.
Thus, for a first "didactical" step, I like to play with three "scalable" sigmoid functions with two parameters (weights?), to scale bending and turning point each.

The question remains for me - how do I design a "simple" neural net with this inputs?

Code: Select all

                    input     hidden        output layer
gamephase*         ---> [i1] ->XX-> [~]-\
kingsafety term    ---> [i2] ->XX-> [~]--\__[score]
kingactivity term  ---> [i3] ->XX-> [~]--/
pawn term          ---> [i4] ->XX-> [~]-/

gamephase* already adjusted by closeness of the pawns 

Thanks for you patience,

Cheers,
Gerd

[Dan Andersson]

If you can get access to MATLAB and the Neural Network Toolbox you can use the existing functions for RBN. Both exact and efficient solutions.
If not there are a lot of papers available. A few:
http://www.anc.ed.ac.uk/rbf/rbf.html
http://www.google.com/search?q=site:cit ... gle+Search
http://citeseer.ist.psu.edu/cis?q=Radia ... works&cs=1
or just search citeseer for function approximation papers:
http://citeseer.ist.psu.edu/cis?q=funct ... ation&cs=1

One of the seminal papers by Chen, Cowan, Grant:
Orthogonal least squares learning algorithm. for radial basis function networks
Or the comprehensive work on learning:
Learning Algorithms for Radial Basis Function Networks: Synthesis, Experiments and Cognitive Modelling
http://citeseer.ist.psu.edu/496615.html

MvH Dan Andersson

[Gerd Isenberg]

Thanks, it will take some time to get the basics.

A lot of papers and tutorials I read so far, somehow suffer from simple practical applications, imho. But of course it is my subjective view due to my rudimentary math skills trying to autodidactic learn such stuff. I almost need a concrete practical catalyzer, to get the abstraction.

I am still not sure whether the mentioned application is suited for neural networks at all, or at least the need of a hidden layer.

Assume the mentioned sigmoid functions are implemented as a simple lookup-table with 128(129) (or whatever fixpoint granulation) fixed point numbers, indexed by gamephase 0...127(128). There are several vectors with several bendings, "indexed" by one particular eval-term or feature to get a sum of products as a final score.

Middlegame-features use monotonically increasing curves, while endgame-features are decreasing curves, implicitly considered in sigmoidTable[feature].

Code: Select all

score = 0;
gamephase = (sumOfMaterial + ...) / maxSumOfMaterialAnd...;
for ( each feature f)
   score += sigmoidTable[f][gamephase] * evalTerm[f];

For each feature, keeping/adjusting bending (B) and turning point (TP) as persistent values would be enough, since we can calculate the sigmoidTable[f] by B(f) and TP(f) at startup-time once.

The brute force "learning"-approach would be to iterate over all possible curves per feature, varying TP(f) by some plausible extend, but trying all possible bendings B(f) (all curves are still monotonically increasing (or decreasing for the endgame bounded terms)), to find out, which are the strongest TP(i),B(i) - probably times different time-controls and opponents.

One might iterate with greater increments to reduce the number of feature recombination, with the risc to miss some successful local maxima.

I guess the "neural" approach is "evolution" by random changes of one or several B(f) and TP(f) - to see whether they are stronger. Guess temporal differences have something to do with that stuff?

Mit freundlichem Gruss,
Gerd

[Oliver]

Hi Gerd,

let me first say what i would do. I would simply export my trainingsdata into a giant csv-file (or a custom MySQL-Table) and then use prefabricated tools to solve this problem.
If you like to fiddle with Neural Networks you might consider SNNS
(http://www-ra.informatik.uni-tuebingen.de/SNNS/).
If you are open to other statistical tools, you may choose R
(http://www.r-project.org/).

But if i evaluate you correctly, you are not the person to like that. You like to program every thing out for yourself. (Thats a good thing).

Regarding the sampleproblem you gave, it is not clearly defined!
One must distinguish supervised and unsupervised learning.
See e.g. here:
http://en.wikipedia.org/wiki/Unsupervised_learning

So the question is: Do you have already high quality scores or not?
Please answers this first, it is important. If not, you have really a problem with it. Otherwise, your problem is conceptual trivial.

Oliver