Giraffe, new release (Aug 17)

[Werewolf]

That's amazing, thank you!!

So you use a neural net cluster to tune the evaluation function of your program?

Do you have any plans to make the program work over a cluster so it's searching by using many machines as it plays?
(like the Rybka Cluster etc)

[matthewlai]

That's amazing, thank you!!

So you use a neural net cluster to tune the evaluation function of your program?

Do you have any plans to make the program work over a cluster so it's searching by using many machines as it plays?
(like the Rybka Cluster etc)

Thanks!

The evaluation function itself IS a neural net . That's the special thing about Giraffe. There were a few previous engines that used neural nets either to tune evaluation functions or combine evaluation terms together, but I believe Giraffe is the first engine that uses a neural net all the way, with almost all knowledge being self-learned. I want to see what it can come up with when it's not constrained by human creativity and how humans think about chess. That seems to have worked out quite well. It has learned a very powerful evaluation function all on its own.

As for cluster playing... it's very difficult, and as it's not the subject of my thesis, I can't really afford to spend a lot of time on that. And I will lose access to the cluster once I submit my thesis and (hopefully) graduate .

[Henk]

If I were you I would switch too move ordering immediately. Use your neural network for move ordering only. But only on a level that gives no computational bottleneck.

Evaluation on leave nodes is a computational bottleneck. And searching deeper always won from a better evaluation. But I might be wrong and this is the worst advice.

[matthewlai]

If I were you I would switch too move ordering immediately. Use your neural network for move ordering only. But only on a level that gives no computational bottleneck.

Evaluation on leave nodes is a computational bottleneck. And searching deeper always won from a better evaluation. But I might be wrong and this is the worst advice.

Searching deeper clearly doesn't always win.

Giraffe searches at 1/5 the speed of Fairy-max, and beats it by 200 Elo.

It searches at about the same speed as Skipper. Feel free to try a few matches .

Even on the highest level, Crafty searches at 5x the speed of Stockfish, and Stockfish still beats it by 400 Elo.

A good evaluation is very very important.

[Henk]

If I were you I would switch too move ordering immediately. Use your neural network for move ordering only. But only on a level that gives no computational bottleneck.

Evaluation on leave nodes is a computational bottleneck. And searching deeper always won from a better evaluation. But I might be wrong and this is the worst advice.

Searching deeper clearly doesn't always win.

Giraffe searches at 1/5 the speed of Fairy-max, and beats it by 200 Elo.

It searches at about the same speed as Skipper. Feel free to try a few matches .

Even on the highest level, Crafty searches at 5x the speed of Stockfish, and Stockfish still beats it by 400 Elo.

A good evaluation is very very important.

I'm not talking about speed but about depth. One extra ply means 50-70 elo. Of course if you have a bad evaluation an extra ply won't help. So I think if your evaluation is good enough better move ordering has more effect than better evaluation.

[matthewlai]

If I were you I would switch too move ordering immediately. Use your neural network for move ordering only. But only on a level that gives no computational bottleneck.

Evaluation on leave nodes is a computational bottleneck. And searching deeper always won from a better evaluation. But I might be wrong and this is the worst advice.

Searching deeper clearly doesn't always win.

Giraffe searches at 1/5 the speed of Fairy-max, and beats it by 200 Elo.

It searches at about the same speed as Skipper. Feel free to try a few matches .

Even on the highest level, Crafty searches at 5x the speed of Stockfish, and Stockfish still beats it by 400 Elo.

A good evaluation is very very important.

I'm not talking about speed but about depth. One extra ply means 50-70 elo. Of course if you have a bad evaluation an extra ply won't help. So I think if your evaluation is good enough better move ordering has more effect than better evaluation.

Giraffe's search is also quite basic, so I'm pretty sure Fairy-max searched deeper as well.

The Elo worth of an extra ply depends on how deep you are searching already. If you are searching reasonably deep already, plies matter less, and eval matters more.

At the depths Stockfish and Crafty are searching for example, it's almost all about eval.

[Henk]

If I were you I would switch too move ordering immediately. Use your neural network for move ordering only. But only on a level that gives no computational bottleneck.

Evaluation on leave nodes is a computational bottleneck. And searching deeper always won from a better evaluation. But I might be wrong and this is the worst advice.

Searching deeper clearly doesn't always win.

Giraffe searches at 1/5 the speed of Fairy-max, and beats it by 200 Elo.

It searches at about the same speed as Skipper. Feel free to try a few matches .

Even on the highest level, Crafty searches at 5x the speed of Stockfish, and Stockfish still beats it by 400 Elo.

A good evaluation is very very important.

It's even more difficult to beat Fairy-max in 60 seconds games. I remember once Skipper was able to win from Fairy-max too but not on a much shorter time control.

[matthewlai]

If I were you I would switch too move ordering immediately. Use your neural network for move ordering only. But only on a level that gives no computational bottleneck.

Evaluation on leave nodes is a computational bottleneck. And searching deeper always won from a better evaluation. But I might be wrong and this is the worst advice.

Searching deeper clearly doesn't always win.

Giraffe searches at 1/5 the speed of Fairy-max, and beats it by 200 Elo.

It searches at about the same speed as Skipper. Feel free to try a few matches .

Even on the highest level, Crafty searches at 5x the speed of Stockfish, and Stockfish still beats it by 400 Elo.

A good evaluation is very very important.

It's even more difficult to beat Fairy-max in 60 seconds games. I remember once Skipper was able to win from Fairy-max too but not on a much shorter time control.

1 minute games.

Code: Select all

Rank Name                   Elo    +    - games score oppo. draws 
   1 Giraffe 63f71c3eb204   126   61   52    38   83%  -126   13% 
   2 Fairy-Max 4.8S        -126   52   61    38   17%   126   13% 

[Henk]

If I were you I would switch too move ordering immediately. Use your neural network for move ordering only. But only on a level that gives no computational bottleneck.

Evaluation on leave nodes is a computational bottleneck. And searching deeper always won from a better evaluation. But I might be wrong and this is the worst advice.

Searching deeper clearly doesn't always win.

Giraffe searches at 1/5 the speed of Fairy-max, and beats it by 200 Elo.

It searches at about the same speed as Skipper. Feel free to try a few matches .

Even on the highest level, Crafty searches at 5x the speed of Stockfish, and Stockfish still beats it by 400 Elo.

A good evaluation is very very important.

It's even more difficult to beat Fairy-max in 60 seconds games. I remember once Skipper was able to win from Fairy-max too but not on a much shorter time control.

1 minute games.

Code: Select all

Rank Name                   Elo    +    - games score oppo. draws 
   1 Giraffe 63f71c3eb204   126   61   52    38   83%  -126   13% 
   2 Fairy-Max 4.8S        -126   52   61    38   17%   126   13% 

Ok some or all engines are totally different.

[Werewolf]

Thanks!

The evaluation function itself IS a neural net . That's the special thing about Giraffe. There were a few previous engines that used neural nets either to tune evaluation functions or combine evaluation terms together, but I believe Giraffe is the first engine that uses a neural net all the way, with almost all knowledge being self-learned. I want to see what it can come up with when it's not constrained by human creativity and how humans think about chess. That seems to have worked out quite well. It has learned a very powerful evaluation function all on its own.

Matthew, this is amazing. It's really excellent that you're doing something new like this. I was under the impression that a neural net had to use lots of computers to function, but I must be wrong, since you've done it in software.

My only concern with all this is - like other posters - even if you get the best evaluation function in the world, if the search is poor there will be a limit on how strong Giraffe is.

Or do you just want to see how high it can go with the search as it is but a better and better evaluation function..?

By the way, when do you graduate?

Keep going!