Is it time for another new move generator?

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

hristo

Re: Is it time for another new move generator?

Post by hristo »

Michael Sherwin wrote:
Aleks Peshkov wrote:

Code: Select all

ts = fs;
do {
    ts += vec;
    if (board[ts]) break;                       
    Link(type, fs, ts);
} while (ts != last) //last legal square
Good point Aleks, I will rework it. Thanks! :D

To others, this is nothing like the move generator in GNUChess. I also have an improved version of that too! :D
Michael,
how much time do you spend in the move-generator while playing a normal game?
Recently, I moved to a 64 bit OS and my perft went down by about 2-3% while the node search (evaluation) went up by almost 10%. Strange.

Admittedly, my evaluation is very similar to running multiple FFTs on the data-stream generated by the positions.

Regards,
Hristo
Michael Sherwin
Posts: 3196
Joined: Fri May 26, 2006 3:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Is it time for another new move generator?

Post by Michael Sherwin »

hristo wrote:
Michael Sherwin wrote:
Aleks Peshkov wrote:

Code: Select all

ts = fs;
do {
    ts += vec;
    if (board[ts]) break;                       
    Link(type, fs, ts);
} while (ts != last) //last legal square
Good point Aleks, I will rework it. Thanks! :D

To others, this is nothing like the move generator in GNUChess. I also have an improved version of that too! :D
Michael,
how much time do you spend in the move-generator while playing a normal game?
Recently, I moved to a 64 bit OS and my perft went down by about 2-3% while the node search (evaluation) went up by almost 10%. Strange.

Admittedly, my evaluation is very similar to running multiple FFTs on the data-stream generated by the positions.

Regards,
Hristo
Hi Hristo,

I write new move generators as sort of a relaxing hobby when I am stuck on developing RomiChess. so far I have developed about a dozen (more if permutations count) move generators that are unique (or uniquely better) but, I have only worked a few of them up to the point of having a perft or simple search. My simple bitboard perft runs about twice as fast as Crafty's (don't know if that is a completely fair comparison) and it is by far the slowest perft that I've made.

It is important for me to have a fast move generator in my program, because, my evaluation is very cheap. A program that has a huge expensive evaluation can not benifit from a fast move generator very much as it spends relitivly little time in it to begin with.

I experianced no change in performance when going from 32 bits to 64 bits on the same machine.

Best,
Mike
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
User avatar
Zach Wegner
Posts: 1922
Joined: Thu Mar 09, 2006 12:51 am
Location: Earth

Re: Is it time for another new move generator?

Post by Zach Wegner »

hristo wrote:Admittedly, my evaluation is very similar to running multiple FFTs on the data-stream generated by the positions.
Very interesting. Care to elaborate?
hristo

Re: Is it time for another new move generator?

Post by hristo »

Zach Wegner wrote:
hristo wrote:Admittedly, my evaluation is very similar to running multiple FFTs on the data-stream generated by the positions.
Very interesting. Care to elaborate?
I'm not ready to elaborate, just yet, simply because it might take too long.

The basic concept is to represent the static position as an energy map of all the pieces on the board. Where the evaluation takes into account the potential energy for every piece, which is normally referred to as "the material", and the kinetic energy at the given moment.
An interesting aspect of this is that for instance the Kings have attenuation field which is applied to the kinetic energy of the opponent's pieces. This makes it possible to "automatically" select attacking and defensive moves.

At any rate, I want to find an algorithm that leads to good moves without having to explicitly code in the so called "chess knowledge". It is just seems to be more fun that way. :-)

Regards,
Hristo
hristo

Re: Is it time for another new move generator?

Post by hristo »

Zach Wegner wrote:
hristo wrote:Admittedly, my evaluation is very similar to running multiple FFTs on the data-stream generated by the positions.
Very interesting. Care to elaborate?
Here is an example of how it plays.
Note that the engine is doing only 5-6 ply depth analysis ... I'm debugging it right now. Also there are no bonuses for open lines, bishop vs knight, etc. etc. this is literally a raw engine that has no explicit chess knowledge built into it. The "eval score" that was recorded is not correct, but the ply depth is. The engine has no opening book capabilities, but Glaurung GUI provided the first three moves.

[Event "?"]
[Site "?"]
[Date "2007.11.13"]
[Round "?"]
[White "Hristo Doichev"]
[Black "dAby (experimental) 2007"]
[Result "1/2-1/2"]

1. e4 c5 2. c3 d6 3. d3 Nf6 4. Nd2 Nc6 {+0.00/4} 5. Be2 d5 {+0.00/4} 6. Ngf3
Qc7 {+0.00/4} 7. O-O e6 {+0.00/5} 8. Re1 Be7 {+0.00/4} 9. c4 dxc4 {+0.00/5} 10.
dxc4 Qd6 {+0.00/5} 11. b3 Ng4 {+0.00/5} 12. Bb2 Bh4 {+0.00/4} 13. g3 Bf6
{+0.00/4} 14. Bxf6 Nxf6 {+0.00/5} 15. Bf1 e5 {+0.00/4} 16. Qc2 Bg4 {+0.00/5}
17. h3 Bxf3 {+0.00/4} 18. Nxf3 O-O-O {+0.00/4} 19. Rad1 Qe7 {+0.00/4} 20. a3
Rxd1 {+0.00/5} 21. Rxd1 Rd8 {+0.00/5} 22. Rxd8+ Qxd8 {+0.00/5} 23. Qd3 Qxd3
{+0.00/3} 24. Bxd3 Kd8 {+0.00/5} 25. Kf1 a5 {+0.00/5} 26. Ke2 Ne8 {+0.00/5} 27.
Ke3 Nd6 {+0.00/5} *

eventually ended up in a draw
Michael Sherwin
Posts: 3196
Joined: Fri May 26, 2006 3:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: Is it time for another new move generator?

Post by Michael Sherwin »

hristo wrote:
Zach Wegner wrote:
hristo wrote:Admittedly, my evaluation is very similar to running multiple FFTs on the data-stream generated by the positions.
Very interesting. Care to elaborate?
Here is an example of how it plays.
Note that the engine is doing only 5-6 ply depth analysis ... I'm debugging it right now. Also there are no bonuses for open lines, bishop vs knight, etc. etc. this is literally a raw engine that has no explicit chess knowledge built into it. The "eval score" that was recorded is not correct, but the ply depth is. The engine has no opening book capabilities, but Glaurung GUI provided the first three moves.

[Event "?"]
[Site "?"]
[Date "2007.11.13"]
[Round "?"]
[White "Hristo Doichev"]
[Black "dAby (experimental) 2007"]
[Result "1/2-1/2"]

1. e4 c5 2. c3 d6 3. d3 Nf6 4. Nd2 Nc6 {+0.00/4} 5. Be2 d5 {+0.00/4} 6. Ngf3
Qc7 {+0.00/4} 7. O-O e6 {+0.00/5} 8. Re1 Be7 {+0.00/4} 9. c4 dxc4 {+0.00/5} 10.
dxc4 Qd6 {+0.00/5} 11. b3 Ng4 {+0.00/5} 12. Bb2 Bh4 {+0.00/4} 13. g3 Bf6
{+0.00/4} 14. Bxf6 Nxf6 {+0.00/5} 15. Bf1 e5 {+0.00/4} 16. Qc2 Bg4 {+0.00/5}
17. h3 Bxf3 {+0.00/4} 18. Nxf3 O-O-O {+0.00/4} 19. Rad1 Qe7 {+0.00/4} 20. a3
Rxd1 {+0.00/5} 21. Rxd1 Rd8 {+0.00/5} 22. Rxd8+ Qxd8 {+0.00/5} 23. Qd3 Qxd3
{+0.00/3} 24. Bxd3 Kd8 {+0.00/5} 25. Kf1 a5 {+0.00/5} 26. Ke2 Ne8 {+0.00/5} 27.
Ke3 Nd6 {+0.00/5} *

eventually ended up in a draw
Very impressive Hristo,

Language?

Node rate?

OBTW, I think that you should name your new program, Goat! :lol:
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
hristo

Re: Is it time for another new move generator?

Post by hristo »

Michael Sherwin wrote:
hristo wrote:
Zach Wegner wrote:
hristo wrote:Admittedly, my evaluation is very similar to running multiple FFTs on the data-stream generated by the positions.
Very interesting. Care to elaborate?
Here is an example of how it plays.
Note that the engine is doing only 5-6 ply depth analysis ... I'm debugging it right now. Also there are no bonuses for open lines, bishop vs knight, etc. etc. this is literally a raw engine that has no explicit chess knowledge built into it. The "eval score" that was recorded is not correct, but the ply depth is. The engine has no opening book capabilities, but Glaurung GUI provided the first three moves.

[Event "?"]
[Site "?"]
[Date "2007.11.13"]
[Round "?"]
[White "Hristo Doichev"]
[Black "dAby (experimental) 2007"]
[Result "1/2-1/2"]

1. e4 c5 2. c3 d6 3. d3 Nf6 4. Nd2 Nc6 {+0.00/4} 5. Be2 d5 {+0.00/4} 6. Ngf3
Qc7 {+0.00/4} 7. O-O e6 {+0.00/5} 8. Re1 Be7 {+0.00/4} 9. c4 dxc4 {+0.00/5} 10.
dxc4 Qd6 {+0.00/5} 11. b3 Ng4 {+0.00/5} 12. Bb2 Bh4 {+0.00/4} 13. g3 Bf6
{+0.00/4} 14. Bxf6 Nxf6 {+0.00/5} 15. Bf1 e5 {+0.00/4} 16. Qc2 Bg4 {+0.00/5}
17. h3 Bxf3 {+0.00/4} 18. Nxf3 O-O-O {+0.00/4} 19. Rad1 Qe7 {+0.00/4} 20. a3
Rxd1 {+0.00/5} 21. Rxd1 Rd8 {+0.00/5} 22. Rxd8+ Qxd8 {+0.00/5} 23. Qd3 Qxd3
{+0.00/3} 24. Bxd3 Kd8 {+0.00/5} 25. Kf1 a5 {+0.00/5} 26. Ke2 Ne8 {+0.00/5} 27.
Ke3 Nd6 {+0.00/5} *

eventually ended up in a draw
Very impressive Hristo,

Language?

Node rate?
100% C++

Starting position is 110KN and in the endgame it often reaches 270-300KN.
This is without *any* optimizations, whatsoever. Just consider that for transposition tables I'm using std::set storing 64 byte (position) + other garbage ... :-) ... even though it is slow, I have perfect aging of the visited nodes, at least it is perfect with respect of what I need the aging to be.
Total of size for transposition tables is 10 MB.

I'm currently using Negascout and am getting hurt by it, I guess because the eval takes so long the rescans are killing me.

BTW,
I'm handling time expiration using exceptions. Basically, there are only few places where time can be handled properly and so I just throw an exception from anywhere within the code when time is up and let the first 'catcher' deal with it. (On top of everything else exceptions don't make the generated code faster. :-) Which reminds me that I probably still have RTTI enabled.)

The application is 64 bit and is running on Core 2 Duo @2.4 GHz (MacBook Pro 17")
Michael Sherwin wrote: OBTW, I think that you should name your new program, Goat! :lol:
I'll give it a consideration. :-)

Regards,
Hristo
Pradu
Posts: 287
Joined: Sat Mar 11, 2006 3:19 am
Location: Atlanta, GA

Re: Is it time for another new move generator?

Post by Pradu »

hristo wrote:The basic concept is to represent the static position as an energy map of all the pieces on the board. Where the evaluation takes into account the potential energy for every piece, which is normally referred to as "the material", and the kinetic energy at the given moment.
An interesting aspect of this is that for instance the Kings have attenuation field which is applied to the kinetic energy of the opponent's pieces. This makes it possible to "automatically" select attacking and defensive moves.

At any rate, I want to find an algorithm that leads to good moves without having to explicitly code in the so called "chess knowledge". It is just seems to be more fun that way. :-)

Regards,
Hristo
This is a very interesting idea... indeed a breakthrough if you don't have to explicitly code chess knowledge and it's performance is as good as an eval with explicitly coded chess knowledge. Some questions though. How do you not explicitly code chess knowledge? Wouldn't whatever function you use to define potential energy and kinetic energy be the same thing as adding chess knowledge by the use of that function? Do you have something like KE + PE = 0 or something like KE + PE = Change in total energy in the system from a move? If so what do you do to generate a score when the KE and PE of a position is known.

Also what information is in the datastream for the FFT and what do you do with the (I'm guessing peak-power) frequencies obtained from power spectrum?

Indeed it sounds like a very expensive eval if you have to generate a datastream for each position and do an FFT on it, but I guess one would need to do some testing to see if the quality of the knowledgeless eval is good enough to offset the loss in search depth.
hristo

Re: Is it time for another new move generator?

Post by hristo »

Pradu wrote:
hristo wrote:The basic concept is to represent the static position as an energy map of all the pieces on the board. Where the evaluation takes into account the potential energy for every piece, which is normally referred to as "the material", and the kinetic energy at the given moment.
An interesting aspect of this is that for instance the Kings have attenuation field which is applied to the kinetic energy of the opponent's pieces. This makes it possible to "automatically" select attacking and defensive moves.

At any rate, I want to find an algorithm that leads to good moves without having to explicitly code in the so called "chess knowledge". It is just seems to be more fun that way. :-)

Regards,
Hristo
This is a very interesting idea... indeed a breakthrough if you don't have to explicitly code chess knowledge and it's performance is as good as an eval with explicitly coded chess knowledge. Some questions though. How do you not explicitly code chess knowledge? Wouldn't whatever function you use to define potential energy and kinetic energy be the same thing as adding chess knowledge by the use of that function?
The functions that generate KE and PE take into account minimum amount of chess rules and a few assumptions, however none of those can be considered "chess knowledge", at least not in the way other programs are using this term.
The KE function(s) (logically speaking there are really just two of them, one for pawns and one for everything else) don't have any special cases with respect to positions or particular piece configurations, those functions require the current position only so the generated energy map can follow the rules of chess. There is no explicit knowledge about "doubled pawns", "rook on the 7th", "trapped bishop", "open line", etc. etc. The only aspects of the position that those functions are concerned with are:
1) Is a square occupied by a piece?
1.1) Is this opponent piece?
1.2) Is this own pawn?

This is it. No other information is extracted from the position during energy map generation.

The reason to use two different KE generation functions is that pawns "behave" drastically different from the other pieces and I couldn't figure out how to merge this behavior into one function.
Pradu wrote: Do you have something like KE + PE = 0 or something like KE + PE = Change in total energy in the system from a move?
:-)
I have tried both. Currently am using the KE+PE = Change in total E, not because it is inherently better, but because it is easier for me to understand.
Pradu wrote: If so what do you do to generate a score when the KE and PE of a position is known.
The easiest way of looking at it is that we have two sets of KE and PE:
For white: wKE + wPE = wTE
For black: bKE + bPE = bTE

and then try to maximize the TE for the side to move while minimizing the TE for the opponent. The ratio between wTE and bTE is what drives the final score.

In the case where the result from the position is known (say a "mate" is found) the return value that I'm using is +INFINITY. Although this is not necessary, since the ratio would be 1.0, i.e. the opponent has no Energy left.
Pradu wrote: Also what information is in the datastream for the FFT and what do you do with the (I'm guessing peak-power) frequencies obtained from power spectrum?
The energy maps is what drives the FFT. These maps are two dimensional (2D FFT) and provide information about energy distribution within the system and not just the peak, i.e. band-power is IMO more valuable than absolute peak-power.

As you can imagine at this point in the evaluation we are completely removed from any notion of "chess knowledge". :-)
Pradu wrote:
Indeed it sounds like a very expensive eval if you have to generate a datastream for each position and do an FFT on it, but I guess one would need to do some testing to see if the quality of the knowledgeless eval is good enough to offset the loss in search depth.
I don't know if this is better than the "built in chess knowledge" approach, but the sheer novelty factor makes it more fun to experiment with. In fact, this approach might not lead to a better playing chess program.

There are many different ways to generate the Energy maps and even more ways to analyze them. For instance the FFTs are not necessary, but it is surely fun to see a spectrogram of a chess position. :-)

Regards and Good morning,
Hristo
User avatar
Zach Wegner
Posts: 1922
Joined: Thu Mar 09, 2006 12:51 am
Location: Earth

Re: Is it time for another new move generator?

Post by Zach Wegner »

hristo wrote:The energy maps is what drives the FFT. These maps are two dimensional (2D FFT) and provide information about energy distribution within the system and not just the peak, i.e. band-power is IMO more valuable than absolute peak-power.
I'm still a little confused. A two dimensional FFT? Does that mean that you do FFTs for each direction separately, like ranks and files? I also don't see how band power would mean anything in this context. Perhaps you mean that by band power you are saying the amount of "energy" put along certain ranks, files, and diagonals? I don't know much about FFTs, just some basic theory, so an example of how a position translates to a wave and then to an FT would great.

You see, nobody really does stuff like this, so everyone is amazed when they hear about it.