Diepeveen's move generator

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Diepeveen's move generator

Post by diep »

Tom Likens wrote:
diep wrote:Had you considered shipping me an e-mail?
Yes, I would think if someone were interested in the Diep move generator, sending you an email
would be the quickest (and safest) way to get it. That *might* even be the best way to get any
questions you might have answered correctly.

My guess is you have 100% of the code for "Diepeveen's move generator" :wink:

regards,
--tom
Most these guys just care for public braqgging rights and discussions. E-mails doesn't fit in this strategy.

A guy whose second first name is Morgan told me not too long ago about all those names showing up at CCC:

"Who are all these guys?"
ZirconiumX
Posts: 1334
Joined: Sun Jul 17, 2011 11:14 am

Re: Diepeveen's move generator

Post by ZirconiumX »

diep wrote:
ZirconiumX wrote:A discussion in the Olympus ICS came up about this.

The result was we thought that it was useless, because he didn't explain how the arrays worked, jus gave code which we weren't even sure worked.

If it did work, and VD would tell us how it worked, then we would use it hands down, because it seems quick.

Matthew:out
What's this for ******* nonsense.

I emailed the entire code, which includes building up the tables, to at least 30 persons past years. To 100% of those who asked.

It's possible i posted it to CCC one day as well - not sure about it.
We simply looked at the code you posted in the "Who's got the fastest movegenerator?" thread. That was all of the code we knew of. As far as we were concerned, that was all the code you were willing to release. The code that was there was unusable.

Looking at your code, it *does* seem fast, if a little (to my eyes) unusual, but that's just me.

So - let me get this straight.

The move generator uses global arrays instead of encapsulated structs to access information. (the structs mean yet more memory to be accessed, so it makes sense.)

It is passed the last move made (for en passant), and the side to move.

It produces the information in yet more global arrays.

Matthew:out
Some believe in the almighty dollar.

I believe in the almighty printf statement.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Diepeveen's move generator

Post by diep »

ZirconiumX wrote:
diep wrote:
ZirconiumX wrote:A discussion in the Olympus ICS came up about this.

The result was we thought that it was useless, because he didn't explain how the arrays worked, jus gave code which we weren't even sure worked.

If it did work, and VD would tell us how it worked, then we would use it hands down, because it seems quick.

Matthew:out
What's this for ******* nonsense.

I emailed the entire code, which includes building up the tables, to at least 30 persons past years. To 100% of those who asked.

It's possible i posted it to CCC one day as well - not sure about it.
We simply looked at the code you posted in the "Who's got the fastest movegenerator?" thread. That was all of the code we knew of. As far as we were concerned, that was all the code you were willing to release. The code that was there was unusable.

Looking at your code, it *does* seem fast, if a little (to my eyes) unusual, but that's just me.

So - let me get this straight.

The move generator uses global arrays instead of encapsulated structs to access information. (the structs mean yet more memory to be accessed, so it makes sense.)

It is passed the last move made (for en passant), and the side to move.

It produces the information in yet more global arrays.

Matthew:out
a) producing the move in a global array is faster of course than first spending 5 cycles or so for each move to get a bit out of a bitboard and then storing it in a global array (like crafty is doing).

Any other method like incremental generating moves and not storing them anywhere except your stack,
that is so so 90s.

b) your processors can prefetch arrays. They are very good in this.
so this comes at nearly 0 cost.

c) the bitboard arrays are HUGE, i use smaller arrays there. You can get them in fact even tinier those arrays if you would want to.

d) you really didn't even bother to ship me an email back then. your 'conclusions' are not shared amongst experts

e) game tree search, if you want to achieve something there, is not for average guys. you need to be clever and intelligent to figure something out there.
Hrvoje Horvatic

Re: Diepeveen's move generator

Post by Hrvoje Horvatic »

Gerd Isenberg wrote:I just found some more code of Vincent's generator in Mridul Muralidharan's Random thought blog:
http://mridulm.blogspot.de/2004/06/perm ... ut-up.html
I made it work... I still have several bugs and many things to solve (we all know that chess programming TAKES TIME), but it seems that this stuff is good...

Very early preliminary results show that my bad adaptation of Vincent's move generator is about 20% faster than crafty... I have to check several more things to be sure, I still don't stand behind this, but this DEFINITELY shows promise...
SuneF
Posts: 127
Joined: Thu Sep 17, 2009 11:19 am

Re: Diepeveen's move generator

Post by SuneF »

Hrvoje Horvatic wrote:
Gerd Isenberg wrote:I just found some more code of Vincent's generator in Mridul Muralidharan's Random thought blog:
http://mridulm.blogspot.de/2004/06/perm ... ut-up.html
I made it work... I still have several bugs and many things to solve (we all know that chess programming TAKES TIME), but it seems that this stuff is good...

Very early preliminary results show that my bad adaptation of Vincent's move generator is about 20% faster than crafty... I have to check several more things to be sure, I still don't stand behind this, but this DEFINITELY shows promise...
Perhaps 20% faster in bulk generation but probably 200% slower in capture generation ;-)
Hrvoje Horvatic

Re: Diepeveen's move generator

Post by Hrvoje Horvatic »

diep wrote:
ZirconiumX wrote:A discussion in the Olympus ICS came up about this.

The result was we thought that it was useless, because he didn't explain how the arrays worked, jus gave code which we weren't even sure worked.

If it did work, and VD would tell us how it worked, then we would use it hands down, because it seems quick.

Matthew:out
What's this for focking nonsense.

I emailed the entire code, which includes building up the tables, to at least 30 persons past years. To 100% of those who asked.

It's possible i posted it to CCC one day as well - not sure about it.
Vincent, Matthew,

guys, we are not going to start a fight, aren't we?

Vincent, it depends what you mean by "entire code"... let me give you small explanation and comparison... 10 years ago I received small bitboard-based move generator from Sune Fischer (Sune, BIG thank you!). All I had to do is press F5 in Visual Studio, and it was flying like a tiger. THAT's "entire" code...

Your code, on the other side, misses complete infrastructure for representing position and making moves... plus it calls some other procedures (like, for checking if castling is possible), which are not included... plus it has some quite weird ways of handling certain things... plus, the names of variables and comments are in dutch...

To make that code work demands quite a lot of experience and strong will... it is not really "plug&play" code... :wink: So it is not really strange that people call your code "useless", and quit, although I personally find it quite useful... and sometimes you have to write "weird" code to get things running fast, so it is not THAT weird to me...

IMHO, if you personally have had put a little bit more effort to make code more "readable" to others, you might have had more success in "sending message across"... And I think that a message is valuable...
Hrvoje Horvatic

Re: Diepeveen's move generator

Post by Hrvoje Horvatic »

SuneF wrote:
Hrvoje Horvatic wrote:
Gerd Isenberg wrote:I just found some more code of Vincent's generator in Mridul Muralidharan's Random thought blog:
http://mridulm.blogspot.de/2004/06/perm ... ut-up.html
I made it work... I still have several bugs and many things to solve (we all know that chess programming TAKES TIME), but it seems that this stuff is good...

Very early preliminary results show that my bad adaptation of Vincent's move generator is about 20% faster than crafty... I have to check several more things to be sure, I still don't stand behind this, but this DEFINITELY shows promise...
Perhaps 20% faster in bulk generation but probably 200% slower in capture generation ;-)
Ha ha, you might be right, I have to check that...

On the other hand, if you don't have qsearch and use static exchange on the leaves (like some older square-oriented engines), you don't need separate capture generation :wink:
Hrvoje Horvatic

Re: Diepeveen's move generator

Post by Hrvoje Horvatic »

Tom Likens wrote:
diep wrote:Had you considered shipping me an e-mail?
Yes, I would think if someone were interested in the Diep move generator, sending you an email
would be the quickest (and safest) way to get it. That *might* even be the best way to get any
questions you might have answered correctly.

My guess is you have 100% of the code for "Diepeveen's move generator" :wink:

regards,
--tom
Tom,

it is a bit more complicated than that... I actually wanted feedback if anybody else has tried it and what results he got... I actually said that explicitly... :wink:

I started playing with chess programming somewhere around year 2000, and even back then Vincent was talking about how his move generator is faster than "beancounters"..., and it looks incredible to me that 12 years later still NOBODY tried to disprove him, or give him a credit if he is right...
Hrvoje Horvatic

Re: Diepeveen's move generator

Post by Hrvoje Horvatic »

diep wrote: Had you considered shipping me an e-mail?

I responded to 100% of everyone who emailed with a C file that has everything.
Vincent, thank you for offering your help, but I've managed to make my way through, no need for your help now...

Your stuff seems to work... I still have to iron out all the bugs and incosistencies, optimize *my* part of code, but it seems to me that you were right...
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Diepeveen's move generator

Post by diep »

Hrvoje Horvatic wrote:
diep wrote: Had you considered shipping me an e-mail?

I responded to 100% of everyone who emailed with a C file that has everything.
Vincent, thank you for offering your help, but I've managed to make my way through, no need for your help now...

Your stuff seems to work... I still have to iron out all the bugs and incosistencies, optimize *my* part of code, but it seems to me that you were right...
Did you notice that between what i posted here and what Stallman got at the time, that the latest incarnation that you do not have running yet, that this one is considerable faster?

I posted back in the 2000s a comparision between crafty's semi legal move generation speed and Diep's.

Diep was 2.2x faster.
This was at a 32 bits processor.
Crafty ran at 32 bits hardware online for a pretty long time.

You cannot compare processors from back then with todays processors.

I posted those results - so it's not like "i did do a claim unfunded",
as you are suggesting. Also it was with a crafty compile i didn't do myself. The fastest possible crafty compile at processors from back then.

Note that in terms of system time semi legal move generation is just a small percentage of the system time. I'm using it bigtime in evaluation though for all sorts of mobility loops where i use knowledge to scan. So not some sort of dumbo bitcount, as each square gets seperately evaluated with chess logics.

I disagree with another weird opinion you write here. What i do in my move generator is simple, effective and straightforward.

The weird thing is bitboards - most use inline assembler code to speedup or system functions from the compiler or inline intrinsics. To get the square you are at, you need to do some sort of weirdo Leadz later on called FirstOne then nowadays it's called LSB or whatever. Now *that* is a weird way of programming.

It's incompatible and in terms of speed comparision you're comparing simple straight forward C code of me and compare that with 64 bits assembler. Are you understanding fully what you compare?

Compatible C code with incompatible hard to read assembler. It's very difficult to use bitboards for a program with a huge evaluation function like Diep has.

Just see how much trouble some who post here have to implement just one simple pattern into bitboards. Takes them weeks to optimize just 1 small thing.

If you do that in a knowledge rich environment you're not making much of a progress.

So the programming model for knowledge has proven itself.

In short for upcoming robotics project i'm about to start here implementing knowledge i'll do in a similar manner like Diep. Bitboards are going to be impossible there (besides that of course all those cheap ARMs are 32 bits).

It's interesting to see how long Diep's move generating tables can withstand the test of time, whereas we have seen by now already 100 revisions in bitboards. And they keep busy with it - whereas you do NOT want to spend big time on move generation of course. It has to work fine in your own code and then it's nice if you can keep using it for another 20 years, which in software engineering is similar to a building that stands for 1000 years.