Worst advice

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
hgm
Posts: 23790
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Worst advice

Post by hgm » Tue Aug 11, 2015 11:08 am

flok wrote:From a professional software developer's point of view that is horrible :-)
Imho code must be correct.
That is a counter-productive approach. If incorrect code would do the job far better than 'correct' code, I would prefer incorrect code any time. Chess engines are full of 'incorrect' code. Zobrist hashing cannot map positions unambiguously upon keys even with 64-bit keys. You cannot search until mate so you use very dubious and often flawed heuristics to calculate a static evaluation. You prune moves that might be good.

Wasting time on hash-move validation rather than just accepting that in one out of a billion nodes you get a nonsense score because of a collision (amongst the 100,000 for which you got nonsense scores for completely different reasons) will almost certainly weaken the engine.

User avatar
Roman Hartmann
Posts: 295
Joined: Wed Mar 08, 2006 7:29 pm
Contact:

Re: Worst advice

Post by Roman Hartmann » Tue Aug 11, 2015 11:18 am

Aspiration Windows never worked for me even though I wasted quite some time with it. I'm still not sure why I couldn't make it work but as LMR is working nicely I'm not going back to Aspiration Windows anyway.

Beside that I didn't try too many things that wouldn't work after some work. Rightnow I'm working on a hybrid engine (0x88 move generation combined with bitboard evaluation) and it's still a bit early to tell if it will work out. In any case it took me a bit longer to get the bitboards getting updated properly when making an unmaking moves. But now it finally works.

Roman

mar
Posts: 2015
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Worst advice

Post by mar » Tue Aug 11, 2015 12:55 pm

Joost Buijs wrote:With a 64 bit key validation is not really necessary.

I have a move validation routine which I only use when I want to check if my transposition table is not malfunctioning.
Sometimes I let it run for an hour or so, and it never detects a single invalid move, and this is in a multithreaded search.
To be 100% on the save side you can of course make sure that an invalid move won't crash your engine.
Hmm, that's interesting (I should retest collisions; I use 64-bit signature as well).

My hash/killer validation code is quite complicated and ugly (also because I use special flags stored with each move that have to be validated, this may not have been the best design decision I admit).
So first I make sure that I have a pseudo-legal move and it's then validated using pseudo-is-legal code which is singificantly more straightforward.
I verified that my islegal check by generating all possible combinations; I had to do this because passing illegal move to do_move would crash my engine.

jwes
Posts: 778
Joined: Sat Jul 01, 2006 5:11 am

Re: Worst advice

Post by jwes » Tue Aug 11, 2015 3:24 pm

hgm wrote:I just make sure that the engine does not crash, no matter what move it finds in the hash table. If it wants to capture an own piece with an empty square? Fine, why would that be a problem? Just do it. The only extra code I needed was when MakeMove performs a castling. Then I also have to save and restore the piece that was captured by the Rook (or by whatever was standing on the square where the Rook was expected). As castlings are rare, the overhead is immesurably small.
Does your program handle side not to move in check or capture of a king without problems? These are the situations where less than full (or no) validation could cause problems.

User avatar
hgm
Posts: 23790
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Worst advice

Post by hgm » Tue Aug 11, 2015 3:48 pm

Indeed this could be a concern. My engines are typically King-capture engines, so pseudo-legal must be tested for capturing the King, and cause a cutoff without further search when they do. The hash move has to be subjected to that test too.

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Worst advice

Post by Ferdy » Tue Aug 11, 2015 3:52 pm

Henk wrote: Using bitboards for move generation was at least one month of misery with these stupid bit shifts that always shifted to the wrong direction.
:) 11110

jdart
Posts: 3842
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Worst advice

Post by jdart » Tue Aug 11, 2015 4:23 pm

The frequency is small but with 64-bit hashes and long time controls on modern hardware, collisions are not ignorable.

I used to store the "in check" status for a position in the hashtable, since it costs a little time to determine this dynamically. But that was unreliable. Once in a while you'd get a hash result where that bit was set wrong. Sure, 99.99% of the time or so it worked. But having it not work was fatal, so I took it out. I did see once in a while crashes, playing for hours on a chess server, before removing this.

--Jon

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Worst advice

Post by Ferdy » Tue Aug 11, 2015 4:37 pm

Henk wrote:When you read all these comments there seems to be many ideas and recommendations to improve your chess engine.

For me worst idea was to use bitboards for move generation. For it was difficult to implement, gave horrible bugs and no speed up.

So question is: What was the worst idea you implemented ?
[d]r4rk1/1p1nqppb/3b1n1p/p2pp3/8/PP1P1NP1/1B1NPPBP/Q1R2RK1 w - - 0 16
Give bonus to a rook in open file because it can penetrate the 7th rank :).





[d]1r1r2k1/3qnpb1/1pn1b1p1/p1ppp3/2P5/PPNPP1P1/1BQ1NPBR/2K4R b - - 0 17
Give more bonus to more than one rook that attacks opp king along open file :).




[d]8/8/8/K1k5/P7/8/8/8 w - - 0 1
Give bonus to a king that is close to his passer so that opp king will not
be able to block it :).

Joost Buijs
Posts: 990
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Worst advice

Post by Joost Buijs » Tue Aug 11, 2015 5:36 pm

mar wrote:
Joost Buijs wrote:With a 64 bit key validation is not really necessary.

I have a move validation routine which I only use when I want to check if my transposition table is not malfunctioning.
Sometimes I let it run for an hour or so, and it never detects a single invalid move, and this is in a multithreaded search.
To be 100% on the save side you can of course make sure that an invalid move won't crash your engine.
Hmm, that's interesting (I should retest collisions; I use 64-bit signature as well).

My hash/killer validation code is quite complicated and ugly (also because I use special flags stored with each move that have to be validated, this may not have been the best design decision I admit).
So first I make sure that I have a pseudo-legal move and it's then validated using pseudo-is-legal code which is singificantly more straightforward.
I verified that my islegal check by generating all possible combinations; I had to do this because passing illegal move to do_move would crash my engine.
In practice I always use the hash move without validation and I never had the engine crash or saw any adversary effect by doing so.
My engine is not insensitive to invalid moves at all, I'm quite sure it will crash immediately when for instance the king gets captured, but it just never happens.

I use a lockless strategy for my transposition table by xorring the key with the data field which are both 64 bits in size to produce the final key that will be stored.
When I store or load a bucket that is 128 bits in size I make use of the interlocked128 functions to make sure that a bucket will always be fully stored or loaded with a full memory fence.
The drawback is that these functions are only supported on Windows 8 and higher, but for me that is not such a big problem.
It will be a problem though if I want to release the engine to the general public because most people are still using Windows 7.

Maybe xorring the key with the data is not necessary anymore since I make use of the interlocked128 functions, I have to check that, these are just remnants of the past.

Joost Buijs
Posts: 990
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Worst advice

Post by Joost Buijs » Tue Aug 11, 2015 7:31 pm

mar wrote: My hash/killer validation code is quite complicated and ugly (also because I use special flags stored with each move that have to be validated, this may not have been the best design decision I admit).
So first I make sure that I have a pseudo-legal move and it's then validated using pseudo-is-legal code which is singificantly more straightforward.
I verified that my islegal check by generating all possible combinations; I had to do this because passing illegal move to do_move would crash my engine.
Basically my scheme is the following:

if available try the hashmove without validation
generate captures and sort according to see()
try captures with see() >= 0
generate non captures
loop through non captures once to find both killers
if found try killer 1
if found try killer 2
try captures with see() < 0
sort non captures according to history[]
try non captures

Somehow trying the loosing captures right after the killers has a positive effect on my engine. I guess most people are trying them last.

This is just a very basic move ordering which can be improved quite a lot.
When I look at it this way, it can be an improvement to use the validation routine on the killers and generate the non captures somewhat later.

Post Reply