EPD destruction tests

[chrisw]

The many components of a chess system all intertwine.
There is the chess GUI which may or may not handle a board with 31 white queens, one white king, 31 black queens and one black king.
There is the chess engine, which might pop his tiny little arrays and spew a 7 gig core dump.
And there is the wicked author of positions who composes things that simply are not possible in the game of chess.

Now, it is possible that GUIs and engines and interfaces can handle any sort of nonsense.
And it is good also if they recognize that they have bitten off more than they can chew and must punt the football.
In such a case, a friendly program will simply call the user dirty names and refuse to accept the position.

There are balky engines that won't even promote to bishop for me. But that is a design choice and I have to live with it or use another engine.
So it is possible for engines to reject even legal moves just because the designer cares only about Elo and not the art of the moves in a special problem.

As in just about everything in life, sometimes there are no simple answers. But we can try to come up with better answers if we attempt it.

Well, if the engine traps rogue epds that would cause it to not know what to do, or hang or something, then the engine actually has to do something, and if the communication protocol doesn’t have some sort of error return capability, we have to work out what we should do or could do.
I’ld be inclined to suggest send a null move stream a1a1 a1a1 a1a1 plus some dummy other necessary or demanded return data, and throw the problem back at the GUI or whatever sent the rogue EPD. Better than hanging or crashing or analysing in a nonsense state.

[Dann Corbit]

Assuming input is perfect without checking is like assuming that we will never make mistakes as humans. But a person might type "yfeeding" when they meant to type "feeding". Similarly, someone producing a perfectly reasonable test set might type the wrong character.

Another thing that happens in real life is that people make illegal moves and it does not get caught so the game contains that sequence. Even engines do it.

Now, I realize that engines may not diagnose every possible problem. But they should not crash because they failed to check for simple problems. That's just lazy.

[Dann Corbit]

See also:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=585270

My friend Les Fernandez wrote me a nice utility.
It does lots of things, but one thing that I use all the time is EPD sanitization.
I don't expect the chess engine to do all of those things. But I also don't expect it to crash and core dump.

If I try to drive a bent nail with a hammer, I don't expect the hammer head to explode.

[chrisw]

Assuming input is perfect without checking is like assuming that we will never make mistakes as humans. But a person might type "yfeeding" when they meant to type "feeding". Similarly, someone producing a perfectly reasonable test set might type the wrong character.

Another thing that happens in real life is that people make illegal moves and it does not get caught so the game contains that sequence. Even engines do it.

Now, I realize that engines may not diagnose every possible problem. But they should not crash because they failed to check for simple problems. That's just lazy.

It’s very obvious, at least to me, that as the engine accepts input data it validates the data. It doesn’t know where the data is from, it can’t guarantee the data integrity, so as the unit on the end of the pipe that’s supposed to process the data and send back some coherent result, and it can’t guarantee any coherence if the data is bad and not checked, it really is up to the engine to check and if necessary raise a complaint, return a null move stream, or something. Anything is better than not checking and crashing.

[bob]

I ran these through Crafty and the only errors it missed deal with the two move numbers... I'll look at fixing that. All the other stuff it has no problems with detecting (although not all are illegal, like the last one, just a tough q-search position to rank root move list... The 30 queen position caused complaints years ago since Crafty won't accept > 9 queens (original + 8 promotions is all you can have).

BTW. Dann: what you are describing is known, in software engineering circles, as "robust code". A user should not be able to enter anything that would cause an issue. One could debate an infinite analysis search of course, since an infinite loop could be considered an issue. but if the end user really wants to search until he has seen enough, it seems to be useful.

I've always done my best to do this in Crafty, although I didn't catch the move half move counter >= 100.

Chris: I have a slightly different perspective on this, as Crafty is designed to be (a) an xboard client program AND (b) a stand-alone program that can be run in a console window (for those cases (such as the old Cray machines) we we could not use xboard due to various security issues Cray enforced. So text interface it was. And there we talk directly with the user and had to check anything that could cause a problem, and at least provide error messages for those things that did not hurt anything but were still wrong.

[Dann Corbit]

I ran these through Crafty and the only errors it missed deal with the two move numbers... I'll look at fixing that. All the other stuff it has no problems with detecting (although not all are illegal, like the last one, just a tough q-search position to rank root move list... The 30 queen position caused complaints years ago since Crafty won't accept > 9 queens (original + 8 promotions is all you can have).

BTW. Dann: what you are describing is known, in software engineering circles, as "robust code". A user should not be able to enter anything that would cause an issue. One could debate an infinite analysis search of course, since an infinite loop could be considered an issue. but if the end user really wants to search until he has seen enough, it seems to be useful.

I've always done my best to do this in Crafty, although I didn't catch the move half move counter >= 100.

Chris: I have a slightly different perspective on this, as Crafty is designed to be (a) an xboard client program AND (b) a stand-alone program that can be run in a console window (for those cases (such as the old Cray machines) we we could not use xboard due to various security issues Cray enforced. So text interface it was. And there we talk directly with the user and had to check anything that could cause a problem, and at least provide error messages for those things that did not hurt anything but were still wrong.

Yes, this is the right attitude for software development.
Obviously, IMO-YMMV.

[bob]

BTW, one question. I can't find any restriction on the half-move counter in the FEN spec. Obviously 0 means a non-reversible move was made immediately prior to producing this FEN string. And anything up to 99 seems (somewhat) sensible. But what about 100? That would mean the current position is already a 50 move rule draw unless the moving side chooses to reset the counter by playing a non-reversible move. Should 100 be allowed? Also I don't see where the move number should have any restriction other than > 0...

[chrisw]

BTW, one question. I can't find any restriction on the half-move counter in the FEN spec. Obviously 0 means a non-reversible move was made immediately prior to producing this FEN string. And anything up to 99 seems (somewhat) sensible. But what about 100? That would mean the current position is already a 50 move rule draw unless the moving side chooses to reset the counter by playing a non-reversible move. Should 100 be allowed? Also I don't see where the move number should have any restriction other than > 0...

It’s a logical restriction that the move number has to be greater the 50-move rule count of reversibles. Which is why non move numbered EPDs default to 0 1 in that field.
I find those numbers not very useful actually. I like to use the 50 counter as basis for tracking back looking for draws, but if the EPD comes with, say 20 30, but no move history, it’s less than useful.
With uci position move stream format, one can reconstitute the game history (or part if it), so I started just overruling the EPD movenums and substituting what can be got from the move stream.

[Dann Corbit]

BTW, one question. I can't find any restriction on the half-move counter in the FEN spec. Obviously 0 means a non-reversible move was made immediately prior to producing this FEN string. And anything up to 99 seems (somewhat) sensible. But what about 100? That would mean the current position is already a 50 move rule draw unless the moving side chooses to reset the counter by playing a non-reversible move. Should 100 be allowed? Also I don't see where the move number should have any restriction other than > 0...

The specification does not state the domain other than "nonnegative":

16.1.3.5: Halfmove clock

The fifth field is a nonnegative integer representing the halfmove clock. This
number is the count of halfmoves (or ply) since the last pawn advance or
capturing move. This value is used for the fifty move draw rule.

I know that some draws must be claimed. If this is one of those cases, then the number can be arbitrarily large.

[Dann Corbit]

I notice something else -- the rule does not mention a castle move which is also not reversible.