CCRL Engine Authors' Tournament (will start October 1st)

[Graham Banks]

I'll see if I can get some of the Winboard engine authors to come and post that there isn't a problem so that I can get you guys off my back.

Email sent to all authors of the Winboard engines that are currently in my Amateur divisions, asking them to read this thread and post.

[Modern Times]

All you're doing is putting me off testing Winboard engines full-stop.

Whilst I respect an author's decision to use the winboard protocol, Arena has bugs and I've never really taken to it, I refuse to use adaptors like wb2UCI and polyglot, so that rules out the Winboard GUI itself and things like Shredder GUI and Fritz GUI, so if there is a problem with ChessGUI (and I am very far from convinced that there is) it simply means I will never run winboard engines for conventional chess, and that is perfectly OK with me. Chess960 is an exception, because this alleged bug would not affect chess960 games played by winboard engines under ChessGUI as there is no book and the engines move from move 1.

[ZirconiumX]

I'll see if I can get some of the Winboard engine authors to come and post that there isn't a problem so that I can get you guys off my back.

Email sent to all authors of the Winboard engines that are currently in my Amateur divisions, asking them to read this thread and post.

Can somebody provide a summary of what the problem actually is?

I've read through the thread and don't understand anything.

[Patrice Duhamel]

I'm not sure to understand the problem, maybe Dennis could create a new thread in the programming section, explain the problem with details and the full debug file of the game ?

If previous versions of Schooner doesn't have this problem, then it should be easy to fix it.

[stegemma]

Just trying out the Group E engines under ChessGUI.

Jumbo seems to run as it should.

Pigeon moves pretty much instantly every move.

Dorpsgek seems to run as it should.

Satana plays okay, but without displaying any information.

Neurone moves pretty much instantly every move.

Soberango moves instantly first move out of book, but is okay after that.

I've never try Satana in ChessGUI. I'll do it but I can't work on my engine for another few months.

[Stan Arts]

This affected Nemeton as well.:
http://www.talkchess.com/forum/viewtopi ... 710#599710

Nemeton handles it now by reading the move number out of setboard but ofcourse we still don't know the openingmoves or possible repetitions and knowing what's going on is the whole point of having a Winboard engine.
Not to blame Graham or Matthias as it's a tremendous job of both and again such a shame Matthias rashly deleted his source as a solution would seem relatively straightforward making ChessGUI even more impressive.

[Sergio Martinez]

I am not even sure the Joker version you use does support setboard. I remember having implemented that only very late, basically when I already stopped working on Joker itself, but in its spinoff Joker80 because there are so many different initialpositions for 10x8 Chess (Capablanca, Carrera, Bird, Gothic, Embassy, Schoolbook). Once I had the FEN reader I also ported that back toJoker, but I don't remember if I ever officially released that. Which version exactly are you using?

I checked the 1.1.14 games.
I guess it's possible that I may have used the wb2uci adapters though.

A last note here. This is one reason why only CCRL can efficiently check the game databases for this issue.
'We' don't know when WB2UCI was used and the second reason is that 40/4 games are not available with comments,
the third one would be that we cannot know when ChessGUI was used, except perhaps by typical PGN output.
For programs capable of UCI + WB I guess CCRL always uses UCI?

40/4 games are run in Arena GUI, like I said in previous post.

I dont added comment option in PGN configuration cause I dont use in my tournaments, unless someone ask me, something that has not happened in this tournament, warned on July 30,two months before the start, so if someone was interested about it, he only could have asked and I didnt see you here asking.

If there are things that you dont know you just have to ask them because it seems you do not want to know the answers and yes create your own response.

But do not worry, for anything you ignore, we are here to answer you, even though it does not appear that you had the slightest interest in the tournament until you've seen something you have not understood, but all question you want will be gladly answered.

By the way, this is a tournament outside the official list of CCRL in which more people participate, here are just Graham and I, and if you want to ask questions about CCRL and not on this tournament I suggest you choose another place , we are talking about the tournament and not on other test or tournaments, and Graham and I we are not all CCRL group, only two members, so the rest of test and database are completed with other members arent here.

Regarding the use of Chessgui, I use Arena, so all results of games published by me, mean that games are run under Arena GUI, as I repeat. As for Chessgui I think Graham and Ray has already answered you.

Your unfriendly comments are completely out of place. This thread is since a while not only about what you or Graham had done in this thread
and I did 'not choose this place' it just happened here.
You can ask moderation about splitting that issue if you hate to see people talking about it in this thread though.
We have just discovered a huge problem which could concern ALL CCRL games (with ChessGUI and xboard engines involved).
Inbetween a few minutes I could prove that at least an other program than Schooner was hitten by that problem.
That's why I mentioned the 40/4 games (ALL of them!) which are w/o comments for download at the CCRL site.
(The 40/40 games can be downloaded with comments to be checked)

Do you really want to say ALL 40/4 games ever played for CCRL are only played with Arena?
May be you can also answer if no game ever at 40/4 or 40/40 for CCRL was played in ChessGUI?
(IMHO a good suggestion would be to add a GUI tag at least for testers/organisations who use several GUIs)

Either you did not follow the thread for a while or you deliberately tried to missunderstand me.

For the record and your information again this and my post before is/was generally about ALL CCRL games...

A last question. I tried to download the 40/40 commented database (which always worked in the past),
but now for days the download always stops between 50 and 100MB long before finished. Any idea about the problem?

Guenther

My comments are neither more nor less friendly than his. You say this has emerged so when the problem, if it really affects all engines winboard low chessgui, which has not been shown by the way, affects CCRL, that problem does not affect CCRL only, can affect anyone chessgui to use, whether or not CCRL not only CCRL, as has you focused on your comments, plus as I say not know who CCRL use chessgui only, but if you want ask all members of CCRL about what software use, like I said, this isnt the correct place. You know (Graham and Ray answer you) who use Chess gui here, so I suggest you ask all people run tournaments what interface use or open a new topic and explain the problem, cause its clear the problem (if it is) is about Chessgui, not CCRL or this tournament.

"We Have just discovered a huge problem Which Could concern CCRL ALL games (with ChessGUI and xboard engines Involved)"

It seems very hasty to make conclusions about the problem emerged when it is not yet clear if it affects ALL engines and to what extent it affects them, but directly you assumed that exists and affects all items of CCRL using chessgui, by I said that it seems that you have to ask is responding more directly.

I'm following the thread, but it seems that you wear it for where you're interested only you. There in lies the problem, this thread was created to this TOURNAMENT not for other issues. I do not think it wrong to talk about the problem, which certainly as I say is not yet clear which engines exactly affects, at least, like I say withouth more information, but is that you're heading to CCRL only by what I see, because if we are in a thread on a tournament and says the games have no comments 40/4 without specifying which items, you usually refer to the games of this tournament.

I am examining the problems in other winboard engines because currently see in some of them the problem here is spoken and as you should affect all engines winboard and therefore I at least before announcing a catastrophe as it seems do you, I prefer to analyze it, and not take two PGN and draw conclusions from them.

There have been problems with other interfaces, such as Fritz, and I see that these problems are discussed in the thread of a tournament. If it affects all engines, I dont understand why the problem is discussed here, as if only affect our tournament, since wanting relate CCRL is unfair as not all members of CCRL use that interface. It would be logical to open a new thread and discuss the problem, as has always been done with other problems. It seems the interest of keeping the argument that the only person in the world who use Chessgui is Graham, here and looks for what you read here that the blame for failures also be Graham. All this seems unfair and absurd, but it seems to say is be not friendly...

[hgm]

All you're doing is putting me off testing Winboard engines full-stop.

That is of course up to you to decide. But it does seem the logical course of action, and in any case highly preferable to generating invalid results which would corrupt the CCRL data base.

If there is a known incompatbility between ChessGUI in the mode you are running it, and a particular WB engine, you should not test that engine in ChessGUI. If a certain problem causing such incompatibility is spread very widely amongst WB engines, (and IMO an overwhelming majority of WB engines would suffer from this), you should only test engines that you are certain they don't have this problem. Otherwise you would be guilty of careless testing.

Like I described, there is a rather simple test for this paricular problem: Just run the engine from the command line two times with the same time on its clock from the same position, once with 100 moves to go (according to the FEN move number),once with only one move to go. If it doesnot think significantly longer in the latter case, it is NOT compatible with your tournaments.

E.g. this is the output from KingSlayer (in red, in response to the input in black):

xboard
protover 2
feature ping=1 setboard=1 colors=0 usermove=1 memory=1 debug=1 reuse=0 myname="Simple 0.0"
feature option="Resign -check 0"
feature option="Contempt -spin 0 -200 200"
feature option="Evaluate piece mobility -check 0"
feature option="Evaluate trapped pieces -check 0"
feature option="Evaluate drawish material -check 0"
feature option="Evaluate specific endings -check 0"
feature option="Evaluate pawn structure -check 0"
feature option="Evaluate king shelter -check 0"
feature option="Evaluate king seige -check 0"
feature done=1
new
post
level 100 100 0
setboard rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
time 10000
go
1 10 0 2 b1a3
1 34 0 3 b1c3
2 0 0 42 b1c3 b8c6
3 27 0 199 b1c3 b8c6 g1f3
4 0 0 388 b1c3 b8c6 g1f3 g8f6
5 7 1 3700 b1c3 b8c6 g1f3 g8f6 d2d4
6 0 1 8551 b1c3 b8c6 g1f3 g8f6 d2d4 a7a5
7 25 7 61645 b1c3 b8c6 g1f3 d7d5 d2d4 c8f5 c1f4
8 0 18 148996 b1c3 b8c6 g1f3 d7d5 d2d4 c8f5 c1f4 g8f6
9 4 25 190567 b1c3 b8c6 g1f3 d7d5 d2d4 g8f6 c1f4
10 2 120 1056745 b1c3 b8c6 e2e4 g8f6 g1f3 d7d6 d2d4 a7a5 c1f4 c8e6
11 2 256 2302693 b1c3 d7d5 g1f3 b8c6 d2d4 g8f6 c1f4 c8f5 a2a4 f6e4 c3e4 f5e4
move b1c3
quit

C:\cygwin\home\simple-09a18b9>simple
xboard
protover 2
feature ping=1 setboard=1 colors=0 usermove=1 memory=1 debug=1 reuse=0 myname="Simple 0.0"
feature option="Resign -check 0"
feature option="Contempt -spin 0 -200 200"
feature option="Evaluate piece mobility -check 0"
feature option="Evaluate trapped pieces -check 0"
feature option="Evaluate drawish material -check 0"
feature option="Evaluate specific endings -check 0"
feature option="Evaluate pawn structure -check 0"
feature option="Evaluate king shelter -check 0"
feature option="Evaluate king seige -check 0"
feature done=1
new
post
level 100 100 0
setboard rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 99
time 10000
go
1 10 0 2 b1a3
1 34 0 3 b1c3
2 0 0 42 b1c3 b8c6
3 27 0 199 b1c3 b8c6 g1f3
4 0 0 388 b1c3 b8c6 g1f3 g8f6
5 7 0 3700 b1c3 b8c6 g1f3 g8f6 d2d4
6 0 1 8551 b1c3 b8c6 g1f3 g8f6 d2d4 a7a5
7 25 7 61645 b1c3 b8c6 g1f3 d7d5 d2d4 c8f5 c1f4
8 0 18 148996 b1c3 b8c6 g1f3 d7d5 d2d4 c8f5 c1f4 g8f6
9 4 23 190567 b1c3 b8c6 g1f3 d7d5 d2d4 g8f6 c1f4
10 2 118 1056745 b1c3 b8c6 e2e4 g8f6 g1f3 d7d6 d2d4 a7a5 c1f4 c8e6
11 2 254 2302693 b1c3 d7d5 g1f3 b8c6 d2d4 g8f6 c1f4 c8f5 a2a4 f6e4 c3e4 f5e4
move b1c3

You can see that in both cases it was thinking only ~2.5 sec (boldface), while it was told it was in a 100 moves/session game with 100 sec on its clock. So obviously it does not pay attention to the move number (also boldface), which in the latter case told it it had only a single move to go. Thinking only 2.5 sec when you still have 100 sec on your clock, and expect to get 100 minutes of new time after that ('level 100 100 0') is not making good use of the time, and would result in very sub-optimal play. Even engines that want to keep a large time buffer should think about 25 sec in such a case (except perhaps when they have only a single legal move).

[Sven]

Just run the engine from the command line two times with the same time on its clock from the same position, once with 100 moves to go (according to the FEN move number),once with only one move to go. [...]

setboard rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 99

[...] So obviously it does not pay attention to the move number (also boldface), which in the latter case told it it had only a single move to go.

There is one minor "nitpicking" detail that does not change the result of your excellent analysis:

In the FEN above the full move number should be 100 instead of 99 if you want to create a situation with only one move left to go. The next move of White (which is the moving side) will be no. 100. In the standard opening position the full move number is 1, with 100 moves to go in this example.

Please note that Dennis has opened a new thread for this topic in the programmer's subforum. I suggest that the technical part of the discussion should be continued there.

[hgm]

Well, I was not sure Graham would read there, as he suggested the creation of such a thread only to get 'everyone of his back'.

And you are right about the 100; counting starts at 1 if there are still zero moves played. So I tested a situation where were two moves to go. That doesn't matter very much; either an engine has a policy to use up all its time, and then it would have used 99 sec with 1 move to go, and ~50 sec with 2 moves to go, or it would keep a safety margin of 3 or 4 nominal move times (so that in case of a fail low it could exceed nominal time fy a factor 4 or 5 without forfeiting), and then it would not matter so much if there was still 1or 2 moves to go, because the margin time was much longer than the moves anyway.