Cutechess-cli: A command line tool for engine-engine matches

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Gian-Carlo Pascutto
Posts: 1231
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by Gian-Carlo Pascutto » Fri Apr 10, 2009 8:48 am

Hmm, dumping the weights didn't reveal anything special, I also don't see a bug in the code.

Still I have the impression much more offbeat openings are played than I would expect. It might be simply a PolyGlot thing.

Maybe its an interesting test to make a book from GM games, play a gazillion games, and then look at the move statistics from the games to see if there's a mismatch with the original DB.

Gian-Carlo Pascutto
Posts: 1231
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by Gian-Carlo Pascutto » Fri Apr 10, 2009 8:50 am

Small cosmetic bug:

If I use tc=15+0.15, the PGN will have the timecontrol as 15+0.149. Seems like floating point is somewhere where it shouldn't.

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by ilari » Fri Apr 10, 2009 9:47 am

Gian-Carlo Pascutto wrote:Hmm, dumping the weights didn't reveal anything special, I also don't see a bug in the code.

Still I have the impression much more offbeat openings are played than I would expect. It might be simply a PolyGlot thing.

Maybe its an interesting test to make a book from GM games, play a gazillion games, and then look at the move statistics from the games to see if there's a mismatch with the original DB.
If the problem isn't in your books, it could also be in Qt's random number generator.

Did you look at your books with Scid? It should show a probability for each move. Openings like b4 should have a probablity of less than 0.5% which is rounded to zero by Scid. If the openings played by Cutechess-cli significantly differ from that, then we have a problem.

It wouldn't take too long to play a gazillion games, because the moves after the book wouldn't matter. So you could use a time control of 0.1 sec per game or something.

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by ilari » Fri Apr 10, 2009 9:50 am

Gian-Carlo Pascutto wrote:Small cosmetic bug:

If I use tc=15+0.15, the PGN will have the timecontrol as 15+0.149. Seems like floating point is somewhere where it shouldn't.
I only use floating point temporarily to convert between string format and the time values. Looks like it needs some fixin'.

Gian-Carlo Pascutto
Posts: 1231
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by Gian-Carlo Pascutto » Fri Apr 10, 2009 10:14 am

ilari wrote: If the problem isn't in your books, it could also be in Qt's random number generator.
I think it's a bug in PolyGlot's book builder. See the other thread I posted.

bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by bob » Sat Apr 11, 2009 1:20 am

ilari wrote:
bob wrote:I suppose I should release my code at some point, but it is really tied to unix. It is what I use on our cluster and will handle using multiple starting positions, play as many games per position as you want, alternating colors (I use 2 here), and creates a PGN file containing all the games with results. I have another program that will take a bunch of scripts that use the above program, and execute as many at once as you want, so that you can use all your CPUs or just 1/2 of them or whatever. But again, all is pure Unix-based, no windows at all, and documentation is <nil> :)
It would be interesting to see that code, even without documentation. Did you write it all in C?
What else? :)



Btw, Cutechess-cli doesn't let your program resign because Crafty outputs an unprintable control character at the beginning of the command. Like this: "1-0 {Black resigns}"
In Crafty's logs it looks fine and it's also not visible in the console, but in Cutechess-cli's and also Xboard's logfile it shows.

I'm using a 64-bit Linux, and I tested versions 20.14 and 22.9 of Crafty.
Try adding "ansi off" in your crafty.rc file. Crafty highlights its move by using the usual reverse-video characters so that the move stands out in a tournament when we use console mode. I probably should make that default to off and will do so in 23.1...

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by sje » Sat Apr 11, 2009 8:18 am

bob wrote:Try adding "ansi off" in your crafty.rc file. Crafty highlights its move by using the usual reverse-video characters so that the move stands out in a tournament when we use console mode. I probably should make that default to off and will do so in 23.1...
Symbolic uses ANSI escape sequences to generate color diagram graphics. Of course, it would be a bad idea to send such to a GUI or to a log file. So the program uses the trusty isatty() function to selectively disable ANSI escape sequence output along with prompts, sign-on and sign-off messages, etc.

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by ilari » Sat Apr 11, 2009 10:12 am

bob wrote:
ilari wrote: It would be interesting to see that code, even without documentation. Did you write it all in C?
What else? :)
:shock: Respect. Well, at least you only had to write it for Unix. Now try doing the same with WinAPI threads and pipes. Not fun at all.

Try adding "ansi off" in your crafty.rc file. Crafty highlights its move by using the usual reverse-video characters so that the move stands out in a tournament when we use console mode. I probably should make that default to off and will do so in 23.1...
Thanks, I'll try that. For some reason the aforementioned resignation command is the only one that has an unprintable character, everything else works.

bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by bob » Sat Apr 11, 2009 3:45 pm

sje wrote:
bob wrote:Try adding "ansi off" in your crafty.rc file. Crafty highlights its move by using the usual reverse-video characters so that the move stands out in a tournament when we use console mode. I probably should make that default to off and will do so in 23.1...
Symbolic uses ANSI escape sequences to generate color diagram graphics. Of course, it would be a bad idea to send such to a GUI or to a log file. So the program uses the trusty isatty() function to selectively disable ANSI escape sequence output along with prompts, sign-on and sign-off messages, etc.
I don't send them to a log file, but I do send them to stdout.

isatty() is a non-portable function. I've kept crafty compatible with every platform on the planet I am aware of, windows, unix, and everything in between...

bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by bob » Sat Apr 11, 2009 3:49 pm

ilari wrote:
bob wrote:
ilari wrote: It would be interesting to see that code, even without documentation. Did you write it all in C?
What else? :)
:shock: Respect. Well, at least you only had to write it for Unix. Now try doing the same with WinAPI threads and pipes. Not fun at all.
I don't think the windows API is that bad, since crafty already works with windows threads and such. But this was a one-off project for my cluster testing and I had no intention of trying to distribute or support it outside of my lab here. Which meant a quick-and-dirty unix implementatoin, which is a system I know inside-out.


Try adding "ansi off" in your crafty.rc file. Crafty highlights its move by using the usual reverse-video characters so that the move stands out in a tournament when we use console mode. I probably should make that default to off and will do so in 23.1...
Thanks, I'll try that. For some reason the aforementioned resignation command is the only one that has an unprintable character, everything else works.

Post Reply