cutechess-cli in python

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
lucasart
Posts: 3037
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

cutechess-cli in python

Post by lucasart » Thu Jun 16, 2016 2:59 am

Code speaks louder than words, so here it is:
github.com/lucasart/zinc

Please note that:
* I do not intend to replicate all the functionalities of cutechess-cli. Only the ones useful for engine testing.
* This script is intended for programmers, not end-users, so I will not write a CLI (let alone a GUI), with idiot-proof input validation code. I will write some documentation, and try to keep the code clean and self-documentating, of course.

Feedback, suggestions, bug reports, bug fixes, are welcome. So are pull requests for new functionalities, provided they don't deviate from the aforementioned project goal of simplicity and minimalism.

Enjoy!

PS: Huge credit goes to Niklas Fiekas, for his python-chess library, without which this project would not exist!
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

User avatar
lucasart
Posts: 3037
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: cutechess-cli in python

Post by lucasart » Wed Jun 22, 2016 11:49 am

Some good progress was made on the project, as you can see on github: https://github.com/lucasart/zinc.git. Kudos to Andy for his great contributions.

Here's the gap-analysis compared to cutechess-cli:
  • CLI: not implemented. just configure the parameters at the top of the script and run it.
  • JSON: not implemented. not needed for now (no CLI).
  • Variants: not implemented. will only support Chess960.
  • Gaviota TB: not implemented. may add support for Syzygy.
  • Tournaments: not implement. Only 2 engine matches for now.
  • SPRT: not implemented. will implement.
  • ratinginterval: not implemented. will implement.
  • openings: only EPD, sequentially (repeated).
  • pgnout: not implemented.
  • recover: not implemented.
  • repeat: the only behaviour available. EPD positions are always played twice, with colors reversed.
  • restart: engines are never restarted. don't see any reason to restart engines, other than slowing things down, so won't support it.
  • proto: only UCI. won't support CECP.
  • timemargin: not implemented.
  • book: not implemented.
  • ponder: not implemented.
All the rest is available, and behaves the same was as in cutechess-cli, modulo bugs :lol:
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

User avatar
hgm
Posts: 23623
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: cutechess-cli in python

Post by hgm » Wed Jun 22, 2016 12:20 pm

So apart from almost everything, it behaves exactly the same. :wink:

User avatar
lucasart
Posts: 3037
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: cutechess-cli in python

Post by lucasart » Wed Jun 22, 2016 12:46 pm

hgm wrote:So apart from almost everything, it behaves exactly the same. :wink:
Trolling as usual. You did not look at it, nor even looked at what is available in cutechess-cli.

It has all the features of cutechess-cli that are useful for fishtest, and more.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

User avatar
hgm
Posts: 23623
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: cutechess-cli in python

Post by hgm » Wed Jun 22, 2016 2:17 pm

I looked at the list you posted, and almost everything that makes cutichess-cli a useful tool seems to be amongst the 'not supported'...

So the title of the thread is a bit misleading. This isn't like cutichess any more than any tool to play simple matches between UCI engines. As it is based on Python rather than Qt, 'pittichess-noncli' would be a more apt name.

It is a bit like saying that micro-Max is 'Stockfish in plain C'. :lol:

User avatar
lucasart
Posts: 3037
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: cutechess-cli in python

Post by lucasart » Thu Jun 23, 2016 2:00 pm

lucasart wrote:Some good progress was made on the project, as you can see on github: https://github.com/lucasart/zinc.git. Kudos to Andy for his great contributions.

Here's the gap-analysis compared to cutechess-cli:
  • CLI: not implemented. just configure the parameters at the top of the script and run it.
  • JSON: not implemented. not needed for now (no CLI).
  • Variants: not implemented. will only support Chess960.
  • Gaviota TB: not implemented. may add support for Syzygy.
  • Tournaments: not implement. Only 2 engine matches for now.
  • SPRT: not implemented. will implement.
  • ratinginterval: not implemented. will implement.
  • openings: only EPD, sequentially (repeated).
  • pgnout: not implemented.
  • recover: not implemented.
  • repeat: the only behaviour available. EPD positions are always played twice, with colors reversed.
  • restart: engines are never restarted. don't see any reason to restart engines, other than slowing things down, so won't support it.
  • proto: only UCI. won't support CECP.
  • timemargin: not implemented.
  • book: not implemented.
  • ponder: not implemented.
All the rest is available, and behaves the same was as in cutechess-cli, modulo bugs :lol:
Added Chess960 and PolyGlot opening book (weighted random selection up to book depth).
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

User avatar
lucasart
Posts: 3037
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: cutechess-cli in python

Post by lucasart » Thu Jun 23, 2016 2:13 pm

hgm wrote:I looked at the list you posted, and almost everything that makes cutichess-cli a useful tool seems to be amongst the 'not supported'...

So the title of the thread is a bit misleading. This isn't like cutichess any more than any tool to play simple matches between UCI engines. As it is based on Python rather than Qt, 'pittichess-noncli' would be a more apt name.

It is a bit like saying that micro-Max is 'Stockfish in plain C'. :lol:
It's going to replicate all the useful functionalities of cutechess-cli. Of course, it's not finished, but already you can see that development is several orders of magnitude faster than cutechess-cli; precisely because it's in Python, and I don't have to deal with all the bureaucracy of C++ / Qt to get the job done.

In my experience -- and I've used cutechess-cli a lot -- the CLI part of cutechess-cli adds zero value, in terms of usability. It's only necessary because C++ is a compiled language. There are so many parameters you need to put in cutechess-cli, that you end up having to write a shell script (and a JSON) around it, edit that script, and run it. That's exactly what you do with zinc: you edit the config at the top of the Python script and run it.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

mar
Posts: 1992
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: cutechess-cli in python

Post by mar » Thu Jun 23, 2016 3:09 pm

lucasart wrote:It's going to replicate all the useful functionalities of cutechess-cli. Of course, it's not finished, but already you can see that development is several orders of magnitude faster than cutechess-cli; precisely because it's in Python, and I don't have to deal with all the bureaucracy of C++ / Qt to get the job done.
:lol: well, Python has the slowest interpreter I've ever seen. This is not about performance, sure (or is it? Pypy for the win!)
Plus you depend on python itself, a piece of bloat that has to load first to run your little script.
Why not Lua or something else that's lightweight (which you apparently like)? I guess because Python is cool.

So you've glued together some libs and made something that runs, congrats. How does it relate to the alleged "bureucracy" of C++ is beyond me. Have you just discovered what scripting languages are good for?
Also you don't need to depend on Qt at all to write something like cutechess-cli.

PS this nasty hack should prevent races? :D C'mon, you can do better ;)

Code: Select all

            # HACK: We can't just test jobQueue.empty(), then run jobQueue.get(). Between both
            # operations, another process could steal a job from the queue. That's why we insert
            # some padding 'None' values at the end of the queue
            job = jobQueue.get()
            if job == None:
                return

AndrewGrant
Posts: 472
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: cutechess-cli in python

Post by AndrewGrant » Thu Jun 23, 2016 5:47 pm

Hey, I'm proud of that "nasty" hack! :)

User avatar
lucasart
Posts: 3037
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: cutechess-cli in python

Post by lucasart » Thu Jun 23, 2016 10:46 pm

mar wrote: :lol: well, Python has the slowest interpreter I've ever seen. This is not about performance, sure (or is it? Pypy for the win!)
Plus you depend on python itself, a piece of bloat that has to load first to run your little script.
Have you measured it ? Have you even tried it ? (+same questions on PyPy).
I claim that you won't see a difference in any realistic testing scenario, because almost all the time is spend waiting for I/O operations.
Why not Lua or something else that's lightweight (which you apparently like)? I guess because Python is cool.
I don't know anything about Lua, so I can't comment on that. And yes, Python is cool.
So you've glued together some libs and made something that runs, congrats. How does it relate to the alleged "bureucracy" of C++ is beyond me. Have you just discovered what scripting languages are good for?
I never said it was hard. On the contrary, I'm saying it's easy, thanks to Python itself, and to the python-chess library of Niklas.
Also you don't need to depend on Qt at all to write something like cutechess-cli.
Agreed.
PS this nasty hack should prevent races? :D C'mon, you can do better ;)
Talk is cheap, show us your code...
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

Post Reply