cutechess-cli in python

[lucasart]

Code speaks louder than words, so here it is:
github.com/lucasart/zinc

Please note that:
* I do not intend to replicate all the functionalities of cutechess-cli. Only the ones useful for engine testing.
* This script is intended for programmers, not end-users, so I will not write a CLI (let alone a GUI), with idiot-proof input validation code. I will write some documentation, and try to keep the code clean and self-documentating, of course.

Feedback, suggestions, bug reports, bug fixes, are welcome. So are pull requests for new functionalities, provided they don't deviate from the aforementioned project goal of simplicity and minimalism.

Enjoy!

PS: Huge credit goes to Niklas Fiekas, for his python-chess library, without which this project would not exist!

[lucasart]

Some good progress was made on the project, as you can see on github: https://github.com/lucasart/zinc.git. Kudos to Andy for his great contributions.

Here's the gap-analysis compared to cutechess-cli:

• CLI: not implemented. just configure the parameters at the top of the script and run it.
• JSON: not implemented. not needed for now (no CLI).
• Variants: not implemented. will only support Chess960.
• Gaviota TB: not implemented. may add support for Syzygy.
• Tournaments: not implement. Only 2 engine matches for now.
• SPRT: not implemented. will implement.
• ratinginterval: not implemented. will implement.
• openings: only EPD, sequentially (repeated).
• pgnout: not implemented.
• recover: not implemented.
• repeat: the only behaviour available. EPD positions are always played twice, with colors reversed.
• restart: engines are never restarted. don't see any reason to restart engines, other than slowing things down, so won't support it.
• proto: only UCI. won't support CECP.
• timemargin: not implemented.
• book: not implemented.
• ponder: not implemented.

All the rest is available, and behaves the same was as in cutechess-cli, modulo bugs

[hgm]

So apart from almost everything, it behaves exactly the same.

[lucasart]

So apart from almost everything, it behaves exactly the same.

Trolling as usual. You did not look at it, nor even looked at what is available in cutechess-cli.

It has all the features of cutechess-cli that are useful for fishtest, and more.

[hgm]

I looked at the list you posted, and almost everything that makes cutichess-cli a useful tool seems to be amongst the 'not supported'...

So the title of the thread is a bit misleading. This isn't like cutichess any more than any tool to play simple matches between UCI engines. As it is based on Python rather than Qt, 'pittichess-noncli' would be a more apt name.

It is a bit like saying that micro-Max is 'Stockfish in plain C'.

[lucasart]

Some good progress was made on the project, as you can see on github: https://github.com/lucasart/zinc.git. Kudos to Andy for his great contributions.

Here's the gap-analysis compared to cutechess-cli:

• CLI: not implemented. just configure the parameters at the top of the script and run it.
• JSON: not implemented. not needed for now (no CLI).
• Variants: not implemented. will only support Chess960.
• Gaviota TB: not implemented. may add support for Syzygy.
• Tournaments: not implement. Only 2 engine matches for now.
• SPRT: not implemented. will implement.
• ratinginterval: not implemented. will implement.
• openings: only EPD, sequentially (repeated).
• pgnout: not implemented.
• recover: not implemented.
• repeat: the only behaviour available. EPD positions are always played twice, with colors reversed.
• restart: engines are never restarted. don't see any reason to restart engines, other than slowing things down, so won't support it.
• proto: only UCI. won't support CECP.
• timemargin: not implemented.
• book: not implemented.
• ponder: not implemented.

All the rest is available, and behaves the same was as in cutechess-cli, modulo bugs

Added Chess960 and PolyGlot opening book (weighted random selection up to book depth).

[lucasart]

I looked at the list you posted, and almost everything that makes cutichess-cli a useful tool seems to be amongst the 'not supported'...

So the title of the thread is a bit misleading. This isn't like cutichess any more than any tool to play simple matches between UCI engines. As it is based on Python rather than Qt, 'pittichess-noncli' would be a more apt name.

It is a bit like saying that micro-Max is 'Stockfish in plain C'.

It's going to replicate all the useful functionalities of cutechess-cli. Of course, it's not finished, but already you can see that development is several orders of magnitude faster than cutechess-cli; precisely because it's in Python, and I don't have to deal with all the bureaucracy of C++ / Qt to get the job done.

In my experience -- and I've used cutechess-cli a lot -- the CLI part of cutechess-cli adds zero value, in terms of usability. It's only necessary because C++ is a compiled language. There are so many parameters you need to put in cutechess-cli, that you end up having to write a shell script (and a JSON) around it, edit that script, and run it. That's exactly what you do with zinc: you edit the config at the top of the Python script and run it.

[mar]

It's going to replicate all the useful functionalities of cutechess-cli. Of course, it's not finished, but already you can see that development is several orders of magnitude faster than cutechess-cli; precisely because it's in Python, and I don't have to deal with all the bureaucracy of C++ / Qt to get the job done.

well, Python has the slowest interpreter I've ever seen. This is not about performance, sure (or is it? Pypy for the win!)
Plus you depend on python itself, a piece of bloat that has to load first to run your little script.
Why not Lua or something else that's lightweight (which you apparently like)? I guess because Python is cool.

So you've glued together some libs and made something that runs, congrats. How does it relate to the alleged "bureucracy" of C++ is beyond me. Have you just discovered what scripting languages are good for?
Also you don't need to depend on Qt at all to write something like cutechess-cli.

PS this nasty hack should prevent races? C'mon, you can do better

Code: Select all

            # HACK: We can't just test jobQueue.empty(), then run jobQueue.get(). Between both
            # operations, another process could steal a job from the queue. That's why we insert
            # some padding 'None' values at the end of the queue
            job = jobQueue.get()
            if job == None:
                return

[AndrewGrant]

Hey, I'm proud of that "nasty" hack! :)

[lucasart]

well, Python has the slowest interpreter I've ever seen. This is not about performance, sure (or is it? Pypy for the win!)
Plus you depend on python itself, a piece of bloat that has to load first to run your little script.

Have you measured it ? Have you even tried it ? (+same questions on PyPy).
I claim that you won't see a difference in any realistic testing scenario, because almost all the time is spend waiting for I/O operations.

Why not Lua or something else that's lightweight (which you apparently like)? I guess because Python is cool.

I don't know anything about Lua, so I can't comment on that. And yes, Python is cool.

So you've glued together some libs and made something that runs, congrats. How does it relate to the alleged "bureucracy" of C++ is beyond me. Have you just discovered what scripting languages are good for?

I never said it was hard. On the contrary, I'm saying it's easy, thanks to Python itself, and to the python-chess library of Niklas.

Also you don't need to depend on Qt at all to write something like cutechess-cli.

Agreed.

PS this nasty hack should prevent races? C'mon, you can do better

Talk is cheap, show us your code...