Engines for testing (Linux, fast time control)

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jdart
Posts: 3824
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Engines for testing (Linux, fast time control)

Post by jdart » Sun Nov 19, 2017 4:20 am

I have been using a few engines in the ballpark of Arasan in terms of rating, for testing my engine.

Periodically I revise the list as Arasan gets stronger, and as new engines appear.

I am only interested in Linux, because I moved most of my development to that years ago. I have only one Windows machine.

Also, I can only use engines that are stable at fast time controls, in particular ones that don't crash or hang, and don't lose on time.

It is getting so that is a fairly short list.

Ones I currently use include:

- Texel
- Stockfish (I use 2.2 but all the versions I have tried behave ok)
- Senpai (2.0, have used 1.0 previously)
- Crafty 25.2

I have previously used Toga 3 and Fruit 2.3.1, but those are a bit too weak for me now: they were nice and stable though.

Ones I have tried recently include:

- Pedone - loses on time pretty often. I reported this to the author but it has not been fixed.
- Laser - really didn't work on Linux, games terminate early under cutechess
- Nemorino - v 3 source release didn't compile, dev version does compile but looks like it tries to allocate >1 thread even when Threads option set to 1. Then it dies with a bunch of memory errors.
- Komodo 3 - this works
- Andscacs 0912 - distributed Linux binary does not work on my system. Also doesn't seem to compile on Linux with supplied Makefile.

All these were tested at 0:10+0.1 time control, single threaded, with cutechess-cli.

--Jon

elcabesa
Posts: 815
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: Engines for testing (Linux, fast time control)

Post by elcabesa » Sun Nov 19, 2017 6:29 am

you can try with Vajolet.
I usually test it 6''+0.05'' and doesn't get any time lost.

Xann
Posts: 125
Joined: Sat Jan 22, 2011 6:14 pm
Location: Lille, France

Re: Engines for testing (Linux, fast time control)

Post by Xann » Sun Nov 19, 2017 6:51 am

Hi Jon,
jdart wrote:All these were tested at 0:10+0.1 time control, single threaded, with cutechess-cli.
Senpai is not supposed to play that fast. You should probably change the in-line lag parameter (function time_lag in search.cpp). The default value is as much as your increment ...

Fabien.

User avatar
lucasart
Posts: 3040
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: Engines for testing (Linux, fast time control)

Post by lucasart » Sun Nov 19, 2017 7:45 am

You can try demolito: https://github.com/lucasart/Demolito.
Compiles from source easily on Linux. No problem with extreme time controls, can easily handle 2"+0.02" (at least on my machine). I don't know if it's strong enough though. Current dev version shouls be about on par with Vajolet or Wasp.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

jdart
Posts: 3824
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Engines for testing (Linux, fast time control)

Post by jdart » Sun Nov 19, 2017 3:09 pm

Thanks for the suggestion. I set the lag to 20ms and it seems to play better with that setting. But it was ok with the default, in the sense that it didn't lose on time.

--Jon

Florentino
Posts: 30
Joined: Tue Mar 25, 2014 9:34 pm
Contact:

Re: Engines for testing (Linux, fast time control)

Post by Florentino » Sun Nov 19, 2017 4:36 pm

- Nemorino - v 3 source release didn't compile, dev version does compile but looks like it tries to allocate >1 thread even when Threads option set to 1. Then it dies with a bunch of memory errors.
Hello Jon,
did you use Nemorino in xbpard or in uci mode? I only use uci and it's possible that xboard is broken at the moment.
I tested Nemorino in uci mode on a raspberry pi using raspbian as well as with Windows Subsytem for Linux (which is Ubuntu based) and had no such issues.
Best regards,
Christian

jdart
Posts: 3824
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Engines for testing (Linux, fast time control)

Post by jdart » Sun Nov 19, 2017 5:42 pm

It is using UCI mode.

I have this in the engines.json for cutechess-cli:

Code: Select all

             {
               "name": "Nemorino-3-dev",
               "command": "/home/jdart/chess/engines/nemorino-dev/nemorino",
               "protocol": "uci",
               "workingDirectory": "/home/jdart/chess/engines/nemorino-dev",
               "initStrings": [ "setoption name Hash value 128", "setoption name Threads value 1", "setoption name Ow\
nBook value false", "setoption name SyzygyPath value /home/jdart/chess/syzygy" ]
             },
Then I just started a bunch of matches in parallel (24, because it is a 24 core box). But what I see is the load average spikes above 100. The memory errors mentioned actually seem to be from cutechess-cli. So Nemorino seems to be consuming excessive resources. I do notice when I start it up it says the default Threads setting is 24. But cutechess should be overridding that.

--Jon

Florentino
Posts: 30
Joined: Tue Mar 25, 2014 9:34 pm
Contact:

Re: Engines for testing (Linux, fast time control)

Post by Florentino » Sun Nov 19, 2017 5:58 pm

Hello Jon,
as far as I know are the initStrings send to the engine before the "uci" command is issued. So Nemorino ignores it, as it's not yet in uci-mode.
I'm not sure if you can set UCI options via the engines.json file. I'm always adding something like

Code: Select all

-each option.Threads=1
to my cutechess command line.
Christian

jdart
Posts: 3824
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Engines for testing (Linux, fast time control)

Post by jdart » Sun Nov 19, 2017 7:02 pm

That seems to work, thanks.

--Jon

jwes
Posts: 778
Joined: Sat Jul 01, 2006 5:11 am

Re: Engines for testing (Linux, fast time control)

Post by jwes » Sun Nov 19, 2017 9:32 pm

One idea is to use top engines at time odds. This has the benefit of making matches go quicker. For my engine (which is much weaker than Arasan), it also exposes evaluation problems in particular as at time odds of several hundred to one, my engine can outsearch them.

Post Reply