cutechess-cli 0.7.0

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.7.0

Post by ilari » Sat Aug 29, 2015 8:21 pm

Long time no see, TalkChess! Yes, we had a little hackathon with Arto, which mostly consisted of debugging the Windows version. And we decided to release cutechess-cli 0.7.0 with the following changes (compared to 0.6.0):

* Fix EPD indexing performance on Windows
* Fix Chess 960 castling bug
* Use QElapsedTimer for time control (more accurate timing)
* Print SPRT information along current ratings
* Use 64-bit integers to represent node counts
* Fix alignment and padding of engine names in tournament ratings
* Fix SEGFAUL or SEGILL when an engine process is stopped and deleted

The SEGFAULT/SEGILL problems would only occur under certain circumstances so it took a lot of debugging to reproduce and properly fix them.

Up-to-date binaries are available at https://github.com/cutechess/cutechess
The binaries were just updated today so if you downloaded them before today you do not have the latest version. Running "cutechess-cli --version" will tell which version you have.

Best Regards,
Ilari

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.7.0

Post by ilari » Sat Aug 29, 2015 8:31 pm

Oh, and just like Matthias said, the 32-bit Windows version works fine with 64-bit engines. It wouldn't really add much value to have a 64-bit version on Windows.

Ferdy
Posts: 4077
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: cutechess-cli 0.7.0

Post by Ferdy » Sat Aug 29, 2015 8:49 pm

ilari wrote:Long time no see, TalkChess! Yes, we had a little hackathon with Arto, which mostly consisted of debugging the Windows version. And we decided to release cutechess-cli 0.7.0 with the following changes (compared to 0.6.0):

* Fix EPD indexing performance on Windows
* Fix Chess 960 castling bug
* Use QElapsedTimer for time control (more accurate timing)
* Print SPRT information along current ratings
* Use 64-bit integers to represent node counts
* Fix alignment and padding of engine names in tournament ratings
* Fix SEGFAUL or SEGILL when an engine process is stopped and deleted

The SEGFAULT/SEGILL problems would only occur under certain circumstances so it took a lot of debugging to reproduce and properly fix them.

Up-to-date binaries are available at https://github.com/cutechess/cutechess
The binaries were just updated today so if you downloaded them before today you do not have the latest version. Running "cutechess-cli --version" will tell which version you have.

Best Regards,
Ilari
Thanks for this update.

I did manage to make a compile for windows. Downloading the almost 1GB installation file of QT 5.5 and mingw 4.9.2 :).
Running test now, so far I have not seen a message,

Code: Select all

Warning: EngineProcess: pipe reader was terminated
This is from the cutechess that I compile.

When I run this tour batch file with 0.6.0 I get the message,

Code: Select all

Warning: EngineProcess: pipe reader was terminated
From the link,
Binaries are available for cutechess-cli, the command-line interface of Cute Chess. The latest version is 0.6.0.
Better to change that to 0.7.0.

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.7.0

Post by ilari » Sat Aug 29, 2015 9:06 pm

Ferdy wrote:Running test now, so far I have not seen a message,

Code: Select all

Warning: EngineProcess: pipe reader was terminated
This is from the cutechess that I compile.

When I run this tour batch file with 0.6.0 I get the message,

Code: Select all

Warning: EngineProcess: pipe reader was terminated
Good to hear. I was hoping that the pipe reader warning would get fixed as a result of the SEGFAULT/SEGILL fix. The problem was a WINAPI thread handle that was left open needlessly. Let me know if you still get that warning in some of your tests.
Ferdy wrote:From the link,
Binaries are available for cutechess-cli, the command-line interface of Cute Chess. The latest version is 0.6.0.
Better to change that to 0.7.0.
Oh, well spotted. Fixed!

User avatar
cdani
Posts: 2104
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: cutechess-cli 0.7.0

Post by cdani » Sat Aug 29, 2015 9:26 pm

ilari wrote:Long time no see, TalkChess! Yes, we had a little hackathon with Arto, which mostly consisted of debugging the Windows version. And we decided to release cutechess-cli 0.7.0 with the following changes (compared to 0.6.0):

* Fix EPD indexing performance on Windows
* Fix Chess 960 castling bug
* Use QElapsedTimer for time control (more accurate timing)
* Print SPRT information along current ratings
* Use 64-bit integers to represent node counts
* Fix alignment and padding of engine names in tournament ratings
* Fix SEGFAUL or SEGILL when an engine process is stopped and deleted

The SEGFAULT/SEGILL problems would only occur under certain circumstances so it took a lot of debugging to reproduce and properly fix them.

Up-to-date binaries are available at https://github.com/cutechess/cutechess
The binaries were just updated today so if you downloaded them before today you do not have the latest version. Running "cutechess-cli --version" will tell which version you have.

Best Regards,
Ilari
I have no enough ways of thank you for cutechess-cli :-)
So simply thanks!!

Ferdy
Posts: 4077
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: cutechess-cli 0.7.0

Post by Ferdy » Sat Aug 29, 2015 9:57 pm

It works just fine, Deuterium gauntlet at tc 5s + 100ms inc, no time forfeit, no warnings.

Code: Select all

   # PLAYER            : RATING  ERROR   POINTS  PLAYED    (%)   CFS(next)
   1 Murka_3_64BIT     :   61.5   65.8     42.5      64   66.4%      66
   2 Spark 1.0 x64     :   37.0   62.7     40.5      64   63.3%      93
   3 Quazar_0.4_x64    :  -38.0   55.8     34.0      64   53.1%      75
   4 D2015.1.35.247    :  -60.4   29.6     75.0     192   39.1%     ---

White advantage = 50.95 +/- 23.84
Draw rate (equal opponents) = 27.60 % +/- 3.38
During compilation, I also got the gui. I tried the tournament, does this support concurrency? I tried to find it but could not.

Image

Ferdy
Posts: 4077
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: cutechess-cli 0.7.0

Post by Ferdy » Sat Aug 29, 2015 11:20 pm

ilari wrote:Let me know if you still get that warning in some of your tests.
On a very fast tc of 1s + 50ms inc, I get this,
Started game 306 of 600 (Andscacs 0.72p vs D2015.1.35.247)
Started game 305 of 600 (D2015.1.35.247 vs Andscacs 0.72p)
Started game 307 of 600 (D2015.1.35.247 vs Andscacs 0.72p)
Warning: Can't write to PGN file .\games\g-D2015.1.35.247-t2-tc1s50ms-pgn.pgn
Warning: Can't write to PGN file .\games\g-D2015.1.35.247-t2-tc1s50ms-pgn.pgn
Warning: Can't write to PGN file .\games\g-D2015.1.35.247-t2-tc1s50ms-pgn.pgn
Finished game 301 (D2015.1.35.247 vs DiscoCheck 5.2.1): 0-1 {Black wins by adjud
ication}
Warning: Can't write to PGN file .\games\g-D2015.1.35.247-t2-tc1s50ms-pgn.pgn
Finished game 304 (DiscoCheck 5.2.1 vs D2015.1.35.247): 1-0 {White wins by adjud
ication}
Finished game 307 (D2015.1.35.247 vs Andscacs 0.72p): 0-1 {Black wins by adjudic
ation}
I don't use this kind of tc normally, just tried it in this new version of cutechess.

Ferdy
Posts: 4077
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: cutechess-cli 0.7.0

Post by Ferdy » Sun Aug 30, 2015 7:16 am

This is regarding warnings, from observation of the console output, the message,

Code: Select all

Warning: Can't write to PGN file ....
will show when one engine stalls, or something when engine loses on time.
Setup gauntlet, using epd file at random, tc 5s+100ms inc, concurrency is 4 on 8 physical cores cpu. Before those loses on time termination, there is no warning messages on "Can't write ...".

Code: Select all

[...]
Started game 150 of 900 (Arasan_18_x64_popcnt vs D2015.1.35.247)
Finished game 148 (SmarThink v1.7 vs D2015.1.35.247): 1-0 {White wins by adjudication}
Started game 151 of 900 (D2015.1.35.247 vs DiscoCheck 5.2.1)
Finished game 145 (D2015.1.35.247 vs Gaviota v1.0 64bit): 1-0 {Black's connection stalls}
Finished game 149 (D2015.1.35.247 vs Arasan_18_x64_popcnt): 1-0 {Black loses on time}
Finished game 150 (Arasan_18_x64_popcnt vs D2015.1.35.247): 0-1 {White loses on time}
Started game 152 of 900 (DiscoCheck 5.2.1 vs D2015.1.35.247)
Started game 154 of 900 (Hakkapeliitta 3.0 x64 vs D2015.1.35.247)
Started game 153 of 900 (D2015.1.35.247 vs Hakkapeliitta 3.0 x64)
Started game 156 of 900 (DiscoCheck 5.2.1 vs D2015.1.35.247)
Started game 155 of 900 (D2015.1.35.247 vs DiscoCheck 5.2.1)
Warning: Can't write to PGN file .\games\g-D2015.1.35.247-t1-tc-5splus100ms-epd.pgn
Finished game 151 (D2015.1.35.247 vs DiscoCheck 5.2.1): 0-1 {Black wins by adjudication}
Warning: Can't write to PGN file .\games\g-D2015.1.35.247-t1-tc-5splus100ms-epd.pgn
Finished game 152 (DiscoCheck 5.2.1 vs D2015.1.35.247): 1-0 {White wins by adjudication}
[...]
Even when there is "Can't write..." warning and after the gauntlet, cutechess will report complete game stats, in a summarized table, but in the final pgn output, there are missing games.

BTW indexing on 3.5 million epd lines is fast, less than 1 sec only on i7-2600k cpu, win 7.

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.7.0

Post by ilari » Sun Aug 30, 2015 7:20 am

Ferdy wrote:During compilation, I also got the gui. I tried the tournament, does this support concurrency? I tried to find it but could not.
The GUI is still at a pre-alpha stage and there's no option for changing the concurrency setting. In the source code you can change the default from 1 to whatever value you wish (gamemanager.cpp, line #334).

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.7.0

Post by ilari » Sun Aug 30, 2015 7:35 am

Ferdy wrote:This is regarding warnings, from observation of the console output, the message,

Code: Select all

Warning: Can't write to PGN file ....
will show when one engine stalls, or something when engine loses on time.
The "Can't write to PGN file" warning can be caused by two things:

1. The game doesn't have any tags set
2. The file cannot be opened

I'll see if I can reproduce the problem but in case I can't, could you change the PgnGame::write function so that it would output the reason why the writing won't happen? The function is at pgngame.cpp on line #312. The new function should look like this:

Code: Select all

bool PgnGame::write(const QString& filename, PgnMode mode) const
{
	if (m_tags.isEmpty())
	{
		qWarning("Tags are missing from the game");
		return false;
	}

	QFile file(filename);
	if (!file.open(QIODevice::WriteOnly | QIODevice::Append))
	{
		qWarning("PGN file cannot be opened");
		return false;
	}

	QTextStream out(&file);
	write(out, mode);
	return true;
}

Post Reply