CuteChess-cli "...connection stalls"

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
Steve Maughan
Posts: 1061
Joined: Wed Mar 08, 2006 7:28 pm
Location: Florida, USA
Contact:

CuteChess-cli "...connection stalls"

Post by Steve Maughan » Wed Aug 28, 2013 1:24 pm

Maverick's UCI interface code seems to be stable. I'm not getting reports of crashes and when you play with the engine in analysis mode I can zip back and forth through the game, while its thinking, with no problem. However, when I stress-test it using CuteChess-cli and 8 concurrent games (on a 4 core + HT Intel i7) Maverick falls over and CuteChess-cli reports "Black's connection stalls" and finishes the test. I'm using the ultra fast "tc=40/2+0.01" time control!

Maverick uses one thread to "think" and the other to capture the input. So in the 8 concurrent game scenario there are really 16 threads spinning - I don't know if this is a problem. When I reduce the concurrent games to 4 it seems OK - strange!

I also set the buffers:

Code: Select all

    setbuf(stdout, NULL);
    setbuf(stdin, NULL);
    setvbuf(stdout, NULL, _IONBF, 0);
    setvbuf(stdin, NULL, _IONBF, 0);
Could this be a problem - are the buffers being overloaded and information lost?

I'm assuming this is a problem with Maverick (has anyone else had this problem with CuteChess-cli). The most obvious possibility is a rare race condition between the two threads. But it's difficult to debug as I cannot even write to a text file when there are 8 concurrent games going on.

Any thoughts or suggestions would be appreciated!

Thanks,

Steve
http://www.chessprogramming.net - Maverick Chess Engine

tpetzke
Posts: 686
Joined: Thu Mar 03, 2011 3:57 pm
Location: Germany
Contact:

Re: CuteChess-cli "...connection stalls"

Post by tpetzke » Wed Aug 28, 2013 1:38 pm

Hi Steve,

I recommend to use only as much threads as you have real cores, so limit it to 4 in your setup.

Thomas...
Thomas...

=======
http://macechess.blogspot.com - iCE Chess Engine

tvrzsky
Posts: 128
Joined: Sat Sep 23, 2006 5:10 pm
Location: Prague

Re: CuteChess-cli "...connection stalls"

Post by tvrzsky » Fri Aug 30, 2013 3:11 am

Hi Steve,
it seems that I faced some similar problems (buffering issues, synchronization of input, output and working threads, instability of Cutechess). Maybe I could give you few hints.
From your other post I deduce you are running Windows. My expierence with C runtime library and output buffering was bad. With MinGW compiler I was not able to find buffer settings which would be stable and reliable. Many experiments, using fflush(), nothing worked. Always there were broken output lines (I mean lines which reached other side of the pipe broken to more parts). I noticed this also with other engines which rely on C output functions (for example Cheng3). This was big problem when I worked on my own tournament manager because parsing output of those engines was complicated and generated big overhead (5-7 % of CPU time contrary to good behaving ones, where overhead was almost negligible, 0,1-0,2 %). Only solution which worked rock solid for me was using of Windows WriteFile() function for output instead of fwrite() and this stuff.
For debugging was wery usefull to switch on debug logging in tournament manager (my one has almost the same as Cutechess) and add notification of various events in your engine to its stdout, which allows somewhat trace synchronization issues. For very detailed monitoring of threads and timing issues was very usefull to work with Perfmon utility from Microsoft. It shows very clearly that with such fast time controls where you practically have no more than tens of milliseconds per move and are running all CPU cores then stealing of just one time slice from your engine thread by some system process can lead to time forfeit, and you can do very little to prevent this, maybe except of toying with process or thread priorities which is not good solution indeed.
Last but not least I was also unlucky with Cutechess which was in my test setup crashing too often to be usable. I am really ashamed that I did not report this to Cutechess team in time in order to help in development but I was lazy and shy to do it because no one else reported any problems with it. Finally my solution was creation of my own manager. However I do not want to defame Cutechess in any way, it is an excellent tool.
Last edited by tvrzsky on Fri Aug 30, 2013 3:25 am, edited 1 time in total.

User avatar
Steve Maughan
Posts: 1061
Joined: Wed Mar 08, 2006 7:28 pm
Location: Florida, USA
Contact:

Re: CuteChess-cli "...connection stalls"

Post by Steve Maughan » Fri Aug 30, 2013 3:24 am

Thanks Thomas & Filip!

Interesting stuff Filip. I'll look into WriteFIle(). Maybe I should switch to Microsoft's compiler and see if it helps. Maybe it's the depth command for CuteChess.

Thanks,

Steve
http://www.chessprogramming.net - Maverick Chess Engine

tvrzsky
Posts: 128
Joined: Sat Sep 23, 2006 5:10 pm
Location: Prague

Re: CuteChess-cli "...connection stalls"

Post by tvrzsky » Fri Aug 30, 2013 3:40 am

Ooops, my bad, I did not mean Perfmon but Xperf.
Filip

User avatar
hgm
Posts: 23790
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: CuteChess-cli "...connection stalls"

Post by hgm » Fri Aug 30, 2013 6:48 am

Just out of curiosity: do you have similar problems when you run WinBoard (or XBoard) in -noGUI mode?

User avatar
Steve Maughan
Posts: 1061
Joined: Wed Mar 08, 2006 7:28 pm
Location: Florida, USA
Contact:

Re: CuteChess-cli "...connection stalls"

Post by Steve Maughan » Fri Aug 30, 2013 11:02 am

Hi H.G, - I've never used Windboard in -noGUI mode. I'll download and give it a try.

Thanks,

Steve
http://www.chessprogramming.net - Maverick Chess Engine

Post Reply