ScidVsPC/Mac

[stevenaaus]

Indeed. This is a (pretty bad) protocol violation, so don't bother about it.

Ok. Thanks a lot
The other engines all seem ok regards to this, so until i run into more issues i'll leave it alone.

Fixed a couple of bugs with previous commit. Glaurung sometimes doesn't issue a ponder, so handle this case, and make it work with UCI vs Xboard.

It goes ok - but some anomalies - Umko-1.2 can't get through a ponder game without crashing. Arasan 13.4 halts after a couple of moves too, and redqueen segfaults and gives a kernel message! [75235.002883] redqueen[22497]: segfault at bf9d0000 ip 080a7fbc sp 4821c460 error 6 in redqueen[8048000+80000] Not a bad achievement.

[JuLieN]

Hello Steve,

I just download the new svn trunk and compiled it, and ScidVsMac seems to behave much better: no more interface hangs or engine staying in memory.

Still, this is not totally perfect:

- as a matter of test, I tried to launch a 100-engines tournament: ScidVsMac hanged with a 100% CPU use. After 2-3 minutes of waiting, I killed its task.
- then I launched a 15-engines tournament and it started well. But something occurred to me: as now we can't change the engines starting with position 11 and up, breaking my 100+-engines tournament into smaller 15-tournaments is not possible anymore, as the engines at rank 11-15 will always be the same.
- so I decided to only launch 10-engines sub-tournaments, and up to now it's doing fine... With one small problem still: when an engine hangs or crash, ScidVsMake won't auto flag it. So if we aren't next to the computer to click on "white/black wins", the computer will stay all night long like that with a clock in the red and increasing... Do you think you could add the auto flag option? (Maybe you did and I missed it?)

Still, it looks much more better! Thanks for those improvements!

EDIT: hmm... I just saw a new problem, in an Alibaba<>BACE game: BACE tried to promote a pawn, and Scid just displays an "Error reading move: e2e1" message... (Probably because BACE didn't precise any promoted piece. Easy fix: always assume a queen when the engine says nothing) now the clock is in the red and counting...

[hgm]

Not specifying a promotion piece is a violation of WB protocol, but XBoard has always been very forgiving towards all kind of protocol violations (bad policy?), so indeed it uses the rule you mention. Thus inspiring laziness into engine developers...

I must say, however, that in variants without promotion choice, (such as Shatranj), it does seem a bit silly to require a promotion suffix.

[stevenaaus]

I just download the new svn trunk and compiled it, and ScidVsMac seems to behave much better: no more interface hangs or engine staying in memory.

Sweet.

- as a matter of test, I tried to launch a 100-engines tournament: ScidVsMac hanged with a 100% CPU use. After 2-3 minutes of waiting, I killed its task.

Haha. I have imporved this, but 100 still seems a lot.
I have just played a 45 game tournament with "First engine plays others". About 4 engine failures, but it completed fine.

BTW, i hope you are clicking first engine plays others, otherwise a 100 game tournament has

Code: Select all

#tclsh
% proc fact {n} {
if {$n==1} {return 1}
return [expr $n * [fact [incr n -1]]]
}
% fact 100
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000

games! Which may just crash scid

- then I launched a 15-engines tournament and it started well. But something occurred to me: as now we can't change the engines starting with position 11 and up

You may still do this by rearranging the engine order in Options->Engines

With one small problem still: when an engine hangs or crash, ScidVsMake won't auto flag it.

I think i fixed this tonight. In my first implementation, i had no way of catching a dead engine, but now it is simple.

So best practice for a big tournament is...
Enforce a sensible engine time-out period.
Hide game info widget. It uses slight resources, and though i have attempted to fix the autoscroll bug, i'm not confident about it.
Cross fingers!

EDIT: hmm... I just saw a new problem, in an Alibaba<>BACE game: BACE tried to promote a pawn, and Scid just displays an "Error reading move: e2e1" message

This is a poor engine feature. Could you please test this engine and see if it eventually triggers a Game time-out ?
(And should I mark timed-out games as losses (like i have with crashed engine), or leave them as no result ?)

[stevenaaus]

I just gave Strelka 5 a chance at fame. It seems to be working ok (after my earlier reported anomaly), and is taking everyone in it's stride with ponder.

5 engines hung, red queen really crashed nasty (great for testing actually), but the torunament kills it eventually and continues, and Strelka has beaten everyone except Iggorit, and also loses on time to Scorpio (though with a winning position. Scorpio is a bit of a beast actually).

What this program can do is a pretty amazing for an interpreted language.

Code: Select all

Strelka 5

43 games	2012.03.12

                    Won   Drawn   Lost
Results for All games
  White  88.37%   +  35  =   0  -   2    38.0 /  43
  Black   0.00%   +   0  =   0  -   0     0.0 /   0
  Total  88.37%   +  35  =   0  -   2    38.0 /  43

Hmmm - one game didn't get the tags saved properly - though White was still the winner.

[JuLieN]

I just gave Strelka 5 a chance at fame. It seems to be working ok (after my earlier reported anomaly), and is taking everyone in it's stride with ponder.

5 engines hung, red queen really crashed nasty (great for testing actually), but the torunament kills it eventually and continues, and Strelka has beaten everyone except Iggorit, and also loss on time to Scorpio (though with a winning position. Scorpio is a bit of a beast actually).

What this program can do is a pretty amazing for an interpreted language.

Code: Select all

Strelka 5

43 games	2012.03.12

                    Won   Drawn   Lost
Results for All games
  White  88.37%   +  35  =   0  -   2    38.0 /  43
  Black   0.00%   +   0  =   0  -   0     0.0 /   0
  Total  88.37%   +  35  =   0  -   2    38.0 /  43

The first batch of ten engines just finished its tournament, and 4 engines out of ten just didn't produce any move: Belofte, Beowulf, Buzz and Cassandre, so a mix of wb and uci engines, ruling out a protocol problem. those engines work well in a shell... I'll post the result in the tournaments forum and launch the second tournament.

[stevenaaus]

Thanks for testing J. Re engine failures - i'm not what to do.

If you care to , please examine the engine logs and send me any suggestions. I'm no authority of xboard protocol. Hopefully UCI support should be solid.

[hgm]

Code: Select all

#tclsh
% proc fact {n} {
if {$n==1} {return 1}
return [expr $n * [fact [incr n -1]]]
}
% fact 100
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000

This is an exaggeration. And N-player round-robin only requires N*(N-1)/2 games, not N!. Even with 2 games per pairing and 100 players this would take only 9900 games. A long wait, for sure, but it should not crash SCID.

[stevenaaus]

Yes - very my bad
Here's my best 32 bit engines at 12 seconds + 0

Code: Select all

Ponder
?, 2012.03.13
                     Score        1    2    3    4    5    6
-------------------------------------------------------------
 1: Strelka 5      15.5 / 20   XXXX 0110 1110 11== 1=11 1111
 2: Stockfish 2.2  12.5 / 20   1001 XXXX 1000 ==11 1011 111=
 3: Scorpio        11.5 / 20   0001 0111 XXXX 1=11 =010 1=01
 4: Critter         9.5 / 20   00== ==00 0=00 XXXX 1111 1011
 5: Komodo 2.03     5.5 / 20   0=00 0100 =101 0000 XXXX 0=01
 6: Robbo           5.5 / 20   0000 000= 0=10 0100 1=10 XXXX
-------------------------------------------------------------
60 games: +30 =10 -20

[JuLieN]

Code: Select all

#tclsh
% proc fact {n} {
if {$n==1} {return 1}
return [expr $n * [fact [incr n -1]]]
}
% fact 100
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000

This is an exaggeration. And N-player round-robin only requires N*(N-1)/2 games, not N!. Even with 2 games per pairing and 100 players this would take only 9900 games. A long wait, for sure, but it should not crash SCID.

Exactly my own calculation... So I didn't understood where Stevens got his figures.

A 100 engines round-robin tournament with 5mn blitzes (5x2=10mn at most) with two rounds between two engines (one as black and one as whiteà takes at most:

(100x99) / 2 * 2 * 10 = 68 days and 18 hours,

as yet stated:

I'm about to launch a big (>100 engines)

A hundred! What time control are you thinking ?
Anyway, I'll try to look at it this weekend - but it may be complicated.

In the beginning I thought about 5mn blitzes... so 10min/game... with 2 games between two engines (one with black, one with white) with a computed duration of ((100x99) / 2) * 2 * 10 = 69 days....

Maybe a bit too long, especially if one can't stop and restart a tournament (another suggestion of improvement btw ). So breaking the tournament into 7 smaller tournaments of 15 engines and then promoting the 5 best engines of each tournament is a better idea (the first 7 smaller tournaments will "only" take 70 hours). I just want to produce a crude estimation of the strength of those engines for the moment.