Sjaak II - beta release!

[Evert]

I have uploaded a beta for the new version of my general variant engine Sjaak (it identifies as Sjaak II). This is almost a complete re-write, although it reuses some of the code of the previous version, as well as some code Jazz (mainly the interface driver).

I have used a few ideas I saw in Senpai by Fabien Letouzey, in particular replacing the quiescence search by a static evaluation under certain conditions, making the evaluation colour-symmetric by looping over black and white and negating the eval terms at the end of the loop and the form of the king-safety term (I suspect that other programs use or used similar constructs, that's just where I saw them myself).

In terms of playing strength, in my testing this version is much stronger than the last release (524), but weaker than Jazz (but not hopelessly so, it stands a fair chance). If I were asked to put a number on it, I would guestimate a strength of ~2200 Elo on CCRL 40/40 scale, but take it with a healthy portion of salt since my main testing has been against the last version of Sjaak and Jazz. The strength difference probably also depends on the variant.

I'll give some technical details further down, for those who are interested.

What's new?

• Sjaak now understands both CECP and UCI protocol (also USI and UCCI, but those are not tested very well). For variants, UCI is not very useful because the GUI will probably not understand the rules for different variants (with the exception of Chess960), but you can select any variant in UCI mode anyway.
• Uses features that are new in XBoard 4.8, for the purpose of defining engine defined variants.
• Many new variants, including: Shatar (traditional Mongolian chess), Sittuyin (Burmese chess), Crazyhouse, Chessgi, Ai-Wok, King-of-the-hill, Los Alamos (6x6 mini-chess) and different Shogi variants (Japanese chess, it plays normal Shogi, mini Shogi, Sho Shogi and Tori Shogi).
• Improved terminal interface, for playing without a GUI. Sjaak also has a "rules" command that will display the rules of the current variant, including the moves of different pieces.

The full list of build-in variants is

Code: Select all

 'chess'         (8x8)
 'shatar'        (8x8)
 'makruk'        (8x8)
 'shatranj'      (8x8)
 'sittuyin'      (8x8+6)
 'crazyhouse'    (8x8+6)
 'chessgi'       (8x8+6)
 'asean'         (8x8)
 'ai-wok'        (8x8)
 'spartan'       (8x8)
 'pocketknight'  (8x8+1)
 'kingofthehill' (8x8)
 'knightmate'    (8x8)
 'amazon'        (8x8)
 'chancellor'    (8x8)
 'berolina'      (8x8)
 'losalamos'     (6x6)
 'micro'         (5x5)
 'capablanca'    (10x8)
 'gothic'        (10x8)
 'embassy'       (10x8)
 'courier'       (12x8)
 'grand'         (10x10)
 'opulent'       (10x10)
 'minishogi'     (5x5+5)
 'shoshogi'      (9x9)
 'shogi'         (9x9+8)
 'torishogi'     (7x7+6)
 'xiangqi'       (9x10)

Notes about specific variants
This mostly refers to running under X/WinBoard:

• Shatar must be played with "legality testing" switched off. The reason is that XBoard is unaware of some of the special rules for this variant (a knight may not give mate, a sequence of checks leading to mate must include at least one check by rook, queen or knight)
• King-of-the-hill must also be played with legality testing off, for similar reasons.
• Sho Shogi may need to be played with legality testing off, depending on you Win/XBoard version. The initial release of XBoard 4.8 had a bug that made it fail to recognise the promoted Elephant as a second king.
• Sittuyin must be played with legality testing off. Even then, black must begin the game by placing his king (other moves are rejected as invalid). There are two work-arounds: 1) start from a pre-defined position that already has the king in place, or 2) disable the option "Send 'piece' descriptions" in the engine options dialog
• You can re-define the meaning of a particular variant name. For instance, Fairy-Max has an option to play Makruk using Ai-Wok rules. In Sjaak, you can set "makruk=ai-wok" will cause it to play Ai-Wok when asked to play makruk.

Let me know about any other issues that show up!

What's missing?
Or: why is this release called a "beta"?

• Seirawan chess
• The command-line referee program Sjef has not yet been updated
• USI and UCCI modes have not been tested very well
• Windows port hasn't been tested other than using the build-in move generator tests (which it passes), so it should be ok, but there may be unforeseen issues. It also hasn't been tested on an actual Windows system
• The "rules" command is a bit spotty in places
• Piece descriptions for custom variants need to be improved so they can be specified in Betza notation.

Technical
I mentioned this is almost a complete re-write. In particular, the move generator was largely re-written from scratch, as were the search and evaluation. The reason for this is code maintainability: the old version was written in C, and used separate code-paths for small (64 square) and large (>64 square) boards, which duplicated a lot of code and had branches all over the place. The current version uses C++ templates instead.

The code is a bit of a Frankenstein monster in terms of style though, mixing C, C-style-C++ and cleanish C++ code. This is a result of re-using some of the existing code, and I will clean this up gradually over time.

This mixed heritage means that in order to compile the code, you'll need a compiler that can deal with C99 as well as C++ code. Clang and GCC handle this well (on OS X and Linux, also cross-compiling for Windows), but I don't know about other compilers.

Getting it
It's at http://www.eglebbk.dds.nl/program/chess-download.html
Note that currently I only have 64 bit binaries. I'll update the archive with 32 bit ones (at least for Windows and Linux) after I've made sure those work properly.

Please let me know if and how it runs. Tips, suggestions and feed-back welcome!

[hgm]

Beware that the rules of ASEAN Chess seem to have changed from the original 2011 proposal: rather than automatically promoting to the worthless Ferz-Queen, Pawns now have full promotion choice. Which in practice means you would promote to Rook.

[Evert]

Beware that the rules of ASEAN Chess seem to have changed from the original 2011 proposal: rather than automatically promoting to the worthless Ferz-Queen, Pawns now have full promotion choice. Which in practice means you would promote to Rook.

Ah, thanks! I hadn't realised this.

Not sure what to think of that change though. On the one hand, it fixes one of the most glaring defects in the game (promotion on 8th rank to a piece barely worth more than the original pawn) but it makes the Ferz an even more obviously useless piece, to the point where you have to wonder why it's even in the game. It also moves the game further away from a "synthesis of SE-Asian chess" (but then again it was barely that to begin with).

In Sittuyin pawns promote on the main diagonals (to Ferz), but you can only ever have one Ferz. To make it a bit more interesting, you can also promote a pawn that is on the diagonal by just replacing it. It's apparently not uncommon to simply forego promotion entirely even if it's possible because the pay-off is so minimal.

[hgm]

I only heard about this change a few days ago, receiving an e-mail from Thailand requesting to make Fairy-Max promote to Rook. It will certainly alter the character of the game very drastically. And it is a pity when you can only have one Ferz, because end-games with several Ferz can be interesting. E.g. KFFFK or KNFFK with like Ferz. Have you tried if Sjaak can win those?

Did you manage to register Sjaak for the UEC Cup? I already sent a WinBoard package with Shokidoki, Lima and TJshogi to Takeshi Ito, but I didn't have a Sjaak binary to include at the time.

Does the Sjaak license allow redistribution? Since it can play mini-Shogi, I would be interested to include it as a 7th engine in the WinBoard mini-Shogi package I distribute. (Next to Shokidoki, Nebiyu, Lima, Deep Nikita, GNU miniShogi and a link to Petit Shogi.)

[Edit] Oh, and one suggestion for an improvement: could you make an option to suppress sending of fail-low and fail-high PVs. Preferably one that suppresses this by default. For 99.9% of the users these are very annoying.

[Evert]

I only heard about this change a few days ago, receiving an e-mail from Thailand requesting to make Fairy-Max promote to Rook. It will certainly alter the character of the game very drastically.

I'm having trouble finding any sort of on-line reference for the rules that lists this change. Maybe I'm not picking the correct keywords for google.

And it is a pity when you can only have one Ferz, because end-games with several Ferz can be interesting. E.g. KFFFK or KNFFK with like Ferz. Have you tried if Sjaak can win those?

Not at the moment.
The development version actually doesn't have any "mop-up" evaluation and can't handle KBNK either. It will find KFFFK if it has enough time to search deep and by random chance manages to drive the defending king from the centre. Otherwise it just shuffles its pieces around until 50 moves have passed. It's on the list.
The old version knows to drive the king to a corner, but it just picks whichever one it can drive it to first.

Did you manage to register Sjaak for the UEC Cup? I already sent a WinBoard package with Shokidoki, Lima and TJshogi to Takeshi Ito, but I didn't have a Sjaak binary to include at the time.

Oh, damn. I meant to, but it slipped my mind. I'll check if there's any possibility at all at this stage.

Does the Sjaak license allow redistribution? Since it can play mini-Shogi, I would be interested to include it as a 7th engine in the WinBoard mini-Shogi package I distribute. (Next to Shokidoki, Nebiyu, Lima, Deep Nikita, GNU miniShogi and a link to Petit Shogi.)

Sure. It's GPL.

EDIT

Oh, and one suggestion for an improvement: could you make an option to suppress sending of fail-low and fail-high PVs. Preferably one that suppresses this by default. For 99.9% of the users these are very annoying.

Sure.

[Evert]

I finally found a reference for the updated ASEAN rules, with the new promotion rule: http://www.ncf-phil.org/downloadables/2 ... Nov_26.doc

There seem to be some other minor changes as well (mainly with respect to counting rules).

[Evert]

I have updated the download link for beta 2 (still at http://www.eglebbk.dds.nl/program/chess-download.html). Changes in this version:

• Several fixes/updates to make compiling in 32 bit mode work correctly
• Update the rules of ASEAN chess to the November 2011 amended version rather than the original March 2011 version.
• Add options to show/suppress fail high/fail low in XBoard.

The archive now contains 32 bit binaries for all platforms, with SSE3 or SSE 4.2, as appropriate. Once again, please let me know about any issues.

[hgm]

I tried to build SjaakII on Ubuntu 10.4, but after cmake .. , make I get the error message:

Code: Select all

hgm@hgm-xboard:~/SjaakII/Build$ make
Scanning dependencies of target libsjaak
[  6%] Building C object CMakeFiles/libsjaak.dir/src/misc/aligned_malloc.o
[ 13%] Building C object CMakeFiles/libsjaak.dir/src/misc/cfgpath.o
[ 20%] Building C object CMakeFiles/libsjaak.dir/src/misc/genrand.o
[ 26%] Building C object CMakeFiles/libsjaak.dir/src/misc/keypressed.o
[ 33%] Building C object CMakeFiles/libsjaak.dir/src/misc/softexp.o
[ 40%] Building CXX object CMakeFiles/libsjaak.dir/src/eval/pst.o
[ 46%] Building CXX object CMakeFiles/libsjaak.dir/src/rules/game.o
[ 53%] Building CXX object CMakeFiles/libsjaak.dir/src/rules/move.o
[ 60%] Building CXX object CMakeFiles/libsjaak.dir/src/rules/san.o
[ 66%] Building CXX object CMakeFiles/libsjaak.dir/src/rules/squares.o
[ 73%] Building C object CMakeFiles/libsjaak.dir/src/hash/hashkey.o
[ 80%] Building C object CMakeFiles/libsjaak.dir/src/hash/hashtable.o
[ 86%] Building C object CMakeFiles/libsjaak.dir/src/hash/evalhash.o
[ 93%] Building C object CMakeFiles/libsjaak.dir/src/timer/timer.o
Linking CXX static library liblibsjaak.a
[ 93%] Built target libsjaak
Scanning dependencies of target sjaakii
[100%] Building CXX object CMakeFiles/sjaakii.dir/src/xboard.o
Linking CXX executable sjaakii
liblibsjaak.a(timer.o): In function `get_timer':
/home/hgm/SjaakII/src/timer/timer.c:35: undefined reference to `clock_gettime'
collect2: ld returned 1 exit status
make[2]: *** [sjaakii] Error 1
make[1]: *** [CMakeFiles/sjaakii.dir/all] Error 2
make: *** [all] Error 2

I have also some request about the build process; I don't know cmake, which seems to be responsible for the install, so I don't know if this is possible:

*) Would it be possible to add one that copies sjaakii to $(DESTDIR)/usr/games (and also use $(DESTDIR) prefixes for other files it installs)?

*) Could you include a file sjaakii.eng in the package containing the 3 lines

Code: Select all

plugin spec 0.0
sjaakii
chess,shogi,xiangqi,shatranj,capablanca,dropchess,fairychess,shogivariants,asean

that is installed in $(DESTDIR)/usr/share/games/plugins/xboard/ ?

*) Could you add a sjaakii.png logo file (e.g. 100x50 or 130x65), and have it installed in $(DESTDIR)/usr/share/games/plugins/logos/ ?

*) Finally (and much less important than the others) could you provide a Linux man page with it?

The reason I ask all this is that I think SjaakII, due to some of its unique capablities (like Shatar and Sittuyin), deserves to be in Debian, and want to approach a Debian maintainer to package it.

[Evert]

I tried to build SjaakII on Ubuntu 10.4, but after cmake .. , make I get the error message:

Code: Select all

hgm@hgm-xboard:~/SjaakII/Build$ make
Scanning dependencies of target libsjaak
[  6%] Building C object CMakeFiles/libsjaak.dir/src/misc/aligned_malloc.o
[ 13%] Building C object CMakeFiles/libsjaak.dir/src/misc/cfgpath.o
[ 20%] Building C object CMakeFiles/libsjaak.dir/src/misc/genrand.o
[ 26%] Building C object CMakeFiles/libsjaak.dir/src/misc/keypressed.o
[ 33%] Building C object CMakeFiles/libsjaak.dir/src/misc/softexp.o
[ 40%] Building CXX object CMakeFiles/libsjaak.dir/src/eval/pst.o
[ 46%] Building CXX object CMakeFiles/libsjaak.dir/src/rules/game.o
[ 53%] Building CXX object CMakeFiles/libsjaak.dir/src/rules/move.o
[ 60%] Building CXX object CMakeFiles/libsjaak.dir/src/rules/san.o
[ 66%] Building CXX object CMakeFiles/libsjaak.dir/src/rules/squares.o
[ 73%] Building C object CMakeFiles/libsjaak.dir/src/hash/hashkey.o
[ 80%] Building C object CMakeFiles/libsjaak.dir/src/hash/hashtable.o
[ 86%] Building C object CMakeFiles/libsjaak.dir/src/hash/evalhash.o
[ 93%] Building C object CMakeFiles/libsjaak.dir/src/timer/timer.o
Linking CXX static library liblibsjaak.a
[ 93%] Built target libsjaak
Scanning dependencies of target sjaakii
[100%] Building CXX object CMakeFiles/sjaakii.dir/src/xboard.o
Linking CXX executable sjaakii
liblibsjaak.a(timer.o): In function `get_timer':
/home/hgm/SjaakII/src/timer/timer.c:35: undefined reference to `clock_gettime'
collect2: ld returned 1 exit status
make[2]: *** [sjaakii] Error 1
make[1]: *** [CMakeFiles/sjaakii.dir/all] Error 2
make: *** [all] Error 2

Ok, that's not how it should be. The culprit is this (in src/timer/timer.c):

Code: Select all

#include <unistd.h>
#if defined _POSIX_TIMERS && _POSIX_TIMERS>0 && defined _POSIX_MONOTONIC_CLOCK
#define USE_CLOCK_GETTIME
#include <time.h>
#else
#include <sys/time.h>
#endif

[…]

/* returns the current value of the counter, in micro seconds */
uint64_t get_timer(void)
{
#ifdef USE_CLOCK_GETTIME
   struct timespec t;
   clock_gettime(0, &t);

   return t.tv_sec*1000000 + t.tv_nsec/1000; 
#else
   struct timeval t;
   gettimeofday(&t, NULL);
   
   return t.tv_sec*1000000 + t.tv_usec; 
#endif
}

The pre-processor tests are to verify that clock_gettime is defined on the system, but they're clearly not detecting that properly. Changing

Code: Select all

-#if defined _POSIX_TIMERS && _POSIX_TIMERS>0 && defined _POSIX_MONOTONIC_CLOCK
+#if 0

should make it compile, while I look for a proper solution.

I have also some request about the build process; I don't know cmake, which seems to be responsible for the install, so I don't know if this is possible:

*) Would it be possible to add one that copies sjaakii to $(DESTDIR)/usr/games (and also use $(DESTDIR) prefixes for other files it installs)?

*) Could you include a file sjaakii.eng in the package containing the 3 lines

Code: Select all

plugin spec 0.0
sjaakii
chess,shogi,xiangqi,shatranj,capablanca,dropchess,fairychess,shogivariants,asean

that is installed in $(DESTDIR)/usr/share/games/plugins/xboard/ ?

*) Could you add a sjaakii.png logo file (e.g. 100x50 or 130x65), and have it installed in $(DESTDIR)/usr/share/games/plugins/logos/ ?

Yes, I've done that before for another project. I just need to add the relevant "install" target options to CMake, which will then install things in $(CMAKE_INSTALL_PREFIX) (it's a standard CMake variable).
I'll need to get my hands on a logo though.

*) Finally (and much less important than the others) could you provide a Linux man page with it?

Yes. I've done that in the past, but I'll need to refresh my memory of how to make a manpage. It's not too hard, from what I remember (I didn't use groff though).

The reason I ask all this is that I think SjaakII, due to some of its unique capablities (like Shatar and Sittuyin), deserves to be in Debian, and want to approach a Debian maintainer to package it.

Sure, that'd be great!

[Evert]

Actually, the correct fix seems to be

Code: Select all

Index: src/timer/timer.c
===================================================================
--- src/timer/timer.c	(revision 116)
+++ src/timer/timer.c	(working copy)
@@ -15,7 +15,7 @@
  *  along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 #include <unistd.h>
-#if defined _POSIX_TIMERS && _POSIX_TIMERS>0 && defined _POSIX_MONOTONIC_CLOCK
+#if defined _POSIX_C_SOURCE && _POSIX_C_SOURCE >= 199309L && defined _POSIX_TIMERS && _POSIX_TIMERS>0 && defined _POSIX_MONOTONIC_CLOCK
 #define USE_CLOCK_GETTIME
 #include <time.h>
 #else

I'll update the source archive with this fix in it in a minute.