Crazyhouse Computer Championships 2016

[Ferdy]

After 1600 games 0.1.2 beat 0.1.1 by 55.3%, so I have good hopes it is indeed an improvement.

In blitz after 128 games it gained around 59 rating points over 0.1.1.

Code: Select all

   # PLAYER                          :  RATING    +/-   GAMES  PTS(%)
   1 Stockfish zh 30Nov2016 64bit    :  3157.9   94.1     184      89
   2 Stockfish zh 15Nov2016 64bit    :  3081.8   57.4     402      86
   3 Stockfish zh 02Nov2016 64bit    :  3004.7   50.1     454      79
   4 Imortal 2.21 64bit              :  2915.0   65.3     176      60
   5 Imortal 2.17 64bit              :  2876.7   33.8     930      66
   6 Imortal 2.9001 32bit            :  2823.5   28.8    1569      71
   7 TJchess 1.3 64bit               :  2758.8   25.5    1839      64
   8 Sunsetter 9 32bit               :  2657.2   25.8    1649      49
   9 Imortal 2.0 32bit               :  2638.1   45.3     363      68
  10 TJchess 1.1 64bit               :  2577.1   57.2     220      61
  11 Sunsetter 7g 64bit              :  2522.5   49.3     296      52
  12 CrazyWa 0.1.2 32bit             :  2490.2   76.1     128      43
  13 Imortal 1.0 32bit               :  2485.3   57.2     202      53
  14 CrazyWa 0.1.1 32bit             :  2431.5   36.6     630      43
  15 CrazyWa 0.0.8 32bit             :  2397.3   40.6     420      47
  16 CrazyWa 0.0.9 32bit             :  2380.7   59.4     224      33
  17 CrazyWa 0.0.8a 32bit            :  2355.2   58.8     160      54
  18 CrazyWa 0.1.0 32bit             :  2347.0   41.1     478      30
  19 CrazyWa 0.0.7 32bit             :  2308.1   60.4     216      49
  20 Sjaak II 1.4.1 64bit            :  2301.2   60.0     240      23
  21 Sjeng 11.2 32bit                :  2300.0   22.6    1516      46
  22 NebiyuAlien 1.45a 32bit         :  2275.2   22.7    1478      40
  23 CrazyWa 0.0.6 32bit             :  2254.5   52.8     192      63
  24 Sjaak II 1.3.1a 64bit           :  2253.2   24.0    1435      39
  25 CrazyWa 0.0.4 32bit             :  2229.7   44.7     336      42
  26 Sjaak II 1.4.0 64bit            :  2222.8   42.0     550      23
  27 CrazyWa 0.0.1 32bit             :  2208.5   66.9     129      56
  28 Pulsar 2009 9b 32bit            :  2107.3   37.1     575      40
  29 NebiyuAlien 1.45 32bit          :  2053.6   56.3     204      37
  30 CrazyWa 0.0.0 32bit             :  1995.0   77.8      96      32
  31 TSCP zh 1.1 32bit               :  1985.2   39.7     555      28
  32 KKFChess 2.6.7 beta 32bit       :  1953.8   41.1     566      25
  33 PyChess 0.12.4 32bit            :  1570.1  130.2     116       7

White advantage = 56.28 +/- 4.53
Draw rate (equal opponents) = 1.53 % +/- 0.15

[hgm]

In blitz after 128 games it gained around 59 rating points over 0.1.1.

That is not bad, for just adding two characters (3*)!

This brings me close to the point where I wanted to be, although originally I had hoped to get at the level of Sunsetter 7g. What especially pleases me is that the evaluation still is almost completely free of variant-specific knowledge (other than piece values). In particular in the King Safety term: it just uses number of legal King moves, number of friendly attacked pieces adjacent to King, number of empty squares adjacent to King, and total value of the opponent hand.

This should make implementation of the other variants CrazyWa is supposed to play pretty easy, and I will now focus on that. I still have to think a bit on automatic assignment of PST. In Wa Shogi, with 31 different piece types, I don't want to have to design or tune a table for each type by hand! So similar types should share tables.

[styx]

HGM I'm so sorry to always bother you with the same question, but do you have any plans to release CrazyWa for Linux users too?

Btw.: what does the "Wa" stand for in "CrazyWa"?

[hgm]

'Wa' comes from 'Wa Shogi', a historic version of Japanese Chess invented in the 17th century. It seems to mean 'Peaceful Chess'. Although 'Pieceful Chess' would have been more applicable, as it has 27 pieces per player (11 Pawns plus 16 others, all different) on an 11 x 11 board. Where 14 of the piece types can promote to yet other piece types.

The historic rule descriptions do not mention whether Wa Shogi should be played with drops (like modern Shogi) or without (like the previously dominant form of Chess in Japan, Chu Shogi). It appeared just at the time where games with drops started to rival those without drops in popularity. So today people distinguish "Wa Shogi with drops" and "Wa Shogi without drops", which (of course) are as different fromeach other as Crazyhouse and orthodox Chess. In analogy with the latter, I figured that CrazyWa would be a good more-compact name for "Wa Shogi with drops".

I adopted that as name for the engine, because it was basically designed with the goal of being able to handle Wa Shogi efficiently. Which is not trivial because of the rather large board and excessive number of piece types. In addition I designed the move encoding such that it would be easy to have special moves like castling and e.p. capture. So that it also can do Crazyhouse, something that would be very hard to implement in my existing Shogi engine 'Shokidoki' (which is limited to 9x9 boards, and thus cannot do Wa Shogi either).

The time pressure due to the upcoming zh championship has forced me to largely neglect the Shogi-specific parts of the engine. E.g. it does not detect yet whether a repetition is a perpetual check (to score it as a loss instead of a draw). That brings me to your first question: the version now on my website should not really be considered an official release; it is more a pre-release of an unfinished product, done only for the purpose of entering it in the zh championship. I will do an official release and announcements when I have also completed the Shogi part, and played some test games with that to make sure it works. Then I will provide a Linux binary. I might even make it open source; I have not decided on that yet.

I am not that far from having the Shogi work; apart from the perpetual-check detection I just need to design some useful piece-square tables, and assign these to the various piece types explicitly, or design some heuristic to pick them based on the piece properties.

[Ferdy]

In blitz after 128 games it gained around 59 rating points over 0.1.1.

That is not bad, for just adding two characters (3*)!

This brings me close to the point where I wanted to be, although originally I had hoped to get at the level of Sunsetter 7g. What especially pleases me is that the evaluation still is almost completely free of variant-specific knowledge (other than piece values). In particular in the King Safety term: it just uses number of legal King moves, number of friendly attacked pieces adjacent to King, number of empty squares adjacent to King, and total value of the opponent hand.

This should make implementation of the other variants CrazyWa is supposed to play pretty easy, and I will now focus on that. I still have to think a bit on automatic assignment of PST. In Wa Shogi, with 31 different piece types, I don't want to have to design or tune a table for each type by hand! So similar types should share tables.

Tweaking king safety eval is fun and is very responsive, the search could not get a decent depth, probably the eval is trying to help in this area. Another idea is to attack own king with own pieces, so that when the king moves, the square it vacated is still defended. It seems like a cover check evasion moves are not effective because the attacker will just capture it and ready to be dropped later.

[Ferdy]

Result at stage 1, group A, pairing 1:
TC 1h + 10s/move, best in 24 games
Sjaak - Pulsar : 13 - 1

https://sites.google.com/site/zhassocia ... dpairings#
Scroll down and below that link there are sample embedded games courtesy from Lichess.
Next match will be CrazyWa vs KKFChess

Code: Select all

Head to head statistics:

1) Sjaak II 1.4.1 64bit 2562.7 :     14 (+13,=0,-1),  92.9 %

   vs.                         :  games (  +, =, -),   (%) :    Diff,    SD, CFS (%)
   Pulsar 2009 9b 32bit        :     14 ( 13, 0, 1),  92.9 :  +455.7, 117.7,  100.0

2) Pulsar 2009 9b 32bit 2107.0 :     14 (+1,=0,-13),   7.1 %

   vs.                         :  games ( +, =,  -),   (%) :    Diff,    SD, CFS (%)
   Sjaak II 1.4.1 64bit        :     14 ( 1, 0, 13),   7.1 :  -455.7, 117.7,    0.0

[Evert]

What especially pleases me is that the evaluation still is almost completely free of variant-specific knowledge (other than piece values).

Impressive indeed! I'll need to steal some of your ideas.

In particular in the King Safety term: it just uses number of legal King moves, number of friendly attacked pieces adjacent to King, number of empty squares adjacent to King, and total value of the opponent hand.

I think I have most of those as well, in addition to a "shelter" term, which I think takes the form of occupied squares around the king in drop variants.

I still have to think a bit on automatic assignment of PST. In Wa Shogi, with 31 different piece types, I don't want to have to design or tune a table for each type by hand! So similar types should share tables.

In SjaakII I use centralisation for leapers and something based on mobility on an empty board for sliders. That reproduces centralisation for diagonal sliders, but does nothing for rooks. It's a good starting point, but very crude.

[hgm]

Impressive indeed! I'll need to steal some of your ideas.

This is why I did not want to release the source before the championship!

In SjaakII I use centralisation for leapers and something based on mobility on an empty board for sliders. That reproduces centralisation for diagonal sliders, but does nothing for rooks. It's a good starting point, but very crude.

Centralizing and neutral are definitely two choices that should be on the menu. For Crazyhouse it seemed useful to keep the Queen away from the 3rd-6th rank, as it usually is just chased there with drops, wasting many tempos. And of course one to keep the King on the backrank. And for Pawns giving a high bonus for 7th and 6th rank.

The latter is related to promotion, but Wa calls for a more general and subtle way to handle that. Wa has these excessively weak pieces (Owl and Crow) that have very strong promotions. Not unlike FIDE Pawns, of course, except that they can also stand inside the zone. But there are also pieces with weaker, although significant promotion, and pieces that hardly gain on promotion at all (such as Silver -> Gold). Also, I would expect pieces like a Side Mover (sRvW) to be worth quite a bit more inside the zone than elsewhere, even when they do not promote. Normally the battle goes forward, but in the zone it goes sideways. To a somewhat lesser extent this should also apply to Rooks, as they have two sideway moves, and only one forward move. I guess this is the equivalent of Rook-on-7th. Not sure if it should also be worth more in your own zone, to defend it with its sideway moves. I guess I will start with 2 x 3 different promotion tables, with bonuses of 30, 90 and 180 cP in the zone, respectively. Crow/Owl would then be worth 120 initially, 300 in the zone, and promote to pieces worth 420 or 540.

[Isaac]

Hello people, I have 2 questions :
Is Stockfish-zh still the only multi threaded engine for zh?
Would it be possible to include the latest SF-zh running on 2 cores in this tournament or the next one? I'm really interested to see how much a zh engine gains when running on more cores.
Thanks!

[Evert]

Impressive indeed! I'll need to steal some of your ideas.

This is why I did not want to release the source before the championship!

Much of SjaakII's evaluation is parameterised using "this is good enough for now", so it can probably be improved quite a bit. Most of it is also not tuned for drop games, and none of it is tuned for Crazyhouse (except for the tempo bonus, and I guess I changed the piece values in the last release, but those are also not optimised and just verified to be better than the previous set). I suspect it'll be even worse at Wa Shogi, but at least that's a defined variant rather than a build-in variant, so I can pretend it's not really my fault.

Centralizing and neutral are definitely two choices that should be on the menu. For Crazyhouse it seemed useful to keep the Queen away from the 3rd-6th rank, as it usually is just chased there with drops, wasting many tempos. And of course one to keep the King on the backrank. And for Pawns giving a high bonus for 7th and 6th rank.

I like defining piece square tables generically as a linear combination of advance tables (with a linear and a quadratic term in rank; linear is not so good for pushing passed pawns) and centralisation tables (again with linear and quadratic terms, although I don't think the linear term actually does anything useful in cases I've tested). For rooks central files might count for something too. I then tweak the linear weights for piece types. Other types of bonuses (like 7th rank bonuses for rooks) I prefer to define in terms of pawn structure/enemy king location so that they can scale naturaly to changing positions.

The latter is related to promotion, but Wa calls for a more general and subtle way to handle that.

Indeed.
It's not so difficult to give a bonus for being close to the promotion zone based on whether a piece can promote rather than whether it happens to be a pawn though. I think I do this in SjaakII even taking into account the actual number of steps a piece needs to reach the promotion zone rather than the rank it's on (because promotion zones can be anything). Because you calculate this once at start-up it's not such a big deal, and if you fold it into the PST it's even free (I don't, for some reason).

Crow/Owl would then be worth 120 initially, 300 in the zone, and promote to pieces worth 420 or 540.

SjaakII guestimates both of those at about 200 points. I guess that's not a bad average of 120 and 300, but that's not what it's actually based on.
It evaluates a promoted Owl at 1157 points though, which may be due to Eagle being defined on input as worth 850 (seems a lot though). Not sure where that number comes from.