The Stockfish of shogi

[lkaufman]

I recently got a very strong, free shogi engine called "Dolphin - Kristallweizen-kai" that is based on Stockfish, adapted for shogi. It is just incredibly strong. It is rated about a thousand elo above the world's top human player, and based on games with 2017 computer champ Elmo it is about equal to Alpha Zero. It is way stronger than all NN shogi engines being tested. I'm rather amazed that adapting Stockfish to shogi works so well, after all they are very different games, shogi being more like "CrazyHouse" chess than it is like regular chess. Obviously the evaluation function must be completely different, so the strength must come from the search.
When I play against it, with 15' + 30" per move after time out vs 5' + 10" for engine, I need rook + bishop handicap to have decent chances to win. In shogi those are the two strongest pieces, and you only have one of each! To appreciate what this means, I have beaten several former world champions and near-world champions at just rook handicap, and beat one of them at just bishop handicap. Even at age 72, I'm still the top rated US player, and am considered to be an expert in handicap shogi. So this is really remarkable. Based on some simulations I've run, Dolphin should be able to come out ahead giving Bishop handicap to the Meijin (World Champ), at least at rapid time limits like 30' + 30" per move after time out. I haven't heard of such games being played, but since the recent two stone handicap (minus komi) match in GO between a top engine and a top pro (result 1 to 1) it seems reasonable for such a match to take place.

[Fabian Fichter]

Actually many of the top shogi engines are very similar and based on Stockfish (Dolphin, YaneuraOu, Apery, ...), but their strength comes from the evaluation files they use. The evaluation files are no (deep) neural networks but as far as I know just some coefficients for low-order polynomials of piece squares, distances or counts, but I do not know the exact details (if someone has a link to a detailed description, please let me know). Since most engines even share the same evaluation file interface, they are closely related in both the search engine and the evaluation.

If you want to see the level of shogi a "pure" Stockfish engine plays, you can check my multi-variant engine Fairy-Stockfish https://github.com/ianfab/Fairy-Stockfish, which among others supports shogi. It uses an evaluation function that was only slightly adapted compared to chess and it is almost identical to the evaluation function the engine uses for crazyhouse. Its playing strength should be somewhere at the amateur dan level, but I do not know exactly, since I am only 1 kyu. One also needs to consider that the engine could easily be around 5 times faster (i.e., ~200 Elo stronger) if it was implemented as a pure shogi engine and not as a generic variant engine. You can try a time-handicapped (limited to 1s/move) version at https://pychess-variants.herokuapp.com/, or otherwise install the engine locally as a UCI, USI or XBoard engine depending on the GUI you want to use.

[lkaufman]

Actually many of the top shogi engines are very similar and based on Stockfish (Dolphin, YaneuraOu, Apery, ...), but their strength comes from the evaluation files they use. The evaluation files are no (deep) neural networks but as far as I know just some coefficients for low-order polynomials of piece squares, distances or counts, but I do not know the exact details (if someone has a link to a detailed description, please let me know). Since most engines even share the same evaluation file interface, they are closely related in both the search engine and the evaluation.

If you want to see the level of shogi a "pure" Stockfish engine plays, you can check my multi-variant engine Fairy-Stockfish https://github.com/ianfab/Fairy-Stockfish, which among others supports shogi. It uses an evaluation function that was only slightly adapted compared to chess and it is almost identical to the evaluation function the engine uses for crazyhouse. Its playing strength should be somewhere at the amateur dan level, but I do not know exactly, since I am only 1 kyu. One also needs to consider that the engine could easily be around 5 times faster (i.e., ~200 Elo stronger) if it was implemented as a pure shogi engine and not as a generic variant engine. You can try a time-handicapped (limited to 1s/move) version at https://pychess-variants.herokuapp.com/, or otherwise install the engine locally as a UCI, USI or XBoard engine depending on the GUI you want to use.

How can the evaluation function for a shogi program be "almost identical" to crazyhouse, when most of the pieces are different, they have different values in hand than on the board, most can promote, etc? Perhaps you just mean they have similar types of terms? If so how were the shogi engines params tuned?

[Fabian Fichter]

Yes, that was a bit of an overstatement, I meant that the terms are the same, while the piece values (apart from rook and bishop) and a few other things of course differ. However, the evaluation function is quite generic, i.e., it chooses/modifies the evaluation terms based on the rules/features of the game (e.g., whether captured pieces go to hand, on which rank pawns/pieces promote, etc.), and does not have dedicated evaluation functions for specific variants. That is why a change in the evaluation function for crazyhouse (based on the rule that captured piece go to the hand) directly takes effect also in shogi since it shares this rule, so you might see where my statement was coming from.

I ran some quick tuning with SPSA for the piece values and took a compromise between the results for shogi, minishogi and euroshogi as the final values (since again piece values for same pieces are shared between variants), but apart from that the evaluation mostly inherits the differences compared to the chess evaluation function from crazyhouse, e.g., regarding king safety, and I did much more optimization for crazyhouse than for shogi.

[lkaufman]

Yes, that was a bit of an overstatement, I meant that the terms are the same, while the piece values (apart from rook and bishop) and a few other things of course differ. However, the evaluation function is quite generic, i.e., it chooses/modifies the evaluation terms based on the rules/features of the game (e.g., whether captured pieces go to hand, on which rank pawns/pieces promote, etc.), and does not have dedicated evaluation functions for specific variants. That is why a change in the evaluation function for crazyhouse (based on the rule that captured piece go to the hand) directly takes effect also in shogi since it shares this rule, so you might see where my statement was coming from.

I ran some quick tuning with SPSA for the piece values and took a compromise between the results for shogi, minishogi and euroshogi as the final values (since again piece values for same pieces are shared between variants), but apart from that the evaluation mostly inherits the differences compared to the chess evaluation function from crazyhouse, e.g., regarding king safety, and I did much more optimization for crazyhouse than for shogi.

I'm curious whether you just used chess piece values for crazyhouse or tuned them with SPSA, and in the later case how they compare with normal chess piece values? I would imagine that the relative values of the chessmen would be very different in crazyhouse, and presumably also very different for pieces in hand vs. pieces on the board. I would think that tuning with SPSA would add hundreds of elo vs. using chess values (maybe even a thousand or more elo). For shogi if you used chess pawn structure, the results would be rather silly. Isolated pawns aren't even a known concept in shogi, and passed pawns can usually be made "unpassed" by a pawn drop. King proximity to enemy passed pawns would generally be bad rather than good, etc. Is the shogi engine rated on any of the shogi rating lists?

[hgm]

Definitely the Rook and Bishop value in mini-Shogi should be much smaller (using Gold as a standard ) than in regular Shogi. In mini-Shogi B and G have about the same value. I learned this the hard way, as I was too stupid to foresee it: when I participated with my Shogi engine in the computer Olympiad in Kanazawa, I learned about the existence of mini-Shogi only during the tournament, and as a tourney for it was planned on a day that I had no other games, I dicided to convert my Shogi engine to mini-Shogi overnight in the hotel. The first few games in the tourney it lost horribly, because it though trading general + Pawn for a Rook or Bishop was a good deal, while in mini-Shogi it is an almost certain loss. Only then I realized that the small board of course strongly reduces slider value, as even a Rook has at maximum 8 moves on 5x5, only 4 of them unblockable. After I strongly reduced the slider values and assigned a bonus to a general on the central square during the lunch break, the engine won all other games, and finished second.

[Ovyron]

Leela plays some awesome chess without storing chess' piece values anywhere, I wonder if an engine could be made like this (the mobility of a piece would determine its value?) and then a shoji engine would play very strongly without having to tell it our best guess of what pieces are worth for.

[Raphexon]

Leela plays some awesome chess without storing chess' piece values anywhere, I wonder if an engine could be made like this (the mobility of a piece would determine its value?) and then a shoji engine would play very strongly without having to tell it our best guess of what pieces are worth for.

Fairly sure her networks store chess piece values.

And those super small nets (16x2 and 24x3) have some issues with not always knowing the proper value of a piece leading to very hilarious queen sacs.
Still stronger than any human will ever be, but can still make horrible blunders.

[Raphexon]

Definitely the Rook and Bishop value in mini-Shogi should be much smaller (using Gold as a standard ) than in regular Shogi. In mini-Shogi B and G have about the same value. I learned this the hard way, as I was too stupid to foresee it: when I participated with my Shogi engine in the computer Olympiad in Kanazawa, I learned about the existence of mini-Shogi only during the tournament, and as a tourney for it was planned on a day that I had no other games, I dicided to convert my Shogi engine to mini-Shogi overnight in the hotel. The first few games in the tourney it lost horribly, because it though trading general + Pawn for a Rook or Bishop was a good deal, while in mini-Shogi it is an almost certain loss. Only then I realized that the small board of course strongly reduces slider value, as even a Rook has at maximum 8 moves on 5x5, only 4 of them unblockable. After I strongly reduced the slider values and assigned a bonus to a general on the central square during the lunch break, the engine won all other games, and finished second.

Wouldn't Texel tuning have easily solved that issue?

[hgm]

How would you get the positions to tune on? In a single night?

An additional problem is that games with piece drops do not really have quiet positions, and in mini-Shogi is worst of all, in that respect.