The Stockfish of shogi

[Fabian Fichter]

Actually many of the top shogi engines are very similar and based on Stockfish (Dolphin, YaneuraOu, Apery, ...), but their strength comes from the evaluation files they use. The evaluation files are no (deep) neural networks but as far as I know just some coefficients for low-order polynomials of piece squares, distances or counts, but I do not know the exact details (if someone has a link to a detailed description, please let me know). Since most engines even share the same evaluation file interface, they are closely related in both the search engine and the evaluation.

If you want to see the level of shogi a "pure" Stockfish engine plays, you can check my multi-variant engine Fairy-Stockfish https://github.com/ianfab/Fairy-Stockfish, which among others supports shogi. It uses an evaluation function that was only slightly adapted compared to chess and it is almost identical to the evaluation function the engine uses for crazyhouse. Its playing strength should be somewhere at the amateur dan level, but I do not know exactly, since I am only 1 kyu. One also needs to consider that the engine could easily be around 5 times faster (i.e., ~200 Elo stronger) if it was implemented as a pure shogi engine and not as a generic variant engine. You can try a time-handicapped (limited to 1s/move) version at https://pychess-variants.herokuapp.com/, or otherwise install the engine locally as a UCI, USI or XBoard engine depending on the GUI you want to use.

[lkaufman]

Actually many of the top shogi engines are very similar and based on Stockfish (Dolphin, YaneuraOu, Apery, ...), but their strength comes from the evaluation files they use. The evaluation files are no (deep) neural networks but as far as I know just some coefficients for low-order polynomials of piece squares, distances or counts, but I do not know the exact details (if someone has a link to a detailed description, please let me know). Since most engines even share the same evaluation file interface, they are closely related in both the search engine and the evaluation.

If you want to see the level of shogi a "pure" Stockfish engine plays, you can check my multi-variant engine Fairy-Stockfish https://github.com/ianfab/Fairy-Stockfish, which among others supports shogi. It uses an evaluation function that was only slightly adapted compared to chess and it is almost identical to the evaluation function the engine uses for crazyhouse. Its playing strength should be somewhere at the amateur dan level, but I do not know exactly, since I am only 1 kyu. One also needs to consider that the engine could easily be around 5 times faster (i.e., ~200 Elo stronger) if it was implemented as a pure shogi engine and not as a generic variant engine. You can try a time-handicapped (limited to 1s/move) version at https://pychess-variants.herokuapp.com/, or otherwise install the engine locally as a UCI, USI or XBoard engine depending on the GUI you want to use.

How can the evaluation function for a shogi program be "almost identical" to crazyhouse, when most of the pieces are different, they have different values in hand than on the board, most can promote, etc? Perhaps you just mean they have similar types of terms? If so how were the shogi engines params tuned?

[Fabian Fichter]

Yes, that was a bit of an overstatement, I meant that the terms are the same, while the piece values (apart from rook and bishop) and a few other things of course differ. However, the evaluation function is quite generic, i.e., it chooses/modifies the evaluation terms based on the rules/features of the game (e.g., whether captured pieces go to hand, on which rank pawns/pieces promote, etc.), and does not have dedicated evaluation functions for specific variants. That is why a change in the evaluation function for crazyhouse (based on the rule that captured piece go to the hand) directly takes effect also in shogi since it shares this rule, so you might see where my statement was coming from.

I ran some quick tuning with SPSA for the piece values and took a compromise between the results for shogi, minishogi and euroshogi as the final values (since again piece values for same pieces are shared between variants), but apart from that the evaluation mostly inherits the differences compared to the chess evaluation function from crazyhouse, e.g., regarding king safety, and I did much more optimization for crazyhouse than for shogi.

[lkaufman]

Yes, that was a bit of an overstatement, I meant that the terms are the same, while the piece values (apart from rook and bishop) and a few other things of course differ. However, the evaluation function is quite generic, i.e., it chooses/modifies the evaluation terms based on the rules/features of the game (e.g., whether captured pieces go to hand, on which rank pawns/pieces promote, etc.), and does not have dedicated evaluation functions for specific variants. That is why a change in the evaluation function for crazyhouse (based on the rule that captured piece go to the hand) directly takes effect also in shogi since it shares this rule, so you might see where my statement was coming from.

I ran some quick tuning with SPSA for the piece values and took a compromise between the results for shogi, minishogi and euroshogi as the final values (since again piece values for same pieces are shared between variants), but apart from that the evaluation mostly inherits the differences compared to the chess evaluation function from crazyhouse, e.g., regarding king safety, and I did much more optimization for crazyhouse than for shogi.

I'm curious whether you just used chess piece values for crazyhouse or tuned them with SPSA, and in the later case how they compare with normal chess piece values? I would imagine that the relative values of the chessmen would be very different in crazyhouse, and presumably also very different for pieces in hand vs. pieces on the board. I would think that tuning with SPSA would add hundreds of elo vs. using chess values (maybe even a thousand or more elo). For shogi if you used chess pawn structure, the results would be rather silly. Isolated pawns aren't even a known concept in shogi, and passed pawns can usually be made "unpassed" by a pawn drop. King proximity to enemy passed pawns would generally be bad rather than good, etc. Is the shogi engine rated on any of the shogi rating lists?

[hgm]

Definitely the Rook and Bishop value in mini-Shogi should be much smaller (using Gold as a standard ) than in regular Shogi. In mini-Shogi B and G have about the same value. I learned this the hard way, as I was too stupid to foresee it: when I participated with my Shogi engine in the computer Olympiad in Kanazawa, I learned about the existence of mini-Shogi only during the tournament, and as a tourney for it was planned on a day that I had no other games, I dicided to convert my Shogi engine to mini-Shogi overnight in the hotel. The first few games in the tourney it lost horribly, because it though trading general + Pawn for a Rook or Bishop was a good deal, while in mini-Shogi it is an almost certain loss. Only then I realized that the small board of course strongly reduces slider value, as even a Rook has at maximum 8 moves on 5x5, only 4 of them unblockable. After I strongly reduced the slider values and assigned a bonus to a general on the central square during the lunch break, the engine won all other games, and finished second.

[Ovyron]

Leela plays some awesome chess without storing chess' piece values anywhere, I wonder if an engine could be made like this (the mobility of a piece would determine its value?) and then a shoji engine would play very strongly without having to tell it our best guess of what pieces are worth for.

[Raphexon]

Leela plays some awesome chess without storing chess' piece values anywhere, I wonder if an engine could be made like this (the mobility of a piece would determine its value?) and then a shoji engine would play very strongly without having to tell it our best guess of what pieces are worth for.

Fairly sure her networks store chess piece values.

And those super small nets (16x2 and 24x3) have some issues with not always knowing the proper value of a piece leading to very hilarious queen sacs.
Still stronger than any human will ever be, but can still make horrible blunders.

[Raphexon]

Definitely the Rook and Bishop value in mini-Shogi should be much smaller (using Gold as a standard ) than in regular Shogi. In mini-Shogi B and G have about the same value. I learned this the hard way, as I was too stupid to foresee it: when I participated with my Shogi engine in the computer Olympiad in Kanazawa, I learned about the existence of mini-Shogi only during the tournament, and as a tourney for it was planned on a day that I had no other games, I dicided to convert my Shogi engine to mini-Shogi overnight in the hotel. The first few games in the tourney it lost horribly, because it though trading general + Pawn for a Rook or Bishop was a good deal, while in mini-Shogi it is an almost certain loss. Only then I realized that the small board of course strongly reduces slider value, as even a Rook has at maximum 8 moves on 5x5, only 4 of them unblockable. After I strongly reduced the slider values and assigned a bonus to a general on the central square during the lunch break, the engine won all other games, and finished second.

Wouldn't Texel tuning have easily solved that issue?

[hgm]

How would you get the positions to tune on? In a single night?

An additional problem is that games with piece drops do not really have quiet positions, and in mini-Shogi is worst of all, in that respect.

[Fabian Fichter]

In Fairy-Stockfish the base piece values are the same for all variants, they are only adjusted for a few rules that can heavily influence dynamics like losing chess rules, piece drops, and board size (for sliders). E.g., for drop games, the piece values are scaled by a v_max/(v_max+v) formula, where v_max is around 3 times the value of a queen, so pieces with a high value (v) lose relative strength in drop games. Additionally the piece values are halved for drop games to have a more natural scale for thresholds in futility pruning, razoring, SEE, etc., but this of course does not change their relative value.

In crazyhouse Fairy-Stockfish is only around 200 Elo weaker (~100 of which due to speed) than the multi-variant Stockfish used on lichess for which we heavily tuned dozens/hundreds of parameters (including piece values) specifically for crazyhouse, so the generic adaptions already seem to work well. However, I have not much doubt that playing strength could be increased a lot for shogi by improving the evaluation, especially since the Stockfish-based engines using evaluation files show that with the same search engine you can get a ~1500 Elo stronger engine, but currently my focus is on adding more features/variants.

Back to the original topic of the thread, I found some more info on the most recent generation of evaluation files for shogi at https://github.com/ynasu87/nnue/blob/ma ... s/nnue.pdf (in Japanese).

Abstract
Most of the strongest shogi programs nowadays employ a linear evaluation function, which is computationally efficient but lacks nonlinear modeling capability. This report presents a new class of neural-network-based
nonlinear evaluation functions for computer shogi, called NNUE (Efficiently Updatable Neural-Network-based
evaluation functions). NNUE evaluation functions are designed to run efficiently on CPU using various acceleration techniques, including incremental computation. The first shogi program with a NNUE evaluation
function, the end of genesis T.N.K.evolution turbo type D, will be unveiled at the 28th World Computer Shogi
Championship.