The Stockfish of shogi

[Gian-Carlo Pascutto]

I don't speak Japanese but their architecture seems to be:

W1 = 125388 x 256
W2 = 512 x 32
W3 = 32 x 32
W4 = 32 x 1

And I can make some educated guesses: they exploit the fact that W1 doesn't change much to compute the result of that layer incrementally, which is coincidentally the heaviest layer. The other layers have a structure that's perfect for SIMD optimization.

However, much of the devil is in the details I'm sure, they also seem to exploit the fact that the white or black inputs to W1 don't change per turn so you can just flip them after a move, but captures would be an issue there?

Also, *what* are the inputs exactly? This is critical. 125388 inputs is a lot, so this is in itself the product of something. Piece-on-square x piece-on-square?

Note also that W1 output does not match up with W2 input, it's only half.

[aphirst]

I'm a (bad) amateur shōgi player, and came upon this thread while trying to find some information about the NNUE format for shogi engines, along with having already asked myself the following two questions:

• "How come this approach isn't being used for FIDE chess?"
• "How many parameters would be needed in an NNUE nn.bin for FIDE chess"

For the second one, it's necessary to understand where the magic 125388 for the shogi evaluation comes from, but even after attempting to read the (earlier linked) NNUE.pdf paper, I remain ignorant of how this comes from the board state. To be honest, I haven't even been able to work out how the older KKP/KPP formats work - english documentation is between scarce and nonexistent.

For the 125388 I tried to break it down into prime factors - (2^2) * (3^6) * 43, and then tried to make some very basic observations

• 81 squares = 9*9 = 3^4 factor for whatever the per-square representation is
• empty + (black, white)*(pawn, tokin, lance, lance+, knight, knight+, silver, silver+, gold, bishop, bishop+, rook, rook+, king) = 1 + (2*14) = 29 options per square, could either be wasteful and have a sparse 29-bit pattern per square OR use a compact 5-bit (2^5 > 29)
• Whose turn it is = 1 bit
• Something for the piece stands = 2*(38, all pieces minus kings) or a bit-representation of the number of each piece-in-hand?

As you can see, I get lost very quickly, so there's currently no chance of me working out what the equivalent would be for FIDE chess. I'd be very interested to see this tried out though, as NNUE-shogi engines are now obscenely powerful - orqha1018+dolphin1 is no AlphaZero but it's still a monster.

[lkaufman]

As you can see, I get lost very quickly, so there's currently no chance of me working out what the equivalent would be for FIDE chess. I'd be very interested to see this tried out though, as NNUE-shogi engines are now obscenely powerful - orqha1018+dolphin1 is no AlphaZero but it's still a monster.

From what I read, orqha/dolphin beats Elmo by about the same margin as AlphaZero did, so what is the basis for your statement that it "is no AlphaZero"? It's different, sure, but apparently just as strong, without using a GPU. I've played lots of games with it, and while I can usually win if I'm careful with rook and bishop handicap, I almost invariably lose with rook and lance handicap. Against the top human pros, I've generally scored well with just a rook handicap. I would love to see a top pro take a bishop handicap from dolphin.

[jp]

From what I read, orqha/dolphin beats Elmo by about the same margin as AlphaZero did, so what is the basis for your statement that it "is no AlphaZero"? It's different, sure, but apparently just as strong, without using a GPU.

If its results are as good on much weaker hardware, then it's actually much stronger than A0.

There were complaints from the Elmo/shogi community that were similar to the complaints from the SF/chess community about the conditions of the Elmo-A0 games too.

[aphirst]

lkaufman? As in, Larry Kaufman? I'm honoured, I'm sure.

what is the basis for your statement that it "is no AlphaZero"? It's different, sure, but apparently just as strong, without using a GPU

It's entirely possible that my understanding of its strength is outdated. I was under the impression that the AlphaZero published ELO was still higher even than orqha1018/dolphin1, though with a much narrower gap.

On qhapaq's site the listed ELO for dolphin1/orqha1018 is 4393. AlphaZero[1] Shogi's ELO is apparently around 4400, and DeepMind's MuZero[2] apparently has a shogi ELO of about 4700. I had to eyeball those last two since I can't see an explict ELO figure given - presumably I'm overlooking something. If someone has exact figures, I hope they can correct me.

[1] https://arxiv.org/abs/1712.01815
[2] https://arxiv.org/abs/1911.08265

(As an aside, MuZero seems to not have the rules pre-programmed. Interesting! I hadn't heard of it until I started searching for AlphaZero ELO.)

I'm certainly not qualified to judge the fairness of the ELO evaluations, and I'm aware that's a fiercely-contended (and rightfully so) aspect, but this would seem to suggest that orqha1018/dolphin1 is at least approximately level with the 2017 AlphaZero publication, but currently outclassed by MuZero by a similar degree to how AlphaZero outclassed Elmo (et al). I will of course defer to the experts - that's the reason I came here, after all.

[lkaufman]

lkaufman? As in, Larry Kaufman? I'm honoured, I'm sure.

what is the basis for your statement that it "is no AlphaZero"? It's different, sure, but apparently just as strong, without using a GPU

It's entirely possible that my understanding of its strength is outdated. I was under the impression that the AlphaZero published ELO was still higher even than orqha1018/dolphin1, though with a much narrower gap.

On qhapaq's site the listed ELO for dolphin1/orqha1018 is 4393. AlphaZero[1] Shogi's ELO is apparently around 4400, and DeepMind's MuZero[2] apparently has a shogi ELO of about 4700. I had to eyeball those last two since I can't see an explict ELO figure given - presumably I'm overlooking something. If someone has exact figures, I hope they can correct me.

[1] https://arxiv.org/abs/1712.01815
[2] https://arxiv.org/abs/1911.08265

(As an aside, MuZero seems to not have the rules pre-programmed. Interesting! I hadn't heard of it until I started searching for AlphaZero ELO.)

I'm certainly not qualified to judge the fairness of the ELO evaluations, and I'm aware that's a fiercely-contended (and rightfully so) aspect, but this would seem to suggest that orqha1018/dolphin1 is at least approximately level with the 2017 AlphaZero publication, but currently outclassed by MuZero by a similar degree to how AlphaZero outclassed Elmo (et al). I will of course defer to the experts - that's the reason I came here, after all.

Yes, I'm Larry. As you say, the elos quoted for AlphaZero and Dolphin are virtually the same, but I think that the Dolphin rating may be on less than optimal hardware, I don't think it was based on anything like a 32 or 64 core threadripper, so it may be that given the best pc you can buy for around $10k or so Dolphin is already closer to MuZero (which I hadn't heard about) than to AlphaZero.

[jp]

On qhapaq's site the listed ELO for dolphin1/orqha1018 is 4393. AlphaZero[1] Shogi's ELO is apparently around 4400, and DeepMind's MuZero[2] apparently has a shogi ELO of about 4700. I had to eyeball those last two since I can't see an explict ELO figure given - presumably I'm overlooking something. If someone has exact figures, I hope they can correct me.

You should be very cautious interpreting what DM claim. There's been endless discussion on this board and outside it about the conditions of AZ's games against SF8. What is less known is that similar complaints were made by computer Shogi people about the conditions of the Elmo-AZ games.

If dolphin1/orqha1018 played other Shogi engines in "fairer" conditions, that alone would put it higher than AZ.

You should not look at [1] either. If you want to read it, at least look at the published paper, which has big differences.

[aphirst]

You should be very cautious interpreting what DM claim.

I'm aware, hence the disclaimers I put into my previous post.

You should not look at [1] either. If you want to read it, at least look at the published paper, which has big differences.

OHH, I wasn't aware that the arxiv preprint was substantially different. Do I have the correct link [1] for the paper now? If so, I can make a clarification edit to my previous post.

[1] https://science.sciencemag.org/content/362/6419/1140

EDIT: It seems I'm past the cooldown for being able to edit my previous post anyway. I suspect the "report" feature is inappropriate for this. I apologise for any confusion this will provide to future readers.

[MonteCarlo]

On qhapaq's site the listed ELO for dolphin1/orqha1018 is 4393. AlphaZero[1] Shogi's ELO is apparently around 4400, and DeepMind's MuZero[2] apparently has a shogi ELO of about 4700. I had to eyeball those last two since I can't see an explict ELO figure given - presumably I'm overlooking something. If someone has exact figures, I hope they can correct me.

I don't see any indication of a 300 point improvement for MuZero over AlphaZero in the MuZero paper, for the following reasons (in ascending order of strength, burying the lede )

First, the authors only say that MuZero "matched" the performance of AlphaZero in Shogi, which seems very unlike them if they actually exceeded AlphaZero's performance by 300 points.

Second, AlphaZero's rating is graphed along with MuZero's (it's the horizontal orange line) and I see no indication at all that MuZero's blue line ever exceeded the AlphaZero orange line at all, much less by 300 points.

Third, the json for the graphs is published with the paper (board_game_elos.json in ancillary files). It shows that in Shogi, the AlphaZero line was 4666, while MuZero maxed out at 4646.

[aphirst]

Evidently I eyeballed it wrong. Thanks for providing clarification!