The Perils of missing sub-promotion

[nczempin]

Funny thing I came across in my Eden silver tests:

[D]8/8/8/8/4Q3/5K1k/6p1/8 b - - 1 56


56. ... g1=N+ {(g2g1N+ Kf3f2 Ng1e2 Kf2xe2 Kh3g3 Ke2e3 Kg3h2) -10.28/6 6 White resigns}
{White resigns} 0-1

Oops!


[Event "Eden 13 silver umax4_0 2"]
[Site "ATZE"]
[Date "2007.09.28"]
[Round "4"]
[White "Umax4_0w"]
[Black "Eden_0.0.13dev03"]
[Result "0-1"]
[Time "normal"]
[Variation "120+6"]
[TimeControl "120+6"]
[SetUp "1"]
[FEN "r1b1kbnr/1pp2ppp/p1p5/8/3NP3/8/PPP2PPP/RNB1K2R b KQkq - 0 1 "]
[Termination "normal"]
[PlyCount "111"]
[WhiteType "program"]
[BlackType "program"]

c5 {(c6c5 Nd4f3 g7g6 00 Bf8h6 Nb1d2 Bh6xd2 Bc1xd2) -0.40/6 10} 2.Ne2 {(d4e2)
-0.12/8 33} Bd7 {(Bc8d7 00 Bd7b5 Nb1c3 Bb5xe2 Nc3xe2) -0.39/6 10} 3.Nbc3 {(b1c3)
+0.15/7 11} O-O-O {(000 00 Kc8b8 Bc1g5 Bf8e7 Bg5xe7 Ng8xe7) -0.38/6 9} 4.O-O {(e1g1)
+0.25/7 8} c4 {(c5c4 Nc3d5 Bd7c6 Nd5c3 Bf8b4 e4e5 Bb4xc3 Ne2xc3) -0.36/6 9} 5.Be3
{(c1e3) +0.03/7 10} Bg4 {(Bd7g4 f2f3 Bg4e6 Ne2f4 Kc8b8 Nf4xe6 f7xe6) -0.54/6 9} 6.f3
{(f2f3) -0.01/7 8} Bd7 {(Bg4d7 Ne2d4 Kc8b8 Be3g5 Bf8e7 Bg5xe7 Ng8xe7) -0.28/6 9} 7.b3
{(b2b3) -0.01/7 12} cxb3 {(c4xb3 a2xb3 Kc8b8 Be3d4 Ng8e7 Rf1d1) +0.01/6 9} 8.axb3
{(a2b3) -0.01/7 4} Kb8 {(Kc8b8 Rf1d1 Bf8d6 Be3d4 f7f6 Nc3d5) +0.01/6 9} 9.Kf2 {(g1f2)
-0.09/6 3} Nf6 {(Ng8f6 Kf2g1 Bf8b4 Rf1d1) +0.29/5 9} 10.e5 {(e4e5) -0.01/7 11} Ne8
{(Nf6e8 Rf1d1 Bf8e7 Kf2g1 Be7b4 Nc3d5) +0.14/6 9} 11.Kg3 {(f2g3) -0.06/6 3} f6 {(f7f6
f3f4 h7h5 Kg3f2 Bd7f5 Ne2d4) +0.54/6 8} 12.Bf4 {(e3f4) -0.08/7 22} fxe5 {(f6xe5
Bf4xe5 h7h5 h2h4 Bf8c5 Rf1e1) +0.75/6 8} 13.Bxe5 {(f4e5) -0.14/7 4} h5 {(h7h5 h2h4
Bf8e7 Rf1d1 Be7c5) +0.75/5 8} 14.Kf4 {(g3f4) -0.12/7 12} Be7 {(Bf8e7 Kf4g3 h5h4+
Kg3f2 Be7c5+ Ne2d4 Bd7f5) +1.13/5 8} 15.Nd5 {(c3d5) -0.07/7 20} Bh4 {(Be7h4 Ne2d4
Rh8g8 Rf1d1 Rg8f8+ Kf4e4) +1.41/5 8} 16.Nd4 {(e2d4) -0.11/6 5} Rf8+ {(Rh8f8+ Kf4e4
g7g6 Rf1d1 Bh4g5 Ke4d3) +1.39/5 8} 17.Ke3 {(f4e3) -0.19/7 13} Bb5 {(Bd7b5 c2c4 Bb5d7
g2g3 Bh4g5+ Ke3f2) +1.00/5 8} 18.c4 {(c2c4) -0.24/6 4} Rxd5 {(Rd8xd5 c4xd5 Bb5xf1
Ra1xf1 Bh4g5+ f3f4 Kb8c8) +0.24/6 8} 19.cxd5 {(c4d5) -0.20/7 13} Bxf1 {(Bb5xf1 Ra1xf1
Kb8c8 Nd4e6 Rf8f7 Ne6xc7) +0.24/6 8} 20.Rxf1 {(a1f1) -0.13/7 5} Kc8 {(Kb8c8 Nd4e6
Rf8f7 Ne6f4 Bh4g5) +0.25/5 8} 21.Ne6 {(d4e6) -0.06/7 6} Rf7 {(Rf8f7 f3f4 Ne8f6 Ke3d4
Kc8d7) +0.25/5 8} 22.Rc1 {(f1c1) -0.06/7 11} Re7 {(Rf7e7 f3f4 a6a5 Rc1a1 b7b6) +0.24/5
7} 23.Bf4 {(e5f4) -0.07/7 10} a5 {(a6a5 Bf4e5 Bh4f6 Be5xf6 Ne8xf6) +0.26/5 7}
24.Kd3 {(e3d3) -0.01/7 9} Bf2 {(Bh4f2 Ne6c5 Re7e1 Rc1xe1 Bf2xe1) +0.26/5 7} 25.Nxc7
{(e6c7) +0.55/6 2} Kd7 {(Kc8d7 Nc7xe8 Kd7xe8 d5d6 Re7e1 Rc1xe1+ Bf2xe1) -1.50/6 7}
26.Ne6 {(c7e6) +1.16/7 10} Rf7 {(Re7f7 Rc1c3 h5h4 Kd3e4 Ne8f6+ Ke4e5) -1.30/5 7}
27.Rf1 {(c1f1) +0.97/6 3} Bb6 {(Bf2b6 Bf4e5 Kd7e7 f3f4 Ne8f6 Ne6g5) -1.31/6 7} 28.Be5
{(f4e5) +0.91/7 9} h4 {(h5h4 f3f4 Rf7f5 Kd3e4 Rf5h5) -1.31/5 7} 29.Kc4 {(d3c4) +0.88/6
2} Nd6+ {(Ne8d6+ Kc4d3 Nd6f5 f3f4 Nf5e3 Ne6g5) -1.30/5 7} 30.Kd3 {(c4d3) +1.00/7
5} Nf5 {(Nd6f5 Rf1c1 g7g6 Ne6c5+ Bb6xc5 Rc1xc5 Rf7e7) -1.36/6 7} 31.Re1 {(f1e1)
+0.99/6 2} Ne7 {(Nf5e7 Kd3e4 g7g6 Ne6d4 Bb6xd4 Be5xd4) -1.37/6 7} 32.Kc4 {(d3c4)
+1.02/7 9} h3 {(h4h3 g2xh3 Bb6f2 Re1e2 b7b5+ Kc4xb5 Ne7xd5 Re2xf2 Kd7xe6) -1.37/6 7}
33.Ng5 {(e6g5) +1.79/7 7} hxg2 {(h3xg2 Ng5xf7 g2g1Q Re1xg1 Bb6xg1 Be5xg7 Bg1xh2)
-1.35/6 7} 34.Nxf7 {(g5f7) +1.13/7 2} Bf2 {(Bb6f2 Re1d1 g2g1Q Rd1xg1 Bf2xg1 Be5xg7)
-1.35/6 7} 35.Rd1 {(e1d1) +1.12/8 8} Nf5 {(Ne7f5 Be5f4 g2g1Q Rd1xg1 Bf2xg1 d5d6)
-1.33/6 7} 36.Kb5 {(c4b5) +1.31/7 4} Ne3 {(Nf5e3 Rd1c1 g2g1Q Rc1xg1 Bf2xg1 d5d6)
-1.32/6 7} 37.Rc1 {(d1c1) +1.15/8 8} g1=Q {(g2g1Q Rc1xg1 Bf2xg1 Be5d4 Kd7e7 Nf7e5)
-1.13/6 7} 38.Rxg1 {(c1g1) +1.74/7 3} Bxg1 {(Bf2xg1 Be5d4 Kd7e7 Nf7e5 Bg1xh2 Ne5g6+
Ke7f7 Ng6h8+ Kf7g8 Bd4xe3 Kg8xh8 Kb5xa5) -1.49/7 7} 39.Bd4 {(e5d4) +1.63/8 3} Ke7
{(Kd7e7 Nf7g5 Ke7d6 Ng5h3 Kd6xd5 Bd4b6 Bg1xh2 Bb6xe3) -2.72/7 7} 40.Bxe3 {(d4e3)
+1.63/8 4} Bxe3 {(Bg1xe3 Nf7e5 Be3g1 Ne5c4 Bg1xh2 Nc4xa5) -1.36/8 6} 41.Ne5 {(f7e5)
+0.99/10 4} Bg1 {(Be3g1 Ne5c4 Bg1xh2 Nc4xa5 Ke7d6 Kb5c4 b7b6) -1.37/7 6} 42.h3 {(h2h3)
+0.81/10 9} a4 {(a5a4 Ne5c4 a4xb3 Kb5b4 b7b5 d5d6+ Ke7e6 Kb4xb5) -0.80/7 6} 43.bxa4
{(b3a4) +1.43/9 2} Kd6 {(Ke7d6 Ne5f7+ Kd6xd5 a4a5 Kd5d4 Nf7d8 Kd4e3 Kb5c4 Ke3xf3
Nd8xb7) -1.36/8 6} 44.Nc4+ {(e5c4) +0.82/10 2} Kxd5 {(Kd6xd5 Nc4a5 b7b6 Na5c4 Kd5d4
Nc4xb6 Kd4e3 a4a5 Ke3xf3) -1.35/8 6} 45.Na5 {(c4a5) +0.95/11 6} Kd4 {(Kd5d4 Na5xb7
Kd4e3 Nb7c5 Ke3xf3 a4a5 Kf3g3 Nc5e6) -1.36/8 6} 46.Nxb7 {(a5b7) +1.11/11 3} Ke3
{(Kd4e3 Nb7c5 Ke3xf3 Nc5e6 Kf3g3 Ne6xg7 Kg3xh3 a4a5) -1.37/9 6} 47.a5 {(a4a5) +1.78/11
8} Kxf3 {(Ke3xf3 Nb7d6 Bg1f2 Nd6f5 g7g5 Nf5d6 g5g4 h3xg4 Kf3xg4) -1.37/8 6} 48.a6
{(a5a6) +1.79/10 2} Kf4 {(Kf3f4 Nb7d8 g7g5 Nd8c6 g5g4 a6a7 Bg1xa7 Nc6xa7 g4xh3) -2.40/8
6} 49.Nc5 {(b7c5) +7.85/11 6} g5 {(g7g5 a6a7 Bg1xc5 Kb5xc5 Kf4g3 a7a8Q Kg3xh3
Kc5d5) -8.99/8 6} 50.a7 {(a6a7) +7.73/11 11} Bxc5 {(Bg1xc5 Kb5xc5 Kf4g3 a7a8Q Kg3xh3
Qa8f3+ Kh3h4 Qf3f2+ Kh4g4 Kc5d5) -8.97/8 6} 51.Kxc5 {(b5c5) +7.99/11 6} Kg3 {(Kf4g3
a7a8Q Kg3xh3 Qa8f3+ Kh3h4 Qf3f2+ Kh4h5 Qf2f5 Kh5h4 Kc5d5) -8.98/8 6} 52.a8=Q {(a7a8)
+8.87/13 3} Kxh3 {(Kg3xh3 Qa8f3+ Kh3h4 Qf3f2+ Kh4h5 Qf2f5 Kh5h4 Kc5d5) -8.98/6 6} 53.Qe4
{(a8e4) +9.00/11 2} g4 {(g5g4 Qe4e6 Kh3g2 Qe6xg4+) -8.80/6 6} 54.Kd4 {(c5d4) +9.02/11
2} g3 {(g4g3 Qe4h7+ Kh3g4 Kd4e4 g3g2 Qh7h2 Kg4g5 Qh2xg2+) -10.22/6 6} 55.Ke3
{(d4e3) 0.00/98} g2 {(g3g2 Ke3f3 g2g1Q Qe4e6+ Qg1g4+ Qe6xg4+) -M5/6 6} 56.Kf3 {(e3f3)
0.00/98} g1=N+ {(g2g1N+ Kf3f2 Ng1e2 Kf2xe2 Kh3g3 Ke2e3 Kg3h2) -10.28/6 6 White resigns}
{White resigns} 0-1

[bob]

I am lost. Why did white resign? Obviously the g1=N move is a mate in 6. But if black just promotes to queen it is an instant mate for white.

So why did he resign?

[nczempin]

I am lost. Why did white resign? Obviously the g1=N move is a mate in 6. But if black just promotes to queen it is an instant mate for white.

So why did he resign?

Must be either a bug in Arena, or in umax 4_0.

Perhaps there is some code that when an illegal move is made (which it is only from the pov of a non-sub-promoting engine), it automatically resigns rather than give an error message.

[Uri Blass]

I am lost. Why did white resign? Obviously the g1=N move is a mate in 6. But if black just promotes to queen it is an instant mate for white.

So why did he resign?

The reason is simply the fact that white does not understand underpromotion and the engine assume always promotion to queen.

The author did not want the engine to play illegal moves so he simply
added code to tell it to resign.

He probably thought that it is not counter productive for games because illegal move is a loss but in this example illegal move is a draw because black has no mating material even against the worst play of white
so if we care only about playing strength the interface should not resign for micromax in case that the opponent has only knight against queen(note that knight against pawn bishop knight or rook can win against bad play of the opponent).

Edit:I can add that there is a reason that the author did not tell the engine the underpromotion rule because the target of the author was not to write stronger engine but to write the strongest possible engine that has less than 2048 chars in the code and with these small number of chars it is better to implement things like hash tables(that he did) and not to implement understanding underpromotions.

Uri

[nczempin]

I am lost. Why did white resign? Obviously the g1=N move is a mate in 6. But if black just promotes to queen it is an instant mate for white.

So why did he resign?

The reason is simply the fact that white does not understand underpromotion and the engine assume always promotion to queen.

The author did not want the engine to play illegal moves so he simply
added code to tell it to resign.

He probably thought that it is not counter productive for games because illegal move is a loss but in this example illegal move is a draw because black has no mating material even against the worst play of white
so if we care only about playing strength the interface should not resign for micromax in case that the opponent has only knight against queen(note that knight against pawn bishop knight or rook can win against bad play of the opponent).


Edit:I can add that there is a reason that the author did not tell the engine the underpromotion rule because the target of the author was not to write stronger engine but to write the strongest possible engine that has less than 2048 chars in the code and with these small number of chars it is better to implement things like hash tables(that he did) and not to implement understanding underpromotions.

Uri

I think we are all aware of the intentions of the micromax project, and that that is the reason for the missing underpromotions.

Also, your explanation makes it clear to me that not the interface is responsible for the resignation, but the engine. AFAIK Arena takes into account insufficient material.

There are other reasons to postpone sub-promotions, all perfectly valid. I think hgm mentions on his site that he has never come across a situation where he would have needed it. Neither have I with Eden, but I had other goals for my engine and thus have included them. Nice to see that one situation where it payed off!

[nczempin]

So in fact it is probably not so much an issue of missing sub-promotion, but of how to handle illegal moves. My engine would complain, ignore the move and make its own, thus actually making an illegal move itself and be forfeited.

[hgm]

This works exactly as designed!

Perhaps some explanation is in order: uMax 4.x does not know underpromotion. It promotes every pawn that reaches 1st or 8th rank to Q. So if the opponent underpromotes to Knight, it cannot guarantee that its subsequent moves will be legal. For instance, in the given position it would play Qe4-h7# (it thinks). Because it assumes there is a Q on g1, and that it thus is not in check.

Now typically opponents under-promote only in about 1% of the games. (This is about an order of magnitude more than I had imagined before I started testing.) And not always in positions that are won to uMax. On the average it costs about 0.5% in score to forfeit on the illegal move or the unjust illegal-move claim that follows a promotion to N. So fixing it, would bring uMax 3.5 Elo. That means if it would take more than 3 characters to fix it, it would be counter-productive in terms of Elo/char, as the current ratio is slightly over 1. So far I have found no way to implement under-promotion with just 3 characters. So I just take the occasional loss.

Now some testers do not accept programs that can play illegal moves. So in order to let uMax qualify for testing, I had to make sure it wil not ever play any illegal moves. Therefore I let the WB interface (which I do not consider an integral part of uMax, but rather as a device to make user-friendly testing of uMax possible) remember when the opponent under-promotes to Knight. If then, on the next move, uMax does not come with a move that captures the Knight (with uMax still thinks is a Queen), the interface intervenes, and prints a resign message. Because we might be in-check, without knowing it. I figured this would be the major reason why opponents pick a Knight anyway. (Q is upward compatible with B or R, so that will hardly ever lead to illegal moves.)

Now this resign policy is a bit too liberal, and some opponents abuse it! In some positions, like

[d] 6Q1/8/3k4/6p1/8/8/3p3K/8 w

uMax sees no reason to hurry in stopping the promotion, and plays 1. Qxh5, with the idea to play 1. ..., d1=Q; 2. Qd8+, Ke5; 3. Qxd1. The opponent, seeing that he is going to lose the Pawn or what it promotes to no matter what, for some warped reason then chooses to promote to Knight! uMax, completely ignorant of this fact, then comes with Qd8+, but the interface substitutes a resign for this, and the vile trick of the opponent pays off! So I really should have the interface test if uMax does wander in check from that Knight, and not resign until it does (or if the opponent moves that Knight, which uMax would not accept).

Note that uMax 1.6 actually does accept under-promotions as input (at the expense of 32 characters), as I wanted it to play fully legal FIDE Chess, to avoid discussions if it really would qualify as the World's smallest Chess program (source-code wise).

[hgm]

He probably thought that it is not counter productive for games because illegal move is a loss but in this example illegal move is a draw because black has no mating material even against the worst play of white

We had a discussion about this on the Chess War chat a month or so ago. The engine Belofte resigned there in the Promo division, when the opponent had a bare King. We wondered if resigning in that position is legal or not. I think in the end Olivier corrected the score to draw.

[MartinBryant]

Unfortunately these non-underpromoting engines sometimes get away with murder!!!

Tao is a particular culprit and I (and several others) simply won't use it because of it.

Ages ago Colossus (playing black) had the following position against Tao 5.6 using the Fritz GUI.

[D]8/7p/1p4p1/P7/7P/1K6/2pq2k1/8 b - - 0 64

Colossus played c1=N+ which is mate in 3.
Now Tao doesn't bother with underpromotions even if the opponent plays them, just assumes all promotions are to queens, so in reply it calmly played the illegal axb
Now unfortunately the Fritz GUI (presumably because it expects only legal moves from engines) allowed it!!!
It then passed the illegal move to Colossus which had a thrombosis, crashed and lost on time!!!! GRRRRRRRRR!!!!!!

GUI WRITERS PLEASE NOTE! CHECK MOVES FROM ENGINES FOR LEGALITY!!

[nczempin]

He probably thought that it is not counter productive for games because illegal move is a loss but in this example illegal move is a draw because black has no mating material even against the worst play of white

We had a discussion about this on the Chess War chat a month or so ago. The engine Belofte resigned there in the Promo division, when the opponent had a bare King. We wondered if resigning in that position is legal or not. I think in the end Olivier corrected the score to draw.

Does it apply here? I will correct it...

umax had a string of 1 draw and then 6 losses at the start of the second round of the the silver gauntlet; I thought something had crashed, like perhaps I had removed the exe by mistake or something

The other Eden wins were legitimate, I'm happy to say! The draw was another three-moves draw in winning position from umax. Not sure if the 3moves is in 4.8, or how many chars it would cost to add it, but I am sure it is costing you quite a few half-points. That, plus pushing those passers and keeping the king from his frequent excursions, will probably be the most significant improvements I see on the eval-side (plus the centralization we talked about).

I think you should update your goal to 2048 characters, it is a much "rounder" figure than 2000 (My roundest birthday was 32, rounder than 30 I thought; at 16 I wasn't thinking in binary yet---something I seem to regress to---, and 64 seems such a long way away----Bob, you there yet?).