Stockfish underpromotes much more often than Komodo

[Laskos]

I tried to see initially which promotes where, but the most striking result was about underpromotions. I equalized the engines to have similar strength. 10,000 games between K and SF without adjudication at close to 10'' + 0.1'' time control (adjusted for strength)

Komodo 10.1 generally promotes more often than Stockfish dev, an important thing in itself, but underpromotes almost an order of magnitude less:

Code: Select all

Komodo                 SF dev
promotions             promotions
(under-pr)             (under-pr)  

2484                   1901
(18)                   (141)

Here are the details on which square promotions (and under-promotions) occur:

Code: Select all

      Komodo    SF dev
a8    482(0)    357(17)    
b8    320(4)    261(12)  
c8    279(3)    227(17)   
d8    220(3)    168(19)  
e8    216(3)    175(20) 
f8    286(3)    184(16)   
g8    322(2)    227(16)  
h8    359(0)    302(24)

And here to what pieces underpromotions occur:

Code: Select all

Komodo:  N=9,  B=4,  R=5
SF dev:  N=35, B=13, R=93

If I am not doing something wrong (I didn't go through the games manually), it seems Stockfish underpromotes a lot, especially to Rook.

[Lyudmil Tsvetkov]

That makes sense.

SF is winning more games through king attacks, and Komodo endgames other engines more, hence Komodo tends to promote more frequently.

Considering underpromotions, I have seen those SF 'underpromotions', and yes, they are very funny, many other engines also do so, probably because of some bug in the search/SEE/MVV-LVA.

Almost all SF underpromotions to rook are in positions where a queen promotion is changes nothing, as the opponent is obliged to capture on the very next move, either the promoted queen, or the underpromoted rook.

SF just prefers that a smaller-value piece(the rook) is captured(attacked)

Nothing changes, but I find SF approach aesthetically uglier for a human to look at, though it basically changes nothing, just evoking a funny feeling...

[Jouni]

I am curious Kai: how much you gave handicap to Komodo?

[Norm Pollock]

SF just prefers that a smaller-value piece(the rook) is captured(attacked)

Nothing changes, but I find SF approach aesthetically uglier for a human to look at, though it basically changes nothing, just evoking a funny feeling...

It makes it easier to spot clones.

[Dann Corbit]

I tried to see initially which promotes where, but the most striking result was about underpromotions. I equalized the engines to have similar strength. 10,000 games between K and SF without adjudication at close to 10'' + 0.1'' time control (adjusted for strength)

Komodo 10.1 generally promotes more often than Stockfish dev, an important thing in itself, but underpromotes almost an order of magnitude less:

Code: Select all

Komodo                 SF dev
promotions             promotions
(under-pr)             (under-pr)  

2484                   1901
(18)                   (141)

Here are the details on which square promotions (and under-promotions) occur:

Code: Select all

      Komodo    SF dev
a8    482(0)    357(17)    
b8    320(4)    261(12)  
c8    279(3)    227(17)   
d8    220(3)    168(19)  
e8    216(3)    175(20) 
f8    286(3)    184(16)   
g8    322(2)    227(16)  
h8    359(0)    302(24)

And here to what pieces underpromotions occur:

Code: Select all

Komodo:  N=9,  B=4,  R=5
SF dev:  N=35, B=13, R=93

If I am not doing something wrong (I didn't go through the games manually), it seems Stockfish underpromotes a lot, especially to Rook.

It would be very interesting to know what percentage of underpromotions are necessary, and what percentage are "I would rather lose a rook than a queen" sillyness.

[matthewlai]

I tried to see initially which promotes where, but the most striking result was about underpromotions. I equalized the engines to have similar strength. 10,000 games between K and SF without adjudication at close to 10'' + 0.1'' time control (adjusted for strength)

Komodo 10.1 generally promotes more often than Stockfish dev, an important thing in itself, but underpromotes almost an order of magnitude less:

Code: Select all

Komodo                 SF dev
promotions             promotions
(under-pr)             (under-pr)  

2484                   1901
(18)                   (141)

Here are the details on which square promotions (and under-promotions) occur:

Code: Select all

      Komodo    SF dev
a8    482(0)    357(17)    
b8    320(4)    261(12)  
c8    279(3)    227(17)   
d8    220(3)    168(19)  
e8    216(3)    175(20) 
f8    286(3)    184(16)   
g8    322(2)    227(16)  
h8    359(0)    302(24)

And here to what pieces underpromotions occur:

Code: Select all

Komodo:  N=9,  B=4,  R=5
SF dev:  N=35, B=13, R=93

If I am not doing something wrong (I didn't go through the games manually), it seems Stockfish underpromotes a lot, especially to Rook.

It would be very interesting to know what percentage of underpromotions are necessary, and what percentage are "I would rather lose a rook than a queen" sillyness.

I don't think it's "I would rather lose a rook than a queen" silliness. It's "it doesn't matter what I promote to because I'll lose it anyways so I'll flip a coin".

In minimax the leaf values are propagated up, so those two options propagate up the exact same value and one is picked randomly, and there's really nothing wrong with that. Unless you just don't like it aesthetically.

[Laskos]

That makes sense.

SF is winning more games through king attacks, and Komodo endgames other engines more, hence Komodo tends to promote more frequently.

Considering underpromotions, I have seen those SF 'underpromotions', and yes, they are very funny, many other engines also do so, probably because of some bug in the search/SEE/MVV-LVA.

Almost all SF underpromotions to rook are in positions where a queen promotion is changes nothing, as the opponent is obliged to capture on the very next move, either the promoted queen, or the underpromoted rook.

SF just prefers that a smaller-value piece(the rook) is captured(attacked)

Nothing changes, but I find SF approach aesthetically uglier for a human to look at, though it basically changes nothing, just evoking a funny feeling...

Indeed, Stockfish seems to win more often before the the endgame, I just plotted for the same database the lengths of won games, Stockfish is a bit quicker to mate:



For underpromotion, wasn't Stockfish code written with the goal to eliminate useless parts, which don't bring ELO? Isn't it easier to promote to Q unless forced by the position?

[Laskos]

It would be very interesting to know what percentage of underpromotions are necessary, and what percentage are "I would rather lose a rook than a queen" sillyness.

Yes, but I am unable to filter that.

[Lyudmil Tsvetkov]

It has long been known that all SF bugs are actually features.

Almost all such underpromotions (>95%) are the funny-looking-logically correct case. It is indeed very rare that an underpromotion to rook or bishop will be necessary because of avoiding study-position stalemates.

Only downside is that not all people run tests and compares stats, there are a handful that also look at what is played on the board, and unnecessary underpromotions somehow send a signal the engine is not able to make a mistake in a limited-depth exchange sequence, which is certainly true, though very weird to humans. We will get accustomed though, as soon as we start calculating as well as engines...

[Jouni]

Can somebody post position, where bishop promotion is THE ONLY winning or drawing move? It is not so easy to find any! This is nice
[pgn]
[Event "Dortmund op-A"]
[Site "Dortmund"]
[Date "1993.??.??"]
[Round "2"]
[White "Dragoslav Tomic"]
[Black "Frank Winzbeck"]
[Result "1-0"]
[ECO "B83"]
[PlyCount "87"]
[EventDate "1993.??.??"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 Nc6 6. Be2 e6 7. Be3 Be7 8.
Qd2 a6 9. O-O O-O 10. Rad1 Qc7 11. f4 Bd7 12. Nxc6 Bxc6 13. Bf3 Rad8 14. Qf2
Rd7 15. Bb6 Qb8 16. g4 d5 17. g5 Nxe4 18. Nxe4 dxe4 19. Rxd7 Bxd7 20. Bxe4 f5
21. Bd3 e5 22. Bc5 Qd8 23. h4 e4 24. Bc4+ Kh8 25. Bd4 Bc6 26. Rd1 Bc5 27. h5
Bxd4 28. Qxd4 Qxd4+ 29. Rxd4 g6 30. h6 a5 31. Kf2 Rc8 32. Ke3 Re8 33. Rd6 Rc8
34. Be6 Re8 35. c4 Rb8 36. b3 Be8 37. a3 b6 38. Bd5 b5 39. c5 b4 40. a4 Rc8 41.
c6 Rd8 42. c7 Rc8 43. Rd8 Rxd8 44. cxd8=B 1-0

[/pgn]
but unfortunately cxd8=N mates as well if longer.