250 ELO in ~ 10 months

[MikeB]

That is crazy , but that's what the SF - ZH (crazyhouse variant) team did.
Daniel Dugovic and Fabian Fichter and others did an incredible amount of work in a very short time period to make Stockfish - CrazyHouse virtually unbeatable by the top human players.

Even on level 7, Stockfish beat Yasser Seirawan (top 3 player in ZH) 36-12 on the 2 last weeks : https://lichess.org/@/yasser-seirawan/s ... order=desc

Daniel 's last post on GitHub " ..I searched for hours and hours for strange things in Stockfish play against humans. I'm done now !
Thanks to everyone for this amazing achievement !"

To celebrate this extraordinary event, I have folded ithe McB code into SF-ZH and with some suprise, found that it seemed to help!

parameters as recommended
ResultSet-EloRating>mm 0 1
ResultSet-EloRating>confidence 0.95

Code: Select all

Rank Name    Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 ZH-McB   3198   0.0   17   17  1684  953.0  56.6  928  706   50  55.1   3.0  3146 
   2 ZH       3176  21.7   17   17  1684  857.0  50.9  832  802   50  49.4   3.0  3165 
   3 CH1      2926 249.6   43   43   400   74.0  18.5   72  324    4  18.0   1.0  3187 
---------------------------------------------------------------------------------------------------------
  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

ResultSet-EloRating>los
        ZH ZH CH
ZH-McB     99100
ZH       0   100
CH1      0  0   
ResultSet-EloRating>

ZH is current SF-CH, CH1 is SF-CH from late December 2016 and ZH-McB is current SF-ZH with McB code overlaid.

Also added keyboard shortcuts and play by ELO - code will be released by end of week. Of course, Daniel and Fabian are free to add McB code if they view it as worthy. I can compile the macOS exe's, if somebody can help with the Linux and Windows versions that would be great.

[Volker Pittlik]

... if somebody can help with the Linux ... versions that would be great.

I can try.

Volker

[Fabian Fichter]

Michael, could you please add the time control and opening book used for the test? I could not find it in your post and it would be helpful to be able to compare the results with our tests.

[MikeB]

Sure - book: 2moves_v1.pgn
tc: 15 sec with 0.25 inc - which shows as zero in the pgn file

zip file of the
https://www.dropbox.com/s/6pceuidzwxr8v ... n.zip?dl=1

final result head to head with ZH-McB

Code: Select all

ResultSet>rp /Users/michaelbyrne/Documents/Results/zh1.pgn 
3119 game(s) loaded
ResultSet>elo
ResultSet-EloRating>mm 0 1
00:00:00,00
ResultSet-EloRating>confidence .95
0.9
ResultSet-EloRating>r
Rank Name    Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 ZH-McB   3106   0.0   12   12  3119 1613.5  51.7 1557 1449  113  49.9   3.6  3094 
   2 ZH       3094  12.1   12   12  3119 1505.5  48.3 1449 1557  113  46.5   3.6  3106 
---------------------------------------------------------------------------------------------------------
  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

ResultSet-EloRating>los
        ZH ZH
ZH-McB     97
ZH       2   
ResultSet-EloRating>

pgn filers for the RR between ZN, CH1 and ZN-McB
same tc, opening book is called 100_Equal.pgn , not sure where I got that from - both books are just normal chess openings - not zh specific books -but engines were playing from both sides.

https://www.dropbox.com/s/z58j67bymoaei ... n.zip?dl=1

Code: Select all

ResultSet-EloRating>x
ResultSet>reset
ResultSet>rp /Users/michaelbyrne/Documents/Results/zh.pgn 
600 game(s) loaded
ResultSet>elo
ResultSet-EloRating>mm 0 1
00:00:00,00
ResultSet-EloRating>confidence .95
0.9
ResultSet-EloRating>r
Rank Name    Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 ZH-McB   3196   0.0   38   38   400  271.0  67.8  268  126    6  67.0   1.5  3052 
   2 ZH       3175  21.2   37   37   400  255.0  63.7  252  142    6  63.0   1.5  3062 
   3 CH1      2928 247.1   43   43   400   74.0  18.5   72  324    4  18.0   1.0  3186 
---------------------------------------------------------------------------------------------------------
  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

amazing work on SF-ZH!

[MikeB]

Also here is the ZH code with the McB code overlaid - did not add the pawn extension since I was pretty sure that would not help ZH, also left out for now some brute force and no null move uci options - might add those two back for the final release - it does have the play by ELO code (limitStrength), and fast play when mate is found and some other McB code. No cerebellum opening book code of course.

https://www.dropbox.com/s/koay2rink9a87 ... B.zip?dl=1

[Fabian Fichter]

Thanks. I think that the opening book is the reason for the disprepancy between your test and our regression test results (they suggest a 50-100% higher Elo difference). A standard chess opening book probably contains many positions that are balanced in standard chess, but unbalanced in crazyhouse. Strongly unbalanced positions usually decrease the Elo difference if the difference in playing strength is big as it helps the weaker engine to win some games.

[MikeB]

Thanks. I think that the opening book is the reason for the disprepancy between your test and our regression test results (they suggest a 50-100% higher Elo difference). A standard chess opening book probably contains many positions that are balanced in standard chess, but unbalanced in crazyhouse. Strongly unbalanced positions usually decrease the Elo difference if the difference in playing strength is big as it helps the weaker engine to win some games.

Makes sense and I knew that it was at least 250 😊👍🤛🏻

[MikeB]

Thanks. I think that the opening book is the reason for the disprepancy between your test and our regression test results (they suggest a 50-100% higher Elo difference). A standard chess opening book probably contains many positions that are balanced in standard chess, but unbalanced in crazyhouse. Strongly unbalanced positions usually decrease the Elo difference if the difference in playing strength is big as it helps the weaker engine to win some games.

Makes sense and I knew that it was at least 250 😊👍🤛🏻

That also suggests that my McBrain code overlay should be worth more than the ELO suggested.😊

[corres]

[quote="Fabian Fichter"]

Strongly unbalanced positions usually decrease the Elo difference if the difference in playing strength is big as it helps the weaker engine to win some games.

[/quote]

??
Why do you think this?
I think just the opposite.
To a certain point the higher the unbalancing in the start positions the higher the chance of the stronger engine to win the competition with higher rate and higher Elo gain.

[jd1]

It surely depends on just how unbalanced the position is.

To take it to the extreme, if the position is easily won even a weak player can score 50% against a much stronger opponent.

On the other hand, I do remember some studies here showing that unbalanced openings increased the % of wins the stronger engine could score against a weaker opponent.

I suppose it's partly because a lot of openings are very drawish at the higher levels of computer chess. Also unbalanced openings gives the stronger engine to show its superiority by winning the favourable position and holding the return game to a draw, whereas with a perfectly equal opening the weaker engine is likely to get two draws.