Page 1 of 3
250 ELO in ~ 10 months
Posted: Thu Oct 26, 2017 6:56 am
by MikeB
That is crazy , but that's what the SF - ZH (crazyhouse variant) team did.
Daniel Dugovic and
Fabian Fichter and others did an incredible amount of work in a very short time period to make Stockfish - CrazyHouse virtually unbeatable by the top human players.
Even on level 7, Stockfish beat Yasser Seirawan (top 3 player in ZH) 36-12 on the 2 last weeks :
https://lichess.org/@/yasser-seirawan/s ... order=desc
Daniel 's last post on GitHub " ..I searched for hours and hours for strange things in Stockfish play against humans. I'm done now !
Thanks to everyone for this amazing achievement !"
To celebrate this extraordinary event, I have folded ithe McB code into SF-ZH and with some suprise, found that it seemed to help!
parameters as recommended
ResultSet-EloRating>mm 0 1
ResultSet-EloRating>confidence 0.95
Code: Select all
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
---------------------------------------------------------------------------------------------------------
1 ZH-McB 3198 0.0 17 17 1684 953.0 56.6 928 706 50 55.1 3.0 3146
2 ZH 3176 21.7 17 17 1684 857.0 50.9 832 802 50 49.4 3.0 3165
3 CH1 2926 249.6 43 43 400 74.0 18.5 72 324 4 18.0 1.0 3187
---------------------------------------------------------------------------------------------------------
Δ = delta from the next higher rated opponent
# = number of games played
Σ = total score, 1 point for win, 1/2 point for draw
ResultSet-EloRating>los
ZH ZH CH
ZH-McB 99100
ZH 0 100
CH1 0 0
ResultSet-EloRating>
ZH is current SF-CH, CH1 is SF-CH from late December 2016 and ZH-McB is current SF-ZH with McB code overlaid.
Also added keyboard shortcuts and play by ELO - code will be released by end of week. Of course, Daniel and Fabian are free to add McB code if they view it as worthy. I can compile the macOS exe's, if somebody can help with the Linux and Windows versions that would be great.
Re: 250 ELO in ~ 10 months
Posted: Thu Oct 26, 2017 7:18 am
by Volker Pittlik
MikeB wrote:... if somebody can help with the Linux ... versions that would be great.
I can try.
Volker
Re: 250 ELO in ~ 10 months
Posted: Thu Oct 26, 2017 9:00 am
by Fabian Fichter
Michael, could you please add the time control and opening book used for the test? I could not find it in your post and it would be helpful to be able to compare the results with our tests.
Re: 250 ELO in ~ 10 months
Posted: Thu Oct 26, 2017 12:50 pm
by MikeB
Sure - book: 2moves_v1.pgn
tc: 15 sec with 0.25 inc - which shows as zero in the pgn file
zip file of the
https://www.dropbox.com/s/6pceuidzwxr8v ... n.zip?dl=1
final result head to head with ZH-McB
Code: Select all
ResultSet>rp /Users/michaelbyrne/Documents/Results/zh1.pgn
3119 game(s) loaded
ResultSet>elo
ResultSet-EloRating>mm 0 1
00:00:00,00
ResultSet-EloRating>confidence .95
0.9
ResultSet-EloRating>r
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
---------------------------------------------------------------------------------------------------------
1 ZH-McB 3106 0.0 12 12 3119 1613.5 51.7 1557 1449 113 49.9 3.6 3094
2 ZH 3094 12.1 12 12 3119 1505.5 48.3 1449 1557 113 46.5 3.6 3106
---------------------------------------------------------------------------------------------------------
Δ = delta from the next higher rated opponent
# = number of games played
Σ = total score, 1 point for win, 1/2 point for draw
ResultSet-EloRating>los
ZH ZH
ZH-McB 97
ZH 2
ResultSet-EloRating>
pgn filers for the RR between ZN, CH1 and ZN-McB
same tc, opening book is called 100_Equal.pgn , not sure where I got that from - both books are just normal chess openings - not zh specific books -but engines were playing from both sides.
https://www.dropbox.com/s/z58j67bymoaei ... n.zip?dl=1
Code: Select all
ResultSet-EloRating>x
ResultSet>reset
ResultSet>rp /Users/michaelbyrne/Documents/Results/zh.pgn
600 game(s) loaded
ResultSet>elo
ResultSet-EloRating>mm 0 1
00:00:00,00
ResultSet-EloRating>confidence .95
0.9
ResultSet-EloRating>r
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
---------------------------------------------------------------------------------------------------------
1 ZH-McB 3196 0.0 38 38 400 271.0 67.8 268 126 6 67.0 1.5 3052
2 ZH 3175 21.2 37 37 400 255.0 63.7 252 142 6 63.0 1.5 3062
3 CH1 2928 247.1 43 43 400 74.0 18.5 72 324 4 18.0 1.0 3186
---------------------------------------------------------------------------------------------------------
Δ = delta from the next higher rated opponent
# = number of games played
Σ = total score, 1 point for win, 1/2 point for draw
amazing work on SF-ZH!
Re: 250 ELO in ~ 10 months
Posted: Thu Oct 26, 2017 1:09 pm
by MikeB
Also here is the ZH code with the McB code overlaid - did not add the pawn extension since I was pretty sure that would not help ZH, also left out for now some brute force and no null move uci options - might add those two back for the final release - it does have the play by ELO code (limitStrength), and fast play when mate is found and some other McB code. No cerebellum opening book code of course.
https://www.dropbox.com/s/koay2rink9a87 ... B.zip?dl=1
Re: 250 ELO in ~ 10 months
Posted: Thu Oct 26, 2017 10:59 pm
by Fabian Fichter
Thanks. I think that the opening book is the reason for the disprepancy between your test and our regression test results (they suggest a 50-100% higher Elo difference). A standard chess opening book probably contains many positions that are balanced in standard chess, but unbalanced in crazyhouse. Strongly unbalanced positions usually decrease the Elo difference if the difference in playing strength is big as it helps the weaker engine to win some games.
Re: 250 ELO in ~ 10 months
Posted: Fri Oct 27, 2017 12:14 am
by MikeB
Fabian Fichter wrote:Thanks. I think that the opening book is the reason for the disprepancy between your test and our regression test results (they suggest a 50-100% higher Elo difference). A standard chess opening book probably contains many positions that are balanced in standard chess, but unbalanced in crazyhouse. Strongly unbalanced positions usually decrease the Elo difference if the difference in playing strength is big as it helps the weaker engine to win some games.
Makes sense and I knew that it was at least 250 😊👍🤛🏻
Re: 250 ELO in ~ 10 months
Posted: Fri Oct 27, 2017 12:16 am
by MikeB
MikeB wrote:Fabian Fichter wrote:Thanks. I think that the opening book is the reason for the disprepancy between your test and our regression test results (they suggest a 50-100% higher Elo difference). A standard chess opening book probably contains many positions that are balanced in standard chess, but unbalanced in crazyhouse. Strongly unbalanced positions usually decrease the Elo difference if the difference in playing strength is big as it helps the weaker engine to win some games.
Makes sense and I knew that it was at least 250 😊👍🤛🏻
That also suggests that my McBrain code overlay should be worth more than the ELO suggested.😊
Re: 250 ELO in ~ 10 months
Posted: Sat Oct 28, 2017 11:34 am
by corres
[quote="Fabian Fichter"]
Strongly unbalanced positions usually decrease the Elo difference if the difference in playing strength is big as it helps the weaker engine to win some games.
[/quote]
??
Why do you think this?
I think just the opposite.
To a certain point the higher the unbalancing in the start positions the higher the chance of the stronger engine to win the competition with higher rate and higher Elo gain.
Re: 250 ELO in ~ 10 months
Posted: Sat Oct 28, 2017 4:06 pm
by jd1
It surely depends on just how unbalanced the position is.
To take it to the extreme, if the position is easily won even a weak player can score 50% against a much stronger opponent.
On the other hand, I do remember some studies here showing that unbalanced openings increased the % of wins the stronger engine could score against a weaker opponent.
I suppose it's partly because a lot of openings are very drawish at the higher levels of computer chess. Also unbalanced openings gives the stronger engine to show its superiority by winning the favourable position and holding the return game to a draw, whereas with a perfectly equal opening the weaker engine is likely to get two draws.