Official Release of Ethereal 12.50

[chysiddh14]

great work ,wish this could be no. 1 chess engine in future ,thanks for andorid version.

[jonkr]

In a quick FRC bullet test, Ethereal 12.50 performed 34 elo better than Ethereal 12.25 against SlowChess 2.3, so FRC play looks clearly improved for this version (though my error bars are high, I only did the 1920 games for 12.50)

I like testing FRC but also always get higher gains there. I tried the 8-moves book and it does seem most drawish, then 2-moves book, then FRC shows largest difference. Possibly I should concentrate on standard more now to have one less performance skew to worry about. (Although with 2-moves book I started to worry that I was partially just arranging the eval to randomly end up reaching good positions in self-play from those positions.)

[ThatsIt]

Note that I use a book called 8moves_v3 for Regression Testing. This book is typically very conservative. For the last few releases, (aside from some Contempt nonsense) I have under estimated the performance via this testing. This is likely due to the drawish nature of the opening book. I could be wrong, but +17 elo is a painful amount at this level, so even the lower end would make me happy.

Test for Standard Chess

Code: Select all

ELO   | 16.97 +- 3.41 (95%)
CONF  | 60.0+0.6s Threads=1 Hash=64MB
Games | N: 11269 W: 1873 L: 1323 D: 8073

Test for Fischer Random Chess

Code: Select all

ELO   | 40.77 +- 4.62 (95%)
CONF  | 60.0+0.6s Threads=1 Hash=64MB
Games | N: 8432 W: 2139 L: 1154 D: 5139

At the moment +11 for our 40/4 + ... list.
But only ~ 600 games are played so far.

https://cegt.forumieren.com/t1354-testi ... -12-50-x64

Best wishes,
G.S.
(CEGT team)

[Modern Times]

In a quick FRC bullet test, Ethereal 12.50 performed 34 elo better than Ethereal 12.25 against SlowChess 2.3, so FRC play looks clearly improved for this version (though my error bars are high, I only did the 1920 games for 12.50)

We're showing more than +50 Elo so far at FRC compared to 12.00 which was the last one tested. A few more opponents to come.

[ThatsIt]

@ CEGT 40/4 (no FRC) so far:

Code: Select all

Ethereal 12.50 x64 1CPU    3291 (1000 games)  + 9 / +38
Ethereal 12.25 x64 1CPU    3282 (1500 games)  +29
Ethereal 12.00 x64 1CPU    3253 (3800 games)

Best wishes,
G.S.
(CEGT team)

[AndrewGrant]

@ CEGT 40/4 (no FRC) so far:

Code: Select all

Ethereal 12.50 x64 1CPU    3291 (1000 games)  + 9 / +38
Ethereal 12.25 x64 1CPU    3282 (1500 games)  +29
Ethereal 12.00 x64 1CPU    3253 (3800 games)

Best wishes,
G.S.
(CEGT team)

I'm disappointed that you are still testing Houdini. You essentially have 3x sets of games between Stockfish and Ethereal :/

[ThatsIt]

I'm disappointed that you are still testing Houdini. You essentially have 3x sets of games between Stockfish and Ethereal :/

3x sets?

Whom do you have in the suspicion too?

Best wishes,
G.S.
(CEGT team)

[AndrewGrant]

I'm disappointed that you are still testing Houdini. You essentially have 3x sets of games between Stockfish and Ethereal :/

3x sets?

Whom do you have in the suspicion too?

Time will tell

[Laskos]

Note that I use a book called 8moves_v3 for Regression Testing. This book is typically very conservative. For the last few releases, (aside from some Contempt nonsense) I have under estimated the performance via this testing. This is likely due to the drawish nature of the opening book. I could be wrong, but +17 elo is a painful amount at this level, so even the lower end would make me happy.

Test for Standard Chess

Code: Select all

ELO   | 16.97 +- 3.41 (95%)
CONF  | 60.0+0.6s Threads=1 Hash=64MB
Games | N: 11269 W: 1873 L: 1323 D: 8073

Test for Fischer Random Chess

Code: Select all

ELO   | 40.77 +- 4.62 (95%)
CONF  | 60.0+0.6s Threads=1 Hash=64MB
Games | N: 8432 W: 2139 L: 1154 D: 5139

You specifically coded the eval for FRC or the openings are generally better?
I guess your 17 Elo points from 8moves_v3 would be some 25 Elo points on 2moves_v1 of the SF testing framework

[AndrewGrant]

Note that I use a book called 8moves_v3 for Regression Testing. This book is typically very conservative. For the last few releases, (aside from some Contempt nonsense) I have under estimated the performance via this testing. This is likely due to the drawish nature of the opening book. I could be wrong, but +17 elo is a painful amount at this level, so even the lower end would make me happy.

Test for Standard Chess

Code: Select all

ELO   | 16.97 +- 3.41 (95%)
CONF  | 60.0+0.6s Threads=1 Hash=64MB
Games | N: 11269 W: 1873 L: 1323 D: 8073

Test for Fischer Random Chess

Code: Select all

ELO   | 40.77 +- 4.62 (95%)
CONF  | 60.0+0.6s Threads=1 Hash=64MB
Games | N: 8432 W: 2139 L: 1154 D: 5139

You specifically coded the eval for FRC or the openings are generally better?
I guess your 17 Elo points from 8moves_v3 would be some 25 Elo points on 2moves_v1 of the SF testing framework

The evaluation was tuned using a ~66% ~33% mix of Standard games and FRC games.
Although the Fischer book does inflate elo more than 8moves_v3 does.