Official Release of Ethereal 12.50

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

AndrewGrant
Posts: 1754
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Official Release of Ethereal 12.50

Post by AndrewGrant »

Hello all. Today we are officially releasing Ethereal 12.50. This version shows modest gains over 12.25, and sharp gains over 12.25 when playing Fischer Random Chess. Tests are still pending, and I will be posting more about that below. This version adds an additional setting named "AnalysisMode", whose sole purpose is to disable some of the Syzygy tablebase filtering tricks used in endgames. This only really serves a purpose if you are using an engine in a < 7 piece position. The major changes since the last release are an updated Tuner, Tuning method, and higher quality and quantity of datasets, refined using a novel approach. You can read more about this here. This is only a draft, and I plan to finish the document up in the near future, and store it in the Ethereal repo.

Binaries for Windows & Android (ARMv7, ARMv8) can be found here. As always, I highly suggest that anyone capable of building their own Ethereal do so. A trivial "make", or "make pext" (For BMI2/PEXT machines) will suffice, under any resonable gcc. Normally I provide a NOPOPCNT, POPCNT, and PEXT binary. This time, I will be adding two other ones to the list: SEE41, and SEE41-PEXT. These may be a bit faster, assuming your CPU can support them. Please try them before defaulting to the POPCNT or PEXT compiles. Again, please compile your own Ethereal if you can.

The following are all the patches:

Code: Select all

12.26: Tune all of the search margins using the skopt tuner
12.27: Penalize Queen's that may be prone to discovered attacks
12.28: Have QueenRelativePin only evaluate truly discovered attacks
12.29: Use knowledge of fail-highs/fail-lows to tweak LMR
12.30: Revert all of the changes from V12.29
12.31: Apply a slight non-functional speedup to Probcut
12.32: Fix a hanging pthread which used up resources over time
12.33: Add UCI option "AnalysisMode" to toggle TB behaviour
12.34: Replace Fathom with Pyrrhic, adding 7-man support
12.35: Kick up the version number due to Pyrrhic
12.36: Update to latest Pyrrhic and fix TB_LARGEST bugs
12.37: Tune all evaluation terms using a new dataset methodology
12.38: Tune and tweak King Safety interactions using a Standard book
12.39: Delete the old tuning code and upload the new AdaGrad tuner
12.40: Add an option to the tuner to print verbatim source code
12.41: Tune every term in the evaluation using a Standard book
12.42: Tune all of the "normal" terms using a larger Standard book
12.43: Increase max hash usage to 128GB
12.44: Redefine how we delta prune in the qsearch
12.45: Tune all of the "complexity" terms using a Standard book
12.46: Tune all of the "normal" terms using a FRC book
12.47: Reduce many of the pruning margins in the search
12.48: Add a 512KB Evaluation Cache to each Thread
12.49: Make the PSQTs Asymmetrical ([64], not [32])
12.50: Tune the PSQT using a mix of FRC + Standard games
OpenBench is still churning away, supporting the development of Ethereal, Laser, Demolito, Weiss, RubiChess, FabChess, Igel, and Winter. OpenBench is a Fishtest-like platform, geared towards supporting engines of all varieties. Currently we are running engines in C, C++, and Rust. We are also pulling down custom NNUE files for development on that front.

Setting up the Client to contribute games is very manageable for Windows, and downright trivial for Linux. I'm happy to answer any questions here, through my email (andrew@grantnet.us), or on Discord (Andrews#9072), for anyone who is interested in supporting the development of the many open-source engines housed on OpenBench.

As always, a big thanks to Noobpwnftw for his support on the framework. All contributors of CPU time to OpenBench can be found here
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
User avatar
Thomas Lagershausen
Posts: 328
Joined: Mon Jun 11, 2007 6:59 pm

Re: Official Release of Ethereal 12.50

Post by Thomas Lagershausen »

i like Ethereal because chess isnt solved by Stockfish nnue !

Look at this:
[d]r5rk/1bq1bpp1/1p1ppn1p/pP5P/2P1P3/P1N2P2/1B1Q1P2/1K1R1BR1 w - - 0 19 bm Nd5

Stockfish nnue &Co didn´t have any clue.

Ethereal12.50-x64-popcnt:
37/65 03:28 1.652.261.891 7.911.000 +1,10 Dd2-e3 e6-e5 Sc3-d5 Lb7xd5 e4xd5 Sf6-d7 f3-f4 Le7-f6 Lf1-d3 Sd7-c5 f4xe5 Lf6xe5 Lb2xe5 d6xe5 Tg1-g4 Dc7-e7 Ld3-c2 Ta8-e8 f2-f3 a5-a4 Td1-g1 De7-d6 Tg1-g2 Dd6-f6 Tg2-g3 Te8-c8 De3-c3 Tc8-e8 Kb1-a2 Te8-a8 Tg3-g1 Ta8-e8 Dc3-e3 Te8-f8 Tg1-g2 Tf8-c8 De3-d2 Tc8-a8 Tg2-g1 Ta8-c8 Tg1-g3 Tc8-d8 Ka2-b1 Td8-e8 Tg3-g1 Te8-f8 Tg4-g3 e5-e4 Dd2-e3 Tf8-e8 f3xe4
38/60+ 05:39 2.525.504.260 7.445.000 +1,24 Sc3-d5
38/63+ 05:59 2.709.374.323 7.542.000 +1,38 Sc3-d5
38/60+ 07:05 3.135.051.741 7.363.000 +1,90 Sc3-d5
38/57 09:06 4.004.090.014 7.333.000 +1,91 Sc3-d5 Lb7xd5 e4xd5 e6-e5 f3-f4 e5-e4 Dd2-e3 Ta8-e8 Td1-c1 Sf6-d7 Lf1-h3 Le7-f6 Lh3xd7 Lf6xb2 Kb1xb2 Dc7xd7 De3xb6 Dd7-f5 Db6-e3 Te8-e7 Kb2-b3 f7-f6 Tg1-h1 Tg8-b8 a3-a4 Tb8-e8 Th1-h4 Df5-d7 c4-c5 d6xc5 Tc1xc5 Dd7-d6 Th4-h3 Kh8-g8 Th3-g3 Te7-d7 Kb3-a2 Te8-d8 De3-d4 Dd6xf4 b5-b6
TL
Frank Quisinsky
Posts: 6808
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Official Release of Ethereal 12.50

Post by Frank Quisinsky »

Andrew,

yes, yes ... I have to start directly the download order!

Ethereal The Fighter (high move-average without resign parameter) but all what Ehtereal produced is great material for my opening analyzes and database stats.

You remember on FCP Toruney-2020 round 2, game vs. Fizbo!
The position I have on my start page ... many people like it.

And Komodo will get a problem to hold place 2.

Andrew, you must invested a lot of time in such a strong engines Ethereal is.
Thanks for your time, your interesting engine.

You can be sure the Komodo team saw that and will start with a newer version the fight for place 2 against the new star on engine-sky.

Keep up the good work!

Best
frank
User avatar
Guenther
Posts: 4606
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: Official Release of Ethereal 12.50

Post by Guenther »

AndrewGrant wrote: Tue Sep 08, 2020 12:05 pm Hello all. Today we are officially releasing Ethereal 12.50.

...
Thanks Andrew!
https://rwbc-chess.de

trollwatch:
Chessqueen + chessica + AlexChess + Eduard + Sylwy
User avatar
Kotlov
Posts: 266
Joined: Fri Jul 10, 2015 9:23 pm
Location: Russia

Re: Official Release of Ethereal 12.50

Post by Kotlov »

AndrewGrant wrote: Tue Sep 08, 2020 12:05 pm Hello all. Today we are officially releasing Ethereal 12.50.
Great work!
Eugene Kotlov
Hedgehog 2.1 64-bit coming soon...
AndrewGrant
Posts: 1754
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: Official Release of Ethereal 12.50

Post by AndrewGrant »

Note that I use a book called 8moves_v3 for Regression Testing. This book is typically very conservative. For the last few releases, (aside from some Contempt nonsense) I have under estimated the performance via this testing. This is likely due to the drawish nature of the opening book. I could be wrong, but +17 elo is a painful amount at this level, so even the lower end would make me happy.

Test for Standard Chess

Code: Select all

ELO   | 16.97 +- 3.41 (95%)
CONF  | 60.0+0.6s Threads=1 Hash=64MB
Games | N: 11269 W: 1873 L: 1323 D: 8073
Test for Fischer Random Chess

Code: Select all

ELO   | 40.77 +- 4.62 (95%)
CONF  | 60.0+0.6s Threads=1 Hash=64MB
Games | N: 8432 W: 2139 L: 1154 D: 5139
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
Terje
Posts: 347
Joined: Tue Nov 19, 2019 4:34 am
Location: https://github.com/TerjeKir/weiss
Full name: Terje Kirstihagen

Re: Official Release of Ethereal 12.50

Post by Terje »

Great progress :)
peter
Posts: 3186
Joined: Sat Feb 16, 2008 7:38 am
Full name: Peter Martan

Re: Official Release of Ethereal 12.50

Post by peter »

Hi Thomas!
Thomas Lagershausen wrote: Tue Sep 08, 2020 12:30 pm i like Ethereal because chess isnt solved by Stockfish nnue !

Look at this:
[d]r5rk/1bq1bpp1/1p1ppn1p/pP5P/2P1P3/P1N2P2/1B1Q1P2/1K1R1BR1 w - - 0 19 bm Nd5

Stockfish nnue &Co didn´t have any clue.
I like Ethereal a lot too, but not any clue is to harsh against SF NNUE as for this one position from Morozevich- Vitiugov, Reggio Emilia 2011.

One more time it may depend on hybrid- eval, and on the net used. This one was Use NNUE Pure and Dietrich Kappe's NightNurse 0.2:

r5rk/1bq1bpp1/1p1ppn1p/pP5P/2P1P3/P1N2P2/1B1Q1P2/1K1R1BR1 w - - 0 1

Analysis by Cfish 040920 64 POPCNT NUMA:

Code: Select all

19.Bd3 Nd7 20.f4 Nc5 21.f5 e5 22.Rh1 
  =  (0.15)   Depth: 7/7   00:00:00  36kN
19.Be2 Qc5 20.Bd3 Qxh5 21.f4 e5 22.Qc2 exf4 
  =  (0.20)   Depth: 8/9   00:00:00  77kN
19.Qe3 Rab8 20.Na4 Ba8 21.Qf4 e5 22.Qf5 Rgc8 23.Bd3 
  =  (0.27)   Depth: 9/11   00:00:00  102kN
...
19.Qe3 Rab8 20.Bd3 Ba8 21.Na4 e5 22.Nc3 Nd7 23.Nd5 Bxd5 24.exd5 Nc5 25.f4 exf4 26.Qxf4 Bg5 27.Qf3 Nxd3 28.Qxd3 Rbc8 29.Qf5 Qe7 30.f4 Bf6 31.Bxf6 Qxf6 32.Qxf6 gxf6 33.Rxg8+ Kxg8 
  +/=  (0.44)   Depth: 25/34   00:00:06  59222kN
19.Qe3 Rab8 
  +/=  (0.36 --)   Depth: 26/38   00:00:09  83158kN
19.Qe3 
  +/=  (0.44 ++)   Depth: 26/38   00:00:09  85716kN
19.Bd3 Rad8 20.Qe3 d5 21.cxd5 exd5 22.Bc2 d4 23.Rxd4 Rxd4 24.Qxd4 Rd8 25.Qe3 Bc5 26.Qxh6+ gxh6 27.Nd5 Qd6 28.Bxf6+ Qxf6 29.Nxf6 Bc8 30.Ng4 Bxg4 31.Rxg4 Rd2 32.Rf4 Bxa3 33.Rxf7 Rxf2 34.Ka2 
  +/=  (0.50)   Depth: 26/40   00:00:12  109MN
...
19.Bd3 e5 20.f4 Nd7 21.Qe3 Qc5 22.Qxc5 Nxc5 23.fxe5 dxe5 24.Bc2 Rac8 25.f3 f6 26.Bc1 Ne6 27.Na4 Nd4 28.Nxb6 Rc7 29.Bd3 Nxf3 30.Rg3 Nd4 31.Nd5 Bxd5 32.exd5 Bd6 33.Re1 Re8 34.Rg6 e4 35.Rxe4 Rxe4 
  +/=  (0.63)   Depth: 31/45   00:00:59  515MN
19.Bd3 e5 
  +/=  (0.55 --)   Depth: 32/48   00:01:01  539MN
19.Bd3 
  +/=  (0.63 ++)   Depth: 32/48   00:01:06  583MN
19.Nd5 
  +/-  (0.76 ++)   Depth: 32/48   00:01:18  682MN
19.Nd5 
  +/-  (0.94 ++)   Depth: 32/48   00:01:23  730MN
19.Nd5 Bxd5 20.exd5 Qc5 21.dxe6 fxe6 22.Bd3 Raf8 23.Qe2 e5 24.Rg6 Rf7 25.f4 Re8 26.fxe5 dxe5 27.Qxe5 Qxe5 28.Bxe5 Bxa3 29.Bd4 Bc5 30.Bxc5 bxc5 31.Rgg1 Ne4 32.Rg2 Rf6 33.Bxe4 Rxe4 34.Rd8+ Kh7 
  +/-  (0.91)   Depth: 32/48   00:01:38  860MN
...
19.Nd5 Bxd5 20.cxd5 e5 21.Rc1 Qd8 22.Rc6 Nh7 23.f4 Bf6 24.Bd3 Re8 25.Bc2 Nf8 26.fxe5 Bxe5 27.Bxe5 Rxe5 28.Qd4 Qf6 29.Qxb6 Rxh5 30.Rxd6 Qe7 31.f4 Rh4 32.Qd4 Ng6 33.Ra6 Rb8 34.d6 Qd7 35.Rxa5 Rxf4 36.Qd3 Ne5 37.Qd5 
  +/-  (1.36)   Depth: 33/50   00:02:29  1301MN
...
19.Nd5 Bxd5 20.cxd5 e5 21.Rc1 Qb8 22.Rc6 Nh7 23.f4 Bf6 24.fxe5 Bxe5 25.Bxe5 dxe5 26.Qe3 Qd8 27.Rxb6 Qh4 28.Bd3 Qh2 29.Rc1 Qxh5 30.Rbc6 Nf6 31.Ka1 Rgd8 32.b6 Rab8 33.Ba6 Qh4 34.f3 Ne8 35.Qc5 Qh3 36.Qxa5 Qxf3 37.Qb4 f5 38.b7 fxe4 39.Rc8 Kh7 40.Rxd8 
  +/-  (1.55)   Depth: 34/51   00:04:36  2397MN
...
19.Nd5 Bxd5 20.cxd5 e5 21.Rc1 Qd8 22.Rc6 Rc8 23.Qe3 Qe8 24.Qxb6 a4 25.Qe3 Nd7 26.f4 f6 27.Bh3 Rb8 28.Be6 Rxb5 29.Bxg8 Kxg8 30.fxe5 Nxe5 31.Qxh6 Qf7 32.Rc3 Bf8 33.Qd2 Qxh5 34.Qc2 Qh2 35.Rc1 Rb8 36.Qxa4 Qxf2 37.R1c2 Qf4 38.Rc8 Qf1+ 39.Ka2 Rb5 40.Ra8 Nf7 41.Rcc8 
  +/-  (1.60)   Depth: 35/52   00:05:54  3071MN
Second best of three trials, best was 50", worst 1'35".
24 threads of 12x3GHz Xeon SSE4.1- popcnt.- compile, 32G hash.

Sorry for hitchhiking the thread and thanks to Andrew for new Ethereal regards
Peter.
Gabor Szots
Posts: 1362
Joined: Sat Jul 21, 2018 7:43 am
Location: Szentendre, Hungary
Full name: Gabor Szots

Re: Official Release of Ethereal 12.50

Post by Gabor Szots »

Thanks Andrew.

For me your PEXT compile is fastest and my own compile using make pext is significantly slower. I guess GCC 10 is not the best ever.
Gabor Szots
CCRL testing group
Paloma
Posts: 1167
Joined: Thu Dec 25, 2008 9:07 pm
Full name: Herbert L

Re: Official Release of Ethereal 12.50

Post by Paloma »

Thanks Andrew!