Official Release of Ethereal9.00

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

AndrewGrant
Posts: 1754
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Official Release of Ethereal9.00

Post by AndrewGrant »

Hello Again Everyone,

Today I am announcing the release of Ethereal9.00. It's been a wild ride since
adding lazy SMP a few months ago in order to participate in TCEC. A big thanks
to Knight Moves and Anton for having me. It's provided inspriation for a few
patches, and for a handful of bug fixes.

Link to the Github Release Page: https://github.com/AndyGrant/Ethereal/r ... /tag/V9.00

Link to the 64bit Windows Binary: https://github.com/AndyGrant/Ethereal/r ... -Win64.exe

Link to the Android Binary: https://github.com/AndyGrant/Ethereal/r ... ndroid.exe

Link to a MAC Binary: If someone will provide one :)

Everytime I take a guess at the elo gains it comes back to bite me, so I'll just
say this; Ethereal is a hell of a lot better with you throw many cores at it. In
fast testing with 4CPUs I am seeing over 250 elo. Obviously that number will not
hold in CCRL conditions, but I expect this version of Ethereal will perform far
better than Ethereal8.61 in conditions like TCEC with 43 cores to work with.

Here is the recap for changes since 8.61, for those interested:

Code: Select all

    8.61 : Fix a bug with how the main scheduling lock was setup
    8.62 : More aggressive time allocation in X/Y games
    8.63 : Allow the aspiration window to be as small as [-1, 1]
    8.64 : Again more aggressive time allocation in X/Y games
    8.65 : Penalty in the eval for a queen attacked by a pawn
    8.66 : Prune weak captures that are likely to fail in QS, with
           a short / cheap version of SEE. Idea by Dennis Sceviour!
    8.67 : Don't apply the weak capture pruning to promotion moves
    8.68 : Better time managment in X+Y and X/Y games
    8.69 : Introduction of ProbCut, inspired by the Stockfish implementation
    8.70 : Apply the weak capture pruning in the search as well as qsearch
    8.71 : More aggressive use of weak capture pruning in the search (depth 5)
    8.72 : Allow ProbCut to refute a null move (Unlike SF)
    8.73 : Allow for a different upper and lower margin on the aspiration window,
           by looking at the last 3 search scores. Minumum bound is 4. I believe
           this idea is unique to Ethereal -- at least I have never seen it elsewhere
    8.74 : Bonus for Pawns and Minors in the King's Safety Area
    8.75 : Fix an issue where helper threads were able to allocate more search time,
           such that with many threads the max time would be used with every move.
           This was found thanks to TCEC, and was worth ~70 ELO in selfplay
    8.76 : Take a guess at the time of the next search and consider stopping now
    8.77 : Lower base time usage, and allow for much more massive time allocations
    8.78 : Add an official Logo for Ethereal, created by DanishStyleFetta!
    8.79 : Clean up some threading code (reduce LOC) and allow for max depth of 128
    8.80 : Fix a bug where all helper threads would always use the aspiration window
           centered around the value zero. This was worth ~80 ELO in selfplay
    8.81 : Skip Weak Capture Pruning when in check
    8.82 : Implement a basic form of Pawn/King Shelter
    8.83 : Use a generalized Piece Square Table for sorting quiet moves
    8.84 : Penalty for Minors attacked by Pawns
    8.85 : Do not place any value on the R value for Null Move Pruning
    8.86 : New time allocation scheme to unify X+Y and X/Y (later reworked...)
    8.87 : Resolve a bug where Ethereal could, on very rare occasion, play an illegal
           move as a result of the PV not being filled AT ALL, even in the root node
    8.88 : Allow beta pruning (static null / futility) when we only have Pawns
    8.89 : When doing early pruning (NMP, Razoring, Static Null, and Probcut), always
           return alpha or beta. The goal was to make more conservative TT entries
    8.90 : Lower the margin for futility pruning / beta pruning to about one pawn
    8.91 : Make Ethereal's history counters more like the Stockfish history counters
    8.92 : Yet another patch to allow for more aggressive time allocation
    8.93 : (CURRENT TM) Allocate three things, an ideal usage, a max usage, and a max
           alloc. We will NEVER exceed max usage, and will never start another iteration
           after exceeding max alloc. Overtime we adjust a score and pv stability factor
           to increase or decrease allocated time with respect to pv stability and scoring
    8.94 : Fixed an issue with true-sharing of the transposition table used counter. Now
           we sample the first entries in the TT and approximate the hashfull. This was a
           huge elo winner with high thread counts
    8.95 : Increase base LMR value by one, and reduce it by one when the best move is
           a quiet move and the TT move has failed or this is our first quiet attempts
    8.96 : Cleanup of capture value pruning decisions and some inconsitencies
    8.97 : Remove a pointless flag from the makefile, thanks Laldon!
    8.98 : New method of Pawn/King Shelter definition. Before we would be okay with a
           King pushing past his pawns, so long as they were still close
    8.99 : One big fat tuning patch to update everything the last tune
    9.00 : Cleanup the Texel Tuning implementation by using about 10% of the memory
           as before, resulting in a 700% speed up in a quick test of tuning

Thanks to everyone who has taken an interest in Ethereal over the last few months.
The ideas and suggestions from the TCEC chat have been incredible.
And, as always, thank you to the testers :)

Thanks,
Andrew Grant
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
jd1
Posts: 269
Joined: Wed Oct 24, 2012 2:07 am

Re: Official Release of Ethereal9.00

Post by jd1 »

AndrewGrant wrote:Hello Again Everyone,

Today I am announcing the release of Ethereal9.00. It's been a wild ride since
adding lazy SMP a few months ago in order to participate in TCEC. A big thanks
to Knight Moves and Anton for having me. It's provided inspriation for a few
patches, and for a handful of bug fixes.

Link to the Github Release Page: https://github.com/AndyGrant/Ethereal/r ... /tag/V9.00

Link to the 64bit Windows Binary: https://github.com/AndyGrant/Ethereal/r ... -Win64.exe

Link to the Android Binary: https://github.com/AndyGrant/Ethereal/r ... ndroid.exe

Link to a MAC Binary: If someone will provide one :)

Everytime I take a guess at the elo gains it comes back to bite me, so I'll just
say this; Ethereal is a hell of a lot better with you throw many cores at it. In
fast testing with 4CPUs I am seeing over 250 elo. Obviously that number will not
hold in CCRL conditions, but I expect this version of Ethereal will perform far
better than Ethereal8.61 in conditions like TCEC with 43 cores to work with.

Here is the recap for changes since 8.61, for those interested:

Code: Select all

    8.61 : Fix a bug with how the main scheduling lock was setup
    8.62 : More aggressive time allocation in X/Y games
    8.63 : Allow the aspiration window to be as small as [-1, 1]
    8.64 : Again more aggressive time allocation in X/Y games
    8.65 : Penalty in the eval for a queen attacked by a pawn
    8.66 : Prune weak captures that are likely to fail in QS, with
           a short / cheap version of SEE. Idea by Dennis Sceviour!
    8.67 : Don't apply the weak capture pruning to promotion moves
    8.68 : Better time managment in X+Y and X/Y games
    8.69 : Introduction of ProbCut, inspired by the Stockfish implementation
    8.70 : Apply the weak capture pruning in the search as well as qsearch
    8.71 : More aggressive use of weak capture pruning in the search (depth 5)
    8.72 : Allow ProbCut to refute a null move (Unlike SF)
    8.73 : Allow for a different upper and lower margin on the aspiration window,
           by looking at the last 3 search scores. Minumum bound is 4. I believe
           this idea is unique to Ethereal -- at least I have never seen it elsewhere
    8.74 : Bonus for Pawns and Minors in the King's Safety Area
    8.75 : Fix an issue where helper threads were able to allocate more search time,
           such that with many threads the max time would be used with every move.
           This was found thanks to TCEC, and was worth ~70 ELO in selfplay
    8.76 : Take a guess at the time of the next search and consider stopping now
    8.77 : Lower base time usage, and allow for much more massive time allocations
    8.78 : Add an official Logo for Ethereal, created by DanishStyleFetta!
    8.79 : Clean up some threading code (reduce LOC) and allow for max depth of 128
    8.80 : Fix a bug where all helper threads would always use the aspiration window
           centered around the value zero. This was worth ~80 ELO in selfplay
    8.81 : Skip Weak Capture Pruning when in check
    8.82 : Implement a basic form of Pawn/King Shelter
    8.83 : Use a generalized Piece Square Table for sorting quiet moves
    8.84 : Penalty for Minors attacked by Pawns
    8.85 : Do not place any value on the R value for Null Move Pruning
    8.86 : New time allocation scheme to unify X+Y and X/Y (later reworked...)
    8.87 : Resolve a bug where Ethereal could, on very rare occasion, play an illegal
           move as a result of the PV not being filled AT ALL, even in the root node
    8.88 : Allow beta pruning (static null / futility) when we only have Pawns
    8.89 : When doing early pruning (NMP, Razoring, Static Null, and Probcut), always
           return alpha or beta. The goal was to make more conservative TT entries
    8.90 : Lower the margin for futility pruning / beta pruning to about one pawn
    8.91 : Make Ethereal's history counters more like the Stockfish history counters
    8.92 : Yet another patch to allow for more aggressive time allocation
    8.93 : (CURRENT TM) Allocate three things, an ideal usage, a max usage, and a max
           alloc. We will NEVER exceed max usage, and will never start another iteration
           after exceeding max alloc. Overtime we adjust a score and pv stability factor
           to increase or decrease allocated time with respect to pv stability and scoring
    8.94 : Fixed an issue with true-sharing of the transposition table used counter. Now
           we sample the first entries in the TT and approximate the hashfull. This was a
           huge elo winner with high thread counts
    8.95 : Increase base LMR value by one, and reduce it by one when the best move is
           a quiet move and the TT move has failed or this is our first quiet attempts
    8.96 : Cleanup of capture value pruning decisions and some inconsitencies
    8.97 : Remove a pointless flag from the makefile, thanks Laldon!
    8.98 : New method of Pawn/King Shelter definition. Before we would be okay with a
           King pushing past his pawns, so long as they were still close
    8.99 : One big fat tuning patch to update everything the last tune
    9.00 : Cleanup the Texel Tuning implementation by using about 10% of the memory
           as before, resulting in a 700% speed up in a quick test of tuning

Thanks to everyone who has taken an interest in Ethereal over the last few months.
The ideas and suggestions from the TCEC chat have been incredible.
And, as always, thank you to the testers :)

Thanks,
Andrew Grant
Congratulations, very impressive progress!
User avatar
Graham Banks
Posts: 41423
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: Official Release of Ethereal9.00

Post by Graham Banks »

jd1 wrote:Congratulations, very impressive progress!
+1
gbanksnz at gmail.com
Damir
Posts: 2801
Joined: Mon Feb 11, 2008 3:53 pm
Location: Denmark
Full name: Damir Desevac

Re: Official Release of Ethereal9.00

Post by Damir »

Thanks a lot for this new release Andrew. :) As Jerry just said, it is very impressive progress. :D :D
mkchan
Posts: 88
Joined: Thu Oct 06, 2016 9:17 pm
Location: India

Re: Official Release of Ethereal9.00

Post by mkchan »

Nice! Congrats Andrew :D