Drofa 2.0.0

[No4b]

Hello all.

After ~2 month of work i feel like i`m ready to publish Drofa 2.0.0.

https://github.com/justNo4b/Drofa/releases/tag/2.0.0

There were ~30 of elo-gaining patches since than. And if the first of them were quickly tested (with ~1000 games), some of the final patches were tested with up to ~20k games ( as you can imagine, my PC was working almost non-stop).

List of the major changes added to the Drofa:

• Fixed two major bugs regarding repetition and pawnHash

• More efficient MovePicker and Evaluation

• Added Queen promotions in the QSearch

• LMR implemented

• Razoring implemented

• LMP implemented

• Reverse futility pruning implemented

• Extended futility pruning

* I know having both should be theoretically unsound, but for some reason i need both for best performance. Maybe i my constants are too bad or smth.

• Un-hashed reduction

(that is what I call IID replacement by Rebel`s idea, dunno why people still reference it as IID, but i may be wrong here )

• More agressive time management (maybe too agressive )

• Implemented Countermoves (both for Quiets move ordering and for LMR-reductions)

• PassedPawn extention on the nodes that are close to leafs

• Fixed many move ordering issues

• Piece values are tuned

As the last time, you can look at Drofa_changelog_2 file to see full list of changes.

Drofa 2.0.0 is ~400 elo stronger than Drofa 1.0.0 and 50-90 elo stronger than Drofa 1.3.1 (2402 elo in CCRL blitz)

Code: Select all

TC 2`+1``:
Score of Drofa_1.3.8 vs Drofa_1.3.1: 180 - 75 - 231  [0.608] 486
Elo difference: 76.27 +/- 22.41
SPRT: llr 2.96, lbound -2.94, ubound 2.94 - H1 was accepted
Finished match

I would like to credit people without whom at least some of this patches would be impossible:

Terje - because i took LMR and LMP formulas from Weiss (while i rewrote LMR to be pow() based instead of log() based, curve is still very close, if not the same. I tried to tweak and tune both formulas, but with 0 success. I also took general logic for mingling LMR code with PVS (but with my own implementation) and about what pruning techniques good engine must use.

I must admit at some point i was worried that Drofa search is too similar to the Weiss, but (especially after latest patches) i feel content that my LMR and some other stuff are divergent enough.

Ferdy - for his great tuning tool. I was able to tune my piece values for ~20 elo without knowing anything about tuning.

Guenther Simon - for helping me with Windows Drofa compiles, because i still was not able to resolve speed drop issue while using cross-compilers.

Future directions:
I think that Drofa search is now decent enough.
Next i want to use Drofa as a tool to learn some machine learning techniques.
Firstly i want to add some more eval parameters and then tune them, but using strictly my own implementation.
Then I want to train NN for a time management (it maybe stupid, but i just think it would be a decent practice)
And finally implement NN pawn eval (this probably will be also elo-gainer )

As you can imagine, this will take a lot of time, and i want to concentrate on my other projects for now soo.....

[No4b]

I also want to share a gauntlet for Drofa 2.0.0 i did for testing purposes:

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   0 Drofa_1.3.8                    23      41     225   53.3%   20.4%
   1 Velvet-v1.0.0                  85      73      75   62.0%   20.0%
   2 Raven1.10                     -51      74      75   42.7%   16.0%
   3 Weiss_0.6                    -105      71      75   35.3%   25.3%
Finished match

And one of the Drofa games i liked the most so far:
[pgn]
[Event "?"]
[Site "?"]
[Date "2020.11.06"]
[Round "2"]
[White "Velvet-v1.0.0"]
[Black "Drofa_1.3.8"]
[Result "0-1"]
[ECO "A06"]
[Opening "Reti Opening"]
[Variation "Nimzowitsch-Larsen, 2...Nf6"]
[TimeControl "60+1"]
[Termination "adjudication"]
[PlyCount "161"]
[WhiteType "human"]
[BlackType "human"]

1. Nf3 {book} d5 {book} 2. b3 {book} Nf6 {book} 3. e3 {-0.10/13 1.7s} e6
{+0.14/15 4.4s} 4. d4 {-0.32/14 2.5s} Bd6 {+0.14/15 5.0s} 5. Bd3 {-0.22/14
2.4s} Nc6 {0.00/14 4.5s} 6. c4 {+0.13/13 1.2s} dxc4 {+0.51/15 3.6s} 7. bxc4
{+0.17/15 2.4s} e5 {+0.45/16 3.2s} 8. Nbd2 {-0.25/13 1.9s} exd4 {+0.81/13
6.5s} 9. exd4 {-0.70/13 2.3s} O-O {+0.35/13 4.0s} 10. O-O {-0.23/12 1.2s}
Re8 {+0.27/13 2.7s} 11. a3 {+0.23/13 2.3s} Bf4 {+0.08/13 5.3s} 12. Bb2
{+0.32/13 1.2s} Qd6 {-0.01/12 3.2s} 13. h3 {+0.33/13 2.3s} b6 {-0.13/12
3.4s} 14. Qc2 {+0.56/14 2.2s} Bb7 {-0.23/12 2.2s} 15. Rfe1 {+0.55/12 1.2s}
Rad8 {-0.23/12 5.2s} 16. Ne4 {+0.68/13 2.2s} Nxe4 {-0.51/15 2.1s} 17. Bxe4
{+0.82/14 1.5s} Qh6 {-0.51/13 1.7s} 18. g3 {+0.63/14 2.1s} Bd6 {-0.46/14
2.3s} 19. Qa4 {+0.98/13 1.7s} Rxe4 {-1.04/13 1.5s} 20. Rxe4 {0.00/14 1.9s}
Nxd4 {0.00/14 1.6s} 21. Rxd4 {+0.23/15 5.4s} Bxf3 {0.00/14 1.3s} 22. Qb3
{+0.23/14 2.0s} Bc6 {+0.26/13 2.3s} 23. Rh4 {+0.25/14 1.3s} Qe6 {+0.17/14
1.3s} 24. Qc3 {+0.37/14 1.9s} f6 {+0.36/13 1.9s} 25. Qd3 {0.00/13 1.9s} f5
{+0.42/13 1.6s} 26. Qc3 {0.00/12 1.4s} Rd7 {+0.50/14 1.4s} 27. a4 {-0.12/11
0.96s} Re7 {+0.71/12 1.8s} 28. Rf1 {+0.10/12 1.9s} Qe2 {+0.72/12 2.1s} 29.
Qb3 {+0.16/13 1.3s} Bxg3 {+1.28/13 1.4s} 30. Qxg3 {-0.24/14 1.4s} Qxb2
{+1.44/15 1.8s} 31. Rxh7 {-0.36/13 0.91s} Qf6 {+1.32/14 1.2s} 32. Rh5
{-0.36/15 1.7s} Be8 {+1.32/14 0.99s} 33. Rg5 {-0.28/14 1.8s} Bxa4 {+1.20/13
1.2s} 34. Rg6 {-0.18/13 0.77s} Qf7 {+1.22/13 1.3s} 35. Rc1 {-0.22/13 1.8s}
f4 {+1.59/13 1.8s} 36. Qg5 {-0.47/12 1.0s} a5 {+1.57/11 1.1s} 37. c5
{+0.26/13 1.3s} Be8 {+0.78/15 1.2s} 38. Rh6 {-0.05/14 1.6s} b5 {+0.91/14
1.7s} 39. Rh4 {0.00/12 1.2s} Re4 {+0.29/14 1.7s} 40. c6 {+0.23/14 1.1s} b4
{+0.16/13 0.95s} 41. Qxa5 {+0.72/13 0.99s} Qe7 {+0.23/12 1.5s} 42. Qd5+
{+0.42/13 1.1s} Bf7 {+0.30/15 0.92s} 43. Qf5 {+0.38/14 1.1s} g6 {+0.42/14
1.5s} 44. Qc8+ {0.00/14 1.6s} Be8 {+0.24/14 0.82s} 45. Rg4 {0.00/14 1.0s}
Kg7 {+0.34/13 1.5s} 46. Qb8 {-0.37/13 1.7s} Bf7 {+0.46/11 0.80s} 47. Qb5
{-0.11/12 1.2s} Be6 {+0.74/13 0.92s} 48. Rg5 {-0.47/14 1.3s} Bxh3 {+0.87/13
1.4s} 49. Qc5 {-0.23/13 1.7s} Qf6 {+1.50/13 0.99s} 50. Qd5 {-1.35/14 5.2s}
Re2 {+1.81/14 0.91s} 51. Kh2 {-1.35/13 1.2s} Be6 {+1.70/13 0.83s} 52. Qc5
{-2.06/13 1.6s} f3 {+2.47/14 1.3s} 53. Rb1 {-2.15/12 1.2s} b3 {+3.17/13
0.72s} 54. Kh1 {-2.80/12 1.6s} b2 {+5.17/15 0.90s} 55. Rbg1 {-5.13/14 7.0s}
Kf7 {+4.84/13 0.85s} 56. Rf1 {-5.38/14 1.1s} Ba2 {+5.05/14 1.1s} 57. Rg4
{-5.48/14 3.6s} b1=Q {+5.07/14 0.75s} 58. Rxb1 {-5.51/13 0.61s} Bxb1
{+5.99/13 0.86s} 59. Qd5+ {-5.43/12 0.94s} Kg7 {+6.40/15 1.1s} 60. Qd7+
{-6.15/12 1.3s} Re7 {+6.35/16 0.74s} 61. Qd4 {-6.13/14 0.84s} Qxd4
{+6.34/17 0.77s} 62. Rxd4 {-6.32/16 0.85s} Be4 {+6.36/19 1.0s} 63. Rc4
{-6.31/18 1.2s} Kf6 {+6.42/21 1.4s} 64. Kh2 {-6.31/17 1.3s} Ke5 {+6.48/21
1.2s} 65. Kg3 {-6.42/16 0.56s} Re6 {+6.48/19 0.99s} 66. Rc5+ {-6.84/16
1.3s} Bd5 {+6.49/19 0.91s} 67. Ra5 {-7.36/17 1.3s} Rxc6 {+6.74/19 1.4s} 68.
Ra1 {-7.44/16 1.0s} Re6 {+6.76/21 0.99s} 69. Rc1 {-6.56/14 0.85s} Kd6
{+6.81/19 0.76s} 70. Kf4 {-7.14/14 0.84s} c5 {+6.82/18 1.5s} 71. Ra1
{-7.84/18 5.0s} c4 {+7.24/17 1.4s} 72. Kg3 {-7.84/17 1.2s} Kc5 {+8.90/20
1.1s} 73. Rg1 {-8.67/16 1.2s} c3 {+11.44/19 0.80s} 74. Rc1 {-8.90/15 0.60s}
Kb4 {+11.46/19 1.4s} 75. Kf4 {-9.52/14 0.75s} Kb3 {+12.97/19 1.3s} 76. Rb1+
{-11.53/16 0.79s} Ka2 {+17.69/18 0.66s} 77. Rg1 {-11.82/16 0.91s} c2
{+21.29/19 0.77s} 78. Kg5 {-12.60/15 0.49s} Kb2 {+21.42/19 0.80s} 79. Rf1
{-13.70/15 1.3s} c1=Q+ {+21.65/17 1.4s} 80. Rxc1 {-24.04/20 0.97s} Kxc1
{+21.68/17 0.77s} 81. Kh6 {-M20/22 1.2s, Black wins by adjudication} 0-1
[/pgn]

[No4b]

Hello all!
Drofa 3.2.0 released.
https://github.com/justNo4b/Drofa/releases/tag/3.2.0

In self-play it is stronger than 3.1.0 approximately by 100 elo.

STC 8'' + 0.08''

Code: Select all

ELO   | 94.21 +- 5.29 (95%)
CONF  | 8.0+0.08s Threads=1 Hash=8MB
GAMES | N: 10000 W: 4273 L: 1626 D: 4101

LTC 40'' + 0.4''

Code: Select all

ELO   | 109.37 +- 5.06 (95%)
CONF  | 40.0+0.40s Threads=1 Hash=64MB
GAMES | N: 10000 W: 4249 L: 1201 D: 4550

The most important changes:

• Rewrite of history bonuses (now they are capped, plus other tweaks)

• Added capture history

• Added countermove history

• Added late capture extension

• Scale evaluation based on winning side pawn count

• Adjust some of the extensions if null move failed

• Scale evaluation for OCB + Rook / Knight endgames

• Some evaluation tweaks / a couple of new eval terms

• Much more agressive LMP, LMR and SEE

• Retuned evaluation with Drofa data (got ~ -7 elo from this, but IMHO tuning on own data worth it)

As usual, all changes with est. elo-gain can be seen in Drofa-changelog on github

Please notify me if I fucked up windows compiles again , hopefully i am not...

There are still many ways to improve Drofa, hopefully by the end of a year I`ll manage to crap another +100 for a release

[Rebel]

For the GRL.

Games : 1000

Elo pool : 3008

As usable at - http://rebel13.nl/a/grl.htm

[AlexChess]

Hello all!
Drofa 3.2.0 released.
https://github.com/justNo4b/Drofa/releases/tag/3.2.0

In self-play it is stronger than 3.1.0 approximately by 100 elo.

STC 8'' + 0.08''

Code: Select all

ELO   | 94.21 +- 5.29 (95%)
CONF  | 8.0+0.08s Threads=1 Hash=8MB
GAMES | N: 10000 W: 4273 L: 1626 D: 4101

LTC 40'' + 0.4''

Code: Select all

ELO   | 109.37 +- 5.06 (95%)
CONF  | 40.0+0.40s Threads=1 Hash=64MB
GAMES | N: 10000 W: 4249 L: 1201 D: 4550

The most important changes:

• Rewrite of history bonuses (now they are capped, plus other tweaks)

• Added capture history

• Added countermove history

• Added late capture extension

• Scale evaluation based on winning side pawn count

• Adjust some of the extensions if null move failed

• Scale evaluation for OCB + Rook / Knight endgames

• Some evaluation tweaks / a couple of new eval terms

• Much more agressive LMP, LMR and SEE

• Retuned evaluation with Drofa data (got ~ -7 elo from this, but IMHO tuning on own data worth it)

As usual, all changes with est. elo-gain can be seen in Drofa-changelog on github

Please notify me if I fucked up windows compiles again , hopefully i am not...

There are still many ways to improve Drofa, hopefully by the end of a year I`ll manage to crap another +100 for a release

Thank you! Since there is also a popcount version, I will test it soon. With NNUE the quantum leap is certain (when you will add it, please make it usable also without AVX or BM2 ) . TIP: I'm dreaming about an engine capable of positional sacrifices (eg: giving a pawn or knight with no material compensation, but leading to a winning position, out from endgames solved by tablebases) . Do you think that it would be possible to add patterns for that?

Best regards, Alex

[No4b]

For the GRL.

Games : 1000

Elo pool : 3008

As usable at - http://rebel13.nl/a/grl.htm

Thank you for testing!
So far going well for Drofa

[No4b]

Thank you! Since there is also a popcount version, I will test it soon. With NNUE the quantum leap is certain (when you will add it, please make it usable also without AVX or BM2 ) . TIP: I'm dreaming about an engine capable of positional sacrifices (eg: giving a pawn or knight with no material compensation, but leading to a winning position, out from endgames solved by tablebases) . Do you think that it would be possible to add patterns for that?

Best regards, Alex

Well, there are no plans on replacing evaluation with any type of NN in the near future (lets say a half a year or so). I would say that Drofa handcrafted eval has some potential and can still be massively improved.

As for style.... Well, I dont really know about the method of doing it. Especially if coupled with the goal of increasing elo

[Brunetti]

Drofa 3.2.0 released.
https://github.com/justNo4b/Drofa/releases/tag/3.2.0

In self-play it is stronger than 3.1.0 approximately by 100 elo.

1'+1" rating ~3100.

Alex

[No4b]

Drofa 3.2.0 released.
https://github.com/justNo4b/Drofa/releases/tag/3.2.0

In self-play it is stronger than 3.1.0 approximately by 100 elo.

1'+1" rating ~3100.

Alex

Thank you for testing!

[Rebel]

For the GRL.

Games : 1000

Elo pool : 3008

As usable at - http://rebel13.nl/a/grl.htm

Thank you for testing!
So far going well for Drofa

Drofa a new 3000+ engine.

Code: Select all

Gambit Rating List
Running      : gauntlet Drofa 3.2.0
Time Control : Time control : 40/120
Games        : 1000
 
Results from file gauntlet-drofa-320.pgn:

No. Name           Win Draw Loss Unf.  Score Games       %
----------------------------------------------------------
  1 Drofa 3.2.0   +441 =345 -214   *0  613.5  1000   61.4%
  2 Cheng 4.41     +57  =59  -84   *0   86.5   200   43.2%
  3 Monolith 2     +46  =71  -83   *0   81.5   200   40.8%
  4 Amoeba 3.3     +37  =85  -78   *0   79.5   200   39.8%
  5 FabChess 1.16  +35  =70  -95   *0   70.0   200   35.0%
  6 Marvin 5.0     +39  =60 -101   *0   69.0   200   34.5%

Total Games:    1000
White Wins:      313 (31.3%)
Black Wins:      342 (34.2%)
Draws:           345 (34.5%)
Unfinished:        0 (0.0%)

Estimated ratings for this elo 3008 pool

   # PLAYER           :  RATING  POINTS  PLAYED   (%)
   1 Drofa 3.2.0      :  3075.8   613.5    1000    61
   2 Cheng 4.41       :  3028.2    86.5     200    43
   3 Monolith 2       :  3010.2    81.5     200    41
   4 Amoeba 3.3       :  3002.9    79.5     200    40
   5 FabChess 1.16    :  2967.3    70.0     200    35
   6 Marvin 5.0       :  2963.5    69.0     200    35

Congrats!