new axolotl

[AxolotlFever]

Hello,

axolotl v1.8 has been released.

https://github.com/louism33/Axolotl

The main changes are bugfixes, but these have made the engine a bit stronger than before. I roughly estimate the elo rating to be about 2400, and I would be grateful to anyone who can give me a more precise number.

The code is open source of course, and I am thankful to anyone who has a look, or benefits from axolotl in any way.

I have a couple of issues I would love some help with:

1. On the STS suite, the weakest challenges by far are King Safety (number 11) and Centre Control (number 12), where the engine performs pretty badly. I am not really sure how to improve at this, beyond trial and error, as my engine is a better chess player than I am.

2. In a "go movetime 10000" from startpos, my engine registers about 250,000 nps (and takes a while to get to this speed). On my machine, stockfish gets 1,250,000 for the same command. My engine is written in Java, so I am not expecting parity, but I think doubling my nps to 500,000 is not overly ambitious. Any feedback on this goal, or help achieving it, is hugely appreciated. (For the record, startpos "perft 7" gives us 60,000,000 nps, with bulk counting, while stockfish gets 166,000,000 on perft 7)

Thanks in advance for any feedback, and if you can give me any ideas about how to make progress, I would be hugely grateful

Louis

[Necromancer]

What are your results for the sts tests? I mean the file with 1500 positions.

I could not run java -jar axolotl_v1.8.jar, my Java version is "1.8.0_101".

[AxolotlFever]

Hi Necromancer, thanks for the quick reply.
Yes, unfortunately you will need at least Java 11 to run axolotl, which is easily available online :
https://openjdk.java.net/projects/jdk/11/
or
https://www.oracle.com/technetwork/java ... 66655.html

I ran the STS suite at 50 ms and 1000ms per position, here are the results:

50 ms per position:
STS1Undermining: Successes: 64 out of 100
STS2OpenFilesAndDiagonals: Successes: 56 out of 100
STS3KnightOutposts: Successes: 67 out of 100
STS4SquareVacancy: Successes: 60 out of 100
STS5BishopVKnight: Successes: 80 out of 100
STS6Recapturing: Successes: 92 out of 100
STS7Simplification: Successes: 65 out of 100
STS8fghPAWNS: Successes: 48 out of 100
STS9abcPAWNS: Successes: 60 out of 100
STS10Simplification: Successes: 88 out of 100
STS11KingActivity: Successes: 53 out of 100
STS12CenterControl: Successes: 44 out of 100
STS13PawnCentre: Successes: 68 out of 100
STS14SeventhRank: Successes: 83 out of 100
STS15PointlessExchange: Successes: 54 out of 100

1000 ms per position:
STS1Undermining: Successes: 83 out of 100
STS2OpenFilesAndDiagonals: Successes: 68 out of 100
STS3KnightOutposts: Successes: 78 out of 100
STS4SquareVacancy: Successes: 75 out of 100
STS5BishopVKnight: Successes: 84 out of 100
STS6Recapturing: Successes: 92 out of 100
STS7Simplification: Successes: 73 out of 100
STS8fghPAWNS: Successes: 59 out of 100
STS9abcPAWNS: Successes: 62 out of 100
STS10Simplification: Successes: 88 out of 100
STS11KingActivity: Successes: 62 out of 100
STS12CenterControl: Successes: 49 out of 100
STS13PawnCentre: Successes: 67 out of 100
STS14SeventhRank: Successes: 85 out of 100
STS15PointlessExchange: Successes: 66 out of 100

For the purposes of this test, I count a success if the engine returns any one of the suggested moves (and I ignore the score associated with that move)

Thanks!
Louis

[Necromancer]

I think it's better to consider only the best move in order to track progress, then you can run a regression to estimate ELO.
http://www.talkchess.com/forum3/viewtopic.php?t=56667

When I change my engine I like to run the STS tests 10 times (at 50 ms per move) to see if it improved, if yes then I run games between the old x new version. To estimate Rating just run the STS with 200ms, then apply the formula:

R = 1439.4 + 25.212* (%correct)

For my engine at 200ms (the current dev version, not the 2471 version on CCRL):

id "STS(v1.0) Undermine 47/100
id "STS(v10.0) Simplification 66/100
id "STS(v11.0) King Activity 41/100
id "STS(v12.0) Center Control 52/100
id "STS(v13.0) Pawn Play in the Center 55/100
id "STS(v14.0) 7th Rank 55/100
id "STS(v15.0) AT 28/100
id "STS(v2.2) Open Files and Diagonals 35/100
id "STS(v3.0) Knight Outposts/Repositioning/Centralization 54/100
id "STS(v4.0) Square Vacancy 43/100
id "STS(v5.0) Bishop vs Knight 59/100
id "STS(v6.0) Recapturing 53/100
id "STS(v7.0) Simplification 51/100
id "STS(v8.0) AKPC 35/100
id "STS(v9.0) Advancement of a/b/c pawns 36/100
Found 710 of 1500 (47.3%)

R = 1439.4 + 25.212 * 47.3
= 2632

which is close to my estimates, but probably a bit too optimistic. This regression would improve a lot if more points were added, but it gives good values I think.
Regarding where to improve, here is what I found in my engine (first version was 2140 ELO):

-speed = more nodes searched = +ELO
-switching to magic bitboards gave me a good speedup
-make_move/undo_move are really important, make them smart and fast
-you should be 100% sure that move generation and evaluation have no bugs (use perft and the mirror board technique to be sure)
-make sure your search works correctly: drawing by repetition, stalemates, etc.
-qsearch is crucial: if your engine tends to get mated against much stronger engines, maybe there's a problem there. If in check I generate all evasions, otherwise only captures. I don't know if that is the best way to go, but it gave me a solid improvement.
-some evaluation stuff are crucial: make sure your king safety and passed pawn evaluation are solid (passed pawn evaluation, not only passed pawn bonus).
-static exchange evaluation helped a lot in qsearch
-cache stuff in the evaluation (for now I only cache attacked squares for each side).
-tuning will gain ELO, but it's not trivial and should be done later I think

If I said something stupid please someone correct me!

[AxolotlFever]

Hi Necromancer,

Thank you so much for taking the time! And thank you also for your really good ideas. How would you go about evaluating your King Safety and Passed Pawn eval? At the moment I mainly use the Strategic test suite, and some Texel Tuning.

Many thanks again, and I will post some more precise STS numbers, using your ideas, tomorrow!

Kind regards,
Louis

[Necromancer]

For king safety I follow more or less what is described here (Attacking King Zone section).
https://www.chessprogramming.org/King_Safety

For each attacking piece it just counts the attack intersection against the enemy king region and do a weighted sum. This is something I also need to improve. I also check the pawn shelter near my king, if they are missing or messed up, there is a penalty.

My passed pawn evaluation is a shameless copy from Stockfish, it gave me +50 ELO. There is not much to invent here: it checks the safety of squares ahead and behind the passed pawn and king distance.