Devlog of Leorik

[lithander]

Congratulations! I'm sure it feels great to have a more pure engine finally and the rating jump for Leorik 2.3 in the CCRL of over 50 Elo is very impressive. I think you've inspired me to get back into chess programming soon, I can't let Leorik get to far ahead

It would be great to have you back!

I know I haven't updated the devlog for a while but I'm actively working on Leorik. And just today I've replaced my Ryzen 3600 CPU with a Ryzen 5900X so instead of 10 games simultaneously I can now run 22 games to confirm quicker that all my "improvements" are indeed rubbish!

But seriously I'm really impressed with the new chip. Out of the box and on stock settings on my ancient 60€ mainboard it already performed on much higher clock rates than advertised. But the VRMs were getting hot (108° in a winter-cold room) so I undervolted the CPU, constrained the PPT to 100W and with just a 5-10% multi-core performance drop the entire system now only drains 180W from the wall-socket, while still boosting to 5Ghz single core and maintaining a stable 4.1Ghz on all-core load.

My only regret so far is that I didn't upgrade earlier!

[Sazgr]

...
Here is a gauntlet I ran with a few engines of similar strength. All Elo values except Leorik's are fixed.

Code: Select all

   # PLAYER           :  RATING  POINTS  PLAYED   (%)
   1 Inanis-1.1.1     :  2767.0   323.5     620    52
   2 odonata-0.6.2    :  2744.0   298.5     618    48
   3 Leorik-2.3       :  2741.3  1960.5    3716    53
   4 zahak-5.0        :  2730.0   295.5     620    48
   5 dumb-1.9         :  2703.0   325.0     620    52
   6 blunder-8.5.5    :  2700.0   255.0     620    41
   7 Supernova-2.4    :  2687.0   258.0     618    42

...

Hi, sorry for being less on topic but I have a question. In cutechess, how do you fix the elo of other players? I have been trying to do that for my gauntlets.

[emadsen]

Hi, sorry for being less on topic but I have a question. In cutechess, how do you fix the elo of other players? I have been trying to do that for my gauntlets.

I do it like this in Ordo:

Code: Select all

-m BulletAnchors.txt

For context, here's the full command:

Code: Select all

ordo-win64.exe -m BulletAnchors.txt -U "0,1,2,6,4,7,8,9,5,11" -o ComputerGamesBullet.txt -j ComputerGamesBulletMatchups.txt -W -D -s 100 -n 16 ComputerGamesBullet.pgn MadChessGauntlet.pgn

BulletAnchors.txt looks like this:

Code: Select all

"Stockfish 9",3480
"iCE 3.0",3080
"Djinn 1.021",2678
"Sungorus 1.4",2310

[lithander]

In cutechess, how do you fix the elo of other players? I have been trying to do that for my gauntlets.

I use Cutechess just for playing the gauntlet. And then I use a 2nd tool called Ordo to compute the scores with the following command:

Code: Select all

.\ordo-win64.exe -p leorik23_gauntlet2_30per40_7threads.pgn -m anchors_2.3.txt

The anchors textfile is just a simple list of the engines with their fixed Elo value:

Code: Select all

"Cheese-3.1.1", 2915
"Fridolin-4.0", 2759
"MadChess-3.1", 2713
"zurichess-neuchatel", 2824
"MinkoChess-1.3", 2849
"OliThink-5.10.1", 2844
"Shallow-5", 2811
"dumb-1.9", 2703
"zahak-5.0", 2730
"odonata-0.6.2", 2744
"Inanis-1.1.1", 2767
"Supernova-2.4", 2687

It may look inconvenient to do it like that but it has the additional advantage that you can look at the score whenever you like. You don't have to stop cutechess to see how the current standings are. And you can aggregate the games in multiple sessions.

[j.t.]

It may look inconvenient to do it like that but it has the additional advantage that you can look at the score whenever you like. You don't have to stop cutechess to see how the current standings are. And you can aggregate the games in multiple sessions.

I maybe missed what you want to do, but in case you didn't know yet, you can add a flag to cutechess "-ratinginterval 50" which would mean that cutechess prints out the current standing every 50 games.

[lithander]

I maybe missed what you want to do, but in case you didn't know yet, you can add a flag to cutechess "-ratinginterval 50" which would mean that cutechess prints out the current standing every 50 games.

I didn't know that feature and it's a good tip! Thanks. The main reason I use Ordo though is that I fix all other engines with their CCRL rating so my own engines score is the only variable. And by that I try to find an Elo value for the tested version that is in the same scale as the CCRL.

[Sazgr]

Thank you! Ordo seems like a nice tool for doing matches in more than one go and for setting elo, I will use it.

[Mike Sherwin]

Thinking about NN for Leorik. What if you keep your current eval for each piece as inputs. Then for each piece there is a hidden layer comprised of chess knowledge. And then there is one score for each piece as output. The outputs are added together to score the position. The hidden layer for each piece is trained from the same dataset that was used to train the PSTBls. How would that compare with NNUE?

[lithander]

Thinking about NN for Leorik.

I still haven't tried your idea of introducing 15x15 king-relative PSTs yet! And that's just one of many incremental steps I want to try before looking into NN-based eval in earnest. Once I'm ready I'll probably just start with the NNUE approach and a really simple topology like 768x64x1 for example! (6 Pieces x (Black + White) x 64 Squares = 768)

But currently I'm just focusing on search optimizations. I hope to get to 2800 CCRL Elo before touching the eval again.

How would that compare with NNUE?

I don't have the slightest idea! I can make all kind of crazy experiments once I know how to get basic NNUE to work in C#. And that's of course not possible without generating all the data and label it with my own engine and train the nets and so on... a lot to learn.

[algerbrex]

It would be great to have you back!

Thansk! Since I'm currently taking a CS course that uses Java, I may put Blunder to the side for a bit and work on creating an engine from scratch there, trying some different stuff, experimenting

I know I haven't updated the devlog for a while but I'm actively working on Leorik. And just today I've replaced my Ryzen 3600 CPU with a Ryzen 5900X so instead of 10 games simultaneously I can now run 22 games to confirm quicker that all my "improvements" are indeed rubbish!

But seriously I'm really impressed with the new chip. Out of the box and on stock settings on my ancient 60€ mainboard it already performed on much higher clock rates than advertised. But the VRMs were getting hot (108° in a winter-cold room) so I undervolted the CPU, constrained the PPT to 100W and with just a 5-10% multi-core performance drop the entire system now only drains 180W from the wall-socket, while still boosting to 5Ghz single core and maintaining a stable 4.1Ghz on all-core load.

My only regret so far is that I didn't upgrade earlier!

Nice! Depending on how much work I'm able to start doing on Blunder/my new engine during this semester, I may have to look into making some upgrades to my setup, as I'm able to run about 8 games comfortably using cutechess.