Beating the devilish MCTS engine...

[ericlangedijk]

I am working on my chessengine 'chessnix' of which my local dev version is now somewhere around 3050 ELO.

That is now still done manual something like: changing something and then play with CuteChess 200 games of 2 minute games against slightly weaker and slightly stronger engines and check if there is improvement.

What surprised me and I find fascinating is the fact that the engine 'Monty' performs much better against chessnix than its rating on CCRL (2945) would suggest. I know that Monty is a MCTS engine.

Does anyone know what would be the main area(s) to investigate to stabilize the search a bit more?
Or is the Monty engine actually stronger than 2945?

My next area to improve is smarter time mangement...

Here some results of my testruns:

Code: Select all

chessnix dev - qapla         (2890) :  72 -  11 -  35 [0.758]
chessnix dev - shallowguess  (2896) :  87 -  22 -  32 [0.730]
chessnix dev - monty         (2940) :  63 -  58 -  79 [0.512]
chessnix dev - sentinel      (2964) :  87 -  34 -  79 [0.632]
chessnix dev - chessnix 1.2  (3019) :  42 -  26 - 132 [0.535]
chessnix dev - stash 27      (3021) :  83 -  39 -  78 [0.610]
chessnix dev - bit-genie     (3099) :  78 -  52 -  70 [0.565]
chessnix dev - stash 29      (3102) :  47 -  66 -  87 [0.453]

[JacquesRW]

Are you actually downloading the version of monty that is on CCRL? It is in an old repo (https://github.com/jw1912/monty/releases/tag/v1.0.0). The results suggest that you are and it is probably an artifact of the time control. v1.0.0 missing a key feature (CPUCT scaling) that lack thereof leads to terrible scaling at longer time controls - i.e. Monty v1.0.0 would be expected to perform much better at shorter time controls than CCRL 40/15.
In the very unlikely case you are building from the active repo (https://github.com/official-monty/monty), current master is something like 600+ elo stronger :p

[ericlangedijk]

Interesting! I indeed have that first one: monty 1.0.0. Still I think that Monty is stronger than 2945. Also at the currently running ccrl live Monty 1.0.0 performs quite good.
Will checkout that stronger version as well. Maybe I can learn of it

[lithander]

In the very unlikely case you are building from the active repo (https://github.com/official-monty/monty), current master is something like 600+ elo stronger :p

So it's the strongest MCTS engine now?

[abgursu]

In terms of CCRL, Jackal performs better. And impressively, far aggressiver. I think Jackal would score a good EAS score.

I don't know how the current masters of them are tho, just shared my two cents.

[JacquesRW]

In terms of CCRL, Jackal performs better. And impressively, far aggressiver. I think Jackal would score a good EAS score.

I don't know how the current masters of them are tho, just shared my two cents.

Jackal is much weaker. And many of its features, network architectures, etc are originally from Monty.

[JacquesRW]

So it's the strongest MCTS engine now?

On consumer CPUs.

[gflohr]

My next area to improve is smarter time mangement...

I think that time management is really underrated. I have stolen Stockfish's time management for my engine Plisco last month (around 15th or 16th of December), and that gave it a push of well over 100 rating points on Lichess, see here: https://lichess.org/@/plisco-bot/all
I believe that the leverage is even bigger for stronger (read: faster) engines. And I don't think that my previous time management was complete nonsense. It was just very risk-loving, and the engine lost too often on time for my taste.

On the other hand, the SPRT results were not impressive, albeit convincing in the end. I'm not sure, but my gut feeling tells me that improvements in time management do not show up clearly at super-fast time controls, especially when your engine is relatively slow, which is definitely true for Plisco, written in Perl.

[ericlangedijk]

It turns out I had one instability in my search, which solved my 'monty issue' together with some other (too aggressive) prunings / heuristics.

Time management is quite difficult, but I managed to make something primitive which is already boosting the ELO a bit.
My target for my 1.x engine is 3200 without eval changes. Then I know the search is really really stable. Currently my guess is I am at 3120.

I am not a c programmer. In stockfish many things are spread all over the place so I can't read its timemanagement code very well. Besides that I do not want to copy it.