Has CrazyAra really improved because of MTGS ?

[IQ_QI]

Hello Chessqueen,

you can find more information on how the figures of our evaluation have been generated in the CrazyAra wiki:

• https://github.com/QueensGambit/CrazyAra/wiki/v0.8.4

Our results suggest that the more simulations you have, the more advantageous it becomes to use a directed acyclic graph (DAG) instead of a tree structure.
Intuitively, this makes sense, as more simulations create more transposition nodes and avoid duplicate subtrees.
If you want to reproduce the experiments for activating Monte-Carlo Graph Search (MCGS), you can set Use_Transposition_Table to true for release 0.8.4 or set Search_Type to MCGS for releases > 0.8.4.
It should be noted that if Search_Type is set to MCTS, no transposition table or buffer is used. This is due to maintainability reasons as the DAG structure requires a different search framework.

One experiment which directly compared DAG with using transposition evaluations as a look-up table is shown in Figure 4.
Here the Elo improvement is lower of course compared to not making use of transpositions at all.

To be fair, using a DAG instead of a tree also presents some implementation challenges.
For instance, you can now longer assume that a node has only one parent, and thus you can now longer back-propagate a value evaluation by simply storing a pointer to the leave node.
As a further consequence, if you want to use smart pointers for memory management, then one can no longer make use of unique pointers, but must rely on shared pointers instead.

Our paper has been reviewed and accepted to the International Conference on Automated Planning 2021 (ICAPS 2021) conference.
Generally, you want to cite the corresponding conference article as it is more reliable as non reviewed preprints on arXiv.

• https://ojs.aaai.org/index.php/ICAPS/article/view/15952

Kind regards,
Johannes Czech