Root move order (again)

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Root move order (again)

Post by bob »

Kempelen wrote:
bob wrote:

Code: Select all


       Bxe4      335949     0 0
         d5     1051118     0 0
       Rfd8      645743     0 0
        Nh5      585859     0 0
         e5      377863     0 0
       Nxe4      197655     0 0
        Ne5       91882     0 1
        Nh7       84857     0 1
        Bd5       39302     1 1
        Ng4       29087     1 1
        Bc6       15676     1 1
        Qc5       13615     1 1
        Qb8       12134     1 1
       Qxc4       12001     1 1
        Nd5       11762     1 1
        Kh7       11133     1 1
       Rce8        8839     1 1
        Ba6        6896     1 1
         g5        6423     1 1
       Rfe8        6301     1 1
        Ba8        6186     1 1
        Ne8        5770     1 1
        Kh8        5561     1 1
         h5        4288     1 1
        Ra8        4219     1 1
         g6        3887     1 1
        Qc6        3731     1 1
        Nb8        3695     1 1
        Rb8        3649     1 1
        Bd8        2960     1 1
        Qd8        2926     1 1
       Rcd8        2792     1 1
        Nc5           0     1 1
Notice anything unusual? It is NOT a bug. Our good friend the TT causes that. zero nodes searched below that node because at ply=2 there'a hash entry waiting.
Hello Bob,

Do you think storing node count in TT could help later in move ordering?. That way you will always have the "true" effort needed for each node. Of course there is a drawback, you have less space in the whole table.
No. Already experimented with that last year. Problem is that we accept signature matches when depth of search remaining is <= draft of table entry. That would cause over-stating the nodes searched when you are searching 20 plies deep, are at ply=10, and get a TT hit on an entry with a draft of 15.

I've essentially given up on node counts being anything other than a statistical performance measure of the search. Gives a clue about how fast the move generation, evaluation and such are when used together. Other than that, however, nada...
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Root move order (again)

Post by bob »

jdart wrote:
Unfortunately, it just doesn't seem to work. I removed that move ordering and reverted to the original crafty approach of a new best move goes to the front, the rest move down one slot, keeping past best moves near the front. Only thing I do a bit differently is that when a move goes to the front, I save an "age" with it (currently using 4). At the end of each iteration, all the ages are reduced by 1, with a lower bound of zero. If a move has a non-zero age, it is not searched in parallel with any other move, nor is it reduced at the root. This handles the flip-flop case that occasionally happens, where the thing keeps changing its mind.
Based on this thread, I tried a variant of this. Arasan actually searches the first few iterations with all moves having a nonzero window, for easy move detection. So I actually have scores for the moves at that point and for those iterations I sort on the scores. This gives a rough starting ordering.

Formerly after the wide window part of the search I sorted on node counts but I tried as you suggested just moving the new PV to the front and pushing all other moves down.

I think from what you said this is a better founded approach. However, in fact, this tested equal with a sort based on node counts. It wasn't any better, but it wasn't worse either.
I've sorted several different ways during recent testing. None really matter as much as simply getting the best move from the previous iteration sorted first. That covers approximately 85% of the searches, since only about 15% of the time we change to a new best move.

However, for me, there was a small gain in parallel search. Remember that I have an adaptive split at the root algorithm, previously controlled by node counts. For moves (after the first) with significant node counts, I would set a flag that said "do not search in parallel" so I could get through the best move candidates and get a solid alpha bound before starting a parallel search at the root. If you do this, there are two outcomes. (1) if you are NOT going to change your mind, this is less efficient than splitting at the root outright, because the latter will have no search overhead. (2) if you do change your mind, you get a better alpha bound before starting a parallel split at the root, reducing overhead. Unfortunately, it turned out the node count approach was avoiding parallel searches on too many root moves, increasing overhead.

I now avoid splitting at the root until I have searched each move that has been best during the 3 previous iterations. Has proven to be MUCH more accurate for me.

If not for that, perhaps no gain. And one other thing I almost forgot, I do a similar trick for reducing at the root. Makes little sense to reduce a candidate best move because that makes it harder for it to pop to the top. So I am probably getting a more accurate "do not reduce" case as well.

If you don't do either of those, root move ordering is not going to be very significant. Unless you are like me and do not try to complete an iteration before stopping. If you stop when time runs out, you have to hope you have searched the good moves already. Better ordering helps a bit there.
jdart
Posts: 4367
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Root move order (again)

Post by jdart »

Interesting. I currently do root move reduction but not for the PV or gainful captures. I have also tried not reducing moves with relatively high nodes counts, but the latest experiment I mentioned doesn't do this (it has all the node count stuff removed).

--Jon
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Root move order (again)

Post by bob »

jdart wrote:Interesting. I currently do root move reduction but not for the PV or gainful captures. I have also tried not reducing moves with relatively high nodes counts, but the latest experiment I mentioned doesn't do this (it has all the node count stuff removed).

--Jon
I think anything based on node counts might be done instead by raplacing the

if (nodecount > something)

by

if (random() > .5)

:)

last year I tried using node counts rather than draft for TT replacement. Simply did not work well at all. Another idea down the tubes.