Testing Transposition Tables

[Tony]

I see, makes sense,

I just timed my engine, to see how long it took to complete ply 10.

A) With 2^20 entries in table:
Depth=10 BEST_MOVE=e3 score=-94 nnode=290,833,381 neval=236,804,413
total_nnode=340,373,635 total_neval=278,244,336
TIME TAKEN = 179.954 seconds

B) With 2^21 entries in table:
Depth=10 BEST_MOVE=e3 score=-94 nnode=441,017,974 neval=359,452,142
total_nnode=492,668,899 total_neval=402,521,651
TIME TAKEN = 263.906 seconds

This doesn't look good, it takes much longer with a bigger table.
But if you divide total_nnode (total number of nodes searched for all plies) by the time, you get 1.87M nps for both A) and B).

Surely this is wrong?
But I can't spot any obvious error.

Is there any simple thing I can do to find error.
Thanks

I would have to say: Start looking for bugs, this seems wrong.

Is this the startposition ? If yes, e3 with a -94 score seems fishy (though consequent).

Start by making a flip function. This function calls evaluate,mirrors the black and white position and calls eval again, to check if the scores are the same.

2. Make a CreateHashnumberFromScratch and check it with the incremental one. (Actually, create everything you store in the hashtable)

3. Check the move from hashtable to see if it's legal. (i've had some mess with castle and ep moves)

etc for any sanitycheck.

Tony

[hgm]

Colin does not use QS, so I would not be worried too much about strange scores.

You should not test from a single position, because sometimes it indeed happens that a larger table gives you a slower search. (Because the move stored in the hash from the previous search turns bad on the current search, and you would have been better off not having the hash hit). On the average, a bigger table should give you shorter search time, though.

To test your hash, KRK or KPK are much more sensitive tests than the opening position.

[Tony]

Ok, in that case:

Implement quiescence search. Even if it is just a stand pat or recapture of the last piece. Because not having anything gives you crappy mainlines, wich are a lot harder to debug.

It really helps if you can follow a main line.

Tony

[cms271828]

Thanks...

As HG said, I don't have QS yet, so being 10 ply, the search will finish on blacks move, so black will be greedy and grab what it can, hence the negative score.

I already have a test that biulds a full zobrist key at each node, then compares to the incremental zobrist key, and that works.

But.. I have forgotten to put the castling and ep rights into both the incremental key, and the full key build, which is why the test passes.
I didn't need them for most of the checkmate puzzles I was trying to solve after putting in the TT.

As the strange results are from the starting position, then clearly both castling and ep can be reached given a 10 ply search.

So hopefully that is the reason, but I would not have expected such a drastic time difference 2^20 (180 seconds) compared to 2^21 (264 seconds)

I'll see if I can sort it out, thanks

[hgm]

I would be surprised if this was the reason.

It is usually wise to print and compare the move, PV, score, time and node count of every iteration. Then you already get a better impression if the change in node count is a systematic event, or if it might be triggered by a PV switch in one of the earlier iterations, which did not occur in the other case.

[Aleks Peshkov]

Implement quiescence search. Even if it is just a stand pat or recapture of the last piece. Because not having anything gives you crappy mainlines, wich are a lot harder to debug.

It really helps if you can follow a main line.

I agree. Transposition Table in middlegame phase is just a modest speed improvement.

Quiescence search is a significant boost to playing strength by itself and it creates natural looking search tree for the move ordering testing. No reason to optimize chess performance without quiescent search.

[cms271828]

Well, I've tried an endgame position...

[d]4k3/8/8/8/8/8/3PP3/4K3 w 1-0

With 2^20 entries, searching upto and including ply 26...

Depth=26 BM=e4 score=1047 nnode=158,699,271 neval=12,164,529
total_nnode=244,238,366 total_neval=36,930,811
TOTAL_PROBES=244,238,366 TOTAL_RECORDS=47,364,106
TIME TAKEN = 123.797 seconds


With 2^21 entries, searching upto and including ply 26...
Depth=26 BM=Kf2 score=1057 nnode=62,491,199 neval=4,437,154
total_nnode=203,663,619 total_neval=40,429,155
PROBES=203,663,619 RECORDS=34,791,791
TIME TAKEN = 97.735 seconds

The time improvement is there, but best move and the score differ for the 2 different table size.
I'm guessing this is wrong, and both move and score should come out the same?

[hgm]

No, they can be different, because the 'looking over the horizon' effect, that might be different in tables of different size. Perhaps you should leave out the d-Pawn, as the current position is so badly lost to black that it hardly matters what white plays. With only the e-Pawn it is still won for white to move, but the choice of moves is much more restricted.

[cms271828]

I played a couple of games (searching to 9ply), one with 2^20 size table, and one 2^21.
Generally the 2^21 was noticably quicker.

I also tried this position again upto depth 30.
[d]4k3/8/8/8/8/8/4P3/4K3 w 1-0

With 2^20 table...
Depth=30 BM=Kd2 score=950 nnode=55,495,919 neval=2,306,905
total_nnode=194,613,554 total_neval=25,021,401
PROBES=194613554 RECORDS=31,333,813
TIME TAKEN = 87.625 seconds

With 2^21 table...
Depth=30 BM=Kd2 score=947 nnode=71,670,160 neval=2,588,359
total_nnode=128,005,184 total_neval=11,416,574
PROBES=128,005,184 RECORDS=24,845,409
TIME TAKEN = 58.734 seconds

So that perhaps it was just bad luck with the starting position, but with this endgame position, theres a significant improvement.

Out of curiosity, does anyone know how many ply it takes to get checkmate in that position, I had engine go upto 36 ply, and it still hadn't found it.

Thanks

[Pradu]

[d]4k3/8/8/8/8/8/4P3/4K3 w 1-0
Out of curiosity, does anyone know how many ply it takes to get checkmate in that position, I had engine go upto 36 ply, and it still hadn't found it.
Thanks

http://www.k4it.de/index.php?topic=egtb&lang=en
Mate in 22, by Kd2 or Kf2