Improving the search

[Tony]

Your pv is cut in your depth prefered search.

Exactly. How can I prevent that?

And why are you doing lazy evals in pawn endgame positions (or is that cached evals) ?

HJ.

Hm, I guess I'm to lazy doing a full eval if one side ends up with a queen surplus?

1) Don't allow cutoffs when you're in the pv. Just take best move.

2) That wouldn't be called a pawn ending

Tony

[Guetti]

Ok, I fixed the problem with the cut PV (at least partially) by not probing the table in the PV.
Still, didn't make much difference, though.

Code: Select all

Fine 70: Always-Replace
Trying 15s
 ply   score     time       nodes    pv
   1     137        0           3    a1b2 
   2     125        0          14    a1b2 a7b6 
   3     129        0          87    a1b2 a7b6 b2c3 
   4     129        0         216    a1b2 a7b6 b2c3 b6c7 
   5     133        0         603    a1b2 a7b6 b2c3 b6c7 c3d3 
   6     131        0        1213    a1b2 a7b6 b2c3 b6c7 c3d3 c7d7 
   7     131        0        2606    a1b2 a7b6 b2b1 b6c7 b1c2 c7d7 c2d3 
   8     133        0        5248    a1b2 a7b6 b2b3 b6c7 b3c4 c7b6 c4d3 b6c7 
   9     133        1        9129    a1b2 a7b6 b2b3 b6c7 b3c4 c7b6 c4d3 b6c7 d3e3 
  10     133        1       13921    a1b2 a7b6 b2a2 b6b7 a2b3 b7a6 b3c4 a6b6 c4d3 b6c7 
  11     133        2       21122    a1b2 a7b6 b2a2 b6b7 a2b3 b7c7 b3c4 c7b6 c4d3 b6c7 d3c4 
  12     133        3       32081    a1b2 a7b6 b2a1 b6b7 a1a2 b7a6 a2b3 a6b6 b3c4 b6a6 c4d3 a6b6 
  13     133        4       46375    a1b2 a7b6 b2a1 b6b7 a1b1 b7c7 b1a2 c7b6 a2b3 b6a6 b3c2 a6b6 c2d3 
  14     133        5       66041    a1b2 a7b6 b2a1 b6a7 a1b1 a7b6 b1a1 b6a7 a1b1 a7b6 b1c2 b6a6 c2d3 a6b6 
  15     133        7       88813    a1b2 a7b6 b2a1 b6a7 a1b1 a7b7 b1a1 b7a6 a1b1 a6b6 b1a2 b6a6 a2b3 a6b6 b3c4 
  16     133        9      121611    a1b2 a7b6 b2a1 b6a7 a1b1 a7a6 b1a1 a6b6 a1b1 b6a6 b1c1 a6b6 c1c2 b6a6 c2d3 a6b6 
  17     133       11      148407    a1b2 a7b6 b2a1 b6a6 a1b1 a6b6 b1a1 b6a6 a1b1 a6b6 b1a1 b6a6 a1b1 a6b6 b1a1 b6a7 a1b2 
  18     133       13      204914    a1b2 a7b6 b2a1 b6a6 a1b1 a6b6 b1a1 b6a6 a1b1 a6b6 b1a1 b6a6 a1b1 a6b6 b1c2 b6a6 c2d3 a6b6 
  19     133       15      267131    a1b2 a7b6 b2a1 b6a6 a1b1 a6b6 b1a1 b6a6 a1b1 a6b6 b1a1 b6a6 a1b1 a6b6 b1c1 b6a6 c1c2 a6b6 c2d3 
  20     133       21      448512    a1b2 a7b6 b2a1 b6a6 a1b1 a6b6 b1a1 b6a6 a1b1 a6b6 b1a1 b6a6 a1b1 a6b6 b1c1 b6a7 c1c2 a7b6 c2d3 b6c7 
  21     133       32      774814    a1b2 a7a8 b2a1 a8a7 a1b1 a7b7 b1a1 b7a7 a1b1 a7b7 b1a1 b7a7 a1b1 a7b7 b1a1 b7a7 a1b1 a7a6 b1c2 a6b6 c2d3 
  22     133       44     1180753    a1b2 a7a8 b2a1 a8a7 a1b1 a7b7 b1a1 b7a6 a1b1 a6a7 b1a1 a7a6 a1b1 a6a7 b1a1 a7a6 a1b1 a6b7 b1c2 b7a6 c2d3 a6b6 
  23     133       49     1320663    a1b2 a7a8 b2a1 a8b7 a1b1 b7a7 b1a1 a7b7 a1b1 b7a7 b1a1 a7b7 a1b1 b7a7 b1a1 a7b7 a1b1 b7a7 b1a1 a7b7 a1b1 b7a7 b1b2 
  24     133       55     1531011    a1b2 a7a8 b2a1 a8a7 a1b1 a7b7 b1a1 b7a7 a1b1 a7b7 b1a1 b7a7 a1b1 a7b7 b1a1 b7a7 a1b1 a7b7 b1c1 b7c7 c1c2 c7d7 c2d3 d7c7 
>>25     183       63     1752077    a1b2 *
  25     247       77     2122128    a1b1 a7b7 b1c1 b7c7 c1d1 c7d7 d1c2 d7c7 c2d3 c7b6 d3e2 b6a6 e2f2 a6b6 f2g3 b6a6 g3h4 a6b6 h4g5 b6c7 g5f5 c7d7 f5e4 d7e7 f4f5 
  26     247       99     2841261    a1b1 a7b7 b1c1 b7c7 c1d1 c7d7 d1c2 d7c7 c2d3 c7b6 d3e3 b6a6 e3f2 a6b6 f2g3 b6b7 g3h4 b7c7 h4g5 c7d8 g5f5 d8e7 f5e4 e7f6 f4f5 f6g5 
  27     247      115     3331018    a1b1 a7b7 b1c1 b7c7 c1d1 c7d7 d1c2 d7c7 c2d3 c7b6 d3e2 b6a6 e2f2 a6b7 f2g3 b7c7 g3h4 c7d7 h4g5 d7e7 g5f5 e7f7 f5g5 f7e7 f4f5 e7d7 f5f6 
  28     251      141     4126158    a1b1 a7b7 b1c1 b7a6 c1d1 a6b6 d1e1 b6c7 e1f2 c7d7 f2g3 d7e7 g3h4 e7f6 h4h5 f6f7 h5g5 f7e7 g5f5 e7f7 f5g5 f7e7 f4f5 e7d7 f5f6 d7e8 g5f5 e8d7 
  29     255      176     5174004    a1b1 a7b7 b1c1 b7c7 c1d1 c7d7 d1c2 d7c7 c2d3 c7b6 d3e2 b6a6 e2f2 a6b6 f2g3 b6b7 g3h4 b7c7 h4g5 c7d7 g5f5 d7e7 f5g6 e7e8 g6g7 
  30     257      223     6626153    a1b1 a7b7 b1c1 b7a6 c1d1 a6b6 d1e2 b6c7 e2f2 c7d7 f2g3 d7e7 g3h4 e7f6 h4h5 f6f7 h5g5 f7e7 g5f5 e7f7 f5g5 f7g7 f4f5 g7f7 f5f6 f7e8 g5f4 e8d7 f4e4 d7c7 
  31     263      295     8858621    a1b1 a7b7 b1c1 b7b6 c1d2 b6b7 d2e1 b7c7 e1f2 c7d7 f2g3 d7e7 g3h4 e7f6 h4h5 f6f7 h5g5 f7g7 g5f5 g7f7 f5g4 f7g6 f4f5 g6f6 g4f4 f6e7 f4g5 e7d8 f5f6 d8e8 
  32     258      335     9986081    a1b1 a7b7 b1c1 b7c7 c1d1 c7d7 d1c2 d7c7 c2d3 c7b6 d3d2 b6a6 d2e1 a6b6 e1f2 b6a6 f2g3 a6b6 g3h4 b6c7 h4g5 c7d8 g5f5 d8e7 f5g5 e7f7 f4f5 f7e7 f5f6 e7d7 g5f4 d7c7 f6f7 
  33     263      447    13303514    a1b1 a7b7 b1c1 b7b6 c1d2 b6b7 d2e1 b7c7 e1f2 c7d7 f2g3 d7e7 g3h4 e7f6 h4h5 f6f7 h5g5 f7g7 g5f5 g7f7 f5g5 
  34     260      588    16894767    a1b1 a7b7 b1c1 b7a6 c1b2 a6b6 b2c2 b6a6 c2d1 a6b6 d1e1 b6a6 e1f2 a6b6 f2f3 b6c7 f3g3 c7d7 g3h4 d7e7 h4g5 e7e8 g5f6 e8f8 f6e6 f8g7 e6d6 g7f6 d6c6 f6g6 d5d6 
  35     263      762    21701929    a1b1 a7a6 b1c2 a6b6 c2d2 b6b7 d2e1 b7c7 e1f2 c7d7 f2g3 d7e7 g3h4 e7f6 h4h5 f6f7 h5g5 f7g7 g5f5 g7f7 f5g4 f7f6 f4f5 f6e7 g4g5 e7f7 f5f6 f7g8 g5g4 g8h7 g4h5 h7g8 h5g5 g8f8 g5f4 

35 (3/3) 2.63  15.01  37778537  a1b1 a7a6 b1c2 a6b6 c2d2 b6b7 d2e1 b7c7 e1f2 c7d7 f2g3 d7e7 g3h4 e7f6 h4h5 f6f7 h5g5 f7g7 g5f5 g7f7 f5g4 f7f6 f4f5 f6e7 g4g5 e7f7 f5f6 f7g8 g5g4 g8h7 g4h5 h7g8 h5g5 g8f8 g5f4 
Score: 2.63  Time: 15.01  Ply: 35/45
Nodes:  37778537  NPS: 2517k
Qnodes:   1471389   3.89% of Nodes
CutOffs:  30390275  80.44% of Nodes
CutOffOnFirst:  29303022  96.42% of CutOffs
SRExtensions:     18134
CheckExtensions:   2156908
Evals:   9092036
LazyEvals:   5782021  63.59% of Evals
Evasions:   2863439
Mates:      1107
Stalemates:      1590
TT-Entries: 1048576  TT-Full: 99.9%
TT-Writes:   11552292  TT-Hits:   29002561
move a1b1 


Fine 70: Depth-Prefer
Trying 15s
 ply   score     time       nodes    pv
   1     137        0           3    a1b2 
   2     125        0          14    a1b2 a7b6 
   3     129        0          87    a1b2 a7b6 b2c3 
   4     129        0         219    a1b2 a7b6 b2c3 b6c7 
   5     133        0         433    a1b2 a7b6 b2c3 b6c7 c3d3 
   6     131        0         706    a1b2 a7b6 b2c3 b6c7 c3d3 c7d7 
   7     131        0        1309    a1b2 a7b6 b2c3 b6c7 c3d3 c7d7 d3e3 
   8     133        0        2274    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6c7 
   9     133        0        3247    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6c7 d3c4 
  10     133        0        4799    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6a6 d3e3 a6b6 
  11     133        1        6559    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6a6 d3e3 a6b6 e3d3 
  12     133        1        8662    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6a6 d3e3 a6b6 e3d3 b6c7 
  13     133        1       11616    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6a6 d3e3 a6b6 e3d3 b6c7 d3c4 
  14     133        2       15138    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6a6 d3e3 a6b6 e3d3 b6a6 d3e3 a6b6 
  15     133        2       19238    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6a6 d3e3 a6b6 e3d3 b6a6 d3e3 a6b6 e3d3 
  16     133        3       25830    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6a6 d3e3 a6b6 e3d3 b6a6 d3e3 a6b6 e3d3 b6c7 
  17     133        3       32146    a1b2 a7b6 b2c3 b6c7 c3c4 c7b6 c4d3 b6a6 d3e3 a6b6 e3d3 b6a6 d3e3 a6b6 e3d3 b6c7 d3c4 
  18     133       20      371166    a1b2 a7a8 b2c2 a8b8 c2c3 b8b7 
  19     133       21      403690    a1b2 a7a8 b2c2 a8b8 c2c3 b8b7 c3d3 b7c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 
  20     133       31      663370    a1b2 a7a8 b2c2 a8b8 c2c3 b8b7 c3d3 b7c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 
  21     133       44     1052309    a1b2 a7a8 b2c2 a8b8 c2c3 b8b7 c3d3 b7c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 d3c4 
  22     133      605    16061750    a1b2 a7a8 b2c2 a8b8 c2c3 b8b7 c3d3 b7c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 

22 (3/3) 1.33  15.00  41663224  a1b2 a7a8 b2c2 a8b8 c2c3 b8b7 c3d3 b7c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 c4d3 b6c7 d3c4 c7b6 
Score: 1.33  Time: 15.00  Ply: 22/26
Nodes:  41663224  NPS: 2777k
Qnodes:    919724   2.21% of Nodes
CutOffs:  28871850  69.30% of Nodes
CutOffOnFirst:  26864050  93.05% of CutOffs
SRExtensions:         9
CheckExtensions:    627006
Evals:  18054958
LazyEvals:   2646343  14.66% of Evals
Evasions:   1417879
TT-Entries: 1048576  TT-Full: 99.9%
TT-Writes:   16680508  TT-Hits:   40621641
move a1b2 

In contrast, in most other position the difference between which table I probe is not that big. Here is the search after e4 from the starting position.

Code: Select all

Always-Replace:
Trying 15s
 ply   score     time       nodes    pv
   1       8        0          22    b8c6 
   2     -12        0         111    b8c6 b1c3 
   3       8        0         669    b8c6 b1c3 g8f6 
   4      -9        3        4454    g8f6 b1c3 d7d5 f1d3 
   5       0       11       25139    b8c6 b1c3 g8f6 g1f3 d7d5 
   6      -9       19       87777    b8c6 b1c3 g8f6 g1f3 d7d5 f1d3 
   7       0       64      655763    e7e5 b1c3 b8c6 g1f3 g8f6 f1c4 f8b4 
   8     -14      282     3575903    d7d5 e4d5 d8d5 g1f3 c8g4 f1e2 g4f3 e2f3 
   9      -4     1172    15755934    e7e5 b1c3 g8f6 g1f3 f8d6 f1c4 e8g8 e1g1 b8c6 

9 (20/20) -0.04  15.01  20256945  e7e5 b1c3 g8f6 g1f3 f8d6 f1c4 e8g8 e1g1 b8c6 
Score: -0.04  Time: 15.01  Ply: 9/31
Nodes:  20256945  NPS: 1349k
Qnodes:   4974473  24.56% of Nodes
CutOffs:  16573964  81.82% of Nodes
CutOffOnFirst:  14750173  89.00% of CutOffs
SRExtensions:     26609
CheckExtensions:    460912
Evals:  15799580
LazyEvals:  10829953  68.55% of Evals
Evasions:   1446496
Mates:      8923
TT-Entries: 1048576  TT-Full: 91.8%
TT-Writes:    2623525  TT-Hits:    1801428
move e7e5 

Depth-Prefer:
Trying 15s
 ply   score     time       nodes    pv
   1       8        0          22    b8c6 
   2     -12        0         111    b8c6 b1c3 
   3       8        0         669    b8c6 b1c3 g8f6 
   4      -9        3        4424    g8f6 b1c3 d7d5 f1d3 
   5       0       10       23681    b8c6 b1c3 g8f6 g1f3 d7d5 
   6      -9       16       79575    b8c6 b1c3 g8f6 g1f3 d7d5 f1d3 
   7       0       52      502161    e7e5 b1c3 b8c6 g1f3 g8f6 f1c4 f8b4 
   8     -14      241     3016745    d7d5 e4d5 d8d5 g1f3 c8g4 f1e2 g4f3 e2f3 
   9      -4     1106    14672355    e7e5 g1f3 g8f6 b1c3 f8d6 f1c4 b8c6 e1g1 e8g8 

9 (20/20) -0.04  15.00  20009981  e7e5 g1f3 g8f6 b1c3 f8d6 f1c4 b8c6 e1g1 e8g8 
Score: -0.04  Time: 15.00  Ply: 9/32
Nodes:  20009981  NPS: 1333k
Qnodes:   5430158  27.14% of Nodes
CutOffs:  16153808  80.73% of Nodes
CutOffOnFirst:  14173517  87.74% of CutOffs
SRExtensions:     27465
CheckExtensions:    462751
Evals:  15559281
LazyEvals:  10828107  69.59% of Evals
Evasions:   1549262
Mates:      9324
TT-Entries: 1048576  TT-Full: 91.8%
TT-Writes:    2625975  TT-Hits:    1582315
move e7e5 

[Guetti]

Your pv is cut in your depth prefered search.

Exactly. How can I prevent that?

And why are you doing lazy evals in pawn endgame positions (or is that cached evals) ?

HJ.

Hm, I guess I'm to lazy doing a full eval if one side ends up with a queen surplus?

1) Don't allow cutoffs when you're in the pv. Just take best move.

2) That wouldn't be called a pawn ending

Tony

1) ok, I tried to hack this in.

2) Sorry, I don't understand. After some plys one side will promote to queen and then the eval will take the lazy_eval cutoff. That's why the search with less depth has much less lazy_eval cutoffs. Do I do something wrong there? What is the sense in doing a full eval if one side is up a queen?

Maybe I can add that the depth-prefer does solve Fine 70, but only after 5 minutes and at ply 26. I'm just curious why the always-replace does get 10 plys more in the same position in 15 seconds.

I also have to add that if the table probed first contains no entry matching the position, the other table is probed. I just switch which table will be preferred.

And if I remove the condition

Code: Select all

if (depth >= prefer->mDepth) {
   // store Entry ....
}

in the TT.store() function for the depth-prefer table I get the same result when probing it as when I probe the always-replace table.

[Guetti]

Well, after pondering a bit more on this I still think the results seem to be logical. With the depth-prefer approach, as further the search diverges form the starting position, the TT entries will be less useful, and old entries are not replaced in the TT because they have a higher depth, or the other way round, new entries matching the position deep in the search tree are not stored because there are old entries in the slots with a higher depth.

In contrast, with the always-replace method, the entries stored at shallow depth in the TT are replaced and the TT contains much more entries useful deeper in the search to make a cutoff.
But maybe I make thinking mistake. This TT stuff is still quite new the me.