Testing LazySMP

[Ras]

Engines that have a faster move generator have a low speed of make/unmake move, and engines that have a high speed of make/unmake move, have a low speed of move generator.

How does that come? Is it because you can have the legality aspect either in the move generator (legal move generator) or defer that to later (pseudo-legal move generator)?

[Rebel]

Yes, I use Zobrist keys in make/unmake move.

Why, since you have stated not to make use of a hash table.

[LazySMP]

Engines that have a faster move generator have a low speed of make/unmake move, and engines that have a high speed of make/unmake move, have a low speed of move generator.

How does that come? Is it because you can have the legality aspect either in the move generator (legal move generator) or defer that to later (pseudo-legal move generator)?

That depends on how the engine implements the legality check. If that works with make move / in-check detection / unmake move, then there has to be make/unmake in the last ply. Otherwise, like in Stockfish, the number of available legal moves can be counted, which is called "bulk counting".

You mentioned earlier about two different types of engines for generating moves. I just compared them and came to the conclusion that there is an inverse relationship between speeds. I don't know the reason.

Why, since you have stated not to make use of a hash table.

I used Zobrist keys because I was considering implementing transposition tables in the new version. But I stopped developing the engine.

[Graham Banks]

.....I stopped developing the engine.

Why? If it's a genuine and original effort on your part, testers would take an interest in your engine..

[hgm]

Bulk counting eliminates make/unmake at the leaves. Not doing work is always much faster than doing work. That's why bulk counting is faster.

Yes, of course. This was never a point of discussion. The point is which type of perft has a comparable ratio of movegens and makemoves to a typical engine: bulk counting or make/unmake of the last ply. The former has a ratio of approximately 1:1. The latter something like 1:35, depending on the branching ratio of the position.

And an engine with good futility does approximately 1:1. So bulk-counting perfts are representative for how much time an engine spends on its basic tree-walking functions.

I called 'ply level' a meaningless concept, because it tries to single out the repeating 'unit cell' of a periodic (tree) structure. This is always ambiguous; you could make the 'level' start at any point, as long as you end it exactly 1 ply deeper. The ambiguity can only be solved if the repetitive structure does not extend infinitely, but ends somewhere. Then you can require that it ends with a complete unit cell, rather than a fractional one.

According to that criterion the concept of 'ply level' you use is flawed. You compare movegens and makemoves that are not at the same level, but belong to a different one. This is why you arrive at faulty conclusions about the ratio of movegens to makemoves, in particular why you keep insisting that the branching ratio would have any impact on this. In reality the makemove leading to the node belongs to the same level as the movegen in that node. Because engines end their branches with movegens to conclude that this did not produce any moves they want to search (i.e. no non-futile good-or-equal captures).

This is also obvious from the code: the loop over moves sequentially calls makemove - search - unmake, and those three are at the same level of the calling structure. The makemoves that the called search function does are one ply level deeper.

[hgm]

Yes, I use Zobrist keys in make/unmake move.

Why, since you have stated not to make use of a hash table.

TSCP updates a Zobrist key, even though it doesn't have a transposition table. It uses it for the purpose of detecting repetitions.

[hgm]

I think that these discussions may be fruitless because the speed of generating moves has an inverse relationship with the speed of making moves. Engines that have a faster move generator have a low speed of make/unmake move, and engines that have a high speed of make/unmake move, have a low speed of move generator.

This is exactly why it is important which type of perft to use when you want to use it as a model for optimizing the speed of your engine. (Nowadays people use perft mostly for testing correctness of their moove generators, but originally it was a 'performance test', which is where the name comes from!) If you would optimize a perft that makes/unmakes the last ply, you would basically optimize the make/unmake speed, possibly at the cost of a much larger detriment to the generation speed, which hardly contributes. So the faster you would make the perft, the slower it could make the engine that used the same algorithm. In a bulk-counting perft you don't have that problem, as it has a very similar ratio of movegen and make/unmake as an engine.

[LazySMP]

Why? If it's a genuine and original effort on your part, testers would take an interest in your engine..

I'd like to respond to your comment, because, well, just because I'm assuming you don't know the answer. Take a look at the kindergarten section and you will find the answer to your question. viewtopic.php?t=84382&start=20

If you would optimize a perft that makes/unmakes the last ply, you would basically optimize the make/unmake speed, possibly at the cost of a much larger detriment to the generation speed, which hardly contributes. So the faster you would make the perft, the slower it could make the engine that used the same algorithm. In a bulk-counting perft you don't have that problem, as it has a very similar ratio of movegen and make/unmake as an engine.

Thank you very much for your help and consideration. I think 70MNPS is enough for a chess engine and it doesn't need more optimization.

Normally they would test private engines if there is not a strong suspicion that these are copies or close derivatives of other engines. Such suspicion can arise from high similarity in move choice or other behaviors, or from the simple fact that people who never programmed an engine before pop up out of nowhere with an engine stronger than 2600 Elo. But trusting someone always entails the risk of being fooled, such as for instance happened in the case of Houdini. For an author that has a history of creating and continually improving chess engines, some of those open source, there would be little reason to believe he is cheating.

I forgot to reply to this comment. I think there is a possibility that they copied their own engines. It's not cheating, but it's not a new engine. However, it will be very difficult to prove it until the engine is in private mode. I may also copy my engine and give it to someone else to register as a new engine. Is there really a way to detect duplicate engines?

[LazySMP]

[pgn][Event "CCRL 40/15"]
[Site "CCRL"]
[Date "2024.08.15"]
[Round "939.1.24"]
[White "Dragon by Komodo 3.3 64-bit"]
[Black "Torch v3 64-bit"]
[Result "0-1"]
[ECO "E24"]
[Opening "Nimzo-Indian"]
[Variation "Saemisch variation"]
[PlyCount "197"]
[WhiteElo "3606"]
[BlackElo "3623"]

1. d4 {book} Nf6 {book} 2. c4 {book} e6 {book} 3. Nc3 {book} Bb4 {book} 4. a3
{book} Bxc3+ {book} 5. bxc3 {book} b6 {+0.00/29 50s} 6. f3 {-0.16/22 44s} Nc6
{+0.16/31 47s} 7. e4 {-0.12/24 34s} Na5 {+0.19/29 15s} 8. c5 {-0.05/25 34s}
bxc5 {+0.23/29 16s} 9. Qa4 {-0.13/26 62s} Nb7 {+0.23/32 19s} 10. Rb1 {-0.15/27
38s} cxd4 {+0.12/33 50s} 11. cxd4 {-0.03/25 21s} O-O {+0.16/33 36s} 12. Ba6
{+0.05/24 16s} Nxe4 {+0.21/32 16s} 13. fxe4 {+0.13/26 30s} Qh4+ {+0.19/29 17s}
14. Kd1 {+0.07/24 22s} Qxe4 {+0.17/30 27s} 15. Qc2 {+0.04/26 17s} Qxd4+
{+0.22/30 13s} 16. Ke1 {+0.00/28 43s} Qc5 {+0.17/33 33s} 17. Bd3 {-0.06/28 20s}
Qxc2 {+0.20/31 18s} 18. Bxc2 {-0.05/28 14s} Nd6 {+0.25/29 21s} 19. Nf3
{+0.00/30 37s} f6 {+0.28/30 39s} 20. Kf2 {+0.01/24 15s} Bb7 {+0.21/29 28s} 21.
Nd2 {+0.14/26 34s} Rfb8 {+0.20/29 26s} 22. a4 {+0.06/23 15s} Nf7 {+0.21/28 22s}
23. Rb3 {+0.03/24 15s} d6 {+0.23/29 23s} 24. Rg3 {+0.14/22 12s} Bd5 {+0.42/28
18s} 25. Ne4 {-0.13/23 19s} Bxe4 {+0.53/26 13s} 26. Bxe4 {-0.16/23 16s} d5
{+0.52/27 16s} 27. Bc2 {-0.34/24 33s} c5 {+0.58/29 23s} 28. Re1 {-0.31/23 85s}
Ne5 {+0.57/26 13s} 29. h4 {-0.54/24 64s} Rb4 {+0.78/26 17s} 30. h5 {-0.85/24
45s} Rd8 {+0.89/26 13s} 31. Ba3 {-0.88/24 21s} Rb7 {+0.87/25 16s} 32. a5
{-1.12/23 15s} d4 {+1.14/28 16s} 33. a6 {-0.91/25 35s} Rb5 {+1.07/28 20s} 34.
h6 {-0.81/27 10s} g6 {+1.27/36 22s} 35. Rb3 {-0.77/28 17s} Rxb3 {+1.21/37 16s}
36. Bxb3 {-0.70/28 9.0s} Nd3+ {+1.17/34 11s} 37. Kf1 {-1.05/30 11s} Nxe1
{+1.18/35 10s} 38. Bxc5 {-0.98/31 11s} Kf7 {+1.29/39 15s} 39. Kxe1 {-0.84/29
11s} Rc8 {+1.32/35 10s} 40. Bxa7 {-1.04/30 12s} Rc6 {+1.26/36 12s} 41. Bxd4
{-0.99/30 7.4s} Rxa6 {+1.36/34 19s} 42. Bd1 {-1.04/30 12s} Ra2 {+1.15/36 34s}
43. Be2 {-1.20/27 16s} Ra4 {+1.32/33 10s} 44. Be3 {-0.73/24 7.4s} e5 {+1.30/35
25s} 45. g3 {-0.93/28 22s} Ke6 {+1.00/40 36s} 46. Kf2 {-1.36/31 43s} Ra2
{+1.19/33 7.9s} 47. Kf1 {-1.15/26 12s} Kf5 {+1.47/29 7.4s} 48. Bf3 {-1.12/24
6.5s} Rc2 {+1.53/31 8.5s} 49. Ke1 {-1.04/23 4.6s} Rc8 {+1.60/29 7.3s} 50. Kf2
{-1.20/25 7.9s} g5 {+1.49/39 26s} 51. g4+ {-1.24/27 6.7s} Kg6 {+1.57/40 17s}
52. Be4+ {-1.27/29 8.8s} Kxh6 {+1.62/38 8.8s} 53. Kf3 {-1.16/29 6.5s} Kg7
{+1.60/38 9.1s} 54. Kg3 {-1.37/27 9.7s} Rh8 {+1.73/37 10s} 55. Kf3 {-1.43/28
8.0s} h5 {+1.75/36 8.1s} 56. gxh5 {-1.39/28 6.6s} Rxh5 {+1.77/36 7.0s} 57. Kg4
{-1.64/31 12s} Rh2 {+1.83/37 7.3s} 58. Kf5 {-1.73/33 10s} Re2 {+1.82/39 8.6s}
59. Bc5 {-1.66/27 3.4s} g4 {+1.83/37 8.7s} 60. Bd3 {-1.68/27 4.4s} Rd2
{+1.90/35 7.4s} 61. Be4 {-1.56/27 5.3s} g3 {+1.93/37 8.9s} 62. Kg4 {-1.81/25
4.0s} Re2 {+1.95/35 7.3s} 63. Bd3 {-1.69/25 4.5s} Rb2 {+1.95/37 8.7s} 64. Kh3
{-1.78/26 9.7s} Rb3 {+1.97/36 10s} 65. Be4 {-1.80/28 19s} Rc3 {+1.93/40 11s}
66. Bb6 {-1.90/29 12s} Kf7 {+1.95/40 9.6s} 67. Bf5 {-1.96/29 4.2s} Rb3
{+1.99/35 6.7s} 68. Bd8 {-1.91/27 8.3s} Rf3 {+1.98/38 9.5s} 69. Be4 {-1.96/26
5.2s} Rc3 {+1.97/41 13s} 70. Bf5 {-2.03/25 4.6s} Rf3 {+1.99/37 9.3s} 71. Be4
{-2.11/25 3.7s} Rf4 {+2.11/36 11s} 72. Bc2 {-1.87/26 7.2s} Rf2 {+2.11/34 5.6s}
73. Be4 {-2.11/24 5.0s} Re2 {+2.12/32 6.5s} 74. Bg2 {-2.13/28 22s} Re3
{+2.22/32 9.0s} 75. Bb6 {-2.29/27 10s} Rc3 {+2.26/31 6.9s} 76. Ba5 {-2.02/26
6.8s} Rd3 {+2.35/30 6.8s} 77. Bf1 {-2.25/26 4.0s} Ra3 {+2.29/30 15s} 78. Bb4
{-2.06/26 4.1s} Ra1 {+2.41/31 8.7s} 79. Bc4+ {-2.25/26 9.0s} Kg6 {+2.44/32
7.5s} 80. Bd3+ {-2.31/27 8.0s} f5 {+2.44/32 5.6s} 81. Kxg3 {-2.30/29 6.7s} Kf6
{+2.48/34 7.6s} 82. Bc5 {-2.70/27 7.2s} Ra5 {+2.63/33 6.9s} 83. Bb4 {-2.79/27
7.6s} Ra4 {+2.71/30 6.9s} 84. Bc5 {-2.97/26 8.3s} Ke6 {+2.77/30 7.1s} 85. Be2
{-3.22/27 18s} Kd5 {+2.91/27 5.8s} 86. Be7 {-3.46/26 8.4s} Ra1 {+3.02/27 8.3s}
87. Bh5 {-3.21/24 4.6s} Rb1 {+3.08/28 6.0s} 88. Be8 {-3.73/24 14s} Ke6
{+3.20/32 8.4s} 89. Bc5 {-3.77/24 4.7s} Kf6 {+3.28/30 8.0s} 90. Ba4 {-4.00/24
6.2s} Ra1 {+3.45/31 6.8s} 91. Bd7 {-4.32/28 13s} Rd1 {+3.55/32 6.0s} 92. Be8
{-4.23/25 6.3s} Rd8 {+3.68/30 9.3s} 93. Ba4 {-4.39/23 4.6s} Rd3+ {+3.76/27
7.5s} 94. Kf2 {-4.67/24 8.6s} f4 {+3.85/29 8.8s} 95. Kf1 {-4.78/24 7.4s} e4
{+4.10/27 8.6s} 96. Bb4 {-5.10/24 3.9s} Kf5 {+4.97/25 9.1s} 97. Bc6 {-5.53/23
7.8s} Rb3 {+5.16/24 7.0s} 98. Bd2 {-6.49/25 7.8s} Ke5 {+7.34/24 6.5s} 99. Ke2
{-7.65/24 9.3s, Black wins by adjudication} 0-1[/pgn]

Which just proves my point. The torch's playing style is more like Ethereal.

[pgn][Event "CCRL 40/15"]
[Site "CCRL"]
[Date "2024.08.04"]
[Round "938.3.341"]
[White "Torch v3 64-bit 4CPU"]
[Black "Ethereal 14.25 64-bit 4CPU"]
[Result "1-0"]
[ECO "B07"]
[Opening "Pirc defence"]
[PlyCount "106"]
[WhiteElo "3635"]
[BlackElo "3599"]

1. e4 {+0.00/1 0s} d6 {+0.00/1 0s} 2. d4 {+0.00/1 0s} Nf6 {+0.00/1 0s} 3. Nc3
{+0.00/1 0s} g6 {+0.00/1 0s} 4. Be3 {+0.00/1 0s} c6 {+0.00/1 0s} 5. h3 {+0.00/1
0s} Nbd7 {+0.00/1 0s} 6. Nf3 {+0.00/1 0s} Qc7 {+0.00/1 0s} 7. a4 {+0.00/1 0s}
Bg7 {+0.00/1 0s} 8. Be2 {+0.00/1 0s} O-O {+0.00/1 0s} 9. Qd2 {+0.69/34 26s} Re8
{(e6) -0.59/35 70s} 10. Ng5 {+0.80/33 20s} e5 {(e5) -0.47/33 14s} 11. dxe5
{(dxe5) +0.86/37 17s} Nxe5 {(Nxe5) -0.55/32 14s} 12. O-O-O {+0.80/39 32s} h6
{(h6) -0.65/33 21s} 13. Nf3 {(Nf3) +0.80/39 40s} Ned7 {(d5) -0.67/34 59s} 14.
Bxh6 {(Bxh6) +0.84/37 15s} Nxe4 {(Nxe4) -0.66/35 14s} 15. Nxe4 {+0.86/37 35s}
Rxe4 {(Rxe4) -0.62/34 11s} 16. Bxg7 {(Bxg7) +1.06/36 16s} Kxg7 {(Kxg7) -0.56/35
21s} 17. Nd4 {+0.96/39 33s} Nf6 {(Nf6) -0.52/34 14s} 18. h4 {(h4) +0.95/40 20s}
c5 {(c5) -0.78/33 14s} 19. Nb5 {(Nb5) +0.98/41 19s} Qe7 {(Qe7) -0.58/34 16s}
20. Nc3 {(Nc3) +1.00/40 30s} Rd4 {(Rd4) -0.96/33 13s} 21. Qg5 {+0.98/42 33s}
Bd7 {(Bd7) -0.67/36 27s} 22. Bf3 {+0.99/39 22s} Re8 {(Re8) -0.76/38 30s} 23. h5
{+1.11/38 15s} Ng4 {(Ng4) -0.60/38 12s} 24. Qxe7 {(h6) +1.23/45 22s} Rxe7
{(Rxe7) -0.73/39 20s} 25. Rxd4 {(Rxd4) +1.29/48 13s} cxd4 {(cxd4) -0.90/40 20s}
26. Nd5 {+1.38/50 23s} Re5 {(Re5) -0.95/38 11s} 27. Bxg4 {(Bxg4) +1.45/43 11s}
Bxg4 {(Bxg4) -0.93/39 13s} 28. h6+ {(h6) +1.57/42 10s} Kh8 {(Kh8) -0.74/37 11s}
29. Nf6 {(Nf6) +1.60/44 15s} Bf5 {(Bf5) -0.74/39 11s} 30. g4 {(g4) +1.64/46
11s} Re6 {(Re6) -0.84/37 10s} 31. g5 {(g5) +1.66/49 12s} Re2 {(Re2) -0.92/39
9s} 32. Rh4 {(Rh4) +1.66/44 12s} Rxc2+ {(Rxc2) -0.81/43 15s} 33. Kd1 {+1.72/44
11s} Rxf2 {(Rxf2) -0.81/45 35s} 34. Rxd4 {(Rxd4) +2.04/48 10s} Rxb2 {(Rxb2)
-0.86/44 10s} 35. Rxd6 {(Rxd6) +2.37/47 12s} Rb1+ {(Rb1) -1.01/49 16s} 36. Kd2
{(Kd2) +2.59/47 10s} Rb2+ {(Rb2) -1.01/50 11s} 37. Kc3 {(Kc3) +2.66/48 16s}
Rc2+ {(Rc2) -1.01/57 9s} 38. Kb4 {(Kb4) +2.66/49 10s} Rc8 {(Rc8) -1.01/59 10s}
39. Rd5 {(a5) +2.69/56 17s} Be6 {(Be6) -1.01/63 24s} 40. Ra5 {(Rd4) +2.71/58
18s} a6 {(a6) -1.01/49 12s} 41. Re5 {(Re5) +2.76/52 9s} b6 {(b6) -1.01/55 26s}
42. Re4 {(Re4) +2.81/52 18s} Rd8 {(Rc5) -1.01/58 18s} 43. a5 {(a5) +3.52/44
17s} Rd6 {(bxa5) -1.01/58 24s} 44. axb6 {(Ka4) +3.83/43 12s} Rxb6+ {(Rxb6)
-1.06/56 25s} 45. Ka5 {(Ka5) +4.00/43 10s} Rb8 {(Rb5) -1.11/53 28s} 46. Kxa6
{(Kxa6) +4.12/42 11s} Rf8 {(Rd8) -1.22/53 32s} 47. Kb7 {(Re3) +7.64/34 11s} Bf5
{(Bf5) -3.54/43 47s} 48. Re7 {+308.52/38 9s} Be6 {(Be6) -2.09/40 9s} 49. Kc7
{(Rc7) +308.90/51 12s} Ra8 {(Ra8) -3.87/40 26s} 50. Re8+ {[%eval #40,55] 12s}
Rxe8 {(Rxe8) -5.51/35 7s} 51. Nxe8 {[%eval #24,55] 11s} f5 {(f5) -9.53/37 18s}
52. Nf6 {(Nf6) [%eval #23,56] 10s} Bf7 {(Bb3) -15.91/36 39s} 53. Nd7 {[%eval
#16,61] 13s} f4 {(f4) -166.89/37 9s} 1-0[/pgn]

This cannot be proven but it is really unfortunate if the engine test is based on their author because it is not a good standard at all.

[chessica]

Code: Select all

    Program                            Score     %    Av.Op.  Elo    +   -    Draws

  1 Spike 1.4                      :  97.5/100  97.5   2183   2783    0 106    5.0 %
  2 SOS-51_Arena                   :  78.5/100  78.5   2217   2442   80  76    9.0 %
  3 Ruffian 1.0.5                  :  75.0/100  75.0   2220   2411   75  72   12.0 %
  4 Yace9987                       :  67.5/100  67.5   2226   2353   71  69    9.0 %
  5 Snitch-1.6.1                   :  61.0/100  61.0   2230   2308   67  66   10.0 %
  6 Nejmet_307                     :  57.5/100  57.5   2233   2285   67  66    9.0 %
  7 Monarch-v1.7                   :  33.5/100  33.5   2248   2129   69  71    7.0 %
  8 Maxwell-v3.1                   :  29.0/100  29.0   2252   2096   69  71   12.0 %
  9 LazySMP_popcnt                 :  22.5/100  22.5   2257   2042   75  79    9.0 %
 10 Colchess_80_64_ja_uci          :  15.0/100  15.0   2265   1964   86  92    8.0 %
 11 Fairy-Max-4.80                 :  13.0/100  13.0   2267   1937   92 100    6.0 %