DQ 1.2 vs Bad Gyal 7 - the power of policy

[dkappe]

Two nets that, as far as I know, use similar source material.

DarkQueen uses lichess games that have been annotated by Stockfish. Bad Gyal 7 uses lichess games annotated with q and policy from Stockfish. The main difference seems to be policy. So far it's a walkover by the smaller Bad Gyal 7. Here a sample game:

[pgn] [Event "?"] [Site "?"] [Date "2019.08.29"] [Round "9"] [White "DQ 1.2"] [Black "Bad Gyal 7"] [Result "0-1"] [ECO "D20"] [GameDuration "00:06:08"] [GameEndTime "2019-08-29T11:19:39.404 CDT"] [GameStartTime "2019-08-29T11:13:31.036 CDT"] [Opening "QGA"] [PlyCount "190"] [TimeControl "2+2"] [Variation "3.e4"] 1. d4 {book} d5 {book} 2. c4 {book} dxc4 {book} 3. e4 {book} c5 {book} 4. Nf3 {+0.28/8 1.8s} cxd4 {-0.29/8 1.0s} 5. Qxd4 {+0.34/9 1.1s} Qxd4 {-0.30/9 1.2s} 6. Nxd4 {+0.30/8 1.4s} a6 {-0.27/8 1.6s} 7. Bxc4 {+0.23/7 2.5s} e6 {-0.33/8 2.6s} 8. O-O {+0.15/6 2.9s} Nf6 {-0.17/7 2.0s} 9. Nc3 {+0.09/6 1.9s} Nbd7 {-0.23/8 2.7s} 10. Be3 {+0.11/5 2.2s} Ng4 {-0.22/8 2.0s} 11. Bd2 {+0.12/6 2.0s} Bd6 {-0.36/7 2.6s} 12. h3 {+0.16/6 1.5s} Ngf6 {-0.36/8 0.90s} 13. Rfe1 {+0.18/5 2.2s} b5 {-0.02/8 1.6s} 14. Bb3 {+0.06/6 1.7s} Bb7 {+0.09/8 1.4s} 15. Rad1 {+0.15/6 2.0s} Rc8 {+0.09/9 1.7s} 16. Bg5 {+0.19/6 2.2s} Bb8 {+0.30/8 2.0s} 17. f3 {+0.10/6 2.1s} Nc5 {+0.31/8 2.5s} 18. Bc2 {0.00/5 1.9s} Ncd7 {+0.28/8 2.2s} 19. Bb3 {+0.02/5 1.9s} h6 {+0.27/9 2.1s} 20. Be3 {+0.05/6 2.2s} Nc5 {+0.28/8 1.7s} 21. Bc2 {+0.03/6 1.8s} O-O {+0.34/8 2.5s} 22. a3 {+0.09/6 2.3s} Ncd7 {+0.38/8 2.2s} 23. Bb3 {+0.17/6 1.9s} Rfe8 {+0.41/7 2.4s} 24. Rd2 {+0.09/5 2.2s} Ne5 {+0.23/7 2.3s} 25. Red1 {+0.07/6 1.5s} Nh5 {+0.14/8 1.5s} 26. Nde2 {+0.07/6 2.5s} g5 {+0.34/7 1.9s} 27. Kf2 {+0.04/5 2.1s} Kh7 {+0.34/7 2.9s} 28. g3 {+0.19/5 2.0s} Nf6 {+0.30/8 1.7s} 29. f4 {+0.28/7 1.7s} Nc4 {+0.37/12 1.0s} 30. Bxc4 {+0.07/10 1.1s} Rxc4 {+0.39/11 1.2s} 31. e5 {+0.18/9 1.8s} Nd5 {+0.39/11 1.3s} 32. Nxd5 {+0.07/9 1.5s} Bxd5 {+0.38/12 0.34s} 33. Nc3 {+0.04/8 1.8s} Bc6 {+0.34/10 2.7s} 34. Rd8 {-0.04/8 3.4s} Rxd8 {+0.62/12 3.6s} 35. Rxd8 {-0.09/9 1.3s} Bc7 {+0.65/13 0.30s} 36. Rc8 {-0.13/8 2.2s} Bb7 {+0.67/13 0.60s} 37. Re8 {-0.18/9 2.3s} gxf4 {+0.92/12 5.0s} 38. gxf4 {-0.24/8 1.4s} b4 {+0.95/12 0.31s} 39. axb4 {-0.51/8 1.6s} Rxb4 {+0.96/12 0.38s} 40. Nd1 {-0.48/8 2.4s} Bb6 {+0.98/11 2.5s} 41. Bxb6 {-0.56/8 2.5s} Rxb6 {+0.96/11 0.50s} 42. Re7 {-0.61/7 2.5s} Kg6 {+1.00/10 3.3s} 43. h4 {-0.59/6 2.4s} Rb3 {+1.08/11 5.1s} 44. h5+ {-0.35/8 1.5s} Kg7 {+1.07/10 0.31s} 45. f5 {-0.28/9 1.5s} exf5 {+1.15/15 2.2s} 46. e6 {-0.35/11 1.2s} Kf6 {+1.17/15 0.40s} 47. Rxf7+ {-0.48/11 1.3s} Kxe6 {+1.16/13 0.70s} 48. Rh7 {-0.55/10 1.3s} Kf6 {+1.20/12 1.1s} 49. Rxh6+ {-0.71/9 1.5s} Kg5 {+1.22/11 1.3s} 50. Rh8 {-0.85/8 4.4s} Bf3 {+1.30/12 3.9s} 51. Rg8+ {-0.87/9 2.0s} Kxh5 {+1.30/12 0.40s} 52. Ne3 {-0.92/8 2.6s} Be4 {+1.46/9 4.6s} 53. Nd1 {-0.83/7 1.4s} a5 {+1.50/9 2.1s} 54. Rg3 {-0.80/6 3.0s} Rb4 {+1.63/8 3.7s} 55. Ra3 {-1.00/6 2.3s} a4 {+1.80/8 2.5s} 56. Ke1 {-1.02/6 1.8s} f4 {+1.88/7 3.2s} 57. Kd2 {-1.17/6 2.1s} Bf5 {+1.86/9 3.0s} 58. Kc3 {-1.19/7 1.4s} Re4 {+1.81/10 0.81s} 59. Kd2 {-1.34/7 2.4s} Rd4+ {+1.82/10 1.5s} 60. Ke1 {-1.33/7 1.3s} Kg5 {+1.92/9 2.8s} 61. Nc3 {-1.45/7 2.6s} Bc2 {+2.02/9 1.1s} 62. Ne2 {-1.81/7 2.5s} Rd1+ {+2.40/10 2.1s} 63. Kf2 {-2.10/8 0.10s} Rb1 {+2.60/8 1.2s} 64. Nd4 {-2.64/7 3.9s} Rxb2 {+2.88/8 2.3s} 65. Kf3 {-2.94/6 2.0s} Bb3 {+5.74/7 2.3s} 66. Ke4 {-3.79/4 1.4s} Ra2 {+7.35/6 1.5s} 67. Rxa2 {-128.00/3 2.4s} Bxa2 {+128.00/2 2.3s} 68. Nf3+ {-12.64/3 1.8s} Kf6 {+2.59/1 0.012s} 69. Kxf4 {-128.00/2 1.6s} a3 {+128.00/1 0.008s} 70. Nd4 {-128.00/3 2.6s} Bf7 {+128.00/2 7.5s} 71. Nc2 {-128.00/2 1.7s} a2 {+128.00/2 0.009s} 72. Na1 {-1.68/10 2.2s} Bg6 {+1.20/1 0.014s} 73. Ke3 {-2.19/11 1.8s} Ke5 {+0.45/1 0.012s} 74. Kd2 {-2.68/10 2.0s} Kd4 {+0.20/1 0.015s} 75. Kd1 {-3.05/8 2.5s} Kc3 {+1.60/9 5.8s} 76. Kc1 {-3.47/7 0.40s} Bd3 {+1.55/9 4.9s} 77. Nc2 {-128.00/2 3.5s} Bb5 {+1.53/8 0.10s} 78. Na1 {-128.00/2 1.0s} Ba4 {+1.35/7 0.057s} 79. Nc2 {-128.00/2 1.5s} Kb3 {+2.32/5 0.019s} 80. Na1+ {-128.00/2 1.7s} Ka3 {+128.00/2 0.011s} 81. Kd2 {-128.00/2 3.5s} Kb2 {+128.00/2 0.10s} 82. Kd3 {-128.00/2 1.9s} Kxa1 {+128.00/2 0.099s} 83. Kc4 {-128.00/2 2.2s} Kb1 {+128.00/2 15s} 84. Kb4 {-128.00/2 1.5s} a1=Q {+128.00/2 2.1s} 85. Kc5 {-25.37/4 2.3s} Qe5+ {+17.22/1 0.013s} 86. Kb6 {-128.00/2 2.3s} Bd7 {+128.00/2 2.2s} 87. Kb7 {-128.00/2 1.8s} Qa5 {+128.00/2 0.10s} 88. Kb8 {-128.00/2 0.002s} Bc8 {+42.69/2 0.004s} 89. Kxc8 {-128.00/2 0.001s} Qa7 {+41.98/1 0.008s} 90. Kd8 {-126.92/1 0.030s} Kc2 {+13.28/9 8.6s} 91. Kc8 {-128.00/2 5.7s} Kd3 {+128.00/4 2.2s} 92. Kd8 {-128.00/2 0.001s} Ke4 {+128.00/2 3.7s} 93. Kc8 {-128.00/2 4.5s} Kd5 {+128.00/2 2.0s} 94. Kd8 {-128.00/2 0.002s} Kd6 {+128.00/2 0.058s} 95. Kc8 {-128.00/2 5.2s} Qa8# {+128.00/2 3.8s, Black mates} 0-1 [/pgn]

[MikeB]

Two nets that, as far as I know, use similar source material.

DarkQueen uses lichess games that have been annotated by Stockfish. Bad Gyal 7 uses lichess games annotated with q and policy from Stockfish. The main difference seems to be policy. So far it's a walkover by the smaller Bad Gyal 7. Here a sample game:

[pgn] [Event "?"] [Site "?"] [Date "2019.08.29"] [Round "9"] [White "DQ 1.2"] [Black "Bad Gyal 7"] [Result "0-1"] [ECO "D20"] [GameDuration "00:06:08"] [GameEndTime "2019-08-29T11:19:39.404 CDT"] [GameStartTime "2019-08-29T11:13:31.036 CDT"] [Opening "QGA"] [PlyCount "190"] [TimeControl "2+2"] [Variation "3.e4"] 1. d4 {book} d5 {book} 2. c4 {book} dxc4 {book} 3. e4 {book} c5 {book} 4. Nf3 {+0.28/8 1.8s} cxd4 {-0.29/8 1.0s} 5. Qxd4 {+0.34/9 1.1s} Qxd4 {-0.30/9 1.2s} 6. Nxd4 {+0.30/8 1.4s} a6 {-0.27/8 1.6s} 7. Bxc4 {+0.23/7 2.5s} e6 {-0.33/8 2.6s} 8. O-O {+0.15/6 2.9s} Nf6 {-0.17/7 2.0s} 9. Nc3 {+0.09/6 1.9s} Nbd7 {-0.23/8 2.7s} 10. Be3 {+0.11/5 2.2s} Ng4 {-0.22/8 2.0s} 11. Bd2 {+0.12/6 2.0s} Bd6 {-0.36/7 2.6s} 12. h3 {+0.16/6 1.5s} Ngf6 {-0.36/8 0.90s} 13. Rfe1 {+0.18/5 2.2s} b5 {-0.02/8 1.6s} 14. Bb3 {+0.06/6 1.7s} Bb7 {+0.09/8 1.4s} 15. Rad1 {+0.15/6 2.0s} Rc8 {+0.09/9 1.7s} 16. Bg5 {+0.19/6 2.2s} Bb8 {+0.30/8 2.0s} 17. f3 {+0.10/6 2.1s} Nc5 {+0.31/8 2.5s} 18. Bc2 {0.00/5 1.9s} Ncd7 {+0.28/8 2.2s} 19. Bb3 {+0.02/5 1.9s} h6 {+0.27/9 2.1s} 20. Be3 {+0.05/6 2.2s} Nc5 {+0.28/8 1.7s} 21. Bc2 {+0.03/6 1.8s} O-O {+0.34/8 2.5s} 22. a3 {+0.09/6 2.3s} Ncd7 {+0.38/8 2.2s} 23. Bb3 {+0.17/6 1.9s} Rfe8 {+0.41/7 2.4s} 24. Rd2 {+0.09/5 2.2s} Ne5 {+0.23/7 2.3s} 25. Red1 {+0.07/6 1.5s} Nh5 {+0.14/8 1.5s} 26. Nde2 {+0.07/6 2.5s} g5 {+0.34/7 1.9s} 27. Kf2 {+0.04/5 2.1s} Kh7 {+0.34/7 2.9s} 28. g3 {+0.19/5 2.0s} Nf6 {+0.30/8 1.7s} 29. f4 {+0.28/7 1.7s} Nc4 {+0.37/12 1.0s} 30. Bxc4 {+0.07/10 1.1s} Rxc4 {+0.39/11 1.2s} 31. e5 {+0.18/9 1.8s} Nd5 {+0.39/11 1.3s} 32. Nxd5 {+0.07/9 1.5s} Bxd5 {+0.38/12 0.34s} 33. Nc3 {+0.04/8 1.8s} Bc6 {+0.34/10 2.7s} 34. Rd8 {-0.04/8 3.4s} Rxd8 {+0.62/12 3.6s} 35. Rxd8 {-0.09/9 1.3s} Bc7 {+0.65/13 0.30s} 36. Rc8 {-0.13/8 2.2s} Bb7 {+0.67/13 0.60s} 37. Re8 {-0.18/9 2.3s} gxf4 {+0.92/12 5.0s} 38. gxf4 {-0.24/8 1.4s} b4 {+0.95/12 0.31s} 39. axb4 {-0.51/8 1.6s} Rxb4 {+0.96/12 0.38s} 40. Nd1 {-0.48/8 2.4s} Bb6 {+0.98/11 2.5s} 41. Bxb6 {-0.56/8 2.5s} Rxb6 {+0.96/11 0.50s} 42. Re7 {-0.61/7 2.5s} Kg6 {+1.00/10 3.3s} 43. h4 {-0.59/6 2.4s} Rb3 {+1.08/11 5.1s} 44. h5+ {-0.35/8 1.5s} Kg7 {+1.07/10 0.31s} 45. f5 {-0.28/9 1.5s} exf5 {+1.15/15 2.2s} 46. e6 {-0.35/11 1.2s} Kf6 {+1.17/15 0.40s} 47. Rxf7+ {-0.48/11 1.3s} Kxe6 {+1.16/13 0.70s} 48. Rh7 {-0.55/10 1.3s} Kf6 {+1.20/12 1.1s} 49. Rxh6+ {-0.71/9 1.5s} Kg5 {+1.22/11 1.3s} 50. Rh8 {-0.85/8 4.4s} Bf3 {+1.30/12 3.9s} 51. Rg8+ {-0.87/9 2.0s} Kxh5 {+1.30/12 0.40s} 52. Ne3 {-0.92/8 2.6s} Be4 {+1.46/9 4.6s} 53. Nd1 {-0.83/7 1.4s} a5 {+1.50/9 2.1s} 54. Rg3 {-0.80/6 3.0s} Rb4 {+1.63/8 3.7s} 55. Ra3 {-1.00/6 2.3s} a4 {+1.80/8 2.5s} 56. Ke1 {-1.02/6 1.8s} f4 {+1.88/7 3.2s} 57. Kd2 {-1.17/6 2.1s} Bf5 {+1.86/9 3.0s} 58. Kc3 {-1.19/7 1.4s} Re4 {+1.81/10 0.81s} 59. Kd2 {-1.34/7 2.4s} Rd4+ {+1.82/10 1.5s} 60. Ke1 {-1.33/7 1.3s} Kg5 {+1.92/9 2.8s} 61. Nc3 {-1.45/7 2.6s} Bc2 {+2.02/9 1.1s} 62. Ne2 {-1.81/7 2.5s} Rd1+ {+2.40/10 2.1s} 63. Kf2 {-2.10/8 0.10s} Rb1 {+2.60/8 1.2s} 64. Nd4 {-2.64/7 3.9s} Rxb2 {+2.88/8 2.3s} 65. Kf3 {-2.94/6 2.0s} Bb3 {+5.74/7 2.3s} 66. Ke4 {-3.79/4 1.4s} Ra2 {+7.35/6 1.5s} 67. Rxa2 {-128.00/3 2.4s} Bxa2 {+128.00/2 2.3s} 68. Nf3+ {-12.64/3 1.8s} Kf6 {+2.59/1 0.012s} 69. Kxf4 {-128.00/2 1.6s} a3 {+128.00/1 0.008s} 70. Nd4 {-128.00/3 2.6s} Bf7 {+128.00/2 7.5s} 71. Nc2 {-128.00/2 1.7s} a2 {+128.00/2 0.009s} 72. Na1 {-1.68/10 2.2s} Bg6 {+1.20/1 0.014s} 73. Ke3 {-2.19/11 1.8s} Ke5 {+0.45/1 0.012s} 74. Kd2 {-2.68/10 2.0s} Kd4 {+0.20/1 0.015s} 75. Kd1 {-3.05/8 2.5s} Kc3 {+1.60/9 5.8s} 76. Kc1 {-3.47/7 0.40s} Bd3 {+1.55/9 4.9s} 77. Nc2 {-128.00/2 3.5s} Bb5 {+1.53/8 0.10s} 78. Na1 {-128.00/2 1.0s} Ba4 {+1.35/7 0.057s} 79. Nc2 {-128.00/2 1.5s} Kb3 {+2.32/5 0.019s} 80. Na1+ {-128.00/2 1.7s} Ka3 {+128.00/2 0.011s} 81. Kd2 {-128.00/2 3.5s} Kb2 {+128.00/2 0.10s} 82. Kd3 {-128.00/2 1.9s} Kxa1 {+128.00/2 0.099s} 83. Kc4 {-128.00/2 2.2s} Kb1 {+128.00/2 15s} 84. Kb4 {-128.00/2 1.5s} a1=Q {+128.00/2 2.1s} 85. Kc5 {-25.37/4 2.3s} Qe5+ {+17.22/1 0.013s} 86. Kb6 {-128.00/2 2.3s} Bd7 {+128.00/2 2.2s} 87. Kb7 {-128.00/2 1.8s} Qa5 {+128.00/2 0.10s} 88. Kb8 {-128.00/2 0.002s} Bc8 {+42.69/2 0.004s} 89. Kxc8 {-128.00/2 0.001s} Qa7 {+41.98/1 0.008s} 90. Kd8 {-126.92/1 0.030s} Kc2 {+13.28/9 8.6s} 91. Kc8 {-128.00/2 5.7s} Kd3 {+128.00/4 2.2s} 92. Kd8 {-128.00/2 0.001s} Ke4 {+128.00/2 3.7s} 93. Kc8 {-128.00/2 4.5s} Kd5 {+128.00/2 2.0s} 94. Kd8 {-128.00/2 0.002s} Kd6 {+128.00/2 0.058s} 95. Kc8 {-128.00/2 5.2s} Qa8# {+128.00/2 3.8s, Black mates} 0-1 [/pgn]

Bad Gyal 7 is surprising strong, it plays at World Champion strength or better on my Mac , close to 2900 - could be stronger.

[brianr]

FWIW, it is quite a lot stronger than that, I think, at least on the hardware below.

I trained a large 320x24 net and played against Stockfish 7 on one CPU (i5 3570K, 3.4GHz, boost to 3.8)
My net was clearly stronger on an RTX 2060 (I know, not a favorable "Leela Ratio" for SF).

Code: Select all

Score of SF7 vs T60-320x24-74000-Steps: 6 - 21 - 13  [0.313] 40
Elo difference: -136.97 +/- 94.70, LOS: 0.19 %, DrawRatio: 32.5 %

Then, I played my net against Bad Gyal 7:

Code: Select all

Score of Bad-Gyal-7-DKappe vs T60-320x24-74000-Steps: 122 - 94 - 284  [0.528] 500
Elo difference: 19.48 +/- 20.00, LOS: 97.16 %, DrawRatio: 56.8 %

On CCRL at 40/40 SF7 1 CPU is 3,245; again, my hardware is quite different.

Although larger nets should be able to learn more, they also take a lot more training time I think.
Thus the much faster speed of Bad Gyal 7 seems to more than make up for its smaller size.

To be sure, one could match SF7 vs Bad Gyal 7 directly.

Conditions:
6 piece Syzygy tablebases
2 move opening book
Time control 0:10+2.0

[dkappe]

If I want to win a correspondence game, I’ll reach for the strongest T40 net. When I want to prepare for OTB against human opposition, I reach for Bad Gyal. It suggests moves and lines that are not always objectively best, but in practice best against human opponents. Plus they tend to be plans I can understand and follow, rather than the baroque maneuvers of a “zero” net.

This may be one of those cases where the objectively best net is not the best tool for the job.

[Modern Times]

Where do you download Bad Gyal 7 ?

[dkappe]

Where do you download Bad Gyal 7 ?

https://github.com/dkappe/leela-chess-w ... i/Bad-Gyal

[Modern Times]

Thanks, I'll have a play with it.

[MikeB]

FWIW, it is quite a lot stronger than that, I think, at least on the hardware below.

I trained a large 320x24 net and played against Stockfish 7 on one CPU (i5 3570K, 3.4GHz, boost to 3.8)
My net was clearly stronger on an RTX 2060 (I know, not a favorable "Leela Ratio" for SF).

Code: Select all

Score of SF7 vs T60-320x24-74000-Steps: 6 - 21 - 13  [0.313] 40
Elo difference: -136.97 +/- 94.70, LOS: 0.19 %, DrawRatio: 32.5 %

Then, I played my net against Bad Gyal 7:

Code: Select all

Score of Bad-Gyal-7-DKappe vs T60-320x24-74000-Steps: 122 - 94 - 284  [0.528] 500
Elo difference: 19.48 +/- 20.00, LOS: 97.16 %, DrawRatio: 56.8 %

On CCRL at 40/40 SF7 1 CPU is 3,245; again, my hardware is quite different.

Although larger nets should be able to learn more, they also take a lot more training time I think.
Thus the much faster speed of Bad Gyal 7 seems to more than make up for its smaller size.

To be sure, one could match SF7 vs Bad Gyal 7 directly.

Conditions:
6 piece Syzygy tablebases
2 move opening book
Time control 0:10+2.0

IMy hardware is limited, I do use OpenCl and get about 4-5K nps or more with BG, rating could be higher, I didn't test it for rating - but for my setup , it is hands down the best net for me. The bigger nets are really really slow, less than 1K nps on my weak setup. And I love the way Bad Gyal plays, very enetertaiing style of play = far more interesting on a consistent basis. It feels like it thinks "outside the box". Great work by dkappe!