More experiments with neural nets

[jonkr]

I've been doing more experiments with neural nets, and thought I'd share my results.

Training neural nets can be done much quicker than in my initial endgame net tests. One big help is my code is now all optimized and all the bugs are (mostly) worked out. There are really a lot of things that can and do go wrong when starting everything from scratch and building out the process.

I also made an "auto-learner" that automates all the steps, kicks off a match with configurable params, creates more training data, updates a net, shows the results, then loops. So I can train while I'm gone and update nets after smaller steps.

Training a Rook + Pawns Endgame net from scratch to -1 elo to SF11 (final test was 5000 games) took me 20 hours. Training it from scratch to stronger the Slow 2.3 was only 5 hours. This is still weaker than my latest rook endgame net which scored +5 elo to SF11 (10000 games @ 0.8 + 0.06) so I didn't continue the test.

I am pretty convinced at this point that neural nets are enough stronger that there isn't much point from a strength perspective to try to do hard-coded eval stuff. I wouldn't say I doubted this before, but I had the impression that they required more compute resources to get strong than they actually do.

General Evaluation

I tried replacing my regular eval with a neural network (in addition to the individual endgame networks.) First tests were training it from scratch with zero training data to see how long it took to learn. I was initially pretty excited because I had assumed that chess was so big it might be hard to see it work at all, but instead I saw quick progress against the older chess engines I tested against (Fruit then Senpai2.0)

10s + .1 2-moves book, first try from scratch, output pasted from my auto-tuner window. I think this was around 40 hours maybe? (using 11-cores)

Code: Select all

1. SlowDev v. Fruit2.2.1  	( 0.0 elo ) 	+0  	-112  	=0  	0.0%
2. SlowDev v. Fruit2.2.1  	( -1111.0 elo ) 	+0  	-299  	=1  	0.2%
3. SlowDev v. Fruit2.2.1  	( -1111.0 elo ) 	+0  	-299  	=1  	0.2%
4. SlowDev v. Fruit2.2.1  	( -869.3 elo ) 	+2  	-298  	=0  	0.7%
5. SlowDev v. Fruit2.2.1  	( -636.4 elo ) 	+6  	-291  	=3  	2.5%
6. SlowDev v. Fruit2.2.1  	( -395.0 elo ) 	+22  	-266  	=12  	9.3%
7. SlowDev v. Fruit2.2.1  	( -242.6 elo ) 	+46  	-227  	=27  	19.8%
8. SlowDev v. Fruit2.2.1  	( -140.4 elo ) 	+73  	-188  	=39  	30.8%
9. SlowDev v. Fruit2.2.1  	( -70.4 elo ) 	+99  	-159  	=42  	40.0%
10. SlowDev v. Fruit2.2.1  	( -74.1 elo ) 	+95  	-158  	=47  	39.5%
11. SlowDev v. Fruit2.2.1  	( -59.6 elo ) 	+101  	-152  	=47  	41.5%
12. SlowDev v. Fruit2.2.1  	( -20.9 elo ) 	+117  	-135  	=48  	47.0%
13. SlowDev v. Fruit2.2.1  	( 4.6 elo ) 	+125  	-121  	=54  	50.7%
14. SlowDev v. Fruit2.2.1  	( -2.3 elo ) 	+120  	-122  	=58  	49.7%
15. SlowDev v. Fruit2.2.1  	( -30.2 elo ) 	+114  	-140  	=46  	45.7%
16. SlowDev v. Fruit2.2.1  	( 19.7 elo ) 	+133  	-116  	=51  	52.8%
17. SlowDev v. Fruit2.2.1  	( 23.2 elo ) 	+135  	-115  	=50  	53.3%
18. SlowDev v. Fruit2.2.1  	( -24.4 elo ) 	+148  	-176  	=76  	46.5%
19. SlowDev v. Fruit2.2.1  	( 61.4 elo ) 	+193  	-123  	=84  	58.7%
20. SlowDev v. Fruit2.2.1  	( -7.8 elo ) 	+159  	-168  	=73  	48.9%
21. SlowDev v. Fruit2.2.1  	( 52.5 elo ) 	+199  	-139  	=62  	57.5%
22. SlowDev v. Fruit2.2.1  	( -11.3 elo ) 	+168  	-181  	=51  	48.4%
23. SlowDev v. Fruit2.2.1  	( 47.2 elo ) 	+192  	-138  	=70  	56.8%
24. SlowDev v. Fruit2.2.1  	( -261.3 elo ) 	+16  	-79  	=4  	18.2%
25. SlowDev v. Fruit2.2.1  	( 12.2 elo ) 	+179  	-165  	=56  	51.8%
26. SlowDev v. Fruit2.2.1  	( 45.4 elo ) 	+188  	-136  	=76  	56.5%
27. SlowDev v. Fruit2.2.1  	( 60.5 elo ) 	+196  	-127  	=77  	58.6%
28. SlowDev v. Fruit2.2.1  	( 41.9 elo ) 	+182  	-134  	=84  	56.0%
29. SlowDev v. Fruit2.2.1  	( 24.4 elo ) 	+183  	-155  	=62  	53.5%
30. SlowDev v. Fruit2.2.1  	( 54.3 elo ) 	+189  	-127  	=84  	57.8%
31. SlowDev v. Senpai2  	( -236.5 elo ) 	+56  	-352  	=92  	20.4%
32. SlowDev v. Senpai2  	( -230.2 elo ) 	+58  	-348  	=94  	21.0%
33. SlowDev v. Senpai2  	( -196.5 elo ) 	+70  	-326  	=104  	24.4%
34. SlowDev v. Senpai2  	( -179.0 elo ) 	+66  	-303  	=131  	26.3%
35. SlowDev v. Senpai2  	( -190.8 elo ) 	+73  	-323  	=104  	25.0%
36. SlowDev v. Senpai2  	( -198.3 elo ) 	+57  	-315  	=128  	24.2%
37. SlowDev v. Senpai2  	( -146.4 elo ) 	+78  	-277  	=145  	30.1%
38. SlowDev v. Senpai2  	( -177.2 elo ) 	+69  	-304  	=127  	26.5%
39. SlowDev v. Senpai2  	( -251.9 elo ) 	+49  	-359  	=92  	19.0%
40. SlowDev v. Senpai2  	( -119.1 elo ) 	+95  	-260  	=145  	33.5%
41. SlowDev v. Senpai2  	( -135.8 elo ) 	+83  	-269  	=148  	31.4%
42. SlowDev v. Senpai2  	( -161.5 elo ) 	+71  	-288  	=141  	28.3%
43. SlowDev v. Senpai2  	( -137.4 elo ) 	+87  	-275  	=138  	31.2%
44. SlowDev v. Senpai2  	( -127.8 elo ) 	+85  	-261  	=154  	32.4%
45. SlowDev v. Senpai2  	( -129.3 elo ) 	+88  	-266  	=146  	32.2%
46. SlowDev v. Senpai2  	( -129.3 elo ) 	+78  	-256  	=166  	32.2%
47. SlowDev v. Senpai2  	( -123.0 elo ) 	+88  	-258  	=154  	33.0%
48. SlowDev v. Senpai2  	( -142.3 elo ) 	+73  	-267  	=160  	30.6%
49. SlowDev v. Senpai2  	( -101.5 elo ) 	+99  	-241  	=160  	35.8%
50. SlowDev v. Senpai2  	( -108.3 elo ) 	+95  	-246  	=159  	34.9%
51. SlowDev v. Senpai2  	( -113.7 elo ) 	+90  	-248  	=162  	34.2%
52. SlowDev v. Senpai2  	( -84.3 elo ) 	+116  	-235  	=149  	38.1%
53. SlowDev v. Senpai2  	( -100.7 elo ) 	+105  	-246  	=149  	35.9%
54. SlowDev v. Senpai2  	( -77.7 elo ) 	+113  	-223  	=164  	39.0%
55. SlowDev v. Senpai2  	( -85.0 elo ) 	+104  	-224  	=172  	38.0%
56. SlowDev v. Senpai2  	( -64.7 elo ) 	+126  	-218  	=156  	40.8%
57. SlowDev v. Senpai2  	( -64.7 elo ) 	+120  	-212  	=168  	40.8%
58. SlowDev v. Senpai2  	( -56.8 elo ) 	+130  	-211  	=159  	41.9%
59. SlowDev v. Senpai2  	( -113.9 elo ) 	+46  	-115  	=57  	34.2%
60. SlowDev v. Senpai2  	( -81.4 elo ) 	+112  	-227  	=161  	38.5%
61. SlowDev v. Fizbo2  	( -201.2 elo ) 	+62  	-323  	=115  	23.9%
62. SlowDev v. Fizbo2  	( -206.0 elo ) 	+61  	-327  	=112  	23.4%
63. SlowDev v. Fizbo2  	( -176.3 elo ) 	+72  	-306  	=122  	26.6%
64. SlowDev v. Fizbo2  	( -171.9 elo ) 	+75  	-304  	=121  	27.1%
65. SlowDev v. Fizbo2  	( -154.7 elo ) 	+66  	-275  	=159  	29.1%
66. SlowDev v. Fizbo2  	( -164.1 elo ) 	+80  	-300  	=120  	28.0%
67. SlowDev v. Fizbo2  	( -165.8 elo ) 	+71  	-293  	=136  	27.8%
68. SlowDev v. Fizbo2  	( -158.1 elo ) 	+74  	-287  	=139  	28.7%
69. SlowDev v. Fizbo2  	( -151.3 elo ) 	+77  	-282  	=141  	29.5%
70. SlowDev v. Fizbo2  	( -175.7 elo ) 	+23  	-114  	=58  	26.7%
71. SlowDev v. Senpai2  	( -70.4 elo ) 	+30  	-55  	=40  	40.0%
72. SlowDev v. Senpai2  	( -48.3 elo ) 	+138  	-207  	=155  	43.1%
73. SlowDev v. Senpai2  	( -11.8 elo ) 	+159  	-176  	=165  	48.3%
74. SlowDev v. Senpai2  	( -29.3 elo ) 	+152  	-194  	=154  	45.8%
75. SlowDev v. Critter_1.6a  	( -143.9 elo ) 	+84  	-280  	=136  	30.4%
76. SlowDev v. Critter_1.6a  	( -138.2 elo ) 	+92  	-281  	=127  	31.1%
77. SlowDev v. Critter_1.6a  	( -144.7 elo ) 	+85  	-282  	=133  	30.3%
78. SlowDev v. Critter_1.6a  	( -127.8 elo ) 	+102  	-278  	=120  	32.4%
79. SlowDev v. Critter_1.6a  	( -122.2 elo ) 	+97  	-266  	=137  	33.1%
80. SlowDev v. Critter_1.6a  	( -115.2 elo ) 	+112  	-272  	=116  	34.0%
81. SlowDev v. Critter_1.6a  	( -136.3 elo ) 	+48  	-138  	=55  	31.3%
82. SlowDev v. Senpai2  	( -11.8 elo ) 	+160  	-177  	=163  	48.3%
83. SlowDev v. Senpai2  	( -34.2 elo ) 	+141  	-190  	=169  	45.1%
84. SlowDev v. Senpai2  	( -9.0 elo ) 	+143  	-156  	=201  	48.7%
85. SlowDev v. Senpai2  	( -14.6 elo ) 	+160  	-181  	=159  	47.9%
86. SlowDev v. Senpai2  	( -12.5 elo ) 	+158  	-176  	=166  	48.2%
87. SlowDev v. Senpai2  	( -39.8 elo ) 	+133  	-190  	=177  	44.3%
88. SlowDev v. Senpai2  	( -44.6 elo ) 	+50  	-74  	=64  	43.6%
89. SlowDev v. Senpai2  	( 40.5 elo ) 	+193  	-135  	=172  	55.8%

Score of SlowDev vs Senpai2: 431 - 324 - 445 [0.545]   15s +.15  2-moves book
Elo difference: 31.06 +/- 15.61

Second from scratch try 4s + .02 (19 hours training time using 11-cores). Also used some stronger opponents.

Code: Select all

0. SlowDev v. Fruit2.2.1  	( 0.0 elo ) 	+0  	-155  	=0  	0.0%
1. SlowDev v. Fruit2.2.1  	( -837.4 elo ) 	+3  	-495  	=2  	0.8%
2. SlowDev v. Fruit2.2.1  	( -816.7 elo ) 	+4  	-495  	=1  	0.9%
3. SlowDev v. Fruit2.2.1  	( -471.9 elo ) 	+26  	-464  	=10  	6.2%
4. SlowDev v. Fruit2.2.1  	( -331.8 elo ) 	+52  	-423  	=25  	12.9%
5. SlowDev v. Fruit2.2.1  	( -313.9 elo ) 	+58  	-417  	=25  	14.1%
6. SlowDev v. Fruit2.2.1  	( -39.1 elo ) 	+182  	-238  	=80  	44.4%
7. SlowDev v. Fruit2.2.1  	( -69.0 elo ) 	+169  	-267  	=64  	40.2%
8. SlowDev v. Fruit2.2.1  	( -12.5 elo ) 	+198  	-216  	=86  	48.2%
9. SlowDev v. Fruit2.2.1  	( -44.0 elo ) 	+179  	-242  	=79  	43.7%
10. SlowDev v. Fruit2.2.1  	( -27.9 elo ) 	+200  	-240  	=60  	46.0%
11. SlowDev v. Fruit2.2.1  	( -7.6 elo ) 	+198  	-209  	=93  	48.9%
12. SlowDev v. Fruit2.2.1  	( 48.3 elo ) 	+238  	-169  	=93  	56.9%
Changed to FRC
13. SlowDev v. Fruit2.2.1  	( -94.0 elo ) 	+199  	-368  	=73  	36.8%
14. SlowDev v. Fruit2.2.1  	( -39.8 elo ) 	+190  	-247  	=63  	44.3%
15. SlowDev v. Fruit2.2.1  	( -52.5 elo ) 	+181  	-256  	=63  	42.5%
16. SlowDev v. Fruit2.2.1  	( -33.5 elo ) 	+199  	-247  	=54  	45.2%
17. SlowDev v. Fruit2.2.1  	( -41.2 elo ) 	+194  	-253  	=53  	44.1%
18. SlowDev v. Fruit2.2.1  	( -46.8 elo ) 	+187  	-254  	=59  	43.3%
19. SlowDev v. Fruit2.2.1  	( -14.6 elo ) 	+201  	-222  	=77  	47.9%
20. SlowDev v. Fruit2.2.1  	( -18.5 elo ) 	+87  	-98  	=22  	47.3%
21. SlowDev v. Stockfish8  	( -887.7 elo ) 	+1  	-495  	=4  	0.6%
22. SlowDev v. Stockfish8  	( -919.5 elo ) 	+1  	-496  	=3  	0.5%
23. SlowDev v. Stockfish8  	( -752.1 elo ) 	+1  	-488  	=11  	1.3%
24. SlowDev v. Stockfish8  	( -603.9 elo ) 	+7  	-477  	=16  	3.0%
25. SlowDev v. Fruit2.2.1  	( 20.9 elo ) 	+236  	-206  	=58  	53.0%
26. SlowDev v. Stockfish8  	( -659.2 elo ) 	+5  	-483  	=12  	2.2%
27. SlowDev v. Stockfish8  	( -667.4 elo ) 	+2  	-481  	=17  	2.1%
28. SlowDev v. Stockfish8  	( -629.4 elo ) 	+4  	-478  	=18  	2.6%
29. SlowDev v. Stockfish8  	( -661.9 elo ) 	+2  	-576  	=22  	2.2%
30. SlowDev v. Stockfish8  	( -636.4 elo ) 	+3  	-573  	=24  	2.5%
31. SlowDev v. Stockfish8  	( -541.1 elo ) 	+9  	-558  	=33  	4.3%
32. SlowDev v. Stockfish8  	( -594.2 elo ) 	+5  	-567  	=28  	3.2%
33. SlowDev v. Stockfish8  	( -552.1 elo ) 	+4  	-556  	=40  	4.0%
34. SlowDev v. Stockfish8  	( -563.7 elo ) 	+7  	-562  	=31  	3.8%
35. SlowDev v. Stockfish8  	( -571.9 elo ) 	+7  	-564  	=29  	3.6%
36. SlowDev v. Stockfish8  	( -537.6 elo ) 	+9  	-557  	=34  	4.3%
37. SlowDev v. Stockfish8  	( -530.7 elo ) 	+9  	-555  	=36  	4.5%
38. SlowDev v. Stockfish8  	( -555.9 elo ) 	+9  	-562  	=29  	3.9%
39. SlowDev v. Stockfish8  	( -567.8 elo ) 	+8  	-564  	=28  	3.7%
40. SlowDev v. Stockfish8  	( -571.9 elo ) 	+2  	-559  	=39  	3.6%
41. SlowDev v. Stockfish8  	( -517.7 elo ) 	+8  	-550  	=42  	4.8%
42. SlowDev v. Stockfish8  	( -563.7 elo ) 	+4  	-559  	=37  	3.8%
43. SlowDev v. Stockfish8  	( -505.5 elo ) 	+10  	-548  	=42  	5.2%
44. SlowDev v. Stockfish8  	( -530.7 elo ) 	+9  	-555  	=36  	4.5%
45. SlowDev v. Stockfish8  	( -511.5 elo ) 	+9  	-549  	=42  	5.0%
46. SlowDev v. Stockfish8  	( -548.4 elo ) 	+8  	-559  	=33  	4.1%
47. SlowDev v. Stockfish8  	( -544.7 elo ) 	+7  	-557  	=36  	4.2%
48. SlowDev v. Stockfish8  	( -520.9 elo ) 	+4  	-547  	=49  	4.8%
49. SlowDev v. Stockfish8  	( -555.9 elo ) 	+7  	-560  	=33  	3.9%
50. SlowDev v. Stockfish8  	( -524.1 elo ) 	+5  	-549  	=46  	4.7%
51. SlowDev v. Stockfish8  	( -499.7 elo ) 	+6  	-542  	=52  	5.3%
52. SlowDev v. Stockfish8  	( -555.9 elo ) 	+7  	-560  	=33  	3.9%
53. SlowDev v. Stockfish8  	( -527.4 elo ) 	+6  	-551  	=43  	4.6%
54. SlowDev v. Stockfish8  	( -494.0 elo ) 	+9  	-543  	=48  	5.5%
55. SlowDev v. Stockfish8  	( -488.5 elo ) 	+10  	-542  	=48  	5.7%
56. SlowDev v. Stockfish8  	( -478.0 elo ) 	+18  	-546  	=36  	6.0%
57. SlowDev v. Stockfish8  	( -514.6 elo ) 	+9  	-550  	=41  	4.9%
58. SlowDev v. Stockfish8  	( -508.5 elo ) 	+8  	-547  	=45  	5.1%
59. SlowDev v. Stockfish8  	( -527.4 elo ) 	+6  	-551  	=43  	4.6%
60. SlowDev v. Stockfish8  	( -541.1 elo ) 	+7  	-556  	=37  	4.3%
61. SlowDev v. Stockfish8  	( -499.7 elo ) 	+3  	-539  	=58  	5.3%
62. SlowDev v. Stockfish8  	( -456.1 elo ) 	+10  	-529  	=61  	6.8%
63. SlowDev v. Stockfish8  	( -480.6 elo ) 	+9  	-538  	=53  	5.9%
64. SlowDev v. Stockfish8  	( -508.5 elo ) 	+7  	-546  	=47  	5.1%
65. SlowDev v. Stockfish8  	( -491.3 elo ) 	+5  	-538  	=57  	5.6%
66. SlowDev v. Stockfish8  	( -465.6 elo ) 	+10  	-533  	=57  	6.4%
67. SlowDev v. Stockfish8  	( -488.5 elo ) 	+8  	-540  	=52  	5.7%
68. SlowDev v. Stockfish8  	( -534.1 elo ) 	+4  	-551  	=45  	4.4%
69. SlowDev v. Stockfish8  	( -463.2 elo ) 	+9  	-531  	=60  	6.5%
70. SlowDev v. Stockfish8  	( -520.9 elo ) 	+6  	-549  	=45  	4.8%
71. SlowDev v. Stockfish8  	( -451.6 elo ) 	+11  	-528  	=61  	6.9%
72. SlowDev v. Stockfish8  	( -480.6 elo ) 	+5  	-534  	=61  	5.9%
73. SlowDev v. Fruit2.2.1  	( 201.8 elo ) 	+211  	-54  	=35  	76.2%
74. SlowDev v. Stockfish8  	( -442.8 elo ) 	+12  	-525  	=63  	7.3%
75. SlowDev v. Stockfish8  	( -451.6 elo ) 	+13  	-530  	=57  	6.9%
76. SlowDev v. Stockfish8  	( -434.4 elo ) 	+13  	-522  	=65  	7.6%
77. SlowDev v. Stockfish8  	( -491.3 elo ) 	+6  	-539  	=55  	5.6%
78. SlowDev v. Stockfish8  	( -438.5 elo ) 	+14  	-525  	=61  	7.4%
79. SlowDev v. Stockfish8  	( -440.6 elo ) 	+11  	-523  	=66  	7.3%
80. SlowDev v. Stockfish8  	( -432.3 elo ) 	+13  	-521  	=66  	7.7%
81. SlowDev v. Stockfish8  	( -428.3 elo ) 	+11  	-517  	=72  	7.8%
82. SlowDev v. Stockfish8  	( -447.1 elo ) 	+11  	-526  	=63  	7.1%
83. SlowDev v. Stockfish8  	( -428.3 elo ) 	+14  	-520  	=66  	7.8%
84. SlowDev v. Stockfish8  	( -451.6 elo ) 	+6  	-523  	=71  	6.9%
85. SlowDev v. Stockfish8  	( -458.5 elo ) 	+8  	-528  	=64  	6.7%
86. SlowDev v. Stockfish8  	( -449.4 elo ) 	+9  	-525  	=66  	7.0%
87. SlowDev v. Stockfish8  	( -460.8 elo ) 	+12  	-533  	=55  	6.6%
88. SlowDev v. Stockfish8  	( -456.1 elo ) 	+16  	-535  	=49  	6.8%
89. SlowDev v. Stockfish8  	( -460.8 elo ) 	+10  	-531  	=59  	6.6%
90. SlowDev v. Stockfish8  	( -409.1 elo ) 	+12  	-508  	=80  	8.7%
91. SlowDev v. Stockfish8  	( -456.1 elo ) 	+9  	-528  	=63  	6.8%
92. SlowDev v. Stockfish8  	( -451.6 elo ) 	+10  	-527  	=63  	6.9%
93. SlowDev v. Stockfish8  	( -424.3 elo ) 	+16  	-520  	=64  	8.0%
94. SlowDev v. Stockfish8  	( -451.6 elo ) 	+8  	-525  	=67  	6.9%
95. SlowDev v. Stockfish8  	( -438.5 elo ) 	+10  	-521  	=69  	7.4%
96. SlowDev v. Stockfish8  	( -418.5 elo ) 	+12  	-513  	=75  	8.3%
97. SlowDev v. Stockfish8  	( -434.4 elo ) 	+16  	-525  	=59  	7.6%
98. SlowDev v. Stockfish8  	( -407.3 elo ) 	+15  	-510  	=75  	8.8%
99. SlowDev v. Stockfish8  	( -451.6 elo ) 	+10  	-527  	=63  	6.9%
100. SlowDev v. Stockfish8  	( -393.3 elo ) 	+19  	-506  	=75  	9.4%
101. SlowDev v. Stockfish8  	( -386.6 elo ) 	+20  	-503  	=77  	9.8%
102. SlowDev v. Fruit2.2.1  	( 249.1 elo ) 	+304  	-58  	=38  	80.8%
103. SlowDev v. Senpai2  	( -102.5 elo ) 	+153  	-325  	=122  	35.7%
104. SlowDev v. Stockfish8  	( -407.3 elo ) 	+19  	-514  	=67  	8.8%
105. SlowDev v. Stockfish8  	( -436.4 elo ) 	+17  	-527  	=56  	7.5%
106. SlowDev v. Stockfish8  	( -383.3 elo ) 	+16  	-497  	=87  	9.9%
107. SlowDev v. Stockfish8  	( -398.4 elo ) 	+21  	-511  	=68  	9.2%
108. SlowDev v. Stockfish8  	( -438.5 elo ) 	+11  	-522  	=67  	7.4%
109. SlowDev v. Stockfish8  	( -358.8 elo ) 	+28  	-493  	=79  	11.3%
110. SlowDev v. Stockfish8  	( -400.2 elo ) 	+13  	-504  	=83  	9.1%
111. SlowDev v. Stockfish8  	( -412.8 elo ) 	+17  	-515  	=68  	8.5%
112. SlowDev v. Stockfish8  	( -369.2 elo ) 	+16  	-488  	=96  	10.7%
113. SlowDev v. Stockfish8  	( -380.1 elo ) 	+18  	-497  	=85  	10.1%
114. SlowDev v. Stockfish8  	( -354.5 elo ) 	+21  	-483  	=96  	11.5%
115. SlowDev v. Stockfish8  	( -321.4 elo ) 	+30  	-467  	=103  	13.6%
116. SlowDev v. Stockfish8  	( -393.3 elo ) 	+18  	-505  	=77  	9.4%
117. SlowDev v. Stockfish8  	( -407.3 elo ) 	+14  	-509  	=77  	8.8%
118. SlowDev v. Stockfish8  	( -383.3 elo ) 	+14  	-495  	=91  	9.9%
119. SlowDev v. Stockfish8  	( -351.7 elo ) 	+24  	-484  	=92  	11.7%
120. SlowDev v. Stockfish8  	( -363.2 elo ) 	+21  	-489  	=90  	11.0%
121. SlowDev v. Senpai2  	( -71.6 elo ) 	+166  	-288  	=146  	39.8%
122. SlowDev v. Stockfish8  	( -322.7 elo ) 	+15  	-453  	=132  	13.5%
123. SlowDev v. Stockfish8  	( -338.0 elo ) 	+16  	-466  	=118  	12.5%
124. SlowDev v. Stockfish8  	( -347.5 elo ) 	+28  	-485  	=87  	11.9%
125. SlowDev v. Stockfish8  	( -627.3 elo ) 	+1  	-127  	=5  	2.6%
126. SlowDev v. Stockfish8  	( -471.5 elo ) 	+22  	-783  	=64  	6.2%
127. SlowDev v. Stockfish8  	( -511.5 elo ) 	+1  	-28  	=1  	5.0%
128. SlowDev v. Stockfish8  	( -350.9 elo ) 	+12  	-267  	=54  	11.7%
-- Score all positions by 50% Search, 50% Result
129. SlowDev v. Stockfish8  	( -254.3 elo ) 	+92  	-770  	=224  	18.8%
130. SlowDev v. Senpai2  	( 47.2 elo ) 	+175  	-121  	=104  	56.8%
131. SlowDev v. Stockfish8  	( -252.8 elo ) 	+55  	-428  	=117  	18.9%
132. SlowDev v. Stockfish8  	( -249.1 elo ) 	+51  	-420  	=129  	19.3%
133. SlowDev v. Stockfish8  	( -248.2 elo ) 	+58  	-426  	=116  	19.3%
134. SlowDev v. Stockfish8  	( -241.7 elo ) 	+50  	-411  	=139  	19.9%
135. SlowDev v. Stockfish8  	( -262.4 elo ) 	+38  	-421  	=141  	18.1%
136. SlowDev v. Stockfish8  	( -249.1 elo ) 	+49  	-418  	=133  	19.3%
137. SlowDev v. Stockfish8  	( -274.4 elo ) 	+48  	-443  	=109  	17.1%
138. SlowDev v. Stockfish11  	( -384.9 elo ) 	+22  	-504  	=74  	9.8%
139. SlowDev v. Stockfish11  	( -342.0 elo ) 	+32  	-485  	=83  	12.3%
140. SlowDev v. Stockfish11  	( -351.7 elo ) 	+31  	-491  	=78  	11.7%
141. SlowDev v. Stockfish11  	( -355.9 elo ) 	+27  	-490  	=83  	11.4%
142. SlowDev v. Stockfish11  	( -388.2 elo ) 	+17  	-501  	=82  	9.7%
143. SlowDev v. Stockfish8  	( -238.1 elo ) 	+47  	-404  	=149  	20.3%
144. SlowDev v. Stockfish8  	( -235.4 elo ) 	+51  	-405  	=144  	20.5%
145. SlowDev v. Stockfish8  	( -289.1 elo ) 	+32  	-441  	=127  	15.9%
146. SlowDev v. Stockfish8  	( -253.8 elo ) 	+55  	-429  	=116  	18.8%
147. SlowDev v. Fruit2.2.1  	( 499.7 elo ) 	+554  	-18  	=28  	94.7%
148. SlowDev v. Senpai2  	( 68.6 elo ) 	+292  	-175  	=133  	59.8%

Ran it longer to see how rate of improvement held up. Was able to beat Slow 2.1 after about 24 hours more.

Code: Select all

0. SlowDev v. Stockfish8  	( -214.8 elo ) 	+69  	-399  	=132  	22.5%
1. SlowDev v. Stockfish8  	( -236.3 elo ) 	+47  	-402  	=151  	20.4%
2. SlowDev v. Stockfish8  	( -228.4 elo ) 	+51  	-397  	=152  	21.2%
3. SlowDev v. Stockfish7  	( -155.5 elo ) 	+95  	-347  	=158  	29.0%
4. SlowDev v. Stockfish7  	( -144.4 elo ) 	+99  	-335  	=166  	30.3%
5. SlowDev v. Stockfish7  	( -143.8 elo ) 	+102  	-337  	=161  	30.4%
6. SlowDev v. Stockfish8  	( -206.7 elo ) 	+68  	-388  	=144  	23.3%
7. SlowDev v. Stockfish8  	( -241.7 elo ) 	+56  	-417  	=127  	19.9%
8. SlowDev v. Stockfish8  	( -214.0 elo ) 	+62  	-391  	=147  	22.6%
9. SlowDev v. Stockfish8  	( -239.9 elo ) 	+51  	-410  	=139  	20.1%
10. SlowDev v. Stockfish8  	( -238.1 elo ) 	+51  	-408  	=141  	20.3%
11. SlowDev v. Stockfish8  	( -232.8 elo ) 	+57  	-408  	=135  	20.8%
12. SlowDev v. Stockfish8  	( -254.7 elo ) 	+48  	-423  	=129  	18.8%
13. SlowDev v. Stockfish8  	( -262.3 elo ) 	+15  	-216  	=84  	18.1%
14. SlowDev v. Stockfish8  	( -206.7 elo ) 	+68  	-388  	=144  	23.3%
15. SlowDev v. Stockfish8  	( -227.6 elo ) 	+50  	-395  	=155  	21.3%
16. SlowDev v. Stockfish12  	( -511.5 elo ) 	+9  	-549  	=42  	5.0%
17. SlowDev v. Stockfish12  	( -470.4 elo ) 	+15  	-540  	=45  	6.3%
18. SlowDev v. Stockfish12  	( -463.2 elo ) 	+14  	-536  	=50  	6.5%
19. SlowDev v. Stockfish12  	( -453.9 elo ) 	+14  	-532  	=54  	6.8%
20. SlowDev v. Stockfish12  	( -465.6 elo ) 	+11  	-534  	=55  	6.4%
21. SlowDev v. Stockfish11  	( -346.1 elo ) 	+27  	-483  	=90  	12.0%
22. SlowDev v. Stockfish7  	( -112.7 elo ) 	+125  	-313  	=162  	34.3%
23. SlowDev v. Stockfish8  	( -193.2 elo ) 	+76  	-379  	=145  	24.8%
24. SlowDev v. Ethereal12  	( -121.1 elo ) 	+131  	-332  	=137  	33.3%
25. SlowDev v. Ethereal12  	( -145.1 elo ) 	+114  	-351  	=135  	30.3%
26. SlowDev v. Ethereal12  	( -94.3 elo ) 	+142  	-301  	=157  	36.8%
27. SlowDev v. Ethereal12  	( -106.3 elo ) 	+142  	-320  	=138  	35.2%
28. SlowDev v. Senpai2  	( 95.6 elo ) 	+306  	-145  	=149  	63.4%
29. SlowDev v. Fruit2.2.1  	( 589.5 elo ) 	+191  	-4  	=5  	96.8%
30. SlowDev v. Stockfish8  	( -186.7 elo ) 	+117  	-608  	=275  	25.5%
31. SlowDev v. Stockfish8  	( -207.0 elo ) 	+102  	-636  	=262  	23.3%
32. SlowDev v. Stockfish8  	( -206.5 elo ) 	+113  	-646  	=241  	23.4%
33. SlowDev v. Stockfish8  	( -211.4 elo ) 	+104  	-647  	=249  	22.8%
34. SlowDev v. Stockfish8  	( -174.6 elo ) 	+124  	-588  	=288  	26.8%
35. SlowDev v. Stockfish8  	( -184.9 elo ) 	+138  	-625  	=237  	25.7%
36. SlowDev v. Stockfish8  	( -188.1 elo ) 	+119  	-613  	=268  	25.3%
37. SlowDev v. Stockfish8  	( -179.5 elo ) 	+146  	-621  	=233  	26.2%
38. SlowDev v. Stockfish8  	( -169.7 elo ) 	+128  	-581  	=291  	27.4%
39. SlowDev v. Stockfish8  	( -197.4 elo ) 	+108  	-622  	=270  	24.3%
40. SlowDev v. Stockfish8  	( -176.8 elo ) 	+134  	-603  	=263  	26.6%
41. SlowDev v. Stockfish8  	( -180.8 elo ) 	+124  	-602  	=274  	26.1%
42. SlowDev v. Stockfish8  	( -188.1 elo ) 	+120  	-614  	=266  	25.3%
43. SlowDev v. Stockfish8  	( -188.1 elo ) 	+130  	-624  	=246  	25.3%
44. SlowDev v. Stockfish8  	( -175.4 elo ) 	+121  	-587  	=292  	26.7%
45. SlowDev v. Stockfish8  	( -184.0 elo ) 	+128  	-613  	=259  	25.8%
46. SlowDev v. Stockfish8  	( -177.7 elo ) 	+133  	-604  	=263  	26.4%
47. SlowDev v. Stockfish8  	( -179.9 elo ) 	+124  	-600  	=276  	26.2%
48. SlowDev v. Stockfish8  	( -157.2 elo ) 	+149  	-573  	=278  	28.8%
49. SlowDev v. Stockfish8  	( -173.7 elo ) 	+126  	-588  	=286  	26.9%
50. SlowDev v. Stockfish8  	( -159.8 elo ) 	+149  	-579  	=272  	28.5%
51. SlowDev v. Stockfish8  	( -178.1 elo ) 	+125  	-597  	=278  	26.4%
52. SlowDev v. Stockfish8  	( -167.1 elo ) 	+132  	-579  	=289  	27.6%
53. SlowDev v. Stockfish8  	( -165.8 elo ) 	+152  	-596  	=252  	27.8%
54. SlowDev v. Stockfish8  	( -161.1 elo ) 	+133  	-566  	=301  	28.3%
55. SlowDev v. Stockfish8  	( -180.8 elo ) 	+125  	-603  	=272  	26.1%
56. SlowDev v. Stockfish8  	( -172.3 elo ) 	+135  	-594  	=271  	27.1%
57. SlowDev v. Stockfish8  	( -192.2 elo ) 	+112  	-615  	=273  	24.9%
58. SlowDev v. Stockfish8  	( -170.1 elo ) 	+125  	-579  	=296  	27.3%
59. SlowDev v. Stockfish8  	( -184.4 elo ) 	+120  	-606  	=274  	25.7%
60. SlowDev v. Stockfish8  	( -177.2 elo ) 	+132  	-602  	=266  	26.5%
61. SlowDev v. Stockfish8  	( -168.4 elo ) 	+129  	-579  	=292  	27.5%
62. SlowDev v. Stockfish8  	( -159.8 elo ) 	+140  	-570  	=290  	28.5%
63. SlowDev v. Stockfish8  	( -159.8 elo ) 	+147  	-577  	=276  	28.5%
64. SlowDev v. Stockfish8  	( -170.6 elo ) 	+144  	-599  	=257  	27.3%
65. SlowDev v. Stockfish8  	( -149.7 elo ) 	+145  	-551  	=304  	29.7%
66. SlowDev v. Stockfish8  	( -161.1 elo ) 	+138  	-571  	=291  	28.3%
67. SlowDev v. Stockfish8  	( -161.1 elo ) 	+141  	-574  	=285  	28.3%
68. SlowDev v. Stockfish8  	( -177.7 elo ) 	+115  	-586  	=299  	26.4%
69. SlowDev v. Stockfish8  	( -158.5 elo ) 	+140  	-567  	=293  	28.7%
70. SlowDev v. Stockfish8  	( -167.1 elo ) 	+133  	-580  	=287  	27.6%
71. SlowDev v. Stockfish8  	( -158.5 elo ) 	+136  	-563  	=301  	28.7%
72. SlowDev v. Stockfish11  	( -265.8 elo ) 	+82  	-726  	=192  	17.8%
73. SlowDev v. Stockfish11  	( -254.2 elo ) 	+82  	-706  	=212  	18.8%
74. SlowDev v. Stockfish11  	( -281.1 elo ) 	+62  	-731  	=207  	16.5%
75. SlowDev v. Stockfish11  	( -265.8 elo ) 	+68  	-712  	=220  	17.8%
76. SlowDev v. Stockfish11  	( -268.2 elo ) 	+77  	-725  	=198  	17.6%
77. SlowDev v. Stockfish11  	( -245.2 elo ) 	+76  	-684  	=240  	19.6%
78. SlowDev v. Stockfish11  	( -251.9 elo ) 	+76  	-696  	=228  	19.0%
79. SlowDev v. Stockfish11  	( -286.1 elo ) 	+65  	-742  	=193  	16.2%
80. SlowDev v. Stockfish11  	( -257.0 elo ) 	+85  	-714  	=201  	18.5%
81. SlowDev v. Stockfish11  	( -263.4 elo ) 	+76  	-716  	=208  	18.0%
82. SlowDev v. Stockfish11  	( -292.0 elo ) 	+65  	-751  	=184  	15.7%
83. SlowDev v. Stockfish11  	( -267.6 elo ) 	+68  	-715  	=217  	17.6%
84. SlowDev v. Stockfish11  	( -273.0 elo ) 	+78  	-734  	=188  	17.2%
85. SlowDev v. Stockfish11  	( -277.9 elo ) 	+61  	-725  	=214  	16.8%
86. SlowDev v. Stockfish11  	( -267.0 elo ) 	+79  	-725  	=196  	17.7%
87. SlowDev v. Stockfish11  	( -275.5 elo ) 	+66  	-726  	=208  	17.0%
88. SlowDev v. Stockfish11  	( -264.0 elo ) 	+79  	-720  	=201  	18.0%
89. SlowDev v. Stockfish11  	( -278.5 elo ) 	+58  	-723  	=219  	16.8%
90. SlowDev v. Stockfish11  	( -267.0 elo ) 	+67  	-713  	=220  	17.7%
91. SlowDev v. Stockfish11  	( -269.4 elo ) 	+81  	-731  	=188  	17.5%
92. SlowDev v. Stockfish11  	( -260.5 elo ) 	+74  	-709  	=217  	18.3%
93. SlowDev v. Stockfish11  	( -260.5 elo ) 	+79  	-714  	=207  	18.3%
94. SlowDev v. Stockfish11  	( -259.3 elo ) 	+78  	-711  	=211  	18.4%
95. SlowDev v. Stockfish11  	( -251.9 elo ) 	+87  	-707  	=206  	19.0%
96. SlowDev v. Stockfish11  	( -263.4 elo ) 	+86  	-726  	=188  	18.0%
97. SlowDev v. Stockfish11  	( -251.9 elo ) 	+79  	-699  	=222  	19.0%
98. SlowDev v. Stockfish11  	( -266.4 elo ) 	+76  	-721  	=203  	17.8%
99. SlowDev v. Stockfish11  	( -264.0 elo ) 	+76  	-717  	=207  	18.0%
100. SlowDev v. Stockfish11  	( -263.4 elo ) 	+72  	-712  	=216  	18.0%
101. SlowDev v. Stockfish8  	( -135.4 elo ) 	+166  	-537  	=297  	31.5%
102. SlowDev v. Stockfish7  	( -45.8 elo ) 	+287  	-418  	=295  	43.5%
103. SlowDev v. Fruit2.2.1  	( 685.2 elo ) 	+973  	-11  	=16  	98.1%
104. SlowDev v. Senpai2  	( 159.8 elo ) 	+595  	-165  	=240  	71.5%
105. SlowDev v  Slow2.1		( 18.3 elo)     +216	-181    =268    52.5%

Important factors in making it quicker

• I kept all endgame knowledge. Tablebases, Mating Eval, and the full endgame neural nets.
  The endgame nets trained individually from zero and didn't want to extend time to retrain. Since result is used in scoring having accurate endgame results should be huge help in knowing what is actually good earlier.
• Score by search was huge in increasing training speed. The first time I tried it I saw a 100 elo increase compared to the result only score, using the same from-zero data against Fruit and Senpai2.
  I used 60% search, 40% result. Search scoring was using the net being trained, so was still starting from zero. It seems to help generalize / even out the values and can be used to re-score earlier games as the evaluation gets better (much quicker than throwing them out and playing new games.)
• I used horizontal and SideToMove/Vertical symmetry.
• Knowledgeable opponents to train against. If one side knows what to do the results shouldn't ever be that bad. It was able to beat Fruit 2.2.1 quite quickly but then the rate of elo increase slowed down. After switching to Senpai2 training got elo increase got faster again.
• The time control can be super quick. I started with 3s + .02s games. It's possible that longer games are better or become needed at a certain level. But in the beginning it speeds up the process quite a bit, since playing games is by far the longest step.
• The search is already pretty strong/optimized, so against older programs it probably has a big search advantage.

I expect with enough training data most of this is no longer that important, also probably a lot is only relevant to quickly starting the learning, not improving to the best possible network. Like score by search I couldn't measure if it actually was better than just by result when the net was already strong with more data.
However for training on one reasonably priced home computer on just my own data, speed of training is quite important.

Training Randomness

• Sometimes you randomly roll a clearly weaker or stronger net (weights start randomly when training connections, and depending on your process and when/how you create training data there might be more random factors.)
  But this is a larger issue the less data and larger game-space the net covers. When I had very little data there was sometimes a clear 40+ elo difference between eval nets. I even once randomly got a dud net that was -100 elo the previous.
  I started auto-saving numbered nets for each training so as to not lose a good one. But as training data increased variation became smaller and it was harder to pick out what was real and what was just testing noise.
• I also experimented with splitting training and validation data in tensor flow. This does somewhat follow the weaker/stronger nets, but also hard to say for sure the correlation with the amount of testing I can do, and variation became smaller the more data I had, so in the end I didn't incorporate this into my training process.
• Error seemed to reduce some small amount as more weights were added to net structure, but where they were added didn't matter much. I had some thought to force connections with more specific chess inputs and structures, but now I'm leaning away from this. I still have some things I want to try. It's hard to predict strength from error anyway.

Boot-strapping from Zero with Self-Play

If you are not specifically trying to keep it pure, I think it's worthwhile just taking a half an hour to do a simple eval (material + sqs and passed pawns or whatever) to jump start training. Also adding mating evaluation and tablebases. (Or training endgame first.)
Then when one net beats the current base by a certain margin, copy that net and use it as the new base opponent and keep playing. I think Alpha-zero did something like this.
I did enough testing of this process to see that works to get stronger quicker than I initially expected, but I haven't tried getting it strong enough to compete with my current eval. When it was still hundreds of elo weaker and progress started to slow down some, I decided I was more interested in if the experiment would work than trying to find a maximum level.

Actually Improving the Eval

Next I spent some time working to gather and score a large numbers of positions (Using my old HCE training data, and lot of old pgns from testing sitting around.)
Today I am able to get some match wins vs Stockfish 8 (+5 elo @ 4000 games 10s+.1 in 2-moves book.)
Unfortunately this didn't generalize even to 8-moves book (-10 elo) and FRC was pretty bad (abandoned after starting out -60 elo.)
A lot of my old pgns were from testing with 2-moves book, so training had the most positions starting from there.
Training total was 9.8 million positions not including symmetries. They are scored with a blend of search and result. For the training sets I chose between 0% and 50% result weighted depending on the expected quality of the game. (eg. I ran some 1 + .01s games to generate more positions, so these were 0% result, just search.) My training code does multi-threaded search scoring so time needed not that bad, but still a few hours to do all 10 million positions. I unscientifically was trying depth 8 and depth 10 on sets while importing.
I tried both using Slow hard-coded eval, or having the eval net on. Leaving the eval net on seemed better once the eval got decent, but might be some issues where overfitted or missing evals not corrected.

This is the net structure I'm using for the general net :

Code: Select all

	network.SetInputCount(whiteInputCount + blackInputCount + 1);
	network.AddLayer(320, eActivation::RELU, eLayout::SPARSE_INPUTS);
	network.AddLayer(32, eActivation::RELU);
	network.AddLayer(32, eActivation::RELU);
	network.AddLayer(1);

I found Stockfish 11 and Stockfish 8 performance wasn't good at *extremely* short time controls like 2s + .02s. For testing I mainly did 10s + .1s instead where they are as strong as expected. I'm not sure exactly why, I didn't see obviously broken time usage looking at the pgn. Stockfish 12 is still monstrously strong at any time control.

I copied the incremental update code from my general code in my checkers program and pasted it into chess for a very worthwhile, but not huge, speed increase.

Neural net pawn hash : A few layers using only pawns + kings leading to a few output values that are then combined into the general network.
This is pretty much just a speed improvement, but small/moderate speed improvements are a large part of chess. Initial tests showed this is likely an improvement. If I switch to it for general network then will try keeping the same pawn/king network for the endgame networks too. Also maybe try putting general inputs in hash too.

Separate Side updates : I haven't tried this yet. A quiet move would only have to update one side. I think maybe NNUE does this if I remember the diagram? (I have seen Wiki entry and talk chess posts but haven't looked at code to see details of how NNUE works.)

Othello

I trained a neural net othello program based on the GuiNN checkers code that used 4 Edge and 4 corner pattern neural nets that fed into two final layers. I trained by starting out playing random games and with perfect endgame with 16 moves left, then working back in stages. This has been a common way to train othello programs from zero knowledge for a couple decades so not really anything new.
The strength was pretty similar, except for the endgame solving was clearly weaker which means probably a bit weaker overall. I didn't build out a way to automatically play against my old program so hard to say exactly how it compares. I'd consider this result a partial success.
It's harder to be efficient in othello neural nets (more and more markers on board until 64 squares filled, and lines of them flipping each move.) And the GuiNN checkers search is okay but not the best. So I wasn't really expecting to quickly get neural net super strength, just make sure the process did generalize and learn about different net structures
I haven't released the neural net othello, but in the process I put my super old othello program Pointy Stone 3.5 on GitHub, and updated it with a faster 64-bit compile.

Future

I learned a lot about neural nets, and got comfortable with Tensorflow, and experimented with image recognition and other common neural net tasks in addition to the game playing tasks.

Probably I will soon release a Slow 2.5 that is purely neural net, since it seems already clearly stronger (except FRC?) though unlikely to actually be Stockfish 8 level, and for best performance will now need AVX2.
But as commonly happens there is less motivation after some excitement of learning something new and it actually working. My initial curiosity feels satisfied and making actual progress seems fairly slow and a function of time (mostly computer-time, but also deciding what to tweak or train.)

[jstanback]

Hi Jon,

SlowChess is my favorite engine and this is the most interesting post I've seen recently. I'm surprised that there have been no replies. What are the 320 inputs that you use? I would think that 6x64 = 384 inputs (or maybe 352 if only 32 squares are used for king location) would be a minimum. It's amazing to me that a 320 x 32 x 32 x 1 net can play so well!

About 6 months ago I experimented with creating tiny NN's and training them and made some decent progress using just Piece/Sq as inputs but after some amount of training using Wasp's HCE as a target plus some additional training using game result, the strength leveled off. For some reason (probably bugs in my implementation) I didn't see any benefit in going beyond 4 nodes for the two hidden layers. I then realized that I could use the NN back-propogation technique to greatly simplify and speed up tuning for my HCE terms, so I abandoned my NN experiments and re-wrote and re-tuned Wasp's HCE and gained about 70 Elo over Wasp 4.0. Now, based on your post and watching SlowChess clobber Wasp (especially in the endgame) I think I might start working on NN's again.

John

[jonkr]

For the general neural net, the 320 is the first hidden layer, so with input layer it's 705x320x32x32x1.
(The endgame nets are Inputs Count x 256 x 32 x 32 x 1, where input count varies depending on how many different piece types can be in the net.)
So the total inputs are 705, here's my general neural net init code without the top part clipped out :

Code: Select all

	whiteInputCount = 32 + 48 + 64 + 64 + 64 + 64; // white uses horizontal symmetry 32 squares for king
	blackInputCount = 64 + 48 + 64 + 64 + 64 + 64; 
	BuildInputMap(); // fill array to map [piece][sq] to input idx
	
	network.SetInputCount(whiteInputCount + blackInputCount + 1); // +1 for STM
	network.AddLayer(320, eActivation::RELU, eLayout::SPARSE_INPUTS);
	network.AddLayer(32, eActivation::RELU);
	network.AddLayer(32, eActivation::RELU);
	network.AddLayer(1);
	network.Build();
	

It is still surprising to me how strong it can play with just pure board inputs, and no overparameterization or anything, the size on disk of the net is 474Kb, so much smaller than standard NNUE. Also for the endgame nets how many positions they just recognize as draws or likely wins that would be very tedious to program by hand.

More Info on where I was at when I took break from Chess programming :

One thing I started trying, which seemed likely to be an improvement, was having a couple net layers with king+pawns input only in a pawn/king hash table. Also was thinking of adding castle flags to this layer, and maybe general material inputs not accounting for squares (which only would have to be updated on captures or promotions.) The idea is mainly having some weights in a cheap layer where I can experiment more with size and not hit speed much.

This isn't in released version because (beyond wanting to take a break from chess) I minorly messed up my training data somehow when I was experimenting, and all my general nets were testing like 15 to 25 elo worse than the best one I had saved. So even though my initial test of this structure showed some elo gain, it wasn't better than my best net without it. To find the general net in 2.5 I ran 3000 game test matches for a day with my auto-trainer (gen net, test, update data, repeat), and then ran more matches with the 2 that had positive score against Stockfish 8 to choose one.

Also I did try combining all my endgame nets into just one general endgame net and lost like 30 elo (in endgame starting positions test) first try, so while maybe I went overboard with the specific endgames which would matter less, and with more experimenting the general endgame test net could get better, I didn't want to test this further.

I am pretty curious how much trying do things like the standard NNUE architecture and trainer would help elo-wise but still undecided if I will try it out or not, since I do like the feeling of just testing out stuff on my own, and always have list of experiments to try.

[jstanback]

Thanks for the clarification! It was hard to fathom that SlowChess could be so strong with only 320 inputs and two 32 node layers. I guess that your NN topology with 705 inputs and 320 nodes in the first hidden layer is sufficient to roughly equal the best HCE. That's really cool! I know very little about NN's and nothing about libraries to efficiently implement and train them. So my fiddling is all just done with simple C code and no SSE or AVX stuff and will be very slow using your network topology. Maybe I should try KRPKR first and see what I get

John

[Rein Halbersma]

Interesting read! Do you use any random exploration during the training games? (like AlphaZero did in the opening).

Oh, and did I read that correctly that you have extended cutechess-cli to also work with 8x8 checkers and Othello? I once tried that, IIRC, there was some documentation on how you can subclass a Player class (don't recall the exact name) and implement game specific stuff there.

[Joost Buijs]

I've been working on a network with a similar structure as Jonathan is using. Currently it has 736 inputs, the layout is: 736:512x32x32x1. The results that I get with it are significantly better as with my HCE, maybe for a large part because my HCE is 10 years old and not up to modern standards.

For training I use C++ with libTorch. I've been experimenting with PyTorch, the data-loader from PyTorch is so slow that it is unusable. Writing a data-loader in Python is not done either, I've never seen a language as slow as PyThon, It makes me wonder why most data-scientists insist on using this.

Developing a LibTorch trainer is not very easy because the documentation is very cryptic and there are hardly any examples to find on the internet. After fighting for days with several bugs I finally managed to get it working. I still don know how to create the input tensor directly on the GPU, although in my view this should be possible. I still have to create it in main memory and copy it to the GPU afterwards which makes each batch run roughly 3 times as slow as it could be according to experiments I did. I don't use sparse inputs yet which makes it slow too. I still have to find out how to do this with libTorch.

At the moment I use about 400M positions for training, each epoch takes ~8 minutes, usually the error is already quite low after 30 epochs, after 85 epochs the error doesn't change much anymore. Training the network from scratch takes 4 to 11 hours on my RTX2060 Super.

[Rein Halbersma]

For training I use C++ with libTorch. I've been experimenting with PyTorch, the data-loader from PyTorch is so slow that it is unusable. Writing a data-loader in Python is not done either, I've never seen a language as slow as PyThon, It makes me wonder why most data-scientists insist on using this.

Because most data scientist know when and how to call into NumPy / SciPy / Pandas and other heavily optimized libraries for performance sensitive code. You can get near-native performance when reading in multi-Gb files from disk using NumPy. The Python code is just the top-level wrappers around the training loop. There is hardly any cost to training NNs in Python. Generating training games is another matter of course.

[Joost Buijs]

For training I use C++ with libTorch. I've been experimenting with PyTorch, the data-loader from PyTorch is so slow that it is unusable. Writing a data-loader in Python is not done either, I've never seen a language as slow as PyThon, It makes me wonder why most data-scientists insist on using this.

Because most data scientist know when and how to call into NumPy / SciPy / Pandas and other heavily optimized libraries for performance sensitive code. You can get near-native performance when reading in multi-Gb files from disk using NumPy. The Python code is just the top-level wrappers around the training loop. There is hardly any cost to training NNs in Python. Generating training games is another matter of course.

I agree that it won't make much difference if you only call library functions that are written in C and C++ anyway, but for simple things like e.g. parsing PGN's or converting chess positions to tensors it is way to slow. For C++ there are also very heavily optimized scientific libraries, and if you really want to use a scripting language Julia might be a better alternative.

[derjack]

Nice. I am experimenting with neural networks as well. I made engine for the game of breakthrough. Small net, but beats my handcrafted eval 99% of time

My NN inputs have some knowledge, not only raw representation. For example there are 64 squares, but each square is not only empty, white pawn, black pawn, but also wether it is attacked by either side or both, so each square can have 12 states. That makes 768 inputs in which exactly 64 are ones. Also instead of using separate input for a side to move, I use de facto different neural network, so the total inputs are 1536. To make it fast for now, there are only 32 hidden units in 1 layer. Also, like nnue, it's partially updateable, because at most 6 squares are affected each move.

I trained using the scaled handracted eval + d plies search. Then again with NN-learned eval. It yielded very good results. Then I experimented and trained from scratch. I made semi-random games and recorded only endgames and trained on those endgames. Then in next iterations the rounds from the endgames increased and so on. I was in shock when it was making the openings similar presented to the breakthrough game paper, all this learned on its own!

Right now I'm learning those python frameworks and libraries, so I can get muchly optimized learning pipeline. So far it's dificult because input is sparse and I haven't found some good tutorial on my case.

I think this type of NN would work for checkers as well, if you put for input not only the checker type per square, but some of its characteristic as well.

[jonkr]

I guess that your NN topology with 705 inputs and 320 nodes in the first hidden layer is sufficient to roughly equal the best HCE. That's really cool!

I do wonder how it actually compares, definitely performs better than my best HCE. Slow 2.5 still scores like -160 elo to Stockfish 11 in my 15s+.15s match test, not sure how much elo Stockfish's super strong search gains, my net eval must be still somewhat weaker, but I'm also sure it's possible to get at least somewhat better training of the net. In late endgames the endgame net evals must be stronger than SF11 since even with weaker Slow search they can equal or outperform, but that's a lot smaller space of game then the whole mid/endgame.

Interesting read! Do you use any random exploration during the training games? (like AlphaZero did in the opening).

Oh, and did I read that correctly that you have extended cutechess-cli to also work with 8x8 checkers and Othello? I once tried that, IIRC, there was some documentation on how you can subclass a Player class (don't recall the exact name) and implement game specific stuff there.

Unfortunately what happened was that while starting to create a general game playing/learning system I found that I really missed CuteChess-cli with all its options (and the general chess ecosystem with so many engines, and I think biggest community) and went back to testing in chess I was cobbling together some self-play stuff for Othello (and planning to for checkers) and thinking of writing a bare bones match manager program, but kept thinking of more things I would want in it to make it on par with what I use from cutechess.

No forced randomization/exploration in the programs own moves. I did vary openings (8-moves book, 2-moves book, some special test positions, all w/ random order.) My auto trainer had dropdown for options to choose to send cutechess for openings, timecontrol, opponents, and could add multiple options from a list to cycle through. That coupled with the auto-retraining of the neural networks after every n-games, was enough to get decent variation in games.

Joost :

For my tensorflow script I use the numPy loading of a binary file with 1-byte per on-off input, it was surprisingly quick even for 10+ gigabyte files. I export the binary training data from my dev slow.exe which includes my auto-trainer code. My first try with tensorflow I loaded and converted text but that was like 100 times slower.
I have about 10M positions for my general net, I think it takes like 5 minutes for me to train which is nice to play around with. I'm sure in general more is better, my position library was slowly growing as I ran my tests/training.

DerJack :

Sounds like excellent progress, NN these days are definitely surprisingly powerful, I still had some impression of them as uncompetitive from over a decade ago until first Alpha Zero then Stockfish NNUE made it super obvious how strong they were.

For 8x8 checkers I got the impression that it was well suited for pure board inputs even more-so than chess, with fewer pieces/sqs/ no sliding moves and some important patterns based on board position (most obvious being back row, and trapped kings etc.) Other structures would be interesting though, and as you mentioned some could capture the features in a way that is speedier to update.

I hope to eventually go back to checkers mainly to try out a pure and automated from-zero training approach and try to beat my previous GuiNN 2.04 with that.