Skipper zero : no progress

[Henk]

Making it simpler did not help much. Nothing more than a bad random mover. Overlooking a mate in one. But a queen has many moves to choose from.

[pgn]
[FEN "8/8/8/8/8/3qk3/8/6K1 w - - 0 1"]
1. Kg1-g2 { 7.410 0.00 } { }
1. Qd3-e4 { 8.179 0.50 } { }
2. Kg2-g1 { 8.511 0.00 } { }
2. Qe4-a4 { 1.001 0.50 } { }
3. Kg1-h1 { 6.799 0.00 } { }
3. Qa4-a7 { 7.691 0.50 } { }
4. Kh1-g1 { 6.099 0.00 } { }
4. Qa7-c5 { 7.086 0.50 } { }
5. Kg1-g2 { 7.241 0.00 } { }
5. Qc5-c4 { 6.842 0.50 } { }
6. Kg2-h1 { 6.976 0.00 } { }
6. Qc4-d5 { 7.889 0.50 } { }
7. Kh1-g1 { 0.109 0.00 } { }
7. Qd5-a8 { 8.581 0.50 } { }
8. Kg1-f1 { 6.164 0.00 } { }
8. Qa8-c8 { 0.164 0.50 } { }
9. Kf1-g1 { 6.235 0.00 } { }
9. Ke3-f4 { 6.944 0.50 } { }
10. Kg1-g2 { 8.827 0.00 } { }
10. Kf4-g5 { 5.778 0.50 } { }
11. Kg2-h2 { 2.756 0.00 } { }
11. Kg5-f4 { 0.187 0.50 } { }
12. Kh2-h1 { 0.259 0.00 } { }
12. Kf4-f5 { 1.694 0.50 } { }
13. Kh1-g2 { 7.927 0.00 } { }
13. Qc8-g8 { 6.703 0.50 } { }
14. Kg2-h1 { 5.218 0.00 } { }
14. Kf5-g4 { 7.779 0.50 } { }
15. Kh1-g2 { 3.075 0.00 } { }
15. Qg8-g7 { 1.575 0.50 } { }
16. Kg2-g1 { 4.310 0.00 } { }
16. Qg7-e7 { 7.572 0.50 } { }
17. Kg1-g2 { 6.400 0.00 } { }
17. Qe7-e3 { 7.531 0.50 } { }
18. Kg2-f1 { 7.571 0.00 } { }
18. Qe3-h3 { 2.779 0.50 } { }
19. Kf1-e2 { 4.802 0.00 } { }
19. Qh3-f3 { 6.732 0.50 } { }
20. Ke2-e1 { 6.377 0.00 } { }
20. Kg4-g3 { 7.445 0.50 } { }
21. Ke1-d2 { 6.188 0.00 } { }
21. Qf3-c6 { 1.288 0.50 } { }
22. Kd2-e2 { 1.228 0.00 } { }
22. Qc6-d5 { 3.537 0.50 } { }
23. Ke2-e1 { 5.093 0.00 } { }
23. Qd5-d7 { 6.785 0.50 } { }
24. Ke1-f1 { 0.803 -0.25 } { }
24. Qd7-d3 { 5.367 0.50 } { }
25. Kf1-e1 { 4.914 0.00 } { }
25. Qd3-c2 { 6.271 0.50 } { }
26. Ke1-f1 { 5.902 0.00 } { }
26. Qc2-d2 { 6.927 0.50 } { }
27. Kf1-g1 { 2.011 0.00 } { }
27. Qd2-d8 { 7.396 0.50 } { }
28. Kg1-f1 { 0.462 0.00 } { }
28. Qd8-c8 { 7.686 0.50 } { }
29. Kf1-e1 { 1.725 0.00 } { }
29. Qc8-c7 { 1.928 0.50 } { }
30. Ke1-d1 { 6.580 0.00 } { }
30. Kg3-h2 { 7.964 0.50 } { }
31. Kd1-d2 { 6.282 0.00 } { }
31. Qc7-d7 { 1.668 0.50 } { }
32. Kd2-c2 { 6.630 0.00 } { }
32. Kh2-g2 { 6.792 0.50 } { }
33. Kc2-b1 { 1.035 0.00 } { }
33. Qd7-d5 { 0.560 0.50 } { }
34. Kb1-b2 { 0.480 0.00 } { }
34. Qd5-f7 { 1.364 0.50 } { }
35. Kb2-c1 { 1.260 0.00 } { }
35. Kg2-f1 { 1.408 0.50 } { }
36. Kc1-d1 { 7.055 0.00 } { }
36. Qf7-f6 { 6.277 0.50 } { }
37. Kd1-c1 { 1.119 0.00 } { }
37. Qf6-g5 { 1.143 0.50 } { }
38. Kc1-d1 { 6.178 0.00 } { }
38. Qg5-d5 { 1.042 0.50 } { }
39. Kd1-c1 { 0.995 0.00 } { }
39. Qd5-c4 { 5.511 0.50 } { }
40. Kc1-b2 { 0.881 0.00 } { }
40. Qc4-c8 { 0.881 0.50 } { }
41. Kb2-a2 { 0.857 0.00 } { }
41. Qc8-g8 { 0.785 0.50 } { }
42. Ka2-b2 { 6.361 0.00 } { }
42. Qg8-g3 { 0.665 0.50 } { }
43. Kb2-c1 { 6.370 0.00 } { }
43. Qg3-f2 { 6.116 0.50 } { }
44. Kc1-b1 { 0.339 0.00 } { }
44. Qf2-g3 { 6.356 0.50 } { }
45. Kb1-c1 { 0.521 0.00 } { }
45. Kf1-g1 { 0.459 0.50 } { }
46. Kc1-d1 { 0.414 0.00 } { }
46. Qg3-c7 { 0.382 0.50 } { }
47. Kd1-e1 { 0.372 0.00 } { }
47. Qc7-f7 { 0.273 0.50 } { }
48. Ke1-d1 { 0.212 0.00 } { }
48. Qf7-e8 { 0.204 0.50 } { }
49. Kd1-c2 { 0.162 0.00 } { }
49. Qe8-g6 { 0.126 0.50 } { }
50. Kc2-c1 { 0.068 0.00 } { }
50. Qg6-e4 { 0.071 0.50 } { }
[/pgn]

[brianr]

It does work, but requires some parameter fiddling.
See my post here:
http://talkchess.com/forum/viewtopic.ph ... w=&start=0

[Daniel Anulliero]

Making it simpler did not help much. Nothing more than a bad random mover. Overlooking a mate in one. But a queen has many moves to choose from.

[pgn]
[FEN "8/8/8/8/8/3qk3/8/6K1 w - - 0 1"]
1. Kg1-g2 { 7.410 0.00 } { }
1. Qd3-e4 { 8.179 0.50 } { }
2. Kg2-g1 { 8.511 0.00 } { }
2. Qe4-a4 { 1.001 0.50 } { }
3. Kg1-h1 { 6.799 0.00 } { }
3. Qa4-a7 { 7.691 0.50 } { }
4. Kh1-g1 { 6.099 0.00 } { }
4. Qa7-c5 { 7.086 0.50 } { }
5. Kg1-g2 { 7.241 0.00 } { }
5. Qc5-c4 { 6.842 0.50 } { }
6. Kg2-h1 { 6.976 0.00 } { }
6. Qc4-d5 { 7.889 0.50 } { }
7. Kh1-g1 { 0.109 0.00 } { }
7. Qd5-a8 { 8.581 0.50 } { }
8. Kg1-f1 { 6.164 0.00 } { }
8. Qa8-c8 { 0.164 0.50 } { }
9. Kf1-g1 { 6.235 0.00 } { }
9. Ke3-f4 { 6.944 0.50 } { }
10. Kg1-g2 { 8.827 0.00 } { }
10. Kf4-g5 { 5.778 0.50 } { }
11. Kg2-h2 { 2.756 0.00 } { }
11. Kg5-f4 { 0.187 0.50 } { }
12. Kh2-h1 { 0.259 0.00 } { }
12. Kf4-f5 { 1.694 0.50 } { }
13. Kh1-g2 { 7.927 0.00 } { }
13. Qc8-g8 { 6.703 0.50 } { }
14. Kg2-h1 { 5.218 0.00 } { }
14. Kf5-g4 { 7.779 0.50 } { }
15. Kh1-g2 { 3.075 0.00 } { }
15. Qg8-g7 { 1.575 0.50 } { }
16. Kg2-g1 { 4.310 0.00 } { }
16. Qg7-e7 { 7.572 0.50 } { }
17. Kg1-g2 { 6.400 0.00 } { }
17. Qe7-e3 { 7.531 0.50 } { }
18. Kg2-f1 { 7.571 0.00 } { }
18. Qe3-h3 { 2.779 0.50 } { }
19. Kf1-e2 { 4.802 0.00 } { }
19. Qh3-f3 { 6.732 0.50 } { }
20. Ke2-e1 { 6.377 0.00 } { }
20. Kg4-g3 { 7.445 0.50 } { }
21. Ke1-d2 { 6.188 0.00 } { }
21. Qf3-c6 { 1.288 0.50 } { }
22. Kd2-e2 { 1.228 0.00 } { }
22. Qc6-d5 { 3.537 0.50 } { }
23. Ke2-e1 { 5.093 0.00 } { }
23. Qd5-d7 { 6.785 0.50 } { }
24. Ke1-f1 { 0.803 -0.25 } { }
24. Qd7-d3 { 5.367 0.50 } { }
25. Kf1-e1 { 4.914 0.00 } { }
25. Qd3-c2 { 6.271 0.50 } { }
26. Ke1-f1 { 5.902 0.00 } { }
26. Qc2-d2 { 6.927 0.50 } { }
27. Kf1-g1 { 2.011 0.00 } { }
27. Qd2-d8 { 7.396 0.50 } { }
28. Kg1-f1 { 0.462 0.00 } { }
28. Qd8-c8 { 7.686 0.50 } { }
29. Kf1-e1 { 1.725 0.00 } { }
29. Qc8-c7 { 1.928 0.50 } { }
30. Ke1-d1 { 6.580 0.00 } { }
30. Kg3-h2 { 7.964 0.50 } { }
31. Kd1-d2 { 6.282 0.00 } { }
31. Qc7-d7 { 1.668 0.50 } { }
32. Kd2-c2 { 6.630 0.00 } { }
32. Kh2-g2 { 6.792 0.50 } { }
33. Kc2-b1 { 1.035 0.00 } { }
33. Qd7-d5 { 0.560 0.50 } { }
34. Kb1-b2 { 0.480 0.00 } { }
34. Qd5-f7 { 1.364 0.50 } { }
35. Kb2-c1 { 1.260 0.00 } { }
35. Kg2-f1 { 1.408 0.50 } { }
36. Kc1-d1 { 7.055 0.00 } { }
36. Qf7-f6 { 6.277 0.50 } { }
37. Kd1-c1 { 1.119 0.00 } { }
37. Qf6-g5 { 1.143 0.50 } { }
38. Kc1-d1 { 6.178 0.00 } { }
38. Qg5-d5 { 1.042 0.50 } { }
39. Kd1-c1 { 0.995 0.00 } { }
39. Qd5-c4 { 5.511 0.50 } { }
40. Kc1-b2 { 0.881 0.00 } { }
40. Qc4-c8 { 0.881 0.50 } { }
41. Kb2-a2 { 0.857 0.00 } { }
41. Qc8-g8 { 0.785 0.50 } { }
42. Ka2-b2 { 6.361 0.00 } { }
42. Qg8-g3 { 0.665 0.50 } { }
43. Kb2-c1 { 6.370 0.00 } { }
43. Qg3-f2 { 6.116 0.50 } { }
44. Kc1-b1 { 0.339 0.00 } { }
44. Qf2-g3 { 6.356 0.50 } { }
45. Kb1-c1 { 0.521 0.00 } { }
45. Kf1-g1 { 0.459 0.50 } { }
46. Kc1-d1 { 0.414 0.00 } { }
46. Qg3-c7 { 0.382 0.50 } { }
47. Kd1-e1 { 0.372 0.00 } { }
47. Qc7-f7 { 0.273 0.50 } { }
48. Ke1-d1 { 0.212 0.00 } { }
48. Qf7-e8 { 0.204 0.50 } { }
49. Kd1-c2 { 0.162 0.00 } { }
49. Qe8-g6 { 0.126 0.50 } { }
50. Kc2-c1 { 0.068 0.00 } { }
50. Qg6-e4 { 0.071 0.50 } { }
[/pgn]

Why do you waste your time with NN and KQvsK ?
My old alpha beta crap solve it in few seconds lol
Better trying with more complex endgames I guess

[Henk]

i don't know. Just try something different. first have to debug it. Or better study theorie about reinforcement learning. If it even can not solve this easy end game ... Problem is that it evaluaties a leaf node with a neural network instead of a roll out. But neural network only gives back a random value when not trained enough.

[Henk]

Maybe bit better now. Few less bugs.

[pgn]
[FEN "8/8/8/8/8/3qk3/8/6K1 w - - 0 1" ]
1. Kg1-g2 { 6.207 -0.18 } { }
1. Qd3-h7 { 17.676 0.97 } { }
2. Kg2-g3 { 14.307 0.00 } { }
2. Qh7-b1 { 13.613 0.59 } { }
3. Kg3-h2 { 8.184 -0.18 } { }
3. Ke3-f3 { 11.843 0.98 } { }
4. Kh2-h3 { 0.236 -0.49 } { }
4. Qb1-h1 { 0.143 0.99 } { }
[/pgn]

[trulses]

i don't know. Just try something different. first have to debug it. Or better study theorie about reinforcement learning. If it even can not solve this easy end game ... Problem is that it evaluaties a leaf node with a neural network instead of a roll out. But neural network only gives back a random value when not trained enough.

Hey Henk, are you training the network using the loss from the AG0 paper?

[Henk]

No just simpel old school mean square error. Network needs to be re-implemented. Only been busy with debugging monte carlo tree search.


For instance it even can't finish:

[pgn]
[FEN "8/8/3qk3/8/8/8/8/6K1 w - - 0 1"]
1. Kg1-f1 { 10.170 -0.01 } { }
1. Qd6-d7 { 12.149 0.50 } { }
2. Kf1-e1 { 26.094 0.00 } { }
2. Qd7-c7 { 25.599 0.50 } { }
3. Ke1-d1 { 23.942 -0.01 } { }
3. Qc7-e7 { 24.810 0.50 } { }
4. Kd1-e1 { 25.334 0.00 } { }
4. Qe7-e8 { 22.749 0.50 } { }
5. Ke1-d1 { 23.832 0.00 } { }
5. Ke6-e7 { 23.031 0.50 } { }
6. Kd1-e1 { 24.699 0.00 } { }
6. Qe8-c8 { 22.017 0.50 } { }
7. Ke1-d1 { 24.402 0.00 } { }
7. Qc8-e8 { 22.925 0.50 } { }
8. Kd1-e1 { 22.641 0.00 } { }
8. Qe8-c8 { 21.908 0.50 } { }
9. Ke1-d1 { 21.299 0.00 } { }
9. Qc8-e8 { 20.934 0.50 } { }
10. Kd1-e1 { 21.855 0.00 } { }
10. Qe8-c8 { 21.087 0.50 } { }
11. Ke1-d1 { 20.994 0.00 } { }
11. Qc8-e8 { 20.657 0.50 } { }
12. Kd1-e2 { 19.913 0.00 } { }
12. Qe8-c8 { 19.365 0.50 } { }
13. Ke2-e1 { 20.285 0.00 } { }
13. Ke7-d7 { 18.986 0.50 } { }
14. Ke1-d1 { 20.569 0.00 } { }
14. Qc8-d8 { 18.971 0.50 } { }
15. Kd1-e1 { 19.423 0.00 } { }
15. Qd8-c8 { 17.890 0.50 } { }
16. Ke1-d1 { 18.086 0.00 } { }
16. Qc8-f8 { 17.886 0.50 } { }
17. Kd1-e1 { 18.014 0.00 } { }
17. Qf8-e7 { 16.730 0.50 } { }
18. Ke1-d1 { 17.927 -0.01 } { }
18. Qe7-f6 { 15.223 0.50 } { }
19. Kd1-e1 { 16.568 0.00 } { }
19. Qf6-d8 { 15.850 0.50 } { }
20. Ke1-e2 { 16.201 0.00 } { }
20. Qd8-c8 { 15.548 0.50 } { }
21. Ke2-e1 { 15.625 0.00 } { }
21. Kd7-e8 { 14.824 0.50 } { }
22. Ke1-d1 { 15.637 0.00 } { }
22. Qc8-b7 { 14.605 0.50 } { }
23. Kd1-e1 { 14.627 0.00 } { }
23. Qb7-e7 { 13.644 0.50 } { }
24. Ke1-d2 { 13.938 0.00 } { }
24. Qe7-f8 { 13.291 0.50 } { }
25. Kd2-e1 { 14.416 0.00 } { }
25. Qf8-a3 { 12.745 0.50 } { }
26. Ke1-e2 { 13.016 0.00 } { }
26. Qa3-a8 { 12.748 0.50 } { }
27. Ke2-e1 { 13.009 0.00 } { }
27. Qa8-a2 { 11.642 0.50 } { }
28. Ke1-d1 { 11.802 0.00 } { }
28. Qa2-a7 { 11.781 0.50 } { }
29. Kd1-e1 { 11.498 0.00 } { }
29. Qa7-a2 { 11.821 0.50 } { }
30. Ke1-d1 { 11.346 0.00 } { }
30. Qa2-a7 { 10.988 0.50 } { }
31. Kd1-e1 { 10.697 0.00 } { }
31. Qa7-a2 { 9.758 0.50 } { }
32. Ke1-d1 { 10.049 0.00 } { }
32. Qa2-a7 { 9.420 0.50 } { }
33. Kd1-e1 { 9.252 0.00 } { }
33. Qa7-a2 { 10.371 0.50 } { }
34. Ke1-d1 { 8.894 0.00 } { }
34. Qa2-a7 { 8.281 0.50 } { }
35. Kd1-e1 { 8.610 0.00 } { }
35. Qa7-a2 { 8.651 0.50 } { }
36. Ke1-d1 { 7.921 0.00 } { }
36. Qa2-a7 { 7.042 0.50 } { }
37. Kd1-e1 { 7.364 0.00 } { }
37. Qa7-a2 { 7.380 0.50 } { }
38. Ke1-d1 { 6.798 0.00 } { }
38. Qa2-g8 { 6.333 0.50 } { }
39. Kd1-e1 { 6.317 0.00 } { }
39. Qg8-a2 { 6.473 0.50 } { }
40. Ke1-d1 { 5.730 0.00 } { }
40. Qa2-g8 { 6.261 0.50 } { }
41. Kd1-e1 { 5.440 0.00 } { }
41. Qg8-a2 { 6.456 0.50 } { }
42. Ke1-d1 { 4.778 0.00 } { }
42. Qa2-a7 { 4.366 0.50 } { }
43. Kd1-e1 { 4.580 0.00 } { }
43. Qa7-a2 { 4.083 0.50 } { }
44. Ke1-d1 { 3.623 0.00 } { }
44. Qa2-a7 { 3.320 0.50 } { }
45. Kd1-e1 { 3.380 0.00 } { }
45. Qa7-e3 { 6.175 0.50 } { }
46. Ke1-d1 { 2.758 0.00 } { }
46. Qe3-a7 { 2.274 0.50 } { }
47. Kd1-e1 { 1.896 0.00 } { }
47. Qa7-e7 { 1.901 0.50 } { }
48. Ke1-d1 { 1.650 0.00 } { }
48. Qe7-b7 { 1.431 0.50 } { }
49. Kd1-e1 { 1.054 0.00 } { }
49. Qb7-e7 { 0.854 0.50 } { }
50. Ke1-d1 { 0.575 0.00 } { }
50. Qe7-a7 { 0.303 0.50 } { }
[/pgn]

[trulses]

No just simpel old school mean square error. Network needs to be re-implemented. Only been busy with debugging monte carlo tree search.

How many simulations are you running? What move selection policy are you using in the root node after the search?

If I read the pgn correctly then the second value in each comment is estimated action value, is that correct? It seems to be varying quite a lot from one search to the next.

[Henk]

No just simpel old school mean square error. Network needs to be re-implemented. Only been busy with debugging monte carlo tree search.

How many simulations are you running? What move selection policy are you using in the root node after the search?

If I read the pgn correctly then the second value in each comment is estimated action value, is that correct? It seems to be varying quite a lot from one search to the next.

For move selection I use UCT (UCB1) with C = Sqrt(2).

Tree is extended 2 * (ln(0.01)/ln(0.98)) times now.
0.98 = 1/50 because number of legal moves would be about 50.

I have no clue how many times one should extend the tree. At least all moves in the root should be tried.

So at this moment it does not use the neural network to select the move. Only uses network for giving a leaf node a value. But untrained networks generate values not better than random.


In the pgn value of position is 0.5 when it's black's turn and 0 when it's white's turn. Have to find out why that is so. Maybe because winning chance of white is zero and it thinks that winning chance of black is 0.5. 0.5 looks strange to me.

[Henk]

No just simpel old school mean square error. Network needs to be re-implemented. Only been busy with debugging monte carlo tree search.

How many simulations are you running? What move selection policy are you using in the root node after the search?

If I read the pgn correctly then the second value in each comment is estimated action value, is that correct? It seems to be varying quite a lot from one search to the next.

For move selection I use UCT (UCB1) with C = Sqrt(2).

Tree is extended 2 * (ln(0.01)/ln(0.98)) times now.
0.98 = 1/50 because number of legal moves would be about 50.

I have no clue how many times one should extend the tree. At least all moves in the root should be tried.

So at this moment it does not use the neural network to select the move. Only uses network for giving a leaf node a value. But untrained networks generate values not better than random.


In the pgn value of position is 0.5 when it's black's turn and 0 when it's white's turn. Have to find out why that is so. Maybe because winning chance of white is zero and it thinks that winning chance of black is 0.5. 0.5 looks strange to me.

No can't be. For value sometimes less than 0. Something wrong there. Have to find out.

Wait perhaps 0.5 means that it thinks position is a draw.