Nalimov EGTB problem related to DTM?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Nalimov EGTB problem related to DTM?

Post by Laskos »

Daniel Mehrmann wrote:
Laskos wrote:I knew that Nalimovs have this problem, but first, I imagined it to be a problem with very rare, freak positions, second, I didn't know that Syzygy gives "perfect play" against "perfect play". I was under some sort of artistic impression that if Nalimov shows me the shortest distance to mate in a given root position, even ignoring the 50-move rule it should perform at least as well as Syzygy on wins from the root. Then several days ago I played (color and reversed) from my hard TB won positions, using Nalimov, Syzygy and Komodo no TB, with these results:

Code: Select all

Rank Name                          ELO   Games   Score   Draws
   1 Syzygy                         69    1032     60%     20%
   2 Nalimov                        43    1032     56%     21%
   3 Komodo No TB                 -115    1032     34%     32%
Finished match
Well, basicly i wouldn't say that Syzygy performs better then Nalimov. A combination of both can be good too. I tested it with Fruit and it looks not so bad.

Probably they are equal with distance zero/mate, but Syzygy performs better, because the files are smaller (timing i/o balance).

Well, if the root position is already inside a database, "Nalimov" engines doesn't do a move order based on the mate scores at root. At least i can say, i don't know such a engine yet. Maybe that could be a reason in some cases if the depth or time is extremely limited. However, i need to do more tests in that case.

Regards
Daniel
The sole advantage of Nalimovs is that they are showing the correct DTM in PV. Might be useful for analysis, otherwise Syzygy bases are to be preferred.
User avatar
velmarin
Posts: 1600
Joined: Mon Feb 21, 2011 9:48 am

Re: Nalimov EGTB problem related to DTM?

Post by velmarin »

Laskos, you are absolutely right.
Nalimov behavior with this table (two bishops and a Knight) is rather disappointing.
I made a small test with Rybka with Nalimov and Stockfish with Syzygy at the Aquarium GUI.
The tablebasses loaded individually on each engine, no award all games until the end.
Nalimov won only one.

Code: Select all

1   stockfish dev             +168   ½1½1½1½1½1½1½1½1½101  14.5/20
2   Deep Rybka 4.1___        -168    ½0½0½0½0½0½0½0½0½010    5.5/20
[pgn]
[Event "[03] laskos"]
[Site "Computadora local"]
[Date "2016.02.16"]
[Round "1"]
[White "Deep Rybka 4.1 SSE42 x64"]
[Black "stockfish dev"]
[Result "1/2-1/2"]
[BlackElo "-"]
[Classes "0"]
[ECO "?"]
[TimeControl "60"]
[WhiteElo "-"]
[SetUp "1"]
[FEN "2Bn4/8/2k5/8/8/1K6/8/4B3 w - - 0 1"]

{Processor: AMD Phenom(tm) II X4 B50 Processor (@3,21GHz)^13 ^10 Blancas: Deep
Rybka 4.1 SSE42 x64, hash size: 512M, opening book: NarrowBook^13 ^10 Negras:
stockfish dev, hash size: 512M, opening book: NarrowBook^13 ^10 } {[%t Long]
Empate. (1/2-1/2)} {Blancas de biblioteca} 1. Kc4 {[%clk 0:00:59][%clko
0:01:00]}{[%eval 29940,59][%meval TB:1k]}{(Kc7)}{Negras de biblioteca} 1...
Nb7 {[%clk 0:00:58][%clko 0:00:59][%emt 0:00:01]}{[%eval 12353,23][%meval
1s|TB:6]} 2. Be6 {[%clk 0:00:59][%clko 0:00:58]}{[%eval 29941,59][%meval
TB:1k]} 2... Nd6+ {[%clk 0:00:58][%clko 0:00:59]}{[%eval 12353,1][%meval TB:7]}
3. Kd4 {[%clk 0:00:58][%clko 0:00:58]}{[%eval 29942,59][%meval TB:304]}{(Nb5)}
3... Nb7 {[%clk 0:00:57][%clko 0:00:58]}{[%eval 12353,1][%meval TB:12]} 4.
Bd5+ {[%clk 0:00:58][%clko 0:00:57]}{[%eval 29943,59][%meval TB:1k]}{(Kb6)}
4... Kc7 {[%clk 0:00:57][%clko 0:00:58]}{[%eval 12353,1][%meval TB:5]} 5. Bg3+
{[%clk 0:00:57][%clko 0:00:57]}{[%eval 29944,59][%meval TB:1k]} 5... Kb6
{[%clk 0:00:56][%clko 0:00:57]}{[%eval 12353,1][%meval TB:5]} 6. Kc3 {[%clk
0:00:57][%clko 0:00:56]}{[%eval 29945,59][%meval TB:1k]}{(Nc5)} 6... Na5
{[%clk 0:00:55][%clko 0:00:57][%emt 0:00:01]}{[%eval 12353,27][%meval 1s|TB:9]}
7. Bf2+ {[%clk 0:00:56][%clko 0:00:55][%emt 0:00:01]}{[%eval 29946,59][%meval
TB:1k]}{(Kb5)} 7... Kc7 {[%clk 0:00:55][%clko 0:00:56]}{[%eval 12353,1][%meval
TB:3]} 8. Kb4 {[%clk 0:00:55][%clko 0:00:55][%emt 0:00:01]}{[%eval
29947,59][%meval TB:1k]}{(Kd6)} 8... Nb7 {[%clk 0:00:55][%clko 0:00:55]}
{[%eval 12353,1][%meval TB:9]} 9. Bg3+ {[%clk 0:00:55][%clko 0:00:55]}{[%eval
29948,59][%meval TB:1k]} 9... Kb6 {[%clk 0:00:54][%clko 0:00:55]}{[%eval
12353,1][%meval TB:5]} 10. Be4 {[%clk 0:00:55][%clko 0:00:54]}{[%eval
29949,59][%meval TB:1k]} 10... Nd8 {[%clk 0:00:54][%clko 0:00:55]}{[%eval
12353,1][%meval TB:6]} 11. Bf2+ {[%clk 0:00:54][%clko 0:00:54]}{[%eval
29950,59][%meval TB:1k]} 11... Kc7 {[%clk 0:00:54][%clko 0:00:54]}{[%eval
12353,1][%meval TB:2]}{(Kc5)} 12. Kb5 {[%clk 0:00:54][%clko 0:00:54]}{[%eval
29951,59][%meval TB:1k]}{(Nf7)} 12... Kd6 {[%clk 0:00:53][%clko 0:00:54]}
{[%eval 12353,1][%meval TB:8]} 13. Bg3+ {[%clk 0:00:53][%clko 0:00:53]}{[%eval
29952,59][%meval TB:1k]} 13... Ke6 {[%clk 0:00:53][%clko 0:00:53]}{[%eval
12353,1][%meval TB:3]} 14. Bc2 {[%clk 0:00:52][%clko 0:00:53][%emt 0:00:01]}
{[%eval 29953,5][%meval TB:31]} 14... Nf7 {[%clk 0:00:52][%clko 0:00:52]}
{[%eval 12353,1][%meval TB:8]} 15. Bb3+ {[%clk 0:00:52][%clko 0:00:52][%emt
0:00:01]}{[%eval 29954,59][%meval TB:1k]} 15... Kf6 {[%clk 0:00:52][%clko
0:00:52]}{[%eval 12353,1][%meval TB:4]} 16. Bh4+ {[%clk 0:00:51][%clko
0:00:52]}{[%eval 29955,59][%meval TB:1k]}{(Ng5)} 16... Kg6 {[%clk
0:00:52][%clko 0:00:51]}{[%eval 12353,1][%meval TB:5]} 17. Kc5 {[%clk
0:00:51][%clko 0:00:52]}{[%eval 29956,59][%meval TB:1k]}{(Ne5)} 17... Ng5
{[%clk 0:00:51][%clko 0:00:51]}{[%eval 12353,1][%meval TB:11]} 18. Bc2+ {[%clk
0:00:50][%clko 0:00:51][%emt 0:00:01]}{[%eval 29957,59][%meval TB:1k]} 18...
Kh5 {[%clk 0:00:51][%clko 0:00:50]}{[%eval 12353,1][%meval TB:6]} 19. Bg3
{[%clk 0:00:50][%clko 0:00:51][%emt 0:00:01]}{[%eval 29958,59][%meval TB:1k]}
19... Kg4 {[%clk 0:00:51][%clko 0:00:50]}{[%eval 12353,1][%meval TB:8]} 20.
Bd6 {[%clk 0:00:49][%clko 0:00:51][%emt 0:00:01]}{[%eval 29959,59][%meval
TB:1k]}{(Nf7)} 20... Kf3 {[%clk 0:00:51][%clko 0:00:49]}{[%eval
12353,1][%meval TB:10]} 21. Kd4 {[%clk 0:00:48][%clko 0:00:51]}{[%eval
29960,59][%meval TB:1k]}{(Kf2)} 21... Kg4 {[%clk 0:00:50][%clko 0:00:48]}
{[%eval 12353,1][%meval TB:9]} 22. Bb8 {[%clk 0:00:48][%clko 0:00:50]}{[%eval
29961,59][%meval TB:1k]} 22... Ne6+ {[%clk 0:00:50][%clko 0:00:48]}{[%eval
12353,1][%meval TB:10]}{(Kd5)} 23. Ke5 {[%clk 0:00:48][%clko 0:00:50]}{[%eval
29962,59][%meval TB:420]} 23... Nf4 {[%clk 0:00:49][%clko 0:00:48]}{[%eval
12353,1][%meval TB:14]} 24. Bc7 {[%clk 0:00:47][%clko 0:00:49]}{[%eval
29963,59][%meval TB:949]} 24... Kf3 {[%clk 0:00:49][%clko 0:00:47]}{[%eval
12353,1][%meval TB:14]} 25. Be4+ {[%clk 0:00:47][%clko 0:00:49]}{[%eval
29964,59][%meval TB:1k]} 25... Kg4 {[%clk 0:00:48][%clko 0:00:47][%emt
0:00:01]}{[%eval 12353,1][%meval TB:5]} 26. Bc6 {[%clk 0:00:46][%clko 0:00:48]}
{[%eval 29965,59][%meval TB:1k]} 26... Ne2 {[%clk 0:00:48][%clko 0:00:46]}
{[%eval 12353,1][%meval TB:13]} 27. Bb6 {[%clk 0:00:46][%clko 0:00:48]}{[%eval
29966,59][%meval TB:1k]}{(Nf4)} 27... Nc3 {[%clk 0:00:48][%clko 0:00:46]}
{[%eval 12353,1][%meval TB:11]} 28. Be8 {[%clk 0:00:46][%clko 0:00:48]}{[%eval
29967,59][%meval TB:1k]} 28... Ne2 {[%clk 0:00:47][%clko 0:00:46]}{[%eval
12353,1][%meval TB:13]}{(Bd7)} 29. Ba4 {[%clk 0:00:45][%clko 0:00:47][%emt
0:00:01]}{[%eval 29968,59][%meval TB:1k]} 29... Kg3 {[%clk 0:00:47][%clko
0:00:45]}{[%eval 12353,1][%meval TB:12]} 30. Kf5 {[%clk 0:00:45][%clko
0:00:47]}{[%eval 29969,59][%meval TB:1k]} 30... Nc3 {[%clk 0:00:47][%clko
0:00:45]}{[%eval 12353,1][%meval TB:10]} 31. Bc6 {[%clk 0:00:44][%clko
0:00:47]}{[%eval 29970,59][%meval TB:1k]}{(Ne2)} 31... Nb1 {[%clk
0:00:46][%clko 0:00:44]}{[%eval 12353,1][%meval TB:11]} 32. Ba5 {[%clk
0:00:44][%clko 0:00:46][%emt 0:00:01]}{[%eval 29971,59][%meval TB:1k]}{(Na3)}
32... Kf2 {[%clk 0:00:46][%clko 0:00:44]}{[%eval 12353,1][%meval TB:7]} 33.
Ke4 {[%clk 0:00:43][%clko 0:00:46]}{[%eval 29972,59][%meval TB:1k]}{(Na3)}
33... Ke2 {[%clk 0:00:45][%clko 0:00:43]}{[%eval 12353,1][%meval TB:8]} 34.
Bb5+ {[%clk 0:00:43][%clko 0:00:45]}{[%eval 29973,59][%meval TB:1k]} 34... Kd1
{[%clk 0:00:45][%clko 0:00:43]}{[%eval 12353,1][%meval TB:2]} 35. Kd3 {[%clk
0:00:43][%clko 0:00:45]}{[%eval 29974,59][%meval TB:1k]} 35... Kc1 {[%clk
0:00:44][%clko 0:00:43][%emt 0:00:01]}{[%eval 12353,1][%meval TB:4]} 36. Bb4
{[%clk 0:00:42][%clko 0:00:44]}{[%eval 29975,59][%meval TB:1k]} 36... Kb2
{[%clk 0:00:44][%clko 0:00:42]}{[%eval 12353,1][%meval TB:5]} 37. Ba4 {[%clk
0:00:42][%clko 0:00:44]}{[%eval 29976,16][%meval TB:247]}{(Kc1)} 37... Ka2
{[%clk 0:00:35][%clko 0:00:42][%emt 0:00:08]}{[%eval 12353,23][%meval 8s|TB:6]}
38. Kc2 {[%clk 0:00:41][%clko 0:00:35][%emt 0:00:01]}{[%eval 29980,59][%meval
TB:1k]} 38... Na3+ {[%clk 0:00:35][%clko 0:00:41]}{[%eval 48,1][%meval TB:4]}
39. Kd2 {[%clk 0:00:40][%clko 0:00:35]}{[%eval 29981,59][%meval TB:366]} 39...
Nb1+ {[%clk 0:00:35][%clko 0:00:40]}{[%eval 48,1][%meval TB:7]} 40. Kc1 {[%clk
0:00:40][%clko 0:00:35]}{[%eval 29982,59][%meval TB:421]} 40... Na3 {[%clk
0:00:34][%clko 0:00:40]}{[%eval 47,1][%meval TB:4]} 41. Bc5 {[%clk
0:00:40][%clko 0:00:34]}{[%eval 29983,59][%meval TB:1k]} 41... Nc4 {[%clk
0:00:34][%clko 0:00:40][%emt 0:00:01]}{[%eval 47,1][%meval TB:5]} 42. Kc2
{[%clk 0:00:39][%clko 0:00:34]}{[%eval 29984,59][%meval TB:1k]} 42... Na3+
{[%clk 0:00:33][%clko 0:00:39]}{[%eval 47,1][%meval TB:9]}{(Kd2)} 43. Kc3
{[%clk 0:00:39][%clko 0:00:33]}{[%eval 29985,59][%meval TB:365]} 43... Nb1+
{[%clk 0:00:32][%clko 0:00:39]}{[%eval 47,1][%meval TB:6]}{(Kd3)} 44. Kb4
{[%clk 0:00:38][%clko 0:00:32]}{[%eval 29986,59][%meval TB:308]} 44... Nd2
{[%clk 0:00:32][%clko 0:00:38]}{[%eval 47,1][%meval TB:5]} 45. Be3 {[%clk
0:00:38][%clko 0:00:32]}{[%eval 29987,59][%meval TB:1k]}{(Nf3)} 45... Ne4
{[%clk 0:00:32][%clko 0:00:38]}{[%eval 47,1][%meval TB:9]} 46. Bc6 {[%clk
0:00:37][%clko 0:00:32]}{[%eval 29988,59][%meval TB:1k]} 46... Ng3 {[%clk
0:00:31][%clko 0:00:37][%emt 0:00:01]}{[%eval 47,1][%meval TB:11]} 47. Kc3
{[%clk 0:00:37][%clko 0:00:31]}{[%eval 29992,59][%meval TB:1k]} 47... Kb1
{[%clk 0:00:31][%clko 0:00:37]}{[%eval 47,1][%meval TB:9]} 48. Kb3 {[%clk
0:00:36][%clko 0:00:31]}{[%eval 29994,59][%meval TB:1k]} 48... Ne2 {[%clk
0:00:30][%clko 0:00:36]}{[%eval 47,1][%meval TB:7]} 49. Be4+ {[%clk
0:00:36][%clko 0:00:30][%emt 0:00:01]}{[%eval 29995,59][%meval TB:1k]} 49...
Ka1 {[%clk 0:00:30][%clko 0:00:36]}{[%eval 47,1][%meval TB:1]} 50. Bc2 {[%clk
0:00:35][%clko 0:00:30][%emt 0:00:01]}{[%eval 29996,59][%meval TB:1k]} 50...
Nc1+ {[%clk 0:00:30][%clko 0:00:35]}{[%eval 47,1][%meval TB:6]} 1/2-1/2

[Event "[03] laskos"]
[Site "Computadora local"]
[Date "2016.02.16"]
[Round "2"]
[White "stockfish dev"]
[Black "Deep Rybka 4.1 SSE42 x64"]
[Result "1-0"]
[BlackElo "-"]
[Classes "0"]
[ECO "?"]
[TimeControl "60"]
[WhiteElo "-"]
[SetUp "1"]
[FEN "2Bn4/8/2k5/8/8/1K6/8/4B3 w - - 0 1"]

{Processor: AMD Phenom(tm) II X4 B50 Processor (@3,21GHz)^13 ^10 Blancas:
stockfish dev, hash size: 512M, opening book: NarrowBook^13 ^10 Negras: Deep
Rybka 4.1 SSE42 x64, hash size: 512M, opening book: NarrowBook^13 ^10 } {[%t
Long] Las blancas dieron mate} {Blancas de biblioteca} 1. Kc4 {[%clk
0:00:59][%clko 0:01:00]}{[%eval 12353,19][%meval 1s|TB:22]}{(Nb7)}{Negras de
biblioteca} 1... Nf7 {[%clk 0:00:59][%clko 0:00:59]}{[%eval 29941,59][%meval
TB:410]} 2. Kd4 {[%clk 0:00:58][%clko 0:00:59][%emt 0:00:01]}{[%eval
12353,24][%meval TB:19]} 2... Nd6 {[%clk 0:00:59][%clko 0:00:58]}{[%eval
29942,59][%meval TB:680]}{(Be6)} 3. Bg4 {[%clk 0:00:57][%clko 0:00:59][%emt
0:00:01]}{[%eval 12353,25][%meval 1s|TB:18]}{(Nb5)} 3... Nb7 {[%clk
0:00:58][%clko 0:00:57]}{[%eval 29943,59][%meval TB:815]} 4. Bf3+ {[%clk
0:00:56][%clko 0:00:58][%emt 0:00:01]}{[%eval 12353,24][%meval TB:22]}{(Kb6)}
4... Kc7 {[%clk 0:00:58][%clko 0:00:56]}{[%eval 29944,59][%meval TB:343]} 5.
Kd5 {[%clk 0:00:55][%clko 0:00:58][%emt 0:00:01]}{[%eval 12353,30][%meval
1s|TB:24]} 5... Kb6 {[%clk 0:00:58][%clko 0:00:55]}{[%eval 29945,59][%meval
TB:613]} 6. Bf2+ {[%clk 0:00:54][%clko 0:00:58][%emt 0:00:01]}{[%eval
12353,28][%meval TB:19]} 6... Kc7 {[%clk 0:00:57][%clko 0:00:54]}{[%eval
29946,59][%meval TB:343]}{(Bd4)} 7. Bg3+ {[%clk 0:00:54][%clko 0:00:57][%emt
0:00:01]}{[%eval 12353,32][%meval TB:21]} 7... Kb6 {[%clk 0:00:57][%clko
0:00:54]}{[%eval 29946,59][%meval TB:343]}{(Be5)} 8. Be4 {[%clk 0:00:53][%clko
0:00:57][%emt 0:00:01]}{[%eval 12353,32][%meval TB:21]}{(Nd8)} 8... Na5 {[%clk
0:00:57][%clko 0:00:53]}{[%eval 29947,59][%meval TB:545]} 9. Bf2+ {[%clk
0:00:52][%clko 0:00:57][%emt 0:00:01]}{[%eval 12353,30][%meval TB:22]} 9...
Kc7 {[%clk 0:00:56][%clko 0:00:52]}{[%eval 29948,59][%meval TB:275]}{(Be3)}
10. Ke6 {[%clk 0:00:51][%clko 0:00:56]}{[%eval 12353,1][%meval TB:22]}{(Nc4)}
10... Nb7 {[%clk 0:00:56][%clko 0:00:51]}{[%eval 29949,59][%meval TB:478]} 11.
Ke7 {[%clk 0:00:51][%clko 0:00:56]}{[%eval 12353,1][%meval TB:27]} 11... Na5
{[%clk 0:00:55][%clko 0:00:51]}{[%eval 29950,59][%meval TB:410]}{(Bc5)} 12.
Bg3+ {[%clk 0:00:50][%clko 0:00:55][%emt 0:00:01]}{[%eval 12353,1][%meval
TB:27]} 12... Kb6 {[%clk 0:00:55][%clko 0:00:50]}{[%eval 29951,59][%meval
TB:140]}{(Ke6)} 13. Kd7 {[%clk 0:00:49][%clko 0:00:55][%emt 0:00:01]}{[%eval
12353,1][%meval TB:30]} 13... Kc5 {[%clk 0:00:55][%clko 0:00:49]}{[%eval
29952,59][%meval TB:545]} 14. Bf2+ {[%clk 0:00:48][%clko 0:00:55][%emt
0:00:01]}{[%eval 12353,26][%meval 1s|TB:28]} 14... Kc4 {[%clk 0:00:54][%clko
0:00:48]}{[%eval 29953,59][%meval TB:208]} 15. Kd6 {[%clk 0:00:47][%clko
0:00:54][%emt 0:00:01]}{[%eval 12353,29][%meval 1s|TB:29]} 15... Nb3 {[%clk
0:00:54][%clko 0:00:47]}{[%eval 29953,59][%meval TB:478]}{(Be3)} 16. Bd5+
{[%clk 0:00:46][%clko 0:00:54][%emt 0:00:01]}{[%eval 12353,25][%meval
1s|TB:28]} 16... Kc3 {[%clk 0:00:53][%clko 0:00:46][%emt 0:00:01]}{[%eval
29954,59][%meval TB:275]}{(Ke5)} 17. Bg8 {[%clk 0:00:45][%clko 0:00:53][%emt
0:00:01]}{[%eval 12353,1][%meval TB:27]} 17... Nd2 {[%clk 0:00:53][%clko
0:00:45]}{[%eval 29955,59][%meval TB:748]} 18. Kd5 {[%clk 0:00:45][%clko
0:00:53]}{[%eval 12353,1][%meval TB:24]} 18... Kd3 {[%clk 0:00:52][%clko
0:00:45]}{[%eval 29956,59][%meval TB:748]} 19. Bh7+ {[%clk 0:00:44][%clko
0:00:52][%emt 0:00:01]}{[%eval 12353,25][%meval 1s|TB:17]} 19... Ke2 {[%clk
0:00:51][%clko 0:00:44][%emt 0:00:01]}{[%eval 29957,59][%meval TB:208]} 20.
Bb6 {[%clk 0:00:43][%clko 0:00:51]}{[%eval 12353,1][%meval TB:22]} 20... Kf3
{[%clk 0:00:51][%clko 0:00:43][%emt 0:00:01]}{[%eval 29958,59][%meval TB:680]}
{(Bd4)} 21. Kd4 {[%clk 0:00:43][%clko 0:00:51]}{[%eval 12353,1][%meval TB:22]}
{(Kf4)} 21... Kg4 {[%clk 0:00:50][%clko 0:00:43]}{[%eval 29959,59][%meval
TB:748]}{(Bc7)} 22. Bc2 {[%clk 0:00:43][%clko 0:00:50]}{[%eval 12353,1][%meval
TB:18]} 22... Nf3+ {[%clk 0:00:50][%clko 0:00:43]}{[%eval 29960,59][%meval
TB:883]}{(Kd5)} 23. Ke3 {[%clk 0:00:42][%clko 0:00:50]}{[%eval 12353,1][%meval
TB:7]} 23... Ng5 {[%clk 0:00:50][%clko 0:00:42]}{[%eval 29961,59][%meval
TB:883]}{(Bd4)} 24. Ba5 {[%clk 0:00:41][%clko 0:00:50][%emt 0:00:01]}{[%eval
12353,1][%meval TB:20]} 24... Nf7 {[%clk 0:00:49][%clko 0:00:41]}{[%eval
29962,59][%meval TB:680]}{(Bb4)} 25. Kd4 {[%clk 0:00:41][%clko 0:00:49]}
{[%eval 12353,1][%meval TB:22]} 25... Ng5 {[%clk 0:00:49][%clko 0:00:41]}
{[%eval 29963,59][%meval TB:883]}{(Bc3)} 26. Ba4 {[%clk 0:00:40][%clko
0:00:49]}{[%eval 12353,1][%meval TB:23]} 26... Nf3+ {[%clk 0:00:49][%clko
0:00:40]}{[%eval 29964,59][%meval TB:883]}{(Ke4)} 27. Ke3 {[%clk
0:00:40][%clko 0:00:49]}{[%eval 12353,1][%meval TB:7]} 27... Ne5 {[%clk
0:00:48][%clko 0:00:40]}{[%eval 29965,59][%meval TB:950]} 28. Ke4 {[%clk
0:00:40][%clko 0:00:48]}{[%eval 12353,1][%meval TB:19]} 28... Nf7 {[%clk
0:00:48][%clko 0:00:40][%emt 0:00:01]}{[%eval 29966,59][%meval TB:815]} 29.
Bd1+ {[%clk 0:00:39][%clko 0:00:48]}{[%eval 12353,1][%meval TB:18]} 29... Kg5
{[%clk 0:00:47][%clko 0:00:39]}{[%eval 29967,59][%meval TB:275]}{(Bd2)} 30.
Bc3 {[%clk 0:00:39][%clko 0:00:47]}{[%eval 12353,1][%meval TB:19]} 30... Nh6
{[%clk 0:00:47][%clko 0:00:39]}{[%eval 29968,59][%meval TB:545]} 31. Bc2
{[%clk 0:00:39][%clko 0:00:47]}{[%eval 12353,1][%meval TB:24]}{(Nf5)} 31...
Ng4 {[%clk 0:00:46][%clko 0:00:39]}{[%eval 29969,59][%meval TB:545]} 32. Bd2+
{[%clk 0:00:38][%clko 0:00:46]}{[%eval 12353,1][%meval TB:20]} 32... Kf6
{[%clk 0:00:46][%clko 0:00:38]}{[%eval 29969,59][%meval TB:343]} 33. Kf4
{[%clk 0:00:38][%clko 0:00:46]}{[%eval 12353,1][%meval TB:19]} 33... Nh6
{[%clk 0:00:46][%clko 0:00:38]}{[%eval 29970,59][%meval TB:613]}{(Bc3)} 34.
Be1 {[%clk 0:00:37][%clko 0:00:46][%emt 0:00:01]}{[%eval 12353,1][%meval
TB:19]} 34... Ke7 {[%clk 0:00:45][%clko 0:00:37]}{[%eval 29971,59][%meval
TB:545]}{(Bb4)} 35. Bh4+ {[%clk 0:00:37][%clko 0:00:45][%emt 0:00:01]}{[%eval
12353,1][%meval TB:22]}{(Kd6)} 35... Kf7 {[%clk 0:00:44][%clko 0:00:37][%emt
0:00:01]}{[%eval 29972,59][%meval TB:410]}{(Bd3)} 36. Kg5 {[%clk
0:00:36][%clko 0:00:44]}{[%eval 12353,1][%meval TB:22]} 36... Kg7 {[%clk
0:00:43][%clko 0:00:36][%emt 0:00:01]}{[%eval 29973,59][%meval TB:613]} 37.
Bf2 {[%clk 0:00:36][%clko 0:00:43]}{[%eval 12353,1][%meval TB:14]} 37... Nf7+
{[%clk 0:00:43][%clko 0:00:36]}{[%eval 29974,59][%meval TB:545]} 38. Kf5
{[%clk 0:00:36][%clko 0:00:43]}{[%eval 12353,1][%meval TB:5]} 38... Nh6+
{[%clk 0:00:43][%clko 0:00:36]}{[%eval 29975,59][%meval TB:748]} 39. Ke6
{[%clk 0:00:35][%clko 0:00:43]}{[%eval 12353,1][%meval TB:5]} 39... Kf8 {[%clk
0:00:42][%clko 0:00:35]}{[%eval 29976,59][%meval TB:478]}{(Bd3)} 40. Bc5+
{[%clk 0:00:34][%clko 0:00:42][%emt 0:00:01]}{[%eval 12353,18][%meval
1s|TB:23]} 40... Kg7 {[%clk 0:00:42][%clko 0:00:34]}{[%eval 29976,59][%meval
TB:208]} 41. Bd4+ {[%clk 0:00:34][%clko 0:00:42]}{[%eval 12353,1][%meval
TB:25]}{(Kg8)} 41... Kf8 {[%clk 0:00:41][%clko 0:00:34]}{[%eval
29977,59][%meval TB:140]} 42. Bg6 {[%clk 0:00:32][%clko 0:00:41][%emt 0:00:01]}
{[%eval 12353,30][%meval 1s|TB:27]}{(Kg8)} 42... Ng8 {[%clk 0:00:41][%clko
0:00:32]}{[%eval 29978,59][%meval TB:343]}{(Be4)} 43. Bh5 {[%clk
0:00:32][%clko 0:00:41]}{[%eval 12353,1][%meval TB:27]} 43... Nh6 {[%clk
0:00:41][%clko 0:00:32]}{[%eval 29979,59][%meval TB:208]}{(Be5)} 44. Kf6
{[%clk 0:00:32][%clko 0:00:41]}{[%eval 12353,1][%meval TB:25]} 44... Ng8+
{[%clk 0:00:41][%clko 0:00:32]}{[%eval 29980,59][%meval TB:343]}{(Ke5)} 45.
Kg6 {[%clk 0:00:31][%clko 0:00:41][%emt 0:00:01]}{[%eval 12353,1][%meval TB:5]}
45... Ke7 {[%clk 0:00:40][%clko 0:00:31]}{[%eval 29981,59][%meval TB:343]}
46. Kg7 {[%clk 0:00:30][%clko 0:00:40]}{[%eval 12353,1][%meval TB:21]} 46...
Ke6 {[%clk 0:00:40][%clko 0:00:30][%emt 0:00:01]}{[%eval 29982,59][%meval
TB:352]} 47. Kxg8 {[%clk 0:00:30][%clko 0:00:40][%emt 0:00:01]}{[%eval
12353,22][%meval TB:23]} 47... Kd5 {[%clk 0:00:39][%clko 0:00:30][%emt
0:00:01]}{[%eval 29983,59][%meval TB:294]}{(Bf2)} 48. Ba7 {[%clk
0:00:29][%clko 0:00:39][%emt 0:00:01]}{[%eval 12353,21][%meval TB:25]}{(Kc4)}
48... Ke4 {[%clk 0:00:39][%clko 0:00:29]}{[%eval 29984,59][%meval TB:352]}
{(Kf7)} 49. Bg6+ {[%clk 0:00:27][%clko 0:00:39][%emt 0:00:01]}{[%eval
12353,25][%meval 1s|TB:19]} 49... Kd5 {[%clk 0:00:39][%clko 0:00:27]}{[%eval
29984,59][%meval TB:236]}{(Be3)} 50. Bd3 {[%clk 0:00:27][%clko 0:00:39][%emt
0:00:01]}{[%eval 29968,27][%meval TB:21]} 50... Ke6 {[%clk 0:00:38][%clko
0:00:27]}{[%eval 29984,59][%meval TB:236]}{(Be3)} 51. Kg7 {[%clk
0:00:26][%clko 0:00:38][%emt 0:00:01]}{[%eval 29970,30][%meval TB:22]} 51...
Ke5 {[%clk 0:00:38][%clko 0:00:26]}{[%eval 29985,59][%meval TB:294]} 52. Kf7
{[%clk 0:00:25][%clko 0:00:38][%emt 0:00:01]}{[%eval 29983,30][%meval
1s|TB:25]} 52... Kd6 {[%clk 0:00:37][%clko 0:00:25]}{[%eval 29986,59][%meval
TB:178]} 53. Kf6 {[%clk 0:00:24][%clko 0:00:37][%emt 0:00:01]}{[%eval
29984,31][%meval TB:24]}{(Kd7)} 53... Kd5 {[%clk 0:00:37][%clko 0:00:24]}
{[%eval 29987,59][%meval TB:236]} 54. Be3 {[%clk 0:00:23][%clko 0:00:37][%emt
0:00:01]}{[%eval 29985,31][%meval TB:24]} 54... Kd6 {[%clk 0:00:37][%clko
0:00:23]}{[%eval 29988,59][%meval TB:120]} 55. Be4 {[%clk 0:00:22][%clko
0:00:37][%emt 0:00:01]}{[%eval 29987,30][%meval TB:27]} 55... Kd7 {[%clk
0:00:36][%clko 0:00:22][%emt 0:00:01]}{[%eval 29989,59][%meval TB:120]}{(Bf4)}
56. Bc5 {[%clk 0:00:21][%clko 0:00:36][%emt 0:00:01]}{[%eval 29988,31][%meval
TB:30]} 56... Kd8 {[%clk 0:00:35][%clko 0:00:21]}{[%eval 29989,59][%meval
TB:236]} 57. Ke6 {[%clk 0:00:20][%clko 0:00:35][%emt 0:00:01]}{[%eval
29989,31][%meval TB:31]} 57... Kc7 {[%clk 0:00:35][%clko 0:00:20]}{[%eval
29990,59][%meval TB:178]}{(Be3)} 58. Bf3 {[%clk 0:00:19][%clko 0:00:35][%emt
0:00:01]}{[%eval 29990,32][%meval TB:30]} 58... Kc8 {[%clk 0:00:35][%clko
0:00:19]}{[%eval 29991,59][%meval TB:178]}{(Bd4)} 59. Bb6 {[%clk
0:00:18][%clko 0:00:35][%emt 0:00:01]}{[%eval 29991,32][%meval TB:29]} 59...
Kb8 {[%clk 0:00:34][%clko 0:00:18]}{[%eval 29992,5][%meval TB:1]} 60. Kd6
{[%clk 0:00:18][%clko 0:00:34]}{[%eval 29992,32][%meval TB:28]} 60... Kc8
{[%clk 0:00:34][%clko 0:00:18]}{[%eval 29993,13][%meval TB:10]}{(Bd5)} 61. Be4
{[%clk 0:00:17][%clko 0:00:34][%emt 0:00:01]}{[%eval 29993,33][%meval TB:26]}
61... Kb8 {[%clk 0:00:34][%clko 0:00:17]}{[%eval 796,4]}{(Bd5)} 62. Kc6 {[%clk
0:00:16][%clko 0:00:34]}{[%eval 29994,37][%meval TB:29]} 62... Kc8 {[%clk
0:00:34][%clko 0:00:16]}{[%eval 29995,59][%meval TB:120]} 63. Bf5+ {[%clk
0:00:16][%clko 0:00:34][%emt 0:00:01]}{[%eval 29995,48][%meval TB:23]} 63...
Kb8 {[%clk 0:00:33][%clko 0:00:16][%emt 0:00:01]}{[%eval 29996,5][%meval TB:1]}
64. Bd4 {[%clk 0:00:14][%clko 0:00:33][%emt 0:00:01]}{[%eval
29996,127][%meval TB:25]} 64... Ka8 {[%clk 0:00:32][%clko 0:00:14][%emt
0:00:01]}{[%eval 29997,14][%meval TB:11]} 65. Kb6 {[%clk 0:00:14][%clko
0:00:32]}{[%eval 29997,127][%meval TB:31]} 65... Kb8 {[%clk 0:00:31][%clko
0:00:14][%emt 0:00:01]}{[%eval 29998,5][%meval TB:1]} 66. Be5+ {[%clk
0:00:13][%clko 0:00:31]}{[%eval 29998,127][%meval TB:27]} 66... Ka8 {[%clk
0:00:30][%clko 0:00:13][%emt 0:00:01]}{[%eval 29999,28][%meval TB:27]} 67.
Be4# {[%clk 0:00:13][%clko 0:00:30][%emt 0:00:01]}{[%eval 29999,127][%meval
TB:30]} 1-0
[/pgn]
User avatar
Daniel Mehrmann
Posts: 858
Joined: Wed Mar 08, 2006 9:24 pm
Location: Germany
Full name: Daniel Mehrmann

Re: Nalimov EGTB problem related to DTM?

Post by Daniel Mehrmann »

Laskos wrote:
Daniel Mehrmann wrote:
Laskos wrote:I knew that Nalimovs have this problem, but first, I imagined it to be a problem with very rare, freak positions, second, I didn't know that Syzygy gives "perfect play" against "perfect play". I was under some sort of artistic impression that if Nalimov shows me the shortest distance to mate in a given root position, even ignoring the 50-move rule it should perform at least as well as Syzygy on wins from the root. Then several days ago I played (color and reversed) from my hard TB won positions, using Nalimov, Syzygy and Komodo no TB, with these results:

Code: Select all

Rank Name                          ELO   Games   Score   Draws
   1 Syzygy                         69    1032     60%     20%
   2 Nalimov                        43    1032     56%     21%
   3 Komodo No TB                 -115    1032     34%     32%
Finished match
Well, basicly i wouldn't say that Syzygy performs better then Nalimov. A combination of both can be good too. I tested it with Fruit and it looks not so bad.

Probably they are equal with distance zero/mate, but Syzygy performs better, because the files are smaller (timing i/o balance).

Well, if the root position is already inside a database, "Nalimov" engines doesn't do a move order based on the mate scores at root. At least i can say, i don't know such a engine yet. Maybe that could be a reason in some cases if the depth or time is extremely limited. However, i need to do more tests in that case.

Regards
Daniel
The sole advantage of Nalimovs is that they are showing the correct DTM in PV. Might be useful for analysis, otherwise Syzygy bases are to be preferred.
I don't think the problem is DTM itself. Okay, you can make things easier for a engine if you take care about the 50-move rule by generation.

Probably the most implementations of Nalimov's EGTB, including Fruit, are simply lazy written or copyed.

I think you can resolve this problem by a little local search on root, like Syzygy does, to take care of the 50-move rule.

I'll do that and we can bet that a engine of your choice will never draw a winning position against Fruit.

Regards
Daniel
User avatar
hgm
Posts: 27788
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Nalimov EGTB problem related to DTM?

Post by hgm »

If you don't search all the way to the mate (i.e. do not use the EGT at all) I don't see how this is possible. You could be in a position that is just barely won (i.e. where the conversion can be forced on the 50th move, but not earlier), and where Nalimov suggests a wrong move that speeds up the mate at the expense of delaying the conversion already on the second move. If you don't search at least 96 ply on every branch at the point where Nalimov suggests the error, you would never see the opponent now can force a 50-move draw. Any lower-depth search that at the leaves probes Nalimov will simply confirm what a 1-ply probing search gives, as there is no inconsistency within the Nalimov EGT. Nalimov will only be contradicted in nodes where the reversible-ply counter hits 100.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Nalimov EGTB problem related to DTM?

Post by bob »

syzygy wrote:
bob wrote:
syzygy wrote:
bob wrote:I am not sure what the problem is here. I tested this position a hundred times against multiple opponents and never saw a loss or draw by any program for the winning side. The knight is captured well before the 50 move rule limit with optimal play
I don't know what you call optimal play, but black can postpone the knight's capture until the49th move.

So the position is won for white under the 50-move rule, but white needs to play carefully and merely following Nalimov might be insufficient.
I call "optimal play" playing the sequence of moves that leads to the shortest possible mate. DTZ handles the 50 move rule where DTM currently does not, of course. But here 50 move is irrelevant. If black tries to postpone the knight loss, he gets mated even quicker however.
Apparently there are lines starting from Kai's position that have DTM-optimal moves by white but do run afoul of the 50-move rule.

And I don't find that surprising given that the position has DTZ=49: white can only afford to lose 1 move in the DTZ-metric. In this respect, your particular DTM-line that has a capture after 46 moves is simply meaningless.

Maybe Kai can give us an example game where white using Nalimov only manages to draw the position.
After thinking about this, I was obviously wrong. Since the king and knight moves have the same distance to mate, they are equal in the eyes of Crafty. Where the human would obviously try to push the DTZ counter as hard as possible to encourage a 1 tempo mistake here or there...
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Nalimov EGTB problem related to DTM?

Post by bob »

Laskos wrote:
syzygy wrote:
bob wrote:
syzygy wrote:
bob wrote:I am not sure what the problem is here. I tested this position a hundred times against multiple opponents and never saw a loss or draw by any program for the winning side. The knight is captured well before the 50 move rule limit with optimal play
I don't know what you call optimal play, but black can postpone the knight's capture until the49th move.

So the position is won for white under the 50-move rule, but white needs to play carefully and merely following Nalimov might be insufficient.
I call "optimal play" playing the sequence of moves that leads to the shortest possible mate. DTZ handles the 50 move rule where DTM currently does not, of course. But here 50 move is irrelevant. If black tries to postpone the knight loss, he gets mated even quicker however.
Apparently there are lines starting from Kai's position that have DTM-optimal moves by white but do run afoul of the 50-move rule.

And I don't find that surprising given that the position has DTZ=49: white can only afford to lose 1 move in the DTZ-metric. In this respect, your particular DTM-line that has a capture after 46 moves is simply meaningless.

Maybe Kai can give us an example game where white using Nalimov only manages to draw the position.
Sure, from the discussed position, a game between Shredder 12 Nalimov (3-4-5) against Komodo 9.3 Syzygy (3-4-5). Michael posted it with Crafty 25.01 Nalimov too.

[pgn][Event "?"]
[Site "?"]
[Date "2016.02.14"]
[Round "1"]
[White "Shredder Nalimov"]
[Black "Komodo Syzygy"]
[Result "1/2-1/2"]
[FEN "8/4K3/8/4B1n1/B7/8/5k2/8 w - - 0 1"]
[PlyCount "100"]
[SetUp "1"]
[TimeControl "240+2.399"]

1. Bf4 {+298.77/1 0.041s} Ne4 {-280.00/29 4.0s} 2. Bc6 {+298.79/1 0.026s} Nc3 {0.00/5 0s} 3. Be5 {+298.81/1 0.026s} Nb1 {0.00/5 0s}
4. Ke6 {+298.83/1 0.026s} Nd2 {0.00/5 0s} 5. Bd4+ {+298.85/1 0.026s}
Kg3 {0.00/5 0s} 6. Kf5 {+298.87/1 0.026s} Nf3 {0.00/5 0.006s}
7. Bb6 {+298.89/1 0.026s} Nh4+ {0.00/5 0.004s} 8. Ke4 {+298.91/1 0.026s} Ng2 {0.00/5 0s} 9. Kd3 {+298.93/1 0.026s} Nf4+ {-280.00/30 4.5s} 10. Kd2 {+298.95/1 0.026s} Ng2 {0.00/5 0s} 11. Bc7+ {+298.97/1 0.026s} Kf2 {0.00/5 0s} 12. Bd5 {+298.99/1 0.026s} Nh4 {0.00/5 0s}
13. Bb6+ {+M99/1 0.026s} Kg3 {0.00/5 0s} 14. Ke2 {+M97/1 0.026s} Kf4 {0.00/5 0s} 15. Bb3 {+M95/1 0.026s} Ng6 {0.00/5 0s} 16. Bc7+ {+M93/1 0.021s} Kf5 {0.00/5 0.001s} 17. Bc2+ {+M91/1 0.026s} Kf6 {0.00/5 0s} 18. Bd8+ {+M89/1 0.031s} Kf7 {0.00/5 0.001s} 19. Ke3 {+M87/1 0.026s} Ne7 {0.00/5 0s} 20. Bb3+ {+M85/1 0.026s} Ke8 {0.00/5 0s} 21. Bc7 {+M83/1 0.026s} Kd7 {0.00/5 0s} 22. Bf4 {+M81/1 0.026s} Kc6 {0.00/5 0.001s} 23. Kd4 {+M79/1 0.026s} Kd7 {0.00/5 0s} 24. Bh2 {+M77/1 0.025s} Nf5+ {0.00/5 0.001s} 25. Ke5 {+M75/1 0.026s} Nd6 {0.00/5 0.007s} 26. Bg3 {+M73/1 0.025s} Kc6 {0.00/5 0s} 27. Bd5+ {+M71/1 0.025s} Kd7 {0.00/5 0s} 28. Bf3 {+M69/1 0.025s} Nb5 {0.00/5 0s} 29. Bf2 {+M67/1 0.025s} Nc3 {0.00/5 0s} 30. Bh5 {+M65/1 0.025s} Nb5 {0.00/5 0.001s} 31. Bd1 {+M63/1 0.025s} Kc7 {0.00/5 0s} 32. Ke6 {+M61/1 0.026s} Nc3 {0.00/5 0s} 33. Bf3 {+M59/1 0.025s} Na2 {0.00/5 0.001s} 34. Be1 {+M57/1 0.025s} Kb6 {0.00/5 0s} 35. Kd5 {+M55/1 0.026s} Kb5 {0.00/5 0s} 36. Be2+ {+M53/1 0.026s} Ka4 {0.00/5 0s}
37. Kc4 {+M51/1 0.026s} Ka3 {0.00/5 0.016s} 38. Bd2 {+M49/1 0.026s}
Kb2 {0.00/5 0.002s} 39. Bd1 {+M47/1 0.026s} Kb1 {-0.48/28 7.8s}
40. Kb3 {+M39/1 0.026s} Nc1+ {0.00/5 0s} 41. Kb4 {+M37/1 0.025s}
Na2+ {0.00/5 0s} 42. Ka3 {+M35/1 0.031s} Nc1 {0.00/5 0s} 43. Be3 {+M33/1 0.025s} Nd3 {0.00/5 0s} 44. Kb3 {+M31/1 0.025s} Nc1+ {0.00/5 0s} 45. Kc3 {+M29/1 0.025s} Na2+ {0.00/5 0s} 46. Kd2 {+M27/1 0.025s} Nb4 {0.00/5 0s} 47. Bc5 {+M25/1 0.025s} Nd5 {0.00/5 0s} 48. Bf3 {+M23/1 0.025s} Nc7 {0.00/5 0.007s} 49. Kc3 {+M15/1 0.026s} Ka2 {0.00/5 0.008s} 50. Kc2 {+M11/1 0.026s} Nb5 {0.00/5 0s, Draw by fifty moves rule} 1/2-1/2[/pgn]
I might experiment with this a bit as I wonder if it is fixable. IE the original is a mate in N score, which for Crafty is > 32000. I might could factor in the 50 move counter inversely so that it would prefer smallest mate score with largest 50 move counter, encouraging the 50 move counter to be reset earlier in the PV path. Since the 50 move counter is the problem, this would factor in a sort of DTZ approximation while preserving the actual mate score. Will need to adjust how I do the N part of mate in N so that it is dominant however.
User avatar
hgm
Posts: 27788
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Nalimov EGTB problem related to DTM?

Post by hgm »

The problem is that the fifty-move counter measures the distance to the previous irreversible move, while what you are interested in is the distance to the point in the future where you can reset it. But the latter is usually way beyond the horizon. Especially when you want to rely on DTM probing.

Note that when the successor end-game after conversion/zeroing is difficult, resetting the counter too early can be just as fatal as resetting it too late. You must be sure that you don't jump into a cursed win of that successor, and that can require careful preparation in the current phase. Nalimov doesn't contain that information, and plain DTZ would not either. Only DTZ50 guarantees this.
Dirt
Posts: 2851
Joined: Wed Mar 08, 2006 10:01 pm
Location: Irvine, CA, USA

Re: Nalimov EGTB problem related to DTM?

Post by Dirt »

bob wrote:I might experiment with this a bit as I wonder if it is fixable.
The (relatively) easy way is to support Syzygy. It's files take up less room, too.
Deasil is the right way to go.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Nalimov EGTB problem related to DTM?

Post by bob »

hgm wrote:The problem is that the fifty-move counter measures the distance to the previous irreversible move, while what you are interested in is the distance to the point in the future where you can reset it. But the latter is usually way beyond the horizon. Especially when you want to rely on DTM probing.

Note that when the successor end-game after conversion/zeroing is difficult, resetting the counter too early can be just as fatal as resetting it too late. You must be sure that you don't jump into a cursed win of that successor, and that can require careful preparation in the current phase. Nalimov doesn't contain that information, and plain DTZ would not either. Only DTZ50 guarantees this.
My thought was that since I don't know when it will be reset in the future, reward resetting it as early in the PV as possible, rather than later, which might avoid this particular problem.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Nalimov EGTB problem related to DTM?

Post by bob »

Dirt wrote:
bob wrote:I might experiment with this a bit as I wonder if it is fixable.
The (relatively) easy way is to support Syzygy. It's files take up less room, too.
ONLY for 6 pieces or less. But this is a general problem that applies with more pieces on the board, hence thinking about something that has a chance,whether it would work or not is not even conjecture at this point.