New Scorpio bitbase files

[ernest]

It's stunning, that all 5 piece bases are compresses now to 229 MB! Is this the record or are shredderbases still smaller?

Where did you find that 229 MB size?

Shredderbases (3-4-5 bitbases) come in 2 sizes:
all345.tbe (high compression) 161220 KB
all345_fast (for fast decompression) 451677 KB

[Jouni]

The size of 180 files is 228 983 kB. BTW the default dll (egbbdll.dll) is working for single CPU.

Jouni

[BBauer]

BTW the default dll (egbbdll.dll) is working for single CPU.

That is also my experience. Under Arena engines crashes when using 2 procs and new bitbases.
Kind regards
Bernhard

[Daniel Shawul]

Hello Bernhard

Does this fix your problem http://dshawul.googlepages.com/egbbdll.dll ?
I just undone what i did some time ago hopefully it fixes it.
regards
Daniel

[Daniel Shawul]

Hello Gabor

You are right. At first I didn't think of the different egbbdlls required for different problem. I guess that authors who use them will have to agree on a certain name convention and use that.

Maybe something like
egbbdll.dll
egbddll.so
egbdll64.dll
egbdll64.so

regards,
Daniel

[Jouni]

Is it possible to handle all 5 piece endings with bitbases? Here's position (mate in 30) which Cyclone didn't succeed to win at all:

[Event "basetest"]
[Site "?"]
[Date "2009.03.15"]
[Round "3"]
[White "Cyclone 3.4"]
[Black "Rybka 3 1-cpu 32-bit"]
[Result "1/2-1/2"]
[Annotator "46.06;2.76"]
[SetUp "1"]
[FEN "8/4q3/4P3/4Q3/3K2k1/8/8/8 w - - 0 1"]

1. Ke3 {46.06/17 13} Qa3+ {2.76/14 19} 2. Ke2 {(Kd2) 44.66/20 14} Qe7 {
(Qa2+) 2.93/15 12} 3. Kd1 {(Kd2) 44.75/20 22} Kf3 {(Qd8+) 2.26/14 11} 4. Qd5+ {
(Kc2) 45.11/20 8} Kf4 {(Ke3) 1.25/15 15} 5. Qd7 {45.13/20 8} Qa3 {1.26/17 10}
6. Kc2 {(Qf7+) 43.04/20 10} Qa2+ {(Qc5+) 1.25/16 12} 7. Kc3 {43.04/20 7} Qa3+ {
1.50/16 19} 8. Kc4 {43.84/20 7} Qa2+ {(Qa6+) 1.50/16 9} 9. Kb4 {
(Kb5) 45.80/17 9} Qb2+ {1.50/17 7} 10. Ka5 {(Kc5) 45.95/17 5} Qc3+ {1.50/15 4}
11. Ka6 {(Ka4) 45.95/18 6} Qc4+ {(Qf6) 1.46/16 6} 12. Kb7 {44.47/20 6} Qb4+ {
1.65/16 7} 13. Kc8 {45.27/20 5} Qf8+ {2.03/16 32} 14. Qd8 {44.47/21 5} Qf5 {
2.03/17 4} 15. Qd4+ {(Qd6+) 45.27/20 6} Kg5 {2.12/14 8} 16. Qg7+ {46.07/20 5}
Kh4 {(Kf4) 3.33/16 39} 17. Kd7 {43.94/20 5} Qd5+ {3.72/15 6} 18. Ke7 {
44.74/20 5} Qb7+ {3.72/15 3} 19. Kf8 {44.74/21 5} Qa8+ {(Qf3+) 3.72/17 4} 20.
Kf7 {45.73/20 6} Qd5 {3.72/17 23} 21. Qg6 {(Qf6+) 45.56/20 5} Kh3 {
(Qb3) 4.51/14 15} 22. Qf6 {(Kg7) 45.74/19 7} Kh2 {(Qb3) 3.72/14 1} 23. Qh4+ {
(Kg6) 46.53/17 4} Kg2 {3.72/12 1} 24. Qg4+ {(Qf4) 46.46/17 3} Kf2 {2.73/10 1}
25. Kf6 {46.41/17 4} Qc6 {3.35/12 2} 26. Qf4+ {(Kg5) 46.75/17 7} Ke2 {
(Kg2) 3.26/9 1} 27. Kf5 {(Qd4) 46.34/17 3} Qc8 {(Qd5+) 1.84/9 1} 28. Qe5+ {
(Qe4+) 46.46/17 4} Kd3 {2.52/9 1} 29. Qb5+ {(Kg4) 46.44/17 3} Kc3 {
(Kc2) 1.69/9 0} 30. Qa4 {(Qa5+) 46.34/17 3} Qf8+ {(Qb7) 1.65/10 2} 31. Ke4 {
45.71/17 3} Qe7 {(Qd6) 1.64/10 1} 32. Qc6+ {46.29/16 3} Kb2 {(Kb4) 1.69/11 2}
33. Qd7 {(Qb6+) 47.26/14 4} Qh4+ {1.82/10 1} 34. Kd5 {46.04/16 3} Qe1 {
(Qh1+) 2.80/10 6} 35. Qg7+ {(Qd6) 49.83/17 2} Ka3 {(Kb3) 2.82/10 1} 36. e7 {
50.71/20 2} Qh1+ {3.53/11 3} 37. Ke6 {48.99/8 3} Qe4+ {3.53/11 1} 38. Kd7 {
(Qe5) 48.01/11 4} Qa4+ {2.48/10 1} 39. Kc7 {46.98/13 3} Qa5+ {(Qc2+) 2.45/10 0}
40. Kd6 {(Kc6) 47.14/13 2} Qb4+ {(Qd2+) 2.45/11 1} 41. Kd5 {43.86/17 3} Qb7+ {
(Kb3) 0.85/11 2} 42. Kc4 {(Kc5) 45.99/17 10} Qb3+ {(Qb4+) 0.01/8 0} 43. Kc5 {
(Kd4) 0.00/19 2} Qb4+ {0.00/12 0} 44. Kc6 {0.00/21 3} Qc4+ {0.00/13 0} 45. Kd6
{(Kb6) 0.00/23 3} Qb4+ {0.00/13 0} 46. Ke6 {(Kd5) 0.00/23 2} Qe4+ {0.00/14 0}
47. Qe5 {47.32/14 3} Qg6+ {(Qxe5+) 3.53/12 1} 48. Qf6 {(Kd7) 47.21/14 5} Qe8 {
(Qg8+) 3.22/8 0} 49. Qc3+ {47.43/14 3} Ka2 {(Ka4) 3.29/9 0} 50. Qb4 {
(Qc2+) 47.51/14 6} Qg6+ {(Qc8+) 3.29/8 0} 51. Kd7 {46.49/16 2} Qf5+ {
(Qd3+) 3.30/8 0} 52. Kc6 {45.72/17 4} Qc8+ {(Qf3+) 3.38/9 0} 53. Kd6 {
46.76/17 2} Qa6+ {3.38/10 0} 54. Ke5 {46.69/17 4} Qc6 {3.38/10 0} 55. Qd6 {
(Qd2+) 46.49/16 5} Qe8 {(Qc3+) 3.35/7 0} 56. Qd5+ {(Kd5) 46.51/16 2} Kb1 {
(Ka3) 3.45/8 0} 57. Qb3+ {47.65/14 2} Ka1 {(Kc1) 3.52/7 0} 58. Qa3+ {
(Qe6) 46.63/16 4} Kb1 {3.12/9 0} 59. Qd3+ {(Qb4+) 46.56/17 2} Kb2 {
(Kc1) 3.13/8 0} 60. Ke6 {(Kd6) 46.64/17 2} Kc1 {(Qg8+) 3.32/9 0} 61. Qd8 {
(Kd6) 46.83/18 10} Qg6+ {3.05/12 96} 62. Kd5 {(Kd7) 43.13/20 26} Qf5+ {
4.64/11 65} 63. Kd6 {(Kc6) 43.91/20 8} Qf4+ {(Qf6+) 4.64/11 44} 64. Ke6 {
(Kc6) 44.65/21 8} Qg4+ {(Qh6+) 4.64/15 30} 65. Ke5 {42.31/21 10} Qg3+ {
(Qg5+) 4.64/13 20} 66. Kf5 {(Kd5) 43.06/21 8} Qf3+ {(Qg7) 3.05/14 1} 67. Kg5 {
(Kg6) 42.16/22 13} Qg3+ {3.05/15 1} 68. Kf6 {42.31/22 7} Qf4+ {(Qh4+) 3.05/16 2
} 69. Kg6 {(Kg7) 42.31/22 7} Qg4+ {3.05/17 13} 70. Kf7 {42.73/21 6} Qh5+ {
3.05/15 8} 71. Kf6 {(Kg8) 42.92/20 6} Qf3+ {(Qh6+) 3.85/17 6} 72. Kg7 {
42.33/22 6} Qg4+ {(Qc3+) 3.05/14 4} 73. Kf8 {42.93/20 6} Qf5+ {4.65/12 2} 74.
Ke8 {42.71/21 6} Qe6 {(Qh5+) 4.04/12 1} 75. Qd3 {(Qd4) 46.76/16 3} Qb6 {
(Qg8+) 4.04/10 0} 76. Kf7 {(Qc3+) 46.71/16 6} Qf2+ {3.31/9 2} 77. Ke6 {
45.70/19 5} Qb6+ {5.11/8 0} 78. Qd6 {45.88/20 14} Qb5 {(Qe3+) 4.64/8 0} 79. Qd5
{(Qd7) 47.30/15 6} Qb6+ {(Qxd5+) 4.45/11 0} 80. Kf5 {46.59/17 5} Qf2+ {
4.45/11 0} 81. Kg6 {46.40/19 3} Qc2+ {(Qg3+) 4.45/12 0} 82. Kg5 {
(Kf6) 45.64/17 7} Qc8 {(Qa4) 3.38/8 0} 83. Qf7 {(Qh1+) 0.00/17 5} Qc5+ {
0.05/5 0} 84. Kf6 {(Kh6) 0.00/26 4} Qc3+ {(Qf2+) 0.00/12 0} 85. Ke6 {0.00/28 2}
Qc6+ {0.00/16 0} 86. Ke5 {Draw accepted 0.00/63 0} 1/2-1/2

Funnily without bases no problem:

[Event "basetest 2."]
[Site "?"]
[Date "2009.03.15"]
[Round "3"]
[White "Cyclone 3.4"]
[Black "Rybka 3 1-cpu 32-bit"]
[Result "1-0"]
[Annotator "2.58;2.23"]
[SetUp "1"]
[FEN "8/4q3/4P3/4Q3/3K2k1/8/8/8 w - - 0 1"]

1. Kc4 {2.58/13 7} Kh3 {2.23/13 10} 2. Qe4 {(Qh5+) 2.60/13 7} Kg3 {
(Kh2) 2.42/15 14} 3. Kb3 {(Qd5) 2.82/14 6} Kh3 {2.66/14 9} 4. Qf3+ {2.82/14 7}
Kh4 {2.76/16 6} 5. Qf7 {(Qf4+) 2.84/14 10} Qd6 {2.77/15 7} 6. e7 {2.84/14 5}
Qb6+ {(Qd3+) 2.82/17 5} 7. Kc4 {2.92/14 5} Qa6+ {(Qc6+) 3.47/17 15} 8. Kd4 {
(Kc5) 2.92/15 8} Qd6+ {(Qb6+) 3.47/16 74} 9. Ke4 {3.31/13 8} Qb4+ {3.47/16 50}
10. Kf5 {3.31/12 7} Qg4+ {3.47/15 34} 11. Ke5 {3.51/12 7} Qg5+ {3.47/15 23} 12.
Ke4 {9.97/12 7} Qg2+ {5.07/11 16} 13. Ke3 {(Qf3) 9.79/12 7} Qg3+ {2.93/12 11}
14. Qf3 {10.18/13 6} Qe1+ {2.93/11 7} 15. Kf4 {10.20/13 4} Qc1+ {2.93/10 5} 16.
Qe3 {10.20/13 6} Qf1+ {10.02/8 2} 17. Ke4 {10.28/13 6} Qg2+ {(Qb1+) 10.34/7 3}
18. Kd3 {10.44/13 7} Qf1+ {10.34/7 1} 19. Kd2 {(Kd4) 10.44/13 6} Qc4 {
(Qg2+) 10.30/6 1} 20. e8=Q {11.46/11 5} Qa2+ {(Qb4+) 10.34/2 0} 21. Kd1 {
11.65/12 5} Qb1+ {(Qa1+) 10.45/3 0} 22. Ke2 {(Qc1) #9/12 5} Qc2+ {10.72/3 0}
23. Qd2 {(Kf3) #8/11 4} Qxd2+ {10.49/2 0} 24. Kxd2 {10.61/7 5} Kg5 {
(Kg4) 10.61/2 0} 25. Qe6 {(Qb5+) #6/10 4} Kf4 {10.59/2 0} 26. Ke2 {
(Kd3) #5/29 3} Kg3 {#7/3 0} 27. Qe4 {(Ke3) #4/63 1} Kh2 {#6/3 0} 28. Kf3 {
#3/63 1} Kg1 {#3/3 0} 29. Qg6+ {(Qd4+) #2/63 0} Kf1 {(Kh2) #2/3 0} 30. Qb1# {
#1/63 0} 1-0

BTW Daniel, what do You mean? Which dll we should use for single, multi/64b systems actually? And thanks for great work!

Jouni

[Daniel Shawul]

Hello Jouni

The bitbases only give you a win / lose / draw information so the engine itself has to effect the win. Sometimes in positions like you shown that could be difficult. Most of the time the engine uses part of the search depth say (2/3) for that purpose and in the tips it just returns the egbb score without doing search. So this way it would be able to progress most of the time.
Unfortunately the search depth was not enough for the particular for this case which was why it was able to mate without them.

As to your other question, when 64bit and 32bit engines ,which use egbbs, are run on a 64bit computer then we need separate egbbdlls for those. The 32bit engines will load egbbdll and the 64bit engines egbbdll64. The default egbbdlls are compiled with SMP on so we don't need separate compiles for those.

regards,
daniel

[BBauer]

Hallo Daniel,
yes this dll works for me.
Do you also have a egbbso.so or sources for a 64-bit linux?
Kind regards
Bernhard

[Denis P. Mendoza]

This link: http://dshawul.googlepages.com/egbbdll.dll didn't fix the crashes using 2-cpu (egbb enabled engines). EGBB usage works well only on single cpu mode (in my case). I've tried all the Togas using the new bitbases in Arena and all failed on 2-cpu at some point on the epd endgame tests.

What code fix is needed to work on this problem? The only thing I saw in the code (from src at DC site) was "Define PARALLEL to enable multi threading" but doesn't have an effect when used during compiling.

Thanks.

[Daniel Shawul]

Hello Denis
It should work now. I have updated it once probably after you downloaded it. In the other thread, Bernhard also mentioned it worked for him, so please try again. You can also find the source code there http://dshawul.googlepages.com/home.

regards,
Daniel