How is work on 8-man tablebases progressing?

Discussion of chess software programming and technical issues.

Moderator: Ras

User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: How is work on 8-man tablebases progressing?

Post by hgm »

Koistinen wrote: Mon Sep 16, 2024 2:11 pmI computed the size requirements without and with reducing using permutations of same pieces to illustrate.
Note that the number for reducing is just an approximation, dividing by the number of permutations of like pieces for those endgames.
Last on each line is the percentage of space you need if you don't use like-piece symmetry.

3: 524288 524288 100
4: 306184192 290979840 105
5: 96921976832 79135331668 122
6: 22360673484800 14305093222384 156
7: 4218203345518592 1947456033067182 216
8: 690330986839277568 213529294167604807 323
9: 101523728307603898368 19652448336691703337 516
10: 13728689646745625296896 1561065282681230985761 879
11: 1734484955477896291418112 109176580993481410741492 1588
12: 207127250281663267673735168 6824378818319881564216564 3035
I suppose the largest reduction is from permuting mupltiple Pawns, and when you would work by P-slice, you would exploit that automatically. So the only real question is how much you would save due to equivalence of permutations of the other pieces.
mosfel24
Posts: 12
Joined: Thu May 02, 2024 4:24 pm
Full name: Moshe Felman

Re: How is work on 8-man tablebases progressing?

Post by mosfel24 »

hgm wrote: Sun Sep 15, 2024 10:32 pm
syzygy wrote: Fri Sep 13, 2024 6:34 pm Well, at least I don't see any point in spending half the effort to do something half.
That is already a really weird attitude; half the job done without introducing any loss of efficiency should be considered a good start.

It becomes an even more counter-productive stance when you apply it to doing, say, 99% of the job for 1% of the effort. Especially if the remaining 1% concerns only cases that no one is interested in.

Your pi metaphor sucks. You should have said "no point in calculating the first 12 digits of pi when you cannot calculate the next million digits". Engineers would not agree...
Engineers seem fine with chess engines, which do 99% of the job for 1% of the effort.
Koistinen
Posts: 29
Joined: Sun May 23, 2021 10:05 pm
Location: Stockholm, Sweden
Full name: Urban Koistinen

Re: How is work on 8-man tablebases progressing?

Post by Koistinen »

hgm wrote: Mon Sep 16, 2024 5:03 pm I suppose the largest reduction is from permuting mupltiple Pawns, and when you would work by P-slice, you would exploit that automatically. So the only real question is how much you would save due to equivalence of permutations of the other pieces.
I did the same computation but reducing pawn symmetry on both sides the same way:
3: 524288 524288 100
4: 301465600 290979840 103
5: 92467625984 79135331668 116
6: 20567155539968 14305093222384 143
7: 3735559724230246 1947456033067182 191
8: 588698109703146693 213529294167604807 275
9: 83439994934544916195 19652448336691703337 424
10: 10887163846326643716271 1561065282681230985761 697
11: 1328958932314921409984354 109176580993481410741492 1217
12: 153546176647563470947658810 6824378818319881564216564 2249
Leo
Posts: 1103
Joined: Fri Sep 16, 2016 6:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: How is work on 8-man tablebases progressing?

Post by Leo »

I found this about 10 years ago.

Many show interest in what is to expect from 8-man endings. First, take note that the longest 6-man mate took 262 moves (KRN-KNN). Moving to 7-man endings doubled this value. Second, 8-man tablebases include much more endings with both sides having relatively equal strength. All this gives us a strong hope to discover a mate in more than 1000 moves in one of 8-man endgames. Unfortunately the size of 8-man tablebases will be 100 times larger than the size of 7-man tablebases. To fully compute them, one will need about 10 PB (10,000 TB) of disk space and 50 TB of RAM. Only the top 10 supercomputers can solve the 8-man problem in 2014. The first 1000-move mate is unlikely to be found until 2020 when a part of a TOP100 supercomputer may be allowed to be used for solving this task.
Advanced Micro Devices fan.
Viz
Posts: 223
Joined: Tue Apr 09, 2024 6:24 am
Full name: Michael Chaly

Re: How is work on 8-man tablebases progressing?

Post by Viz »

Jouni wrote: Thu Sep 12, 2024 5:31 pm I think 8-man can give 0-5 elo gain like 7-man. No practical benefit. Only academic interest. I don't use even 6-man. If there is interesting position I check DTM from Shredderchess :) .
You are wrong.
With current technology 8 man will lose elo because they will slow down engine a lot during probing because of their enormous size.
syzygy
Posts: 5696
Joined: Tue Feb 28, 2012 11:56 pm

Re: How is work on 8-man tablebases progressing?

Post by syzygy »

hgm wrote: Sun Sep 15, 2024 10:32 pm
syzygy wrote: Fri Sep 13, 2024 6:34 pm Well, at least I don't see any point in spending half the effort to do something half.
That is already a really weird attitude; half the job done without introducing any loss of efficiency should be considered a good start.
It is not doing half the job. It is doing the job half.
It becomes an even more counter-productive stance when you apply it to doing, say, 99% of the job for 1% of the effort. Especially if the remaining 1% concerns only cases that no one is interested in.
I am fine with doing 99% of the job. I don't care for doing the whole job with 99% correctness.
Your pi metaphor sucks. You should have said "no point in calculating the first 12 digits of pi when you cannot calculate the next million digits". Engineers would not agree...
But I did not say that, as you noticed. My pi metaphor was spot on.

Calculating the first 12 digits of pi as a first step towards calculating the first trillion digits is like calculating the 3/4/5-piece tables as a first step towards calculating the 8-piece tables. It is a necessary step. It is also necessary to do it with 100% accuracy. There is no point in approximately getting 99.99% of the 5-piece tables right "at half the effort". There is no point in getting 11 of the first 12 or 999 of the first 1000 digits of pi right.

Correctly calculating a well-defined subset of the 8-men tables, for example all positions with blocked pawns, could certainly be potentially worth the effort. Approximately calculating the value of 8-men tables, on the other hand, seems silly to me (unless you do it to solve a specific position or to solve some other problem, and if you have determined that your approximation will do for that specific purpose -- as I already wrote).

Now it would be nice if your next reply would not twist my words just so that you can pretend to outsmart me. Note that I am not twisting your words. So be better. Thx.
syzygy
Posts: 5696
Joined: Tue Feb 28, 2012 11:56 pm

Re: How is work on 8-man tablebases progressing?

Post by syzygy »

Leo wrote: Tue Sep 17, 2024 7:33 pm I found this about 10 years ago.

Many show interest in what is to expect from 8-man endings. First, take note that the longest 6-man mate took 262 moves (KRN-KNN). Moving to 7-man endings doubled this value. Second, 8-man tablebases include much more endings with both sides having relatively equal strength. All this gives us a strong hope to discover a mate in more than 1000 moves in one of 8-man endgames. Unfortunately the size of 8-man tablebases will be 100 times larger than the size of 7-man tablebases. To fully compute them, one will need about 10 PB (10,000 TB) of disk space and 50 TB of RAM. Only the top 10 supercomputers can solve the 8-man problem in 2014. The first 1000-move mate is unlikely to be found until 2020 when a part of a TOP100 supercomputer may be allowed to be used for solving this task.
The idea that the longest mate must double with every extra piece (as someone has "predicted" in a paper) is really based on nothing.

I believe most pawnless candidates for longest mates have already been computed, and the longest mate record was increased only by a bit. See here:
https://en.chessbase.com/post/8-piece-e ... -interview

Mate in 584. It seems the forum's pgn viewer stops at 300 moves.
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: How is work on 8-man tablebases progressing?

Post by hgm »

syzygy wrote: Tue Sep 17, 2024 10:33 pm
hgm wrote: Sun Sep 15, 2024 10:32 pm
syzygy wrote: Fri Sep 13, 2024 6:34 pm Well, at least I don't see any point in spending half the effort to do something half.
That is already a really weird attitude; half the job done without introducing any loss of efficiency should be considered a good start.
It is not doing half the job. It is doing the job half.
It becomes an even more counter-productive stance when you apply it to doing, say, 99% of the job for 1% of the effort. Especially if the remaining 1% concerns only cases that no one is interested in.
I am fine with doing 99% of the job. I don't care for doing the whole job with 99% correctness.
Your pi metaphor sucks. You should have said "no point in calculating the first 12 digits of pi when you cannot calculate the next million digits". Engineers would not agree...
But I did not say that, as you noticed. My pi metaphor was spot on.

Calculating the first 12 digits of pi as a first step towards calculating the first trillion digits is like calculating the 3/4/5-piece tables as a first step towards calculating the 8-piece tables. It is a necessary step. It is also necessary to do it with 100% accuracy. There is no point in approximately getting 99.99% of the 5-piece tables right "at half the effort". There is no point in getting 11 of the first 12 or 999 of the first 1000 digits of pi right.

Correctly calculating a well-defined subset of the 8-men tables, for example all positions with blocked pawns, could certainly be potentially worth the effort. Approximately calculating the value of 8-men tables, on the other hand, seems silly to me (unless you do it to solve a specific position or to solve some other problem, and if you have determined that your approximation will do for that specific purpose -- as I already wrote).

Now it would be nice if your next reply would not twist my words just so that you can pretend to outsmart me. Note that I am not twisting your words. So be better. Thx.
But you are still arguing from the misconception that there would be anything wrong with the correctness of the part that was calculated. Which makes everything you say nonsensical and irrelevant. It's like insisting alpha-beta is no good because you only searched small part of the tree, so that this part must be mostly incorrect.
syzygy
Posts: 5696
Joined: Tue Feb 28, 2012 11:56 pm

Re: How is work on 8-man tablebases progressing?

Post by syzygy »

hgm wrote: Wed Sep 18, 2024 7:44 amBut you are still arguing from the misconception that there would be anything wrong with the correctness of the part that was calculated. Which makes everything you say nonsensical and irrelevant. It's like insisting alpha-beta is no good because you only searched small part of the tree, so that this part must be mostly incorrect.
Are you referring to this post, which is about "usually you will get the correct result" x 10?
viewtopic.php?p=968448#p968448
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: How is work on 8-man tablebases progressing?

Post by hgm »

Apparently not, because I cannot find the word 'correct' in that posting at all...

The point is that in a generally won end-game most P-slices can be solved as total wins even under rules where opponent promotion is an immediate loss, because in practice you won't be able to beat that extra Queen. Only the small fraction of P-slices for which this is not the case you won't be able to solve.

Another way of looking at it, is that instead of a WDL bitbase, one could use a WUL bitbase, where U means 'undefined'. A probing engine would then have to search on when it hits a U position. And guess what, they are already doing that when they hit a D position. So not much changes for the user. Only some of the W or L would now be a U too. Mostly for positions that a game would never reach, such as multiple Pawns on the pre-promotion rank for both sides.