And Intel Optane is coming : https://www.anandtech.com/show/12512/th ... p-review/5Dann Corbit wrote:...
I think it would be fun to have my very own 7 man file set. I have a great big NAS cabinet, and I can just add on 12 TB disks as they get built. I guess that storage costs will continue to fall over time so it may not be as expensive as one might think.
7-men Syzygy attempt
Moderators: hgm, Rebel, chrisw
-
- Posts: 5239
- Joined: Thu Mar 09, 2006 9:40 am
- Full name: Vincent Lejeune
Re: 7-men Syzygy attempt
-
- Posts: 12662
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: 7-men Syzygy attempt
I have two machines with 128 GB, but 1TB RAM is a lot of cash right nowphhnguyen wrote:Look like you have been solving storage problem for 7 men!!!Dann Corbit wrote:Once you get the code fixed, I can do some runs on a high end machine if you like. I just bought a bunch of SSDs including 2x2TB and 5 smaller ones.noobpwnftw wrote:About the "find_val:not found" problem happened at the third reduction, I think this line may need to be changed intoI'm still not quite sure about this.Code: Select all
j = CAPT_CWIN_RED + 1 + i - REDUCE_PLY_RED;
I think it would be fun to have my very own 7 man file set. I have a great big NAS cabinet, and I can just add on 12 TB disks as they get built. I guess that storage costs will continue to fall over time so it may not be as expensive as one might think.
How about RAM? Will you soon have 1 TB?
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 12662
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: 7-men Syzygy attempt
Two Terabyte SSD drives can be bought for $360 now. The prices are finally starting to drop again.syzygy wrote:HDD is fine if they are probed only at the root (although finding the whole winning line can take a while).Nordlandia wrote:Concerning DTM bases. Have you checked whether placing them on regular hard disk (HDD) inflict/cause noticeabe performance drop. Similarly like dtz works fine on HDD without noticeable performance penalty.
Probing DTM during the search really needs them to be on SSD.
Those super-high speed gumstick SSDs can be had for abouit $400 per Terabyte. Probably not necessary for using the SSD files, but I bet they are great for building them.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: 7-men Syzygy attempt
Maybe PCIe or NVMe types of SSDs is preferable for 7-man syzygy.
I'm worried that regular consumer SSDs cam reduce engine speed if probing 7-man. Probing full 7-man set can inflict moderate speed penalty compared to low/slight speed penalty of 6-man.
Dann Corbit: if Ronald's 500 GB estimation for DTM bases is correct, 1 TB SSD is sufficient enough for syzygy tablebases.
500 + 150 = 650. 350 GB available for something else.
I'm worried that regular consumer SSDs cam reduce engine speed if probing 7-man. Probing full 7-man set can inflict moderate speed penalty compared to low/slight speed penalty of 6-man.
Dann Corbit: if Ronald's 500 GB estimation for DTM bases is correct, 1 TB SSD is sufficient enough for syzygy tablebases.
500 + 150 = 650. 350 GB available for something else.
-
- Posts: 12662
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: 7-men Syzygy attempt
He is working on DTM. If those fit on 500GB for 7 men, I will be more than a little bit surprised. In fact astonished is not enough.Nordlandia wrote:Maybe PCIe or NVMe types of SSDs is preferable for 7-man syzygy.
I'm worried that regular consumer SSDs cam reduce engine speed if probing 7-man. Probing full 7-man set can inflict moderate speed penalty compared to low/slight speed penalty of 6-man.
Dann Corbit: if Ronald's 500 GB estimation for DTM bases is correct, 1 TB SSD is sufficient enough for syzygy tablebases.
500 + 150 = 650. 350 GB available for something else.
I also guess that the win from a 7 man probe hit is much greater than 6 man, which is already better than 5 man.
In ten years a TB of RAM will be affordable on desktop PCs for the average consumer.
And if the probes cost too much, then reduce the fraction of times that you probe. For instance, probing only at the root is cheap, and probing only at the root and at pv nodes is also cheap.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 5239
- Joined: Thu Mar 09, 2006 9:40 am
- Full name: Vincent Lejeune
Re: 7-men Syzygy attempt
I just found a video about new Optane products here :Vinvin wrote:And Intel Optane is coming : https://www.anandtech.com/show/12512/th ... p-review/5Dann Corbit wrote:...
I think it would be fun to have my very own 7 man file set. I have a great big NAS cabinet, and I can just add on 12 TB disks as they get built. I guess that storage costs will continue to fall over time so it may not be as expensive as one might think.
-
- Posts: 684
- Joined: Sun Nov 08, 2015 11:10 pm
- Full name: Bojun Guo
Re: 7-men Syzygy attempt
I did change the stat limits according to their pawn version like so.
Now MAX_PLY=768 should be sufficient for 7-men TBs, I will verify the generation process works without problem before moving on to the compression part.
Now MAX_PLY=768 should be sufficient for 7-men TBs, I will verify the generation process works without problem before moving on to the compression part.
-
- Posts: 5647
- Joined: Tue Feb 28, 2012 11:56 pm
Re: 7-men Syzygy attempt
I will add my 16-bit permutation and compression code to github, but I will probably not have time before the weekend.noobpwnftw wrote:I did change the stat limits according to their pawn version like so.
Now MAX_PLY=768 should be sufficient for 7-men TBs, I will verify the generation process works without problem before moving on to the compression part.
The DTZ format has to be adapted to allow 16-bit values in the "map"' array. It would be possible to do this just for those tables that need it, so the tables already generated remain valid.
For pawns it might be good to make a bigger change: compressing data per rank instead of per file. That might lead to somewhat better compression, but it is also a bit more complicated and it will need some amount of programming. I have done this for DTM, but in a kind of brute force way (generating the whole table at once, which is not an option for 7 pieces). I guess the generation part can still be done per file.
-
- Posts: 2582
- Joined: Tue Aug 30, 2016 8:19 pm
- Full name: Rasmus Althoff
Re: 7-men Syzygy attempt
Btw, in tbcore.c open_tb(), I think there is a buffer overflow waiting to happen when copying over the path names.
-
- Posts: 684
- Joined: Sun Nov 08, 2015 11:10 pm
- Full name: Bojun Guo
Re: 7-men Syzygy attempt
Thank you, and I think it is possible to differentiate between file formats with the fifth byte in the header or just use another magic value.I will add my 16-bit permutation and compression code to github, but I will probably not have time before the weekend.
With your new 16-bit compression code available, I'm pretty confident that full set of 7-men Syzygy will be available soon.
I already have the necessary hardware(both steps in physical RAM without --disk) and will provide hosting for all the resulting files.