Systems with sufficient enough ram like tcec for instance, benefit alot from already cached egtb and is much faster to reach than probing it from the ssd.
Current algorithm work great for systems with sufficient ram installed, for systems w/o it will eventually compromise windows stability.
One idea for syzygy is analysis mode. In analysis mode syzygy functions similarly to nalimov - ram is unaffected.
The history of Syzygy tablebases
Moderators: hgm, Rebel, chrisw
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: The history of Syzygy tablebases
Sounds like nonsense. Syzygy doesn't dedicate any RAM to its use; it purely relies on what the OS gives it. If that would go at the expense of other tasks that are dependent on the RAM, and now cannot get it because it was all given away to some low-priority task that did not even request it, the OS would utterly suck.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: The history of Syzygy tablebases
Sure not nonsense to me. I am using WIndows 8.1, I don't know how to tell OS not to mess with idle processes I use later, and are often high RAM consumers, which if cached, take forever to load again. Sure, I am not a disciplined PC user, I reboot my PC once a week-a-month, and re-use some idle tasks after awhile.hgm wrote:Sounds like nonsense. Syzygy doesn't dedicate any RAM to its use; it purely relies on what the OS gives it. If that would go at the expense of other tasks that are dependent on the RAM, and now cannot get it because it was all given away to some low-priority task that did not even request it, the OS would utterly suck.
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: The history of Syzygy tablebases
Sounds like Windows 8.1 has an enormous memory leak.
I still don't see how it would make any difference between using Nalimov and Syzygy, though. In Nalimov everything that is not explicitly cached in the engine would have to come from disk too, and if the OS keeps that in its disk cache, it would eat the memory just the same.
This is not something that can be ascribed to the EGT format, btw. You can just as easily write probing code that uses an internal cache of limited size for Syzygy.
I still don't see how it would make any difference between using Nalimov and Syzygy, though. In Nalimov everything that is not explicitly cached in the engine would have to come from disk too, and if the OS keeps that in its disk cache, it would eat the memory just the same.
This is not something that can be ascribed to the EGT format, btw. You can just as easily write probing code that uses an internal cache of limited size for Syzygy.
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: The history of Syzygy tablebases
If that cache can be rewritten throughout the game based on relevance, that sounds like a good idea. E.g limted cache at move 40 might not be as useful at move 50. Sorting internal cache by priority & relevance.hgm wrote:GT format, btw. You can just as easily write probing code that uses an internal cache of limited size for Syzygy.
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: The history of Syzygy tablebases
Here is the concept visualized.
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: The history of Syzygy tablebases
But I still don't see how that can help. Everything that is in the internal cache will have to be read through the OS from the disk, and the OS will then cache that too. The RAM used by the internal cache is just a duplicat. Even if you flush the internal cache, to make space for new data, it would not inspire the OS to flush that same data from the system cache.
The problem is that the OS uses the 25% you depict in any way it wants. Applications don't have any say in that. So if the OS prefers to use it for storing a duplicate of what is in your internal Syzygy cache, at the expense of code memory for waiting other applications, you are dead. The OS has no idea that you are caching all that data internally. It just know that it is something you have been reading. The only thing to prevent that is not read any data at all.
The problem is that the OS uses the 25% you depict in any way it wants. Applications don't have any say in that. So if the OS prefers to use it for storing a duplicate of what is in your internal Syzygy cache, at the expense of code memory for waiting other applications, you are dead. The OS has no idea that you are caching all that data internally. It just know that it is something you have been reading. The only thing to prevent that is not read any data at all.
-
- Posts: 919
- Joined: Tue Nov 24, 2015 9:11 pm
- Location: upstate
Re: The history of Syzygy tablebases
Nordlandia wrote:Systems with sufficient enough ram like tcec
Sufficient ram, sufficient ram -- could you be more vague, please?Nordlandia wrote:Current algorithm work great for systems with sufficient ram installed
You nailed it. That's what it amounts to in effect, and the same goes for Win7. I haven't seen how Windoze caching performs under Win10 (had it replaced with Win7 within 24 hours on the only box that came with it) but I doubt it's any better.hgm wrote:Sounds like Windows 8.1 has an enormous memory leak.
Last year I looked for a solution to RAM filling up to 99% during extended analysis sessions with 6-men Syzygy and didn't find one. Programs that are supposed to limit Windoze cache, registry tweaks -- none of them worked. Perhaps they used to at some point and then some Windoze update broke them.
The problem is particularly evident on the two of my boxes that have "only" 32GB of RAM. My main desktop has 48 and it's much less pronounced. In fact, I wasn't aware of the issue until I got one of the 32GB ones; if I had noticed any slowdown I must have put it down to some other factor.
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: The history of Syzygy tablebases
Compared to Syzygy TBs, Nalimov needs 17x as much RAM to get anywhere near a decent performance. Nalimov devours RAM, it just doesn't show up as part of the process's shared memory in the task manager.Robert Pope wrote:Not really. Suppose you have two programs: one does tons of disk thrashing if it doesn't have 8GB of RAM available, the other doesn't:hgm wrote:"Does not eat RAM during analysis"... Isn't that the same thing as saying "Fails to profit from RAM that otherwise would be idle, and just leaves that go to waste"?I would say "B plays poorly with less than 8GB of RAM". I would never say "A isn't taking as good advantage of the extra RAM".Code: Select all
4GB 8GB A 2400 2450 B 1800 2480
To get rid of disk thrashing, disable the page file. Problem solved.
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: The history of Syzygy tablebases
Page file advice is good solution but keeping windows oscillating between 97-98% may inflict compromised windows stability.