Page 1 of 1

MEM_LARGE_PAGES

Posted: Wed Sep 18, 2013 8:33 pm
by Cardoso
Hi,
I've been trying to make the hash table to be allocated using MEM_LARGE_PAGES, but I haven't been able to do it successfully.
I have already set the right account privileges manually and run my engine just right after reboot, but it just doesn't work, it allways gives the error:
"GC large_page: No required number of large pages found. Please reboot.....".
I use Windows 7 x64, and a core i7 i950, and 24Gb RAM.
The function to use in the code below is:
void* alloc_large_pages(size_t size, const char* hint) ;
the second parameter 'hint' has no use.
Does anyone have managed to do this successfully?

best regards,
Alvaro

Code: Select all

// make baseNumber multiple of number, without decreasing baseNumber
size_t myMakeMultiple(size_t baseNumber, size_t number) {
	size_t multiple;
	multiple = baseNumber + number - 1;
	multiple -= (multiple % number);
	return multiple;
}

bool set_privilege(HANDLE process, LPCTSTR priv_name, bool is_enable)
{
  HANDLE token;
  TOKEN_PRIVILEGES tp;
  BOOL res = OpenProcessToken(process, TOKEN_ADJUST_PRIVILEGES, &token);
  if(!res){
    return FALSE;
  }
 
  tp.PrivilegeCount = 1;
  tp.Privileges[0].Attributes = is_enable ? SE_PRIVILEGE_ENABLED : 0;
 
  res = LookupPrivilegeValue( NULL, priv_name, &tp.Privileges[0].Luid);
  if(!res){
    CloseHandle(token);
    return FALSE;
  }
 
  if (AdjustTokenPrivileges( token, FALSE, &tp, 0, NULL, 0) == ERROR_NOT_ALL_ASSIGNED) {
    CloseHandle(token);
    return FALSE;
  }
  return TRUE;
}

bool obtain_lock_memory_priv()
{
  HANDLE process = GetCurrentProcess();
  return set_privilege(process, SE_LOCK_MEMORY_NAME, TRUE);
}

bool release_lock_memory_priv()
{
  HANDLE process = GetCurrentProcess();
  return set_privilege(process, SE_LOCK_MEMORY_NAME, FALSE);
}

void* alloc_large_pages(size_t size, const char* hint)
{
  void* alloc_addr = NULL;
  bool lock_memory_enable = obtain_lock_memory_priv();
  
 
  if(lock_memory_enable){
    size_t page_min = GetLargePageMinimum();
    size = myMakeMultiple(size, page_min);
    alloc_addr = VirtualAlloc(NULL, size, MEM_RESERVE | MEM_COMMIT | MEM_LARGE_PAGES, PAGE_READWRITE);
    release_lock_memory_priv();
    if(alloc_addr == NULL){
      Print_to_logfile("GC large_page: No required number of large pages found. Please reboot.....");
      return NULL;
    }else
      return alloc_addr;
  }else{
    Print_to_logfile("GC large_page: Check that you have permissions:\nGC large_page: Control Panel->Administrative Tools->Local Security Settings->->User Rights Assignment->Lock pages in memory.\nGC large_page: Start VM as soon after reboot as possible, because large pages become fragmented and unusable after a while.\nGC large_page: Heap size should be multiple of large page size.");
    return NULL;
  }
}


Re: MEM_LARGE_PAGES

Posted: Wed Sep 18, 2013 9:52 pm
by Joost Buijs
It is working without problems over here.
I use Windows 7 x64, and a core i7 3960X with 16 GB RAM.

This basically is the code I use:

Code: Select all

void Transtable::allocate(int_t sizeTT, int_t sizePT)
{
	numEntrys  = &#40;1 << 20&#41; * sizeTT / &#40;sizeof&#40;transentry_t&#41; * clusterSize&#41;;
	pawnEntrys = &#40;1 << 20&#41; * sizePT / sizeof&#40;pawnentry_t&#41;;

	SIZE_T LargePageMinimum = GetLargePageMinimum&#40;);

	if &#40;LargePageMinimum > 0&#41; &#123;
		//printf&#40;"info string Large page minimum %d\n", LargePageMinimum&#41;; 
		TOKEN_PRIVILEGES tp;
		HANDLE hToken;
	    OpenProcessToken&#40;GetCurrentProcess&#40;), TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY, &hToken&#41;;
	    LookupPrivilegeValue&#40;NULL, "SeLockMemoryPrivilege", &tp.Privileges&#91;0&#93;.Luid&#41;;
	    tp.PrivilegeCount = 1;
        tp.Privileges&#91;0&#93;.Attributes = SE_PRIVILEGE_ENABLED;
	    AdjustTokenPrivileges&#40;hToken, FALSE, &tp, 0, NULL, 0&#41;;

		// allocate the TT
		pTT = &#40;transentry_t*&#41;VirtualAlloc&#40;NULL, sizeof&#40;transentry_t&#41; * clusterSize * numEntrys, MEM_COMMIT | MEM_RESERVE | MEM_LARGE_PAGES, PAGE_READWRITE&#41;;
		if &#40;pTT&#41; printf&#40;"info string Transposition table allocated in large page memory\n");

		// allocate the PT
		for &#40;int_t i = 0; i < CPUS; i++) &#123;
			pPT&#91;i&#93; = &#40;pawnentry_t*) VirtualAlloc&#40;NULL, sizeof&#40;pawnentry_t&#41; * pawnEntrys, MEM_COMMIT | MEM_RESERVE | MEM_LARGE_PAGES, PAGE_READWRITE&#41;;
			if &#40;pPT&#91;i&#93;) printf&#40;"info string Pawn table&#91;%d&#93; allocated in large page memory\n", i&#41;;
		&#125;
	&#125;

	if &#40;pTT == NULL&#41; pTT = &#40;transentry_t*)_aligned_malloc&#40;sizeof&#40;transentry_t&#41; * clusterSize * numEntrys, CACHE_LINE&#41;;

	for &#40;int_t i = 0; i < CPUS; i++)
		if &#40;pPT&#91;i&#93; == NULL&#41; pPT&#91;i&#93; = &#40;pawnentry_t*)_aligned_malloc&#40;sizeof&#40;pawnentry_t&#41; * pawnEntrys, CACHE_LINE&#41;; 
&#125;

Re: MEM_LARGE_PAGES

Posted: Wed Sep 18, 2013 10:31 pm
by Steve Maughan
Hi Joost,

Can I ask a couple of basic questions?

What are Mem Large Pages?

And why are they important?

Cheers,

Steve

Re: MEM_LARGE_PAGES

Posted: Wed Sep 18, 2013 10:54 pm
by Joost Buijs
Steve Maughan wrote: What are Mem Large Pages?

And why are they important?
Hi Steve,

Memory accesses are translated from virtual to physical addresses, this is done by means of a page table. Reading the page table is rather slow, that's why there is a fast translation cache inside the CPU called the TLB (Translation Lookaside Buffer). The whole point is that this buffer has a limited size and with larger pages you will get more TLB hits because there are less page adresses to store in the cache. This increases the speed of memory writes and reads.

I see an increase in speed of about 10% with large pages enabled, this is because I use the transposition table in quiescence. If you don't use the TT in quiescence it won't help much I suppose.

Re: MEM_LARGE_PAGES

Posted: Wed Sep 18, 2013 11:51 pm
by Joost Buijs
Cardoso wrote:Hi,
I've been trying to make the hash table to be allocated using MEM_LARGE_PAGES, but I haven't been able to do it successfully.

Does anyone have managed to do this successfully?
I forgot to mention that you have to run your engine with administrator rights, otherwise it won't work.

Re: MEM_LARGE_PAGES

Posted: Thu Sep 19, 2013 5:32 am
by Steve Maughan
Thanks!

Maverick uses a move dictionary data structure, which is quite memory intensive. So I might give it a go,

Thanks again,

Steve

Re: MEM_LARGE_PAGES

Posted: Thu Sep 19, 2013 7:01 am
by Joost Buijs
It is a bit cumbersome to use large pages.

You can find documentation here:
http://msdn.microsoft.com/en-us/library ... s.85).aspx

You have to enable 'lock pages in memory':
http://msdn.microsoft.com/en-us/library/ms190730.aspx

Most of the time you have to restart Windows before you load the engine, because after some time the memory becomes too fragmented.

And you have to run your engine with administrator rights.

It does not work for 32 bit systems and 32 bit engines.

Re: MEM_LARGE_PAGES

Posted: Thu Sep 19, 2013 7:22 am
by Joost Buijs
Steve Maughan wrote: Maverick uses a move dictionary data structure, which is quite memory intensive. So I might give it a go,
Steve
It is only useful when your data structure is much larger than the default (4kB) page size.

Re: many thanks, I'll test it (NT)

Posted: Thu Sep 19, 2013 12:16 pm
by Cardoso
Joost Buijs wrote:
Cardoso wrote:Hi,
I've been trying to make the hash table to be allocated using MEM_LARGE_PAGES, but I haven't been able to do it successfully.

Does anyone have managed to do this successfully?
I forgot to mention that you have to run your engine with administrator rights, otherwise it won't work.
ok many thanks, I'll test your code tonight,
best regards,
Alvaro