Compiling Stockfish dev for Large Pages

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
schack
Posts: 123
Joined: Thu May 27, 2010 1:32 am
Contact:

Compiling Stockfish dev for Large Pages

Post by schack » Tue Sep 26, 2017 4:08 pm

Hi - does anyone know how to compile the dev versions of Stockfish (using mingw / gcc) for large pages? I'm trying to do this for Ryzen chips, but on a Haswell computer, if this matters.

Thanks.

tpoppins
Posts: 919
Joined: Tue Nov 24, 2015 8:11 pm
Location: upstate

Re: Compiling Stockfish dev for Large Pages

Post by tpoppins » Tue Sep 26, 2017 7:08 pm

See my reply in the "How to add LP to CorChess?" thread from April for the answer.

schack
Posts: 123
Joined: Thu May 27, 2010 1:32 am
Contact:

Re: Compiling Stockfish dev for Large Pages

Post by schack » Tue Sep 26, 2017 7:47 pm

Ah, I suspect we have hit the limit of my capabilities. :) Patching that many files for each compile is perhaps a step too far for this non-programmer.

Thank you for the pointer, though. Now I know what I need to work up to!

tpoppins
Posts: 919
Joined: Tue Nov 24, 2015 8:11 pm
Location: upstate

Re: Compiling Stockfish dev for Large Pages

Post by tpoppins » Wed Sep 27, 2017 4:18 pm

Never think it. With an SVN client like TortoiseSVN it's all nearly automatic. Check Out the source, apply the patch; do Update after a new commit (TSVN can monitor the repo for you, too), the changes from the patch will be incorporated automatically.

Every few months there may be changes to the master branch sufficiently big to break automatic merging on some files. In such cases these files will be highlighted in red in the Update dialog box. Then yeah, you'd have to patch manually. That shouldn't take more than a few minutes, even for a non-programmer.

Here is a complete patch for the current (rev 4868 20170922) master. See the thread linked above for the extra features and the three extra files required for compilation.

Here is a POPCNT compile, with the above-mentioned extra files and the patch in the src subdir. POPCNT is best for Ryzen, BMI will be significantly slower.

syzygy
Posts: 4450
Joined: Tue Feb 28, 2012 10:56 pm

Re: Compiling Stockfish dev for Large Pages

Post by syzygy » Wed Sep 27, 2017 5:14 pm

Or just get a Stockfish variant/clone with LP support.

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: Compiling Stockfish dev for Large Pages

Post by Milos » Wed Sep 27, 2017 6:06 pm

syzygy wrote:Or just get a Stockfish variant/clone with LP support.
Yea, just use Brainfish, it's the fastest compile with added support for LP and NUMA, plus the book, what more you can ask for? :D

User avatar
Eelco de Groot
Posts: 4148
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: Compiling Stockfish dev for Large Pages

Post by Eelco de Groot » Wed Sep 27, 2017 8:26 pm

A book that you can't tune or add moves and lines to is terribly boring, IMO. There is editing software but it will probly be released in 2020 when it has solved chess :? If they think they are going to make money with that... No offence it is their hobby and it generates publicity for other products perhaps.

Brainfish as far as I know does not have Large Pages and special NUMA but you are a user of Brainfish so you should know. I have not seen anyone writing that though. Ronald's CFish does have NUMA and LP and so does asmFish. There is now an asmFish that uses the Cerebellum, in a slightly imperfect way but Thomas Zipproth has written pull request for version that test for moveloops and transpositions I think, or something of that nature. See Stefan Pohl's reporting.
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

syzygy
Posts: 4450
Joined: Tue Feb 28, 2012 10:56 pm

Re: Compiling Stockfish dev for Large Pages

Post by syzygy » Wed Sep 27, 2017 9:28 pm

Eelco de Groot wrote:Brainfish as far as I know does not have Large Pages and special NUMA but you are a user of Brainfish so you should know. I have not seen anyone writing that though.
Brainfish does have LP: http://www.zipproth.de/#Brainfish_exe
"Large Pages" default = false

Activates Large Pages support for the Hashtable with faster access. May work only for some time after a windows restart because of memory fragmentation. The standard memory allocation fallback is activated automatically. Additional user rights like starting in Administrator mode may be necessary.
And Brainfish's misc.cpp includes fragments of NUMA code from Texel, so it seems safe to say it also supports NUMA.

User avatar
Eelco de Groot
Posts: 4148
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: Compiling Stockfish dev for Large Pages

Post by Eelco de Groot » Wed Sep 27, 2017 9:40 pm

Thanks Ronald, I did not know, They are moving away further from Stockfish although they can still say it plays the same moves. It is logical to move to asmFish next. Can an assembler program still be considered GPL? I suppose so if it is possible to create it from some source? It is becoming harder to check clones though. Although, the functionality of a program is maybe shown better in the final assembler and it is not so easy to tinker with assembler. To make something really better. So that is some form of protection.
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

User avatar
Nordlandia
Posts: 2402
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Compiling Stockfish dev for Large Pages

Post by Nordlandia » Thu Sep 28, 2017 4:53 am

Sugar support Large Pages.

I managed to get large page working on one laptop but not on my primary desktop computer.

I did everything in Houdini manual.

Code: Select all

To enable this feature in Windows, you need to modify the Group Policy for your
account:
1. Run: gpedit.msc (or search for "Group Policy").
2.Under "Computer Configuration", "Windows Settings", "Security Settings", "Local
Policies" click on "User Rights Assignment".
3. In the right pane double-click the option "Lock Pages in Memory".
4.Click on "Add User or Group" and add your account or "Everyone".
5. You may have to logoff or reboot for the change to take effect.
For some strange reasons it won't be activated.

Post Reply