Distributing engines for Mac/OSX

[hgm]

In the XBoard help thread it came up how detrimental it is for the Mac user that there is no decent system for packaging engines, so that people often have to limp along with just a bare executable. Which the user then has to unpack and register with the GUI by hand, without any docs whatsoever, (or, if he is very lucky, a README file that will be hidden in a place he would never visit).

If the Mac does not have the benefit of repositories from which you can install programs in a compliant way, (so that they will automatically appear in the engine lists of GUIs that can run them, and their documentation can be displayed by the standard man command or help utility), they should at least be packaged such that the user can mimic this process by unpacking the archives in which the engines are distributed in, say, /usr/local/ . So that included man pages (which then would be accessed through the GUI help functions), data files and auto-install files all go in the compliant places.

Standardization here would also make it possible for GUIs to allow the user to just point at the archive in the engine-install dialog, and order unpacking of the package in the proper place itself. In fact dragging the tar ball on top of the GUI icon should have this effect, and start up the GUI with the engine installed, registered and loaded, ready for use.

Any suggestions from Mac users how we could reach such a state of affairs?

[Evert]

I would suggest treating engines as plugins. That doesn't really help, since I'm not sure where those are supposed to go (I treat my Mac as a generic UNIX system, so I'd dump stuff in /usr/local), but I suspect something like /Library/Chess\ engines/CECP/ might be a reasonable place.

However, to make that work (assuming we'd want to emulate the Linux way of doing this) probably requires that the engine distributor puts it in a .pkg installer.

Alternatively, some programs come on a disk image (.dmg) that users are instructed to mount in the finder, after which they can drag&drop the files to where they want them (typically, applications in /Applications). In this particular case, users would be instructed to drag the folder "Chess engines" to /Library.

Feels a bit clunky though, and it's not as clean as just unzipping an engine binary...

[mvk]

You can treat engines either as plugins or as documents. If treating them as plugins, every application has its own standard. I don't see an easy way to unify plugins. Forget about macports or brew, or any self-conceived directory-dropping tricks. Users just drag a single thing to a single location and that's it. When dragging an .app you can run additional scripts btw, and the same when removing one. But using that mechanism as plugin detection hook looks wrong in my eyes.

If treating engines as documents, through a file type, you can use "Open with" on them, with several applications claiming they can open them (XBoard, Chess Explorer, ...) and one as a preferred default selected by the user. The applications can remember the last directory you used to "open" one as well, or default to "~/Library/Chess Engines".

But it all still feels like a hack. Engines -are- plugins. Not documents. Not applications. I wouldn't mind the application-specific plugin model. Engines are small and nothing is lost if you have to "install" them twice to use them in two applications. Why go through all the trouble to make them available in "all" chess-engine-running applications in one go when nobody uses all engines in "all" interfaces? How many combinations do you really use and how bad is it if you have to do one step for each combination you are intersted in?

If even Chrome and Safari can't agree on such a thing.. Aren't adblockers and media viewer plugins also different for different browsers?

[jdart]

I just use a disk image file (.dmg). An alternative is PackageMaker, which can do install tasks that are more than just file copy. See:

https://developer.apple.com/library/pre ... -TPXREF101

[MikeB]

./username/chess/Engines/enginename.d/engine and tablebases should be in ./username/chess/tb/Nalimov
./username/chess/tb/syzygy etc

The dot d is so engine and folder don't have same name. If everybody follows that script , All engines can be easily automatically or by hand installed and the gui's will know whee to look. Having engines installed in the application like xBoard is horrendous This keeps it simple. You need organization by engine so you don't have a gazzllion engines in one folder and nobody likes to work with long file paths. Bob Hyatt has created a GUIless console app package that lets chess engines play perfectly - xBoard or polyglot play hundreds of thousands games with no errors , ,no crashes , no GUI overHead and Alain created script that displays real time viewing of results of games played with ELO and los. It's amazing how well all the engines run GUI less.

Likewise GUIs should look for openings in ./documents/chess/openings; results should be in ./documents/chess/results ; pgn files should be in ./documents/chess/pgn etc etc

I'm not a UNIX user and all this /local/crap /library/morecrap doesn't make any sense and the paths get way way too long and you can't find then because when you search library - you get a hit of 20 different folders with name library - keep it simple if you want to win users over

[JoshPettus]

I have a couple ideas regarding this I mentioned in the other thread:

Take a folder labeled EngineName.bundle have the engine executable in the root of that folder with any resource files, manual, and logo and have people drag that bundle file to a place where XBoard will find it.
It would have to be /user/home/Documents/ChessEngine or some other sub directory of Documents. I am sorry to say but anything in ~/Library (or any sub directory there of) is simply not accessible and users need to install engines there easily. At least there is a lot of precedence for user documents.
despite being a minor sin.

Or, we can create a new container file .CEngine or something unique and registered by Xboard's plist file. With that we can have xboard automatically register the engine when people double click on the container. I know the dosbox emulator Boxer creates it's own container files called .boxer which are the dos game self-contained in a pseudo executables. There must be a signal we can catch to get the engine directory.

Either way we will have to use the linux style .eng file in the bundle to identify protocol, launch commands and interpreter applications such as python, ruby, java or wine.

I kind of like the second idea, I'll look into it.

[hgm]

./username/chess/Engines/enginename.d/engine and tablebases should be in ./username/chess/tb/Nalimov
./username/chess/tb/syzygy etc

The dot d is so engine and folder don't have same name. If everybody follows that script , All engines can be easily automatically or by hand installed and the gui's will know whee to look.

I suppose that in this model you would put all data files that belong to the engine (ini files, own books, logos, ...) in the same directory as the executable, like in Windows? What does the leading "." represent in your path names?

I am a bit worried that "username" is in the path. This suggests that every user would have to install his own Nalimov EGTs, which seems very wasteful. It would be much more logical to have a user-independent location.

What also seems a worry is how to treat the engine manual files. It is nice if the GUI knows where to look for the engines, so that it can auto-install. But how would the engine documentation get available through the Mac's native help system. Or isn't there such a system?

You mention that people don't like long path names. But the idea of this whole enterprise is to prevent the user would ever have to see the path name at all, and could be unaware of it. Under those circumstances the length of pathnames would hardly matter. On Linux users are not exposed to path names. They just do "apt-get install hachu", or the GUI equivalent, and then they could start XBoard and click 'hachu' in the Load Engine listbox (or type "xboard -fe hachu"), being completely unaware that something like a path name even exists.

[mvk]

Some random thoughts / braindump:

I agree that each "engine" should be a directory of its own or maybe even a bundle. Definitely no scattering of files.

It would be nice to avoid any type of central database files that keep track of what is installed.

I suppose plugins normally wind up in ~Library or /Library through the installation process within the application. We can still make it such that applications use the same plugin location there. Something like "Library/Chess Engines" just feels "right" to me. Maybe there are rules in Library and those should be followed in that case.

As a general strategy, just follow what Chrome, Firefox and Safari are already doing, but try to unify, or monopolise through XBoard.app, the location and structure. Users know how plugins work in other applications and where to look: Preferences -> Extensions.

About multi-user sharing of EGTs: /Library vs ~/Library is already for that in the same way that we have /Applications vs ~/Applications. (There is no /Documents vs ~/Documents...)

Monopolising a new name directly in the user directory (~/chess) seems like a lesser suitable idea to me. That namespace (the visible part of the user folder) is the user's to control and doesn't need more pollution.

Also, suffix-.d and lower-case folder naming are unknown conventions to osx and I would stay away from that for that reason.

[hgm]

What is still not clear to me as a non-Mac-user is how program documentation is organized. On Linux, when I want information on Fairy-Max or Fruit I just type "man fairymax" or "man fruit". Does OSX have something similar to that? Or are all applications responsible for providing their own help, through an item in the Help menu?

In the latter case, as engines are not applications, and thus have no menus of their own, it seems that XBoard (and other GUIs) should have menu items for help on the engines as well as for its own help.

If help would be purely provided through the GUI, it would of course not be a problem if the help/manual files would be bundled with the engine executable in the same directory; there would be no native OSX application that would have to interact with any components in the bundle.

E.g. for Fairy-Max a bundle would contain

Executables:
fairymax
shamax
maxqi
Data files:
fmax.ini
qmax.ini
Logo:
logo.png
Documentation:
manpage
Auto-install info:
xboard/fairymax.eng
xboard/maxqi.eng
xboard/shamax.eng

By placing the bundle in a standard place, like /Library/ChessEngines/ , there would be no need for the engine to broadcast their presence by placing the .eng files in a particular place; the GUI could just scan the /Library/ChessEngines folder for new bundles, and if it finds any, look for the .eng files in the xboard or uci sub-folder to register the engine with the required startup command.

(An alternative to having those sub-folders would be to install CECP and UCI engines in separate folders /Library/ChessEngines/XBoard and /Library/ChessEngines/UCI. But this seemsless desirable, as it expects the user to know what type the engine is. And for dual-protocol engines it would require duplicate instalation of the entire engine, while having both an uci and an xboard sub-folder (possibly with different content in the .eng files) in one bundle is only a minor thing.)

[hgm]

Another loose idea:

If a "bundle" in OSX is just a directory that the desk-top manager handles differently when you double-click it (executing a binary inside it, rather than opening it in the file explorer), wouldn't it be sufficient to enclude a small program/script as the primary executable in an engine bundle, which just moves the bundle to /Library/ChessEngines (and perhaps pops up a message "This engine is now ready for use in your favorite GUI")?

Then people would only have to double-click the bundle after download, and wouldn't have to know anything about /Library/ChessEngines. (Of course we could try to educate them by making the message "This engine has moved itself to /Library/ChessEngines/, making it available for use in any Chess App".