Distributing engines for Mac/OSX

[Evert]

If a "bundle" in OSX is just a directory that the desk-top manager handles differently when you double-click it (executing a binary inside it, rather than opening it in the file explorer), wouldn't it be sufficient to enclude a small program/script as the primary executable in an engine bundle, which just moves the bundle to /Library/ChessEngines (and perhaps pops up a message "This engine is now ready for use in your favorite GUI")?

Then people would only have to double-click the bundle after download, and wouldn't have to know anything about /Library/ChessEngines. (Of course we could try to educate them by making the message "This engine has moved itself to /Library/ChessEngines/, making it available for use in any Chess App".

Well, that's essentially what an installer is supposed to do, isn't it?
As I said, having to launch an installer to install something on OS X always feels a bit weird, since the normal way to operate things is to just drag stuff into the right directory (typically an application bundle into /Applications or ~/Applications).

[MikeB]

./username/chess/Engines/enginename.d/engine and tablebases should be in ./username/chess/tb/Nalimov
./username/chess/tb/syzygy etc

The dot d is so engine and folder don't have same name. If everybody follows that script , All engines can be easily automatically or by hand installed and the gui's will know whee to look.

I suppose that in this model you would put all data files that belong to the engine (ini files, own books, logos, ...) in the same directory as the executable, like in Windows? What does the leading "." represent in your path names?

I am a bit worried that "username" is in the path. This suggests that every user would have to install his own Nalimov EGTs, which seems very wasteful. It would be much more logical to have a user-independent location.

What also seems a worry is how to treat the engine manual files. It is nice if the GUI knows where to look for the engines, so that it can auto-install. But how would the engine documentation get available through the Mac's native help system. Or isn't there such a system?

You mention that people don't like long path names. But the idea of this whole enterprise is to prevent the user would ever have to see the path name at all, and could be unaware of it. Under those circumstances the length of pathnames would hardly matter. On Linux users are not exposed to path names. They just do "apt-get install hachu", or the GUI equivalent, and then they could start XBoard and click 'hachu' in the Load Engine listbox (or type "xboard -fe hachu"), being completely unaware that something like a path name even exists.

Basically we're talking about the difference between theory and practice. In theory, everything runs seamlessly and and with no need for users to interface with things under the hood that have gone wrong or no longer work , because they always work and are totally trouble free. Most apple apps are like that - xBoard is not one of those. It's one where practice is a total disconnect from theory. If I didn't have the capablitiy to get underneath the hood and fix things , I would not be using xBoard because it would simply not work the way I want it to work. You would have to spend a day with me to see what I mean. I don't mind - I'm used to that - I have used xBoard or winboard off and on for 20 years - but except for me , I can't believe users will be drawn to it. Maybe for somebody that is content playing fairymax , it's fine . But that's not me. You mention an enterprise environment , but I'm not an enterprise. I'm a single user on a personal pc , totally different set of wants and needs than enterprise. I mentioned the home directory because that appears to be the safest place to put things without getting into the permission issues that can get nightmarish at time.

[hgm]

If a "bundle" in OSX is just a directory that the desk-top manager handles differently when you double-click it (executing a binary inside it, rather than opening it in the file explorer), wouldn't it be sufficient to enclude a small program/script as the primary executable in an engine bundle, which just moves the bundle to /Library/ChessEngines (and perhaps pops up a message "This engine is now ready for use in your favorite GUI")?

Then people would only have to double-click the bundle after download, and wouldn't have to know anything about /Library/ChessEngines. (Of course we could try to educate them by making the message "This engine has moved itself to /Library/ChessEngines/, making it available for use in any Chess App".

Well, that's essentially what an installer is supposed to do, isn't it?
As I said, having to launch an installer to install something on OS X always feels a bit weird, since the normal way to operate things is to just drag stuff into the right directory (typically an application bundle into /Applications or ~/Applications).

I guess this could indeed be considered a very rudimentary installer. But dragging things to the "right directory" requires the user to have some knowledge, and I thought this was against the Mac philosophy.

The very fact that there is a "right" directory, rather than the software being absolutely "portable" and running just anywhere means that it has to be "properly installed" do become functional. By definition whatever does this is an "installer". Of course sufficiently knowledgeable users will always be able to do exactly the same thing as what an installer program thus, by dragging stuff to the right place.

[mvk]

The purpose of dragging a bundle into Applications is to ensure it is still there when the dmg is gone after eject or reboot. Not to make it functional. You can run an .app directly from the dmg.

About documentation, the following options, from most common to least common:
- Design to make documentation not needed.
- Internet. Most mac users have that.
- Menu -> Help (there is an urban legend that somebody once found what he needed there)
- Document in the .dmg, next to the .app bundle. Personally I never retain (or even read) them.
- man file (only for developers who like to type in Terminal windows)

[hgm]

Basically we're talking about the difference between theory and practice. In theory, everything runs seamlessly and and with no need for users to interface with things under the hood that have gone wrong or no longer work , because they always work and are totally trouble free. Most apple apps are like that - xBoard is not one of those.

But wouldn't it be better if it was?

One thing is certain, though: it wouldn't get so by just observing that it isn't and throwing your hands in the air. Something would have to be done to make it so. And that is up to us.

It's one where practice is a total disconnect from theory. If I didn't have the capablitiy to get underneath the hood and fix things , I would not be using xBoard because it would simply not work the way I want it to work.

XBoard is designed so that all normal functions one would expect in a Chess GUI work self-evidently, without any need to get "under the hood". Although I am sure that someone who really knows how things work can do a lot by going under the hood, I have in fact difficulty imagining a task for which this would be needed, or even helpful.

You would have to spend a day with me to see what I mean. I don't mind - I'm used to that - I have used xBoard or winboard off and on for 20 years - but except for me , I can't believe users will be drawn to it. Maybe for somebody that is content playing fairymax , it's fine .

On Linux should would be OK with all properly packaged engines. You just install the engine from your distro's repository, start XBoard, and click the engine in the Load Engine listbox, and you are all set. If in your experience it is more difficult on a Mac, this seems a specific Mac problem. Exactly the one I address here.

I would love to spend a day with you to see exactly what you mean. Unfortunal geographical issues make that a little difficult, so it might be more efficient if you could just drop a few hints here of what kind of trouble you experience that should not have been there.

But that's not me. You mention an enterprise environment , but I'm not an enterprise. I'm a single user on a personal pc , totally different set of wants and needs than enterprise. I mentioned the home directory because that appears to be the safest place to put things without getting into the permission issues that can get nightmarish at time.

Well, I did not mean the starship. I have no Mac, so I don't know what nightmarish permission issues there could occur on it. On Windows, when I want to install something through the GUI outside my own Documents folder, I just get a popup asking me whether it is OK if the program will make some changes to the computer. I don't really lose any sleep on answering that affirmatively, and that is usually all. I have no idea how it would be on the Mac. There seems little need for the /Library/ChessEngines to have restricted access rights anyway.

[hgm]

The purpose of dragging a bundle into Applications is to ensure it is still there when the dmg is gone after eject or reboot. Not to make it functional. You can run an .app directly from the dmg.

OK, so they could have dragged it anywhere else as well. That is nice. But unfortunately engines have to be in a specific place for the GUI to find them, as it cannot do a scan on the entire file system each time you start it up. And requiring the user to know where it should be seems like asking for trouble (like perhaps ruling out 99% of would-be users in a single blow...). Installers were invented to solve exactly that problem.

About documentation, the following options, from most common to least common:
- Design to make documentation not needed.
- Internet. Most mac users have that.
- Menu -> Help (there is an urban legend that somebody once found what he needed there)
- Document in the .dmg, next to the .app bundle. Personally I never retain (or even read) them.
- man file (only for developers who like to type in Terminal windows)

Well, engines sometimes can do complex things, and for instance Fairy-Max' options for managing the persistent hash cannot be self-evident. So it seems a more elaborate description than the mere name of the option should be available, ruling out the most-common method. XBoard's help-clicks (somewhat comparable to 'tool tips') do seem preferable over forcing people to search on the internet. But it would require the info to be packed with the engine, and XBoard being able to find it.

[MikeB]

First I do like xBoard and it is possible that I may have ignorance to the proper way of doing things through the GUi.

So for two items that I go underneath almost every day. I do a lot of testing with a lot of different engines and different versions. So using the GUi , what is the property way to delete engines and to rearrange the sort of the engines in the engine list without going under the hood ?

[hgm]

There is an Edit Engine List item in the Engine menu. Clicking it makes the list pop up in a text edit, where you can perform the normal editing like deleting lines, re-ordering them, or adding/deleting XBoard options to existing lines.

It can also be used to insert the "# GROUPNAME" and "# end" lines needed to sub-divide the list into collapsible sections.

Note that any changes will only be committed to the settings file when XBoard actually saves settings (i.e. on exit or when pressing Save Settings Now), not when you press OK in that dialog.

[IanO]

In the XBoard help thread it came up how detrimental it is for the Mac user that there is no decent system for packaging engines, so that people often have to limp along with just a bare executable. Which the user then has to unpack and register with the GUI by hand, without any docs whatsoever, (or, if he is very lucky, a README file that will be hidden in a place he would never visit).

If the Mac does not have the benefit of repositories from which you can install programs in a compliant way, (so that they will automatically appear in the engine lists of GUIs that can run them, and their documentation can be displayed by the standard man command or help utility), they should at least be packaged such that the user can mimic this process by unpacking the archives in which the engines are distributed in, say, /usr/local/ . So that included man pages (which then would be accessed through the GUI help functions), data files and auto-install files all go in the compliant places.

Standardization here would also make it possible for GUIs to allow the user to just point at the archive in the engine-install dialog, and order unpacking of the package in the proper place itself. In fact dragging the tar ball on top of the GUI icon should have this effect, and start up the GUI with the engine installed, registered and loaded, ready for use.

Any suggestions from Mac users how we could reach such a state of affairs?

You are never going to get buy-in for such a packaging standard. Too small a niche. Only Mac power users would ever use the Unixy things you mention, like man pages, and they would be nimble enough to find alternate documentation and configuration.

So I fail to see the problem. It is easy to add UCI engines to Hiarcs Explorer, my preferred Mac client. As to organization, I collect engines one per directory in an "Engines" folder I put in the Applications folder.

There is a weakness in Hiarcs currently: you can't specify a custom engine command line. I would like to use polyglot within Hiarcs for two purposes: 1) a shim for the few engines that don't yet support UCI, and 2) a shim to provide an opening book for the increasing number of UCI engines that don't support OwnBook. However, you either need to recompile polyglot to load a particular custom config file and engine, or specify such file on the command line. HIARCS can't do the latter, though I've submitted a feature request.

[Evert]

Well, that it isn't an issue for you doesn't mean that there isn't something to be gained. However, it may be true that it doesn't fit the normal OS X ecosystem.

Under Linux, the automatic registration of engines ("plugins") makes sense because of two common usage scenarios: installing a chess engine from the package manager makes it work automagically with the installed GUIs (which I would actually consider to be expected behaviour) and installing the engine from source ("./configure && make && sudo make install") also makes it work immediately.

OS X, however, lacks a proper package manager (unless you consider the App store the equivalent) and most users will not install from source. Instead, they download a DMG and simply drop stuff where they want it...