The problem is that when XBoard is first started without any arguments, it brings up the GUI according to the saved settings, and starts the default engine. While the intention could have been to start it in -viewer mode (after clicking a .PGN), which might have invoked another engine through -viewerOptions, or to connect to FICS (after clicking a .xop settings file that contained a -ics option), etc. Not all of what it did in the initial startup with wrong options is easy to undo.
Of course a certain way to undo it is always react to the OpenFile signal by launching a new instance, (which will then be correctly started with the file argument through "open -n", knowing the argument from the beginning), and have the original instance commit suicide when only one instance was required (i.e. when 'suppress' was set). This could be achieved by putting
at the end of StartNewXBoard(), and removing the earlier if(suppress) from there. This is extremely ugly, though, and would furthermore lead to the problem that any time you start up XBoard without any arguments, the first time you would click a PGN would kill that XBoard, even if the already running one was playing a game. So to guard against that it should base the suppress decision on the arrival time of the signal: if it arrives immediately after startup it is apparently OS X that has translated clicking on an associated file with an argumentless launch of XBoard followed by a signal, while if the signal comes much later, the initial launch must have been for another (valid) reason.
Of course we could also have XBoard wait for some time after startup, before having it do anything, to see if it gets the OpenFile signal, and suicide only when such a signal arrives during the timeout period. And then continue with the regular startup (bringing up the GUI) if no signal arrived before the timeout.
But I wonder if we cannot do a better job at misguiding OS X into doing the right thing. It seems that using "open -n" on a script does exert its influence also to $EXEC calls inside the script. I.e., when you $EXEC xboard-bin, OS X aparently doesn't again make the decision "I already have an xboard-bin running, so let's make it a signal". So I wonder if we cannot bootstrap ourselves through two levels of scripts: not associate PGN files with the xboard launch script, but with another script (say 'yboard'), which then contains the command "open -n -a xboard --args $@" (which seems to work if we issue it from XBoard with the system() function). The xboard script can then set the environment to be inherited by xboard-bin, as it does now.
Another idea, that might be worth a try: what happens if inside the xboard launch script we write
$EXEC open -n -a xboard-bin --arg $@
?
