New CFish Windows binary (from today's code)

[drj4759]

I confirm that CFish losses on time forfeit occasionally which don't happen with Stockfish and other SF clones. It also has crashes of about 5 in every 1000 games beginnning with version released October 7 or a day earlier.

This had been observed from about a million games where CFish in the form of Goby clone was used primarily against BrainFish/Cerebellum and Stockfish to create the Fishbook Dynamic opening book which already neutralized Cerebellum and Fauzi opening books.

I also confirm that CFish is stronger and faster than Stockfish.

Good work. Keep going.

[syzygy]

I confirm that CFish losses on time forfeit occasionally which don't happen with Stockfish and other SF clones. It also has crashes of about 5 in every 1000 games beginnning with version released October 7 or a day earlier.

The crash I'm aware of and is being fixed. It happens during TB probing at the root.

After that I will look into why time management works out differently.

[drj4759]

I'm not sure if you are aware that CFish does not run under Linux with Arena. It's the same with AsmFish which also have NUMA awareness.

I cloned CFish to make it run in Linux/WINE by disabling NUMA code which now become my primary testing tool against Stockfish and its clones.

Maybe there is a way for you to make a generic version of CFish that will run under Linux without third party alteration.

Thank you.

[syzygy]

You can compile without NUMA code by adding "numa=no":

Code: Select all

$ make profile-build numa=no

It's not completely clear to me what is the problem with Arena.

It's also not clear to me what is the problem with NUMA code. If you are running on a NUMA machine, then disabling it is not a good solution. If you are running on a non-NUMA machine, then the NUMA code should disable itself.

If the problem is that you don't have libnuma headers, then adding "numa=no" should help. But that is independent of Arena.

So I'm not doubting that there are some issues that need to be fixed, but I need to understand the problem first .

[drj4759]

If you had not tried running CFish on Linux before, then do it and see if it will not crash. It if runs successfully, then the problem lies in my 5 Linux computers in which CFish does not like. Stockfish, clones and 98% of the other 300 chess engines that is in the ChessOwl Rating List site run successfully under Linux/Arena.

It pissed me that 2% of the chess engines don't run on Linux/Arena which includes CFish, AsmFish, Fire, Robodini and Chiron. It is frustrating to use Windows 10 just to test these ill-behaved chess engines which is counter-productive. I can ran 7 simultaneous tournaments in one Linux computer, while I can only run 4 tournaments with Windows 10 which is controlled by VirtualBox.

I hope you understand what I mean that CFish crashed under Linux and why productivity is a reason for my request for CFish to run under Linux.

Thanks.

[syzygy]

If you had not tried running CFish on Linux before, then do it and see if it will not crash.

I'm almost exclusively using Linux. But I don't use the Arena GUI (which I would have to run under Wine).

I hope you understand what I mean that CFish crashed under Linux and why productivity is a reason for my request for CFish to run under Linux.

Well, you said something about "disabling" NUMA to make it work. I don't see how that can make a difference.

Did you get Cfish to run under Arena? What did you do exactly that made a difference?

Do you run a Windows binary in Arena (under Wine)? Or a Linux binary?

I need specific answers if you want me to do something.

[petero2]

I'm not sure if you are aware that CFish does not run under Linux with Arena. It's the same with AsmFish which also have NUMA awareness.

I cloned CFish to make it run in Linux/WINE by disabling NUMA code

I can not run a windows compile of texel on my linux NUMA computer if I compile texel with NUMA awareness. The program crashes with the following message:

Code: Select all

wine: Call from 0x7bc60ccc to unimplemented function KERNEL32.dll.GetNumaNodeProcessorMaskEx, aborting
wine: Unimplemented function KERNEL32.dll.GetNumaNodeProcessorMaskEx called at address 0x7bc60ccc (thread 00aa), starting debugger...

So the problem is that wine does not implement the required windows NUMA API functions. I guess CFish suffers from the same wine limitation.

[syzygy]

I'm not sure if you are aware that CFish does not run under Linux with Arena. It's the same with AsmFish which also have NUMA awareness.

I cloned CFish to make it run in Linux/WINE by disabling NUMA code

I can not run a windows compile of texel on my linux NUMA computer if I compile texel with NUMA awareness. The program crashes with the following message:

Code: Select all

wine: Call from 0x7bc60ccc to unimplemented function KERNEL32.dll.GetNumaNodeProcessorMaskEx, aborting
wine: Unimplemented function KERNEL32.dll.GetNumaNodeProcessorMaskEx called at address 0x7bc60ccc (thread 00aa), starting debugger...

So the problem is that wine does not implement the required windows NUMA API functions. I guess CFish suffers from the same wine limitation.

I have no problem running a windows compile of Cfish under Wine on Linux. Cfish uses GetLogicalProcessorInformationEx but first tests whether that function is available.

I just tried loading a Linux binary of Cfish in Arena/Wine. It works OK, except that Cfish is getting restarted on every move. I remember having had this problem before, not with Cfish and probably with an older version of Arena. I don't remember if I found a solution then. I have no reason to use Arena, to be honest (how are you supposed to run a simple engine-engine match... by setting up a whole tournament?!).

[drj4759]

I use Arena exclusively in my rating list production, therefore there is no single Linux binary used in my tournaments because Arena is a Windows only application.

CFish runs under Arena in any Windows platform whether it is standalone Windows or Linux/VirtualBox/Windows computer. It does not run under Linux/WINE/Arena.

To make sure that NUMA is the culprit, I uncommented all the NUMA specific codes that were disabled, recompiled, run and then it crashed.

Possibly the best explanation to this is Peter Osterlund's (a NUMA implementor) comment in which he experienced the crash under Linux/WINE.

Your results vary with mine probably we do not use the same computing environment. I use PCLinuxOS with Linux kernel 4.7.6, 4 core machines, 8GB RAM and VirtualBox where several versions of Windows lives.

You are half-successful with your run with CFISH under Linux/Arena but it is not acceptable. That shows that there is something wrong with NUMA under Linux/Arena.

BrainFish had a NUMA implementation but it does not also run under Linux/Arena so they had a separate separate binary for NUMA and non-NUMA. I am not sure if you will do the same as you have an obvious dislike for Arena.

I may maintain my own non-NUMA CFish clone for primary testing tool.

Just make sure that you squash the bug that caused the intermittent crashes.

Thank you.

[syzygy]

I use Arena exclusively in my rating list production, therefore there is no single Linux binary used in my tournaments because Arena is a Windows only application.

But Arena on Wine runs Linux binaries just fine, except for the weird restarting issue (that may or may not have a simple solution).

CFish runs under Arena in any Windows platform whether it is standalone Windows or Linux/VirtualBox/Windows computer. It does not run under Linux/WINE/Arena.

So that is a problem of how Arena and Wine interact. Wine is not perfect.

Just to be sure: Cfish runs absolutely fine on Linux (I'm talking about the Linux binary obviously), on which it has been developed. It has hardly been tested by me on Windows, but it seems to run OK on Windows too.

To make sure that NUMA is the culprit, I uncommented all the NUMA specific codes that were disabled, recompiled, run and then it crashed.

Possibly the best explanation to this is Peter Osterlund's (a NUMA implementor) comment in which he experienced the crash under Linux/WINE.

But I implemented NUMA for Cfish, and also the Windows NUMA version runs just fine for me on Wine from the command line.

I just tried running a Window binary of Cfish in Arena on Wine and it runs fine as well. NUMA support is compiled in, but it is not used as I don't have a NUMA machine. No crashes. And this time no restarting. So Arena likes windows binaries better. (But why oh why can I load two engines in Arena, but does it not let me start a simple game between them without having to configure a whole tournament? And when I do configure a minimal tournament of 2 games, it lets the same engine play as white twice?!)

You are half-successful with your run with CFISH under Linux/Arena but it is not acceptable. That shows that there is something wrong with NUMA under Linux/Arena.

As I explained, the presence of NUMA code seems to cause no problems for me whatsoever. The restarting issue disappears when I run a Cfish windows binary in Arena/Wine instead of a Cfish Linux binary. NUMA nothing to do with it.

If for whatever reason (perhaps you're using an older version of Wine than I am) the effectively unused NUMA calls in Cfish cause a problem on your system, then the easy way to compile without it is this:

Code: Select all

$ make profile-build numa=no

No need to comment out anything: just add "numa=no".

BrainFish had a NUMA implementation but it does not also run under Linux/Arena so they had a separate separate binary for NUMA and non-NUMA. I am not sure if you will do the same as you have an obvious dislike for Arena.

I do not provide binaries at all. You have to compile it yourself, so just add "numa=no" if that magically makes a difference. (I have just verified that numa=no does not lead to compiliation errors, at least not when cross-compiling for Windows on Linux.)