If making an SMP engine, do NOT use processes.

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Dann Corbit
Posts: 12538
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: If making an SMP engine, do NOT use processes.

Post by Dann Corbit »

Zach Wegner wrote:Well, not too long ago, I found a huge bug in my program's SMP mode, that was causing illegal moves in the PV. It turns out that I was launching the child process(es) before I initialized the bitboard attack data. Thus, the sliders have no attacks, and thus the king cannot be in check...

Just now, I was tuning my split point selection code, and I was trying to make it based on the iteration. I got some strange behavior before I realized that current_depth is a global variable that only gets modified in the master process...

Now I'm trying to decide whether to bite the bullet and convert my whole program to threads (mostly just find and replace board. to board->), or just deal with it for now. I know some people are waiting on my program.

JUST USE THREADS!!!
Gian-Carlo Pascutto was successful with processes with Deep Sjeng, so it can be done.
Spock

Re: If making an SMP engine, do NOT use processes.

Post by Spock »

Dann Corbit wrote: Gian-Carlo Pascutto was successful with processes with Deep Sjeng, so it can be done.
Rybka 2.3.2a also uses processes I believe..
Dann Corbit
Posts: 12538
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: If making an SMP engine, do NOT use processes.

Post by Dann Corbit »

Zach Wegner wrote:Well, not too long ago, I found a huge bug in my program's SMP mode, that was causing illegal moves in the PV. It turns out that I was launching the child process(es) before I initialized the bitboard attack data. Thus, the sliders have no attacks, and thus the king cannot be in check...

Just now, I was tuning my split point selection code, and I was trying to make it based on the iteration. I got some strange behavior before I realized that current_depth is a global variable that only gets modified in the master process...

Now I'm trying to decide whether to bite the bullet and convert my whole program to threads (mostly just find and replace board. to board->), or just deal with it for now. I know some people are waiting on my program.

JUST USE THREADS!!!
Once again, playing devil's advocate, here is something that makes it a bit easier to write a process launching version that runs on both Unix variants and also Windows:

Code: Select all

static int donothing; // an empty translation unit is a non-standard extension.

#ifdef _MSC_VER
#include <windows.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>

int             fork&#40;)
&#123;
    char            szPath&#91;FILENAME_MAX&#93;;
    char            szCommandLine&#91;1024&#93;;
    PROCESS_INFORMATION pi;
    STARTUPINFO     startup_info;
    startup_info.cb = sizeof&#40;STARTUPINFO&#41;;
    startup_info.lpReserved = NULL;
    startup_info.lpDesktop = NULL;
    startup_info.lpTitle = NULL;
    startup_info.dwX = 0;
    startup_info.dwY = 0;
    startup_info.dwXSize = 0;
    startup_info.dwYSize = 0;
    startup_info.cbReserved2 = 0;
    startup_info.lpReserved2 = NULL;
    // make sure the show state is used and has our settings
    startup_info.dwFlags = STARTF_USESHOWWINDOW;
    startup_info.wShowWindow = SW_HIDE;

    szCommandLine&#91;0&#93; = 0;

    GetModuleFileName&#40;NULL, szPath, sizeof&#40;szPath&#41; - 1&#41;;

    sprintf&#40;szCommandLine, ""%s"",szPath&#41;;
    if &#40;0 == &#40;CreateProcess&#40;NULL,
                            szCommandLine,
                            NULL,
                            NULL,
                            TRUE,
                            0,
                            NULL,
                            NULL,
                            &startup_info,
                            &pi&#41;))
        return -1;

    return pi.dwProcessId;      // return PID of the started task.
&#125;
#endif
Of course, this fork() for Windows won't operate identically to the Unix fork() because the auto variables will not be preserved to the child process. So any memory that needs to be shared between processes has to be put into shared memory. But if both programs use that assumption, then you can write a forking version that works on Windows or on Unix without going to two different versions of launch code.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: If making an SMP engine, do NOT use processes.

Post by bob »

Dann Corbit wrote:
Zach Wegner wrote:Well, not too long ago, I found a huge bug in my program's SMP mode, that was causing illegal moves in the PV. It turns out that I was launching the child process(es) before I initialized the bitboard attack data. Thus, the sliders have no attacks, and thus the king cannot be in check...

Just now, I was tuning my split point selection code, and I was trying to make it based on the iteration. I got some strange behavior before I realized that current_depth is a global variable that only gets modified in the master process...

Now I'm trying to decide whether to bite the bullet and convert my whole program to threads (mostly just find and replace board. to board->), or just deal with it for now. I know some people are waiting on my program.

JUST USE THREADS!!!
Gian-Carlo Pascutto was successful with processes with Deep Sjeng, so it can be done.
Crafty's been using fork() for a few years since I got tired of the thread incompatibilities I kept running into between different vendors (all using unix, but things vary anyway).

Processes work just fine, I found no significant speed difference or memory usage since most good operating systems today use copy-on-write anyway so that non-modified data doesn't get replicated for each process. One minor advantage for threads is the egtb.cpp is already thread-ready, if you use processes, it will replicate the egtb data and drive up the I/O rate over what threads will do.