Cray Blitz source (Carey)

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Cray Blitz source (Carey)

Post by bob »

OK, first things first: first output produced by cray blitz since 1994:

Code: Select all

 your move, robert              
go
                 time limit 0:30.
                 depth   time     eval  variation (s 2)
                   1     0:00   -0.221  e5 Nc3 ...
                   1->   0:00   -0.221  e5 Nc3 ...
                   2     0:00   -0.221  e5 Nc3 ...
                   2->   0:00   -0.221  e5 Nc3 ...
                   3     0:00   -0.045  e5 Nf3 Nc6 
                   3->   0:00   -0.045  e5 Nf3 Nc6 
                   4     0:00   -0.211  e5 Nf3 Nc6 Nc3 
                   4->   0:00   -0.211  e5 Nf3 Nc6 Nc3 
                   5     0:00   -0.081  e5 Nf3 Nc6 d4 d6 
                   5a    0:00     ++0   d5                            
                   5     0:00   -0.026  d5 Nf3 Nc6 Nc3 e5 ...
                   5->   0:00   -0.026  d5 Nf3 Nc6 Nc3 e5 ...
                   6     0:00   -0.170  d5 Nf3 Nc6 d4 Nf6 Nc3 
                   6->   0:00   -0.170  d5 Nf3 Nc6 d4 Nf6 Nc3 
                   7     0:00   -0.071  d5 Nf3 Nc6 d4 Nf6 Nc3 e6 
                   7a    0:00     ++0   Nf6                           
                   7     0:00   -0.030  Nf6 Nf3 d6 Nc3 Nbd7 d4 e5 
                   7->   0:00   -0.030  Nf6 Nf3 d6 Nc3 Nbd7 d4 e5 
                   8     0:00   -0.066  Nf6 Nf3 Nc6 d4 d5 Nc3 e6 Bf4 
                   8->   0:00   -0.066  Nf6 Nf3 Nc6 d4 d5 Nc3 e6 Bf4 
                   9     0:01   -0.014  Nf6 Nf3 Nc6 d4 d5 Nc3 e6 Bf4 Bb4 
                   9->   0:01   -0.014  Nf6 Nf3 Nc6 d4 d5 Nc3 e6 Bf4 Bb4 
                  10     0:03   -0.055  Nf6 Nf3 Nc6 d4 d5 Nc3 e6 Be3 Bb4 Qd3 
                  10->   0:04   -0.055  Nf6 Nf3 Nc6 d4 d5 Nc3 e6 Be3 Bb4 Qd3 
                time: 0:04 0:00  1.0p nodes:  3342072 h  9% 85% 80% 829298 nps
 
 my move    1. ...  Nf6       
 time control phase one reached ( 0:30 per move).
                 clock time was  0:04
 your move, robert              

Now for some nitty-gritty.

1. I got the book code to work. It does not read PGN, PGN didn't exist back then. What it wants is a simple string of SAN moves. then, if there is an alternative to one of those moves in the string, you enter the move number of the alternative and then a string of moves that represent an alternative to that move path. example:

e4 e5 nf3 Nc6 Bb5 a6 Ba4
5 Bc4 Bc5
2 c5 Nf3 e6

Note that the "numbers" are absolute move numbers where the first white move is move 1, the first black move is move 2, etc. Messy, but that's life.

You can just enter a ton of games that look like this:

1 e4 e5 nf3 nc6 etc
1 e4 c5 nf3 d6 etc

and so forth so you could probably produce a workable book input file from a PGN file with some luck. I can't, for the life of me, locate the book input we used. I have no idea why it was not in the source files we found at the Ala supercomputer site, but I have so far not been able to find 'em. So this would be issue 1.

2. the "hs" command changes the hash size but it does not work on modern unix systems. we used the "sbrk()" facility but it does not work. If wanted, someone could use the fortran-90 pointer mechanism to make the hash tables (there are 3, normal hash, king safety and pawn structure) work through pointers rather than using an array.

3. I did not try to figure out the way to get time in fortran. we always had a CPU time component, and I just made it use CPU time everywhere. If someone wants to add elapsed time, cptime.f is where this goes.

4. I thought this version (1991 was the date on the backup so that is probably correct) used the non-recursive null-move (one per path). It doesn't, it uses what we use today, except that it defaults to R=1. But I noticed that I had already added the ability to test R=2, probably because Murray Campbell had written a paper that suggested that R=2 needed more investigation. I tried "nm=2" and it worked just fine, but it won't accept anything larger than 2 unless options.f is modified.

Otherwise, it _seems_ to be working. I compiled it with the free Intel fortran compiler for linux and have run several tests.

I'd like to tar it up and send it to you for you to do with as you wish. Aha, one other note. This version is multiple-cpu capable but it will not work without a few changes. Cray used something called "task common" that was process-specific local data. Normal fortran does not have that. Most modern fortran compilers have the ability to say "this common is local" or "this common is global, all others are local" but I did not try to figure that out. Ditto for starting threads. But the 1991-era DTS code is there, although it was modified quite a bit between 1992-1994 as we began to use more and more CPUs.

Also, the singular extension additions are not present. They were added in 1992, 1993 and into 1994, so those are probably lost unless something else is discovered somewhere else. The listing I sent you appears to match this version although I have made a fair number of changes to make it compile. I did have to tell the compiler to default to integers are 8 bytes, to match the Cray, but that works fine on my core-2.

If you have any questions, feel free. Note there is no documentation although one can look up the commands in options.f, or type "help" for a list. There's lots of things I don't remember. Much of the code is pretty messy. Harry was bad about discovering that part of a function was burning a lot of time, so he would convert that into a new function and then rewrite it in assembly. But it left the pure fortran version looking pretty messy. I can tell you that for every question "why was this done like this?" there is an answer that is based on performance issues on the Cray.

I was quite surprised at the speed. It is nowhere near as fast as crafty, but it can easily hit 500-800K nodes per second, even with all the array-based stuff we did to use vectors on the cray. I can include the cray assembly language parts if you want, although they are essentially useless except for historical analysis.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Cray Blitz source (Carey)

Post by bob »

one other thing that does not work yet: pondering. We used two different approaches, one with two processes (one to read, one to play chess), the other was the classic "poll for input every now and then." I have not looked at this code carefully yet to see how it is supposed to work. the two processes probably will be problematic, while polling for input will need some code written.

I will try to fix this quickly...
Carey
Posts: 313
Joined: Wed Mar 08, 2006 8:18 pm

Re: Cray Blitz source (Carey)

Post by Carey »

Bob;

I'm thrilled at the progress you've made. I realize that most people in this forum don't care, but I really like the classics.... :D

Whenever you are ready, you can email it to me at either:

Carey -at- cebridge.net
ClassicChess -at- gmail.com

Or just leave me a personal message here in the forum, (it's set to notify me whenever I get one) and I can ftp it or whatever you want.

And *yes* please do include the CAL code. Even though it can't be used, it at least shows a lot of why CrayBlitz is structured the way it is. Most people these days (including myself) have never actually used a vector computer.

I'll probably include your message text about it. Just to let people know the condition etc.

I'm waist deep in debugging that other classic chess program, so I wont be able to spend as much time with CrayBlitz as I'd like, but you can bet I'll download OpenWatcom's Fortran compiler (for windows) and give it a try. If that doesn't work, I'll try GNU Fortran. If that doesn't work, I'll try f2c. If that doesn't work.... I'll think of something.



So basically, when ever you get ready, go ahead and send me the original unmodified version along with the modified version. And whatever additional code or notes you have handy.

I'll then try to package it up along with the old OCR'ed version (still with massive typos) and post it on my ClassicChess website.

I suppose I should include Blitz, too. I never did get around to getting all the typo's out. Not even sure where my 'work' copy is at anymore.


Carey
Steelman

Re: Cray Blitz source (Carey)

Post by Steelman »

So what was not Cal was Fortan? I assumed in would be written in c.
Because of the Cray?
Carey
Posts: 313
Joined: Wed Mar 08, 2006 8:18 pm

Re: Cray Blitz source (Carey)

Post by Carey »

Steelman wrote:So what was not Cal was Fortan? I assumed in would be written in c.
Because of the Cray?
Bob hasn't answered yet, so I figured I"d give it a try...

Yes, Blitz & CrayBlitz were Fortran.

Blitz was started back in the 70's. Not sure when, but probably early 70s. It was in Fortran. At the time, if you wanted efficiency and portability, you went with Fortran. There weren't any other choices.

(Bob says he's been doing computer chess for 40 years. So that would put the start in 68, right after MacHack VI. I don't know if Blitz came from 68 or was a later program. By 76 he was up to Blitz 4 and 77 was Blitz 5. So assuming one number per year, Blitz was started in 72 and would not have been his first chess program. But it was influenced by MacHack VI because originally it was a selective search program.)

The Cray was basically built for Fortran. For a while that and CAL was all they offered.

Eventually they added C (not sure when) but that was after Blitz became CrayBlitz. Because of C's flexibility and lack of rules, I imagine there were a number of restrictions and limitations if you wanted something vectorized.

Bob could have done a rewrite, of course, but there wasn't much point in doing so. You'd still have to switch to assembler for the key stuff.

As CrayBlitz grew, he probably wasn't in the mood to rewrite it into C even if it had been beneficial.


According to Welsh's second book, Harry Nelson compared CrayBlitz (right after it was put onto the Cray-1) and Chess 4.5 He discovered CrayBlitz was a lot slower. An execution profiler showed CrayBlitz was spending 25% of its time in just one 20 line subroutine. He rewrote that one routine in CAL and the whole program speeded up by 19%.

The Cray was really not well suited for chess, but Bob & Harry Nelson managed to force it and were creative enough they actually made it work efficiently.


After the source gets released, you can look at it yourself... Doing things in Vectors is a little odd if you are used to normal processors like we use.

There's no way you could get CrayBlitz running efficiently on a regular computer. There is a lot of CRAY in CrayBlitz.


I haven't gotten the source yet, and it may take a day or two after I get it before I post it (I may try running it through f2c, so I can provide a portable C version everybody can use), but I'll be sure and let people know when its ready.

Carey
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Cray Blitz source (Carey)

Post by bob »

Steelman wrote:So what was not Cal was Fortan? I assumed in would be written in c.
Because of the Cray?
It was originally FORTRAN as that was the only choice on the system we had in 1968. I started to convert it to PL/1 several times, but each time I thought about it, portability was an issue as PL/1 was really an IBM language at first, and then when the Cray came along in the mid 70's it was obvious that they were never going to be anything but a FORTRAN machine, so I stuck with FORTRAN.

For speed, we did lots of profiling and rewrote the time-critical parts in cray assembly language. We made sure that the CAL code was functionally equivalent to the FORTRAN code so that we could debug things, but the CAL was pretty wild in that it was really tailored to use the architectural tricks available in the Cray, such as vector operations, tons of registers, etc.

I think I will put the whole mess on my ftp box in case anyone is interested. I played one test game between CB and Crafty last night, one CPU, no pondering since CB can't yet ponder, 10 secs per move. Crafty won handily, typically searching 14-15 plies to CB's 9-10. I even used the nm=2 command to set null-move R=2 in CB. Not quite sure why the depth is so different, other than the checks and things CB does in the q-search might be a bit out of control, and/or the extensions might be "too mch". But in any case, it played just fine, although not all compilers are going to be happy compiling it I'd bet...
User avatar
Zach Wegner
Posts: 1922
Joined: Thu Mar 09, 2006 12:51 am
Location: Earth

Re: Cray Blitz source (Carey)

Post by Zach Wegner »

bob wrote:I think I will put the whole mess on my ftp box in case anyone is interested.
Definitely!
Gerd Isenberg
Posts: 2250
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: Cray Blitz source (Carey)

Post by Gerd Isenberg »

Bob, is your 1983 Master's Thesis available online as scanned ps or pdf?

Hyatt, R. 1983. Cray Blitz - A Computer Chess Playing Program. Master's Thesis, University of Southern Mississippi.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Cray Blitz source (Carey)

Post by bob »

Gerd Isenberg wrote:Bob, is your 1983 Master's Thesis available online as scanned ps or pdf?

Hyatt, R. 1983. Cray Blitz - A Computer Chess Playing Program. Master's Thesis, University of Southern Mississippi.
No, but I could probably have it scanned. I have an original, but it doesn't say much about parallel issues since the parallel stuff we were doing back then was pretty primitive...
Gerd Isenberg
Posts: 2250
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: Cray Blitz source (Carey)

Post by Gerd Isenberg »

bob wrote:
Gerd Isenberg wrote:Bob, is your 1983 Master's Thesis available online as scanned ps or pdf?

Hyatt, R. 1983. Cray Blitz - A Computer Chess Playing Program. Master's Thesis, University of Southern Mississippi.
No, but I could probably have it scanned. I have an original, but it doesn't say much about parallel issues since the parallel stuff we were doing back then was pretty primitive...
If you may scan and upload it on some place if you have some time left...
It would be nice to have a scanned pdf for historical reasons.