rewrite generating a new chess project

[Uri Blass]

first things is to define the basic files that I need not only for chess program and it is practically a map about writing a C program.
Note that I do not give the code in every change but I add a lot of comments in the code so people can see exactly when I added a relevant code.

Files to add to the project are:
1)main.cpp(include main function that return 0)
2)protos.h(include nothing) except a line the compiler enter to h files automatically
3)command.cpp(main function is going to call it and first function that I plan to use is readCommands that I will define later)
4)defines.h that has basic definitions(not variables but constants or new name that I like to use for type of variables instead of long existing names in the language)
5)globals.h that is file of global variables
6)extglobals.h that is a file of extra global variables that are basically same variables as 5(reason that I need it twice is because these variables are not external variable to main but external variables to other files that use them.
7)main,cpp is the only file that include globals.h because I want the global variables to be non external to main.cpp
8)every other cpp file include extglobals.h if I want to allow it to use global variables.
I want to allow command.cpp to use global variables so I include extglobals.h in it.
9)globals.h include defines.h because I may use basic definition to describe global variables(for example if I define C to be 589 I can have A[C] instead of A[589] in the global variables)
10)extglobals.h include defines.h for the same reason
11)I want main to include "protos.h" otherwise it does not know the structure of readCommands function and cannot call it.
Note that it make sense to add other files to help debugging but this is only in the next rewrite.
12)define the prototype of the function readCommands in protos.h
13)define function readCommands in command.cpp with empty code.
14)call readCommands in main.cpp
15)define variable nextc in readCommands(nextc means next command)
16)include iostream in command.cpp that I need for the loop at step 17
17)add a loop that simply wait for the user to enter something when only after the user click enter the program quit

I have now a function readCommands that simply wait and finish when the user enter something.
basically I have 2 cases about the value of nextc
a)nextc is the character of a new line so I need to do something with the input
b)nextc is not a new line so I need to memoprize the input

18)adding a code if else for the 2 cases
19)first I want to deal with the else part and memorize the string that the user define
I decide about a constant 256 that is the maximal size of the input
20)I add an array with MAX_CMD_BUFF to globals.h and extglobals.h
21)I add a variable for the size of the input to globals.h and extglobals.h
22)I add inside the else that I entered in command.cpp a code that put the user command in the buffer and warn the user if the buffer if full
23)I add BOOLTYPE to be an integer in defines.h(I want a new function to enter true or false so I do not like to use int
content of defines.h
24)I add the function doCommand in command.cpp that get the user input and return false only when the user asked to quit the program

not sure if I need CMD_BUFF_COUNT = '\0'; when I exit the program but it is in the code they give in the site
http://web.archive.org/web/201201120650 ... 20commands

25)using doCommand inside readCommand and deleting return that I added at step 17 that I now do not need because doCommand decide if to exit the program.
26)adding version number in defines.h
27)printing the version in the beginning of the program.
28)printing wt before the user enter a new command when I finished to take care of the old command
29)printing command not implemented if I did nothing
30)printing some information if the user ask for help or h or ?
31)added the types U64 and BitMap in defines.h in order to work with 64 bit integers without the ugly C code

code now after all of these things is:
1)main.cpp

Code: Select all

#include "globals.h" //added this line in step 7
#include "protos.h"//added this line in step 11
int main()
{
	readCommands();//added this line in step 14
	return 0; //added this file this function and this line in step 1
}

2)defines.h

Code: Select all

#pragma once //added this file in step 4
#define MAX_CMD_BUFF 256   // Console command input buffer added this line in step 19
typedef int BOOLTYPE;//added this line in step 23
#define chess_prog_version "analyzer 0.0, Copyright (C) 2018, Uri Blass Stef Luijten"//added this line in step 26
typedef unsigned long long U64;// added this line in step 31
typedef unsigned long long BitMap;//added this line in step 31

3)globals.h

Code: Select all

#pragma once //added this file in step 5
#include "defines.h" //added this line in step 9
char CMD_BUFF[MAX_CMD_BUFF];//added this line in step 20 and same for extglobals.h
int CMD_BUFF_COUNT = 0;//added this line in step 21 and same for extglobals.h

4)extglobals.h

Code: Select all

#pragma once //added this file in step 6
#include "defines.h"//added this line in step 10
extern char CMD_BUFF[];//added this line in step 20
extern int CMD_BUFF_COUNT; //added this line in step 21

5)protos.h

Code: Select all

#pragma once //added this file in step 2
void        readCommands();//added this line in step 12

6)command.cpp

Code: Select all

//added this file in step 3
#include <iostream>//added this line in step 16
#include "extglobals.h"//added this line in step 8
BOOLTYPE doCommand(const char *buf)//added this function in step 24  
{
	if ((!strcmp(buf, "exit")) || (!strcmp(buf, "quit"))) //added in step 24
	{
		CMD_BUFF_COUNT = '\0'; //added in step 24
		return false;          //added in step 24
	}
	//     ================================================================= //added in step 30
	//  help, h, or ?: show this help                                        //added in step 30
	//     ================================================================= //added in step 30
	if ((!strcmp(buf, "help")) || (!strcmp(buf, "h")) || (!strcmp(buf, "?")))//added in step 30
	{//added in step 30
		printf(" help\n ");//added in step 30
		printf("exit           : exit program\n ");//added in step 30
		printf("help,h or ?    : show this help\n ");//added in step 30
		printf("quit           : exit program\n ");//added in step 30
		CMD_BUFF_COUNT = '\0';//added in step 30
		return true;//added in step 30
	}
	printf("command not implemented:  %s\n", buf);//added this line in step 29
	CMD_BUFF_COUNT = '\0';//added in step 24
	return true;//added in step 24
}
void readCommands() //added this line in step 13
{//added this line in step 13
	int nextc;//added this line in step 15
	printf(chess_prog_version"\n");//added this line in step 27
	printf("wt> ");//added this line in step 28
	while ((nextc = getc(stdin)) != EOF)//added this line in step 17(wait for the user to click something with enter
	{//added this line in step 17
		if (nextc == '\n') //added this line in step 18
		{//added this line in step 18
			CMD_BUFF[CMD_BUFF_COUNT] = '\0';//added this line in step 25
			while (CMD_BUFF_COUNT)//added this loop in step 25
			{
				if (!doCommand(CMD_BUFF)) return;//added this line in step 25
			}
			printf("wt> ");//added this line in step 28
		}//added this line in step 18
		else//added this line in step 18
		{//added this line in step 18
			//added next 6 lines in step 22
			if (CMD_BUFF_COUNT >= MAX_CMD_BUFF - 1)
			{
				printf("Warning command buffer is full !!\n");
				CMD_BUFF_COUNT = 0;
			}
			CMD_BUFF[CMD_BUFF_COUNT++] = nextc;
		}//added this line in step 18
	//added return here in step 17 otherwise it is an infinite loop	
	//deleted return here in step 25 because after step 25 doCommand decide if to exit the program and there is no infinite loop 
	}//added this line in step 17
}//added this line in step 13

[Dann Corbit]

Suggestion:
Do not use any global value which is written to more than once unless you really, really have to.
That will be very important for SMP.
The time to make that design decision is right when the project starts.

Otherwise you will have a bunch of critical sections and semaphores and other cruft that you probably did not really need in the first place.

[Uri Blass]

Only to be sure.
Do you mean for every global variable not to use it in more than one file if there is a choice?

[Dann Corbit]

There are globals that are strictly necessary.
In the C language, it is very hard to get work done without using some or all of stdin, stdout, and stderr, which are global.
However, if you have something global, and it gets written to a lot, that is something you need to look at very carefully.
I see you have a header file called globals.h which (obviously) contains a list of things everyone can see and modify.
When you change your program to SMP (and all of them become that eventually if they are good enough) then you may have a terrible job to either rewrite that part of the code or carefully gate each access.

Now, the main hash table has to be global. There are attempts to use it with no locking, and lockless data structures are definitely something to examine and consider. (Robert Hyatt also has a paper which you have probably read on simply ignoring the errors caused by rare collisions).

The problems arise when two separate threads try to modify an object or when one thread tries to read an object when another thread is trying to write to the object. Both of those situations are very bad. Now you can gate access (in the simplest method with a critical section) but those things slow down your code.

So every beginning chess program should be very, very careful not to put something in global access unless the following conditions are met:
1. The object is written to by only one thread at startup time (e.g. some sort of evaluation tables)
OR
2. It simply has to be shared (e.g. the main hash table)

If we use globals in a lazy way, we will take a terrible price for it.

[Uri Blass]

I have globals.h because the steps are based on the program winglet(with some changes for example I decided to use printf instead of std::cout)

globals variables are basically variables that are not defined inside functions.
I thought only about the variables that I declared directly as global variables but you mentioned stdin as one of the global variables.
I understand that #include <iostream> in commands.cpp means including stdin as one of the global variables.

http://web.archive.org/web/201201121138 ... /index.htm

I wonder what the advice is about using global variables.
You say it is no problem to use them if the object is written to only one thread at start up time.

Does it mean that the array CMD_BUFF should not be global?
Note that I use it every time that the computer read the user command so not only at start up time.

Note that my plan is not to have a chess engine but a chess analyzer to do different things than chess engines(for example solve draw problem and today chess engines are not able to solve draw problems(they can say 0.00 but I do not know if it is a forced draw or simple equality and I also do not know the number of plies the side to move need to force a draw and the number of plies the opponent needs to force a draw).

[Dann Corbit]

{snip}
Does it mean that the array CMD_BUFF should not be global?
Note that I use it every time that the computer read the user command so not only at start up time.
{snip}

Usually, there is a single thread that performs reading and writing from the console. If this is true, then CMD_BUF can be global. But even so, why is it there? If you forget, and read or write to it from other threads while it is being written, then bad things can happen. So why expose it in this way?

Which brings up the point, why in the world would CMD_BUF need to be the global area? That does not make sense to me unless multiple threads are going to use it, and I can think of no good reason why multiple threads would need to be performing I/O.

You should think of your program threads as a bunch of secret agents dealing with highly classified information. Every time someone wants to see or do something, you should give them the stink-eye and say, "Do you have the need to know?"

[Uri Blass]

I do not plan to go for hash at this point but
basically if I understand correctly the principle is that even global variables should be known only to files that need to know them
so there is no reason for file of all global variables that is in winglet.


I decided to try to find global variables of senpai2.0

I look at the code of senpai2
main.cpp include the following files not including libraries
1)bit.hpp
2)common.hpp
3)eval.hpp
4)fen.hpp
5)game.hpp
6)gen.hpp
7)hash.hpp
8)libmy.hpp
9)list.hpp
10)math.hpp
11)move.hpp
12)pawn.hpp
13)pos.hpp
14)search.hpp
15)sort.hpp
16)thread.hpp
17)tt.hpp
18)util.hpp
19)var.hpp

Is it a good design to include so many hpp files that many of them include variables that are probably global variables?

I decided to look also at bit.hpp it already include common.hpp and libmy.hpp so I think that if main include bit.hpp it does not need to include also common.hpp and libmy.hpp

note that bit.hpp has the following external global variables inside namespace bit

Code: Select all

extern Bit Pawn_Squares;
extern Bit Promotion_Squares;
extern Bit Colour_Squares[2];

Note that these variables are known to many files(for example list.cpp or hash.cpp)

I wonder if it is a good design and I wonder if there is a simple way to find a list of all global variables inside senpai2 that is free source code and SMP

[Dann Corbit]

I do not plan to go for hash at this point but
basically if I understand correctly the principle is that even global variables should be known only to files that need to know them
so there is no reason for file of all global variables that is in winglet.

If you intend to create an SMP version, lots of global variables in an include file is a troublesome design.

I decided to try to find global variables of senpai2.0

I look at the code of senpai2
main.cpp include the following files not including libraries
1)bit.hpp
2)common.hpp
3)eval.hpp
4)fen.hpp
5)game.hpp
6)gen.hpp
7)hash.hpp
8)libmy.hpp
9)list.hpp
10)math.hpp
11)move.hpp
12)pawn.hpp
13)pos.hpp
14)search.hpp
15)sort.hpp
16)thread.hpp
17)tt.hpp
18)util.hpp
19)var.hpp

Is it a good design to include so many hpp files that many of them include variables that are probably global variables?

It is a good idea to have a separate include file for every file (usually, each file represents a class, and the header file is the definition for the class).
Typically, these header files are not defining global variables, they are defining classes (another way to think of it is that they are defining types except that instead of compiler defined types like int and std::string, they are describing user defined types known as classes.

I decided to look also at bit.hpp it already include common.hpp and libmy.hpp so I think that if main include bit.hpp it does not need to include also common.hpp and libmy.hpp

note that bit.hpp has the following external global variables inside namespace bit

Code: Select all

extern Bit Pawn_Squares;
extern Bit Promotion_Squares;
extern Bit Colour_Squares[2];

Note that these variables are known to many files(for example list.cpp or hash.cpp)

I wonder if it is a good design and I wonder if there is a simple way to find a list of all global variables inside senpai2 that is free source code and SMP

This is the Bit class referred to by the above references:

Code: Select all

class Bit {

private :

   uint64 p_bit;

public :

   Bit ();
   explicit Bit (uint64 bit);

   operator uint64 () const;

   void operator |= (Bit b);
   void operator &= (uint64 b);
   void operator ^= (Bit b);
};

The bit Pawn_ Squares is referenced only from within their class:
bit.cpp (23): Bit Pawn_Squares;
bit.cpp (69): Pawn_Squares = rect(0, 1, File_Size, Rank_Size - 1);
bit.cpp (70): Promotion_Squares = ~Pawn_Squares;
bit.hpp (14): extern Bit Pawn_Squares;

And Promotion_Squares is referenced as follows:
bit.cpp ( 24): Bit Promotion_Squares;
bit.cpp ( 70): Promotion_Squares = ~Pawn_Squares;
bit.hpp ( 15): extern Bit Promotion_Squares;
gen.cpp (169): add_pawn_moves(list, pos, sd, pos.pawns(sd), pos.empties() & bit::Promotion_Squares);

The file gen.cpp uses that bit for a test. It never writes to it.

The file bit.cpp initialized the data exactly once in lines 69 and 70 like so, in the init function:

Code: Select all

void init() {

   // files and ranks

   for (int s = 0; s < Square_Size; s++) {

      Square sq = square_make(s);

      set(File_[square_file(sq)], sq);
      set(Rank_[square_rank(sq)], sq);

      set(Colour_Squares[square_colour(sq)], sq);
   }

   Pawn_Squares      = rect(0, 1, File_Size, Rank_Size - 1);
   Promotion_Squares = ~Pawn_Squares;

And they are modified in no other places. This is exactly the first category that I mentioned, namely data objects that are initialized once at program startup and then only read from.

[Dann Corbit]

These are all the global variables used in senpai:

Code: Select all

   bit.hpp (     14): extern Bit Pawn_Squares;
   bit.hpp (     15): extern Bit Promotion_Squares;
   bit.hpp (     16): extern Bit Colour_Squares[2];
   pos.hpp (    100): extern Pos Start;
    tt.hpp (     69): extern TT G_TT; // MOVE ME?
   var.hpp (     15): extern bool Ponder;
   var.hpp (     16): extern bool SMP;
   var.hpp (     17): extern int  Threads;
   var.hpp (     18): extern int  Hash;
   var.hpp (     19): extern bool Chess_960;

[Sven]

Jumbo is written in C++. It has zero (0) global variables.