couple of questions about stockfish code ?

[MahmoudUthman]

1-what unit "grain" does SF use for evaluation ? I read the source "7" and a couple of old posts here that made me think that I could be missing something .
2-What advantage does SF gain by combining mid and end scores inside a single INT instead of using 2 shorts "inside a Struct or a union if needed"?
3-I stumbled upon an old post here that claims that storing only the 32bit upper portion of the key degrades performance "I know it causes more collisions" ,yet SF does this ?!
4-what does this code snippet do "does it have something to do with the sign " ?

Code: Select all

inline Score make_score(int mg, int eg) {
  ScoreView v;
  v.half.mg = (int16_t)(mg - (uint16_t(eg) >> 15));
  v.half.eg = (int16_t)eg;
  return Score(v.full);
}

inline Value mg_value(Score s) {
  ScoreView v;
  v.full = s;
  return Value(v.half.mg + (uint16_t(v.half.eg) >> 15));
}

inline Value eg_value(Score s) {
  ScoreView v;
  v.full = s;
  return Value(v.half.eg);
}

5-I saw -"I don't remember where"- someone recommending the usage of SF's mobility weights in his engine directly , wouldn't this be considered unethical if not illegal ? or is copying values "PST,Mob,etc" considered okay ?

[syzygy]

1-what unit "grain" does SF use for evaluation ? I read the source "7" and a couple of old posts here that made me think that I could be missing something .

Code: Select all

  PawnValueMg   = 188,   PawnValueEg   = 248

2-What advantage does SF gain by combining mid and end scores inside a single INT instead of using 2 shorts "inside a Struct or a union if needed"?

Speed, for example.

3-I stumbled upon an old post here that claims that storing only the 32bit upper portion of the key degrades performance "I know it causes more collisions" ,yet SF does this ?!

Storing 32 bits of the key in TT entries should be more than fine, since you would only get a collision once every 4 billion probes (or once every 1 billion probes if you store 4 entries per hash bucket).

However, I think SF stores just 16 bits and that leads to tons of collisions per second. Somehow SF seems to play reasonable chess anyway and there is no real evidence that it affects play negatively. (It tested positively on fishtest, obviously). The advantage of storing fewer bits is of course that it allows you to store more TT entries in the same amount of memory.

4-what does this code snippet do "does it have something to do with the sign " ?

Code: Select all

inline Score make_score(int mg, int eg) {
  ScoreView v;
  v.half.mg = (int16_t)(mg - (uint16_t(eg) >> 15));
  v.half.eg = (int16_t)eg;
  return Score(v.full);
}

...

That is old SF code. Yes, it is something that makes the addition / subtraction of two signed 16-bit ints wrapped inside a single 32-bit int work correctly.

5-I saw -"I don't remember where"- someone recommending the usage of SF's mobility weights in his engine directly , wouldn't this be considered unethical if not illegal ? or is copying values "PST,Mob,etc" considered okay ?

I highly doubt there is copyright on such values which have been found by lots of tuning and are not the expression of creative freedom. So in my view this is legal.

Whether it is ethical depends on who you ask (and also on what you're actually doing with your engine).

[Milos]

1-what unit "grain" does SF use for evaluation ? I read the source "7" and a couple of old posts here that made me think that I could be missing something .

Code: Select all

  PawnValueMg   = 188,   PawnValueEg   = 248

2-What advantage does SF gain by combining mid and end scores inside a single INT instead of using 2 shorts "inside a Struct or a union if needed"?

Speed, for example.

That's pretty weak reason, making code significantly less readable just to gain some really unmeasurable speed advantage. Like your "plyCounters" instead of "pliesFromNull" + "rule50" in Cfish. If you really wanted speed optimization no matter how bad the code readability becomes afterwards there is ton of things to be done in move generator for example instead of just literally copying it 1:1 from SF.

However, I think SF stores just 16 bits and that leads to tons of collisions per second. Somehow SF seems to play reasonable chess anyway and there is no real evidence that it affects play negatively. (It tested positively on fishtest, obviously). The advantage of storing fewer bits is of course that it allows you to store more TT entries in the same amount of memory.

The impact is pretty big, but one thing is compensated with the other, more hash with a lot of collision is worth more than less hash without collision. But that's fishtest only coz it is very much dependent on the test conditions. Increase available hash in fishtest and suddenly you'll notice 16bit version becoming much worse. If you really wanna do it smart, increase key to 24bits and reduce value and eval value to 12bits and you'll see significant jump in performance. But well, I'm not SF developer, so why would I care...

Whether it is ethical depends on who you ask (and also on what you're actually doing with your engine).

Actually not what you are doing with the engine but how strong your engine is. That's the whole point. Do you endanger someone else's interests with it.

[syzygy]

That's pretty weak reason, making code significantly less readable just to gain some really unmeasurable speed advantage. Like your "plyCounters" instead of "pliesFromNull" + "rule50" in Cfish. If you really wanted speed optimization no matter how bad the code readability becomes afterwards there is ton of things to be done in move generator for example instead of just literally copying it 1:1 from SF.

Hey, you can be much more insulting than that! Try again.

[Dann Corbit]

1-what unit "grain" does SF use for evaluation ? I read the source "7" and a couple of old posts here that made me think that I could be missing something .

Code: Select all

  PawnValueMg   = 188,   PawnValueEg   = 248

2-What advantage does SF gain by combining mid and end scores inside a single INT instead of using 2 shorts "inside a Struct or a union if needed"?

Speed, for example.

3-I stumbled upon an old post here that claims that storing only the 32bit upper portion of the key degrades performance "I know it causes more collisions" ,yet SF does this ?!

Storing 32 bits of the key in TT entries should be more than fine, since you would only get a collision once every 4 billion probes (or once every 1 billion probes if you store 4 entries per hash bucket).

However, I think SF stores just 16 bits and that leads to tons of collisions per second. Somehow SF seems to play reasonable chess anyway and there is no real evidence that it affects play negatively. (It tested positively on fishtest, obviously). The advantage of storing fewer bits is of course that it allows you to store more TT entries in the same amount of memory.

4-what does this code snippet do "does it have something to do with the sign " ?

Code: Select all

inline Score make_score(int mg, int eg) {
  ScoreView v;
  v.half.mg = (int16_t)(mg - (uint16_t(eg) >> 15));
  v.half.eg = (int16_t)eg;
  return Score(v.full);
}

...

That is old SF code. Yes, it is something that makes the addition / subtraction of two signed 16-bit ints wrapped inside a single 32-bit int work correctly.

5-I saw -"I don't remember where"- someone recommending the usage of SF's mobility weights in his engine directly , wouldn't this be considered unethical if not illegal ? or is copying values "PST,Mob,etc" considered okay ?

I highly doubt there is copyright on such values which have been found by lots of tuning and are not the expression of creative freedom. So in my view this is legal.

Whether it is ethical depends on who you ask (and also on what you're actually doing with your engine).

Use of pure math is always OK. You can't patent math.

Use of algorithms is always OK, unless there is a patent. I am not aware of any patents on common chess algorithms, but that does not mean that no patents exist.

Use of small bits of code is OK, if attribution is given. This is called "fair use".

To use a whole routine or something of that nature is almost certainly illegal, unless your program is GPL.

Even if your program is GPL, you should still give credit and attribution for any code that is borrowed from another program.

[Milos]

That's pretty weak reason, making code significantly less readable just to gain some really unmeasurable speed advantage. Like your "plyCounters" instead of "pliesFromNull" + "rule50" in Cfish. If you really wanted speed optimization no matter how bad the code readability becomes afterwards there is ton of things to be done in move generator for example instead of just literally copying it 1:1 from SF.

Hey, you can be much more insulting than that! Try again.

You can also be much more pompous than that, Uwe Hoogheid .

[Dann Corbit]

One additional caveat:

If the code you borrow is not from a GPL program like Stockfish, you will have to carefully check the license.

If it is ordinary copyright (like for instance in the case of crafty) you will have to have written permission for anything other than fair use.

If a program is public domain you can do anything you want to with the code. However, it would still be good to give attribution, even if it is not legally demanded. It seems to me that it is morally demanded.

IMO-YMMV, especially on the last paragraph.

[Fulvio]

Code: Select all

inline Score make_score(int mg, int eg) {
  ScoreView v;
  v.half.mg = (int16_t)(mg - (uint16_t(eg) >> 15));
  v.half.eg = (int16_t)eg;
  return Score(v.full);
}

The standard 4.7 interger coverversion says:
1) "If the destination type is signed, the value is unchanged if it can be represented in the destination type; otherwise, the value is implementation-defined."
2) "If the destination type is unsigned, the resulting value is the least unsigned integer congruent to the source integer (modulo 2n where n is the number of bits used to represent the unsigned type)."

and 3.9.1 fundamental types: "Plain ints have the natural size suggested by the architecture of the execution environment"

This means that the above code is not portable and, without knowing the architecture and compiler, it's impossible to say what it will do.

[MahmoudUthman]

1-what unit "grain" does SF use for evaluation ? I read the source "7" and a couple of old posts here that made me think that I could be missing something .

Code: Select all

  PawnValueMg   = 188,   PawnValueEg   = 248

2-What advantage does SF gain by combining mid and end scores inside a single INT instead of using 2 shorts "inside a Struct or a union if needed"?

Speed, for example.

3-I stumbled upon an old post here that claims that storing only the 32bit upper portion of the key degrades performance "I know it causes more collisions" ,yet SF does this ?!

Storing 32 bits of the key in TT entries should be more than fine, since you would only get a collision once every 4 billion probes (or once every 1 billion probes if you store 4 entries per hash bucket).

However, I think SF stores just 16 bits and that leads to tons of collisions per second. Somehow SF seems to play reasonable chess anyway and there is no real evidence that it affects play negatively. (It tested positively on fishtest, obviously). The advantage of storing fewer bits is of course that it allows you to store more TT entries in the same amount of memory.

4-what does this code snippet do "does it have something to do with the sign " ?

Code: Select all

inline Score make_score(int mg, int eg) {
  ScoreView v;
  v.half.mg = (int16_t)(mg - (uint16_t(eg) >> 15));
  v.half.eg = (int16_t)eg;
  return Score(v.full);
}

...

That is old SF code. Yes, it is something that makes the addition / subtraction of two signed 16-bit ints wrapped inside a single 32-bit int work correctly.

5-I saw -"I don't remember where"- someone recommending the usage of SF's mobility weights in his engine directly , wouldn't this be considered unethical if not illegal ? or is copying values "PST,Mob,etc" considered okay ?

I highly doubt there is copyright on such values which have been found by lots of tuning and are not the expression of creative freedom. So in my view this is legal.

Whether it is ethical depends on who you ask (and also on what you're actually doing with your engine).

2-due to locality of reference ?
4-could you explain how/why it works ?

[syzygy]

2-due to locality of reference ?

Adding 2 32-bit ints is faster than two times adding 2 16-bit ints.

4-could you explain how/why it works ?

Well, I guess the important thing here is to realise that the straightforward solution does not work. If you just string together the 2's-complement representations of two ints and start adding, bits from the lower 16-bit parts can overflow into the higher 16-bit part. To correct for that, you need to do something extra. And now to see that the code snippet does get things right, it's best to just write out on a piece of paper what happens when adding two Score values.

In current Stockfish, the code reads like this:

Code: Select all

enum Score : int { SCORE_ZERO };

inline Score make_score(int mg, int eg) {
  return Score&#40;&#40;int&#41;(&#40;unsigned int&#41;eg << 16&#41; + mg&#41;;
&#125;

/// Extracting the signed lower and upper 16 bits is not so trivial because
/// according to the standard a simple cast to short is implementation defined
/// and so is a right shift of a signed integer.
inline Value eg_value&#40;Score s&#41; &#123;

  union &#123; uint16_t u; int16_t s; &#125; eg = &#123; uint16_t&#40;unsigned&#40;s + 0x8000&#41; >> 16&#41; &#125;;
  return Value&#40;eg.s&#41;;
&#125;

inline Value mg_value&#40;Score s&#41; &#123;

  union &#123; uint16_t u; int16_t s; &#125; mg = &#123; uint16_t&#40;unsigned&#40;s&#41;) &#125;;
  return Value&#40;mg.s&#41;;
&#125;