Questions for the Stockfish team

[Daniel Shawul]

I completely agree. Bugs are very bad to have. Many times I was surprised with finding bugs in parts of my engine I never suspected. Parts you write once and never look back like SEE and transposition tables ... but those are exactly the places I recently found huge bugs. My TT replacement was random and SEE completely wrong for half of the calls..

[Dann Corbit]

When you say random eval , do you mean tactics(material eval) + random score for positional terms OR just random score for all... In the later case I can't see how it can reach 1800 elo.. All the engine will do is make random moves throwing away material. It may try to avoid mates when it sees them through search but how can it possibly think intelligently when it is fed garbage at the end.

If I recall correctly, there are a couple of efforts to make a chess engine that simply execute a random legal move. I guess that they will score poorly.

As I recall, you could play chess against Steven J. Edward's SANKIT in that manner.

[Tord Romstad]

When you say random eval , do you mean tactics(material eval) + random score for positional terms OR just random score for all... In the later case I can't see how it can reach 1800 elo..

Yes, must be material + random score, unless Crafty is very different from Stockfish in this respect. I just tried a blitz game against Stockfish with random eval, and won very easily just by capturing undefended pieces all over the place. Of course it's impossible to judge the strength after just a single game, but I doubt it's even 800. It did play a little better than you would expect from a program with a completely random eval, though.

Code: Select all

[Event "Test game"]
[Site "Oslo"]
[Date "2010.07.20"]
[Round "-"]
[White "tord"]
[Black "Stockfish 100720 64bit"]
[Result "1-0"]
[TimeControl "2+1"]

1. e4 c6 2. d4 d5 3. f3 dxe4 4. fxe4 Qa5+ 5. c3 Nf6 6. Bd3 e6 7. Nf3 Nbd7
8. e5 Qd5 9. exf6 a6 10. fxg7 Bxg7 11. O-O a5 12. Be3 Rg8 13. c4 Qh5 14.
Nc3 Be5 15. dxe5 Rg4 16. Ne4 h6 17. Nd6+ Kd8 18. Bd4 Rxg2+ 19. Kxg2 c5 20.
Bf2 Ra6 21. Bg3 Rxd6 22. exd6 b6 23. Qa4 Bb7 24. Rae1 Kc8 25. Be4 Kd8 26.
Bxb7 Ke8 27. Ne5 b5
{Black resigns} 1-0

[mcostalba]

When you say random eval , do you mean tactics(material eval) + random score for positional terms OR just random score for all... In the later case I can't see how it can reach 1800 elo.. All the engine will do is make random moves throwing away material. It may try to avoid mates when it sees them through search but how can it possibly think intelligently when it is fed garbage at the end.

In another thread Vratko suggested that a full random evaluation is a kind of indirect mobiility evaluation.

I try to explain with one example.

Let's say you have 2 leaf positions A and B. In A mobility is 15 (it means you have 15 possible legal moves), in B mobility is 40.

Now you start from A and loop across the 15 moves, after each move let's get the evaluation that is a random number in range, for instance, [0, 1000].

So the evaluation of position A is the biggest among 15 random numbers between 0 and 1000.

Repeat for B, now you have 40 legal moves so the position B score is the biggest across 40 random numbers between 0 and 1000.

It is clear that B has a bigger probability to have a score higher then A because you run the dices 40 times instead of 15, so although evaluation is random in both cases it happens that is more probable that B scores higher then A.

And so it is more probable that a node with bigger mobility scores higher then a node of lower mobility, so at the end this interesting random evaluation is a kind of indirectly score mobility.

Given this is no surprise that the engine with random evaluation is not _completely_ random in his moves, but has a kind of mobility evaluation built-in.

[Daniel Shawul]

I just tried random eval by returning (hash_key % 1000) . Rand() would have returned different score for same position. The result was bad as expected.

[mcostalba]

I just tried random eval by returning (hash_key % 1000) . Rand() would have returned different score for same position. The result was bad as expected.

Perhaps the captures make all very difficult.

A possible experiment would be the following:

1) Take one engine and remove evaluation but the bare material score. Call this engine A

2) Take engine A and add a random evaluation as a second order term, for instance add a random number to the score within the range [-pawnValue / 2, pawnValue / 2], call this engine B

3) Pit A against B, I would guess B is stronger.

[Daniel Shawul]

Ok maybe it does approximate mobility as you say which is one positional term but I really don't see how it can reach 1800 elo without basic material eval , unless I overvalued the elo number. Someone told me strelka can be a 2600 elo engine with material only eval. So my thought was it is possible to reach more than 1800 elo with material only but the sense I got from the discussion is that this comes from no eval..

[Daniel Shawul]

Where can I download the kit ? I remember you mentioning it here before but I never thought it had some sense to do that experimeint.

[mcostalba]

Where can I download the kit ? I remember you mentioning it here before but I never thought it had some sense to do that experimeint.

Which kit ?

[Daniel Shawul]

I googled it http://cfajohnson.com/chess/SAN/