Scaling from FGRL results with top 3 engines

[Dann Corbit]

If an engine scales better, it is most likely search that is better (lower branching factor).

The second most likely thing would be the SMP implementation.

The evaluation will not affect scaling much, except for improvement in the move ordering.

I think that better search does not mean lower branching factor

It is easy to get lower branching factor by dubious pruning.

I think that evaluation is important and I expect top engines not to scale well if you change their evaluation to simple piece square table evaluation.

Every single great advancement is chess engines has been due to a reduction in branching factor. While it is obviously a mistake to prune away good stuff let's take a quick look at the list:

1) Alpha-Beta : Enormous improvement over mini-max
2) Null move reduction: Enormous improvement over plain alpha-beta
3) PVS search: Modest improvement over null move reduction due to zero window searches
4) History Reductions: (As pioneered by Fruit) - huge improvent over plain PVS search
5) Smooth scaling reductions in null move pruning (As, for instance, Stockfish) - significant improvement over ordinary null move
6) Razoring (like Rybka and Strelka): Enormous improvement over plain pvs search
7) Late Move Reductions: (with Tord taking the lead in both effectiveness and publication) -- a huge improvement over not having LMR.

There are, of course, many others that I did not mention here.

It is not a coincidence that the top ten engines all have branching factors of about 2, and it is not a coincidence that most weak engines have a large branching factor.

Now, your point in well taken with individual cases. For instance, ExChess had the best branching factor of all engines at one point. But it was not the strongest engine by far. So poorly tuned reductions are not nearly so beneficial as properly tuned reductions.

But almost every big advancement comes from a reduction in branching factor and the next revolution will come from a reduction in branching factor.

There are, of course, some exceptions. The material imbalance table in Rybka was another revolution, and almost entirely due to evaluation improvement in that case (as a 'for instance'). We can thank Larry Kaufman for that, I think.

[Dann Corbit]

It is also true that better evaluation will reduce branching factor, principally by improvement in move ordering (which is very important to the fundamental alpha-beta step).

There are other things that tangentially improve branching factor like hash tables and IID.

It is also true that pure wood counting is not good enough. But examine the effectiveness of Olithink, which has an incredibly simply eval. It has more than just wood, but an engine can be made very strong almost exclusively through search. I guess that grafting Stockfish evaluation into a minimax engine you will get less than 2000 Elo.

I guess that grafting Olithink eval into Stockfish you will still get more than 3000 Elo.

Note that I did not test this, it is only a gedankenexperiment.

[Dann Corbit]

If an engine scales better, it is most likely search that is better (lower branching factor).

The second most likely thing would be the SMP implementation.

The evaluation will not affect scaling much, except for improvement in the move ordering.

sorry Dann, but that is all BS.

1) SMP is irrelevant, as Andreas' tests are conducted with single thread

2) BF has nothing to do with elo, are you aware of that? an engine with higher BF might have higher elo.

3) evaluation will not affect scaling much, hmm, I said 'king safety' and not 'evaluation', king safety might be related to both evaluation and search(and move ordering too, for that matter).

I have not based my observations on pure speculation, but on following extreme number of games at STC between the tops.

also, see Kai's statistics, seems to be pointing at exactly my hypothesis.

>>
2) BF has nothing to do with elo, are you aware of that? an engine with higher BF might have higher elo.
<<
You're a funny man.

you are even funnier.

Let me put it another way:
This is utterly, ludicrously, and absurdly false:
"2) BF has nothing to do with elo, are you aware of that?"
Even though this is true:
"an engine with higher BF might have higher elo."

It is like saying:
"Being in good physical shape has no correlation to winning an Olympic event. And a fat, out of shape athlete may beat one who is in good shape."

The first part of the sentence is so dumb it is funny, even though the second half is true.

[Lyudmil Tsvetkov]

If an engine scales better, it is most likely search that is better (lower branching factor).

The second most likely thing would be the SMP implementation.

The evaluation will not affect scaling much, except for improvement in the move ordering.

I think that better search does not mean lower branching factor

It is easy to get lower branching factor by dubious pruning.

I think that evaluation is important and I expect top engines not to scale well if you change their evaluation to simple piece square table evaluation.

Every single great advancement is chess engines has been due to a reduction in branching factor. While it is obviously a mistake to prune away good stuff let's take a quick look at the list:

1) Alpha-Beta : Enormous improvement over mini-max
2) Null move reduction: Enormous improvement over plain alpha-beta
3) PVS search: Modest improvement over null move reduction due to zero window searches
4) History Reductions: (As pioneered by Fruit) - huge improvent over plain PVS search
5) Smooth scaling reductions in null move pruning (As, for instance, Stockfish) - significant improvement over ordinary null move
6) Razoring (like Rybka and Strelka): Enormous improvement over plain pvs search
7) Late Move Reductions: (with Tord taking the lead in both effectiveness and publication) -- a huge improvement over not having LMR.

There are, of course, many others that I did not mention here.

It is not a coincidence that the top ten engines all have branching factors of about 2, and it is not a coincidence that most weak engines have a large branching factor.

Now, your point in well taken with individual cases. For instance, ExChess had the best branching factor of all engines at one point. But it was not the strongest engine by far. So poorly tuned reductions are not nearly so beneficial as properly tuned reductions.

But almost every big advancement comes from a reduction in branching factor and the next revolution will come from a reduction in branching factor.

There are, of course, some exceptions. The material imbalance table in Rybka was another revolution, and almost entirely due to evaluation improvement in that case (as a 'for instance'). We can thank Larry Kaufman for that, I think.

so, what makes you think Komodo has better BF than SF?
what is the connection to LTC scaling?

[Lyudmil Tsvetkov]

It is also true that better evaluation will reduce branching factor, principally by improvement in move ordering (which is very important to the fundamental alpha-beta step).

There are other things that tangentially improve branching factor like hash tables and IID.

It is also true that pure wood counting is not good enough. But examine the effectiveness of Olithink, which has an incredibly simply eval. It has more than just wood, but an engine can be made very strong almost exclusively through search. I guess that grafting Stockfish evaluation into a minimax engine you will get less than 2000 Elo.

I guess that grafting Olithink eval into Stockfish you will still get more than 3000 Elo.

Note that I did not test this, it is only a gedankenexperiment.

so, no search without eval.

I guess you are grossly wrong about both the 2000 and 3000 elo mark.

wanna try one of the 2?

Olithink eval into SF will play something like 1500 elo, wanna bet?

I guess it is time to change gedankenexperiment for realitaetsueberpruefung...

[Uri Blass]

It is also true that better evaluation will reduce branching factor, principally by improvement in move ordering (which is very important to the fundamental alpha-beta step).

There are other things that tangentially improve branching factor like hash tables and IID.

It is also true that pure wood counting is not good enough. But examine the effectiveness of Olithink, which has an incredibly simply eval. It has more than just wood, but an engine can be made very strong almost exclusively through search. I guess that grafting Stockfish evaluation into a minimax engine you will get less than 2000 Elo.

I guess that grafting Olithink eval into Stockfish you will still get more than 3000 Elo.

Note that I did not test this, it is only a gedankenexperiment.

so, no search without eval.

I guess you are grossly wrong about both the 2000 and 3000 elo mark.

wanna try one of the 2?

Olithink eval into SF will play something like 1500 elo, wanna bet?

I guess it is time to change gedankenexperiment for realitaetsueberpruefung...

I believe that you are wrong and I expect stockfish with olithink evaluation to be not only better than 2000 but also better than olithink that is better than 2000.

Stockfish's search is of course not tuned to olithink's evaluation but I still think that it is better than olithink's search for stockfish's evaluation.

[Uri Blass]

If an engine scales better, it is most likely search that is better (lower branching factor).

The second most likely thing would be the SMP implementation.

The evaluation will not affect scaling much, except for improvement in the move ordering.

I think that better search does not mean lower branching factor

It is easy to get lower branching factor by dubious pruning.

I think that evaluation is important and I expect top engines not to scale well if you change their evaluation to simple piece square table evaluation.

Every single great advancement is chess engines has been due to a reduction in branching factor. While it is obviously a mistake to prune away good stuff let's take a quick look at the list:

1) Alpha-Beta : Enormous improvement over mini-max
2) Null move reduction: Enormous improvement over plain alpha-beta
3) PVS search: Modest improvement over null move reduction due to zero window searches
4) History Reductions: (As pioneered by Fruit) - huge improvent over plain PVS search
5) Smooth scaling reductions in null move pruning (As, for instance, Stockfish) - significant improvement over ordinary null move
6) Razoring (like Rybka and Strelka): Enormous improvement over plain pvs search
7) Late Move Reductions: (with Tord taking the lead in both effectiveness and publication) -- a huge improvement over not having LMR.

There are, of course, many others that I did not mention here.

It is not a coincidence that the top ten engines all have branching factors of about 2, and it is not a coincidence that most weak engines have a large branching factor.

Now, your point in well taken with individual cases. For instance, ExChess had the best branching factor of all engines at one point. But it was not the strongest engine by far. So poorly tuned reductions are not nearly so beneficial as properly tuned reductions.

But almost every big advancement comes from a reduction in branching factor and the next revolution will come from a reduction in branching factor.

There are, of course, some exceptions. The material imbalance table in Rybka was another revolution, and almost entirely due to evaluation improvement in that case (as a 'for instance'). We can thank Larry Kaufman for that, I think.

I agree about the history.
I do not think it means that always the future is going to be reduction of the branching factor.

The target is to play better and not to reduce the branching factor and I see no reason to assume that the next improvement is going to be more reductions and it also can be more extensions of the right lines.

[Dann Corbit]

It is also true that better evaluation will reduce branching factor, principally by improvement in move ordering (which is very important to the fundamental alpha-beta step).

There are other things that tangentially improve branching factor like hash tables and IID.

It is also true that pure wood counting is not good enough. But examine the effectiveness of Olithink, which has an incredibly simply eval. It has more than just wood, but an engine can be made very strong almost exclusively through search. I guess that grafting Stockfish evaluation into a minimax engine you will get less than 2000 Elo.

I guess that grafting Olithink eval into Stockfish you will still get more than 3000 Elo.

Note that I did not test this, it is only a gedankenexperiment.

so, no search without eval.

I guess you are grossly wrong about both the 2000 and 3000 elo mark.

wanna try one of the 2?

Olithink eval into SF will play something like 1500 elo, wanna bet?

I guess it is time to change gedankenexperiment for realitaetsueberpruefung...

From CCRL 40/40;
216 OliThink 5.3.2 64-bit 2372 +19 −19 48.3% +12.5 25.6% 1011

With a super simple eval and a fairly simple search, it is already 2372.
Adding the incredible, sophisticated search of Stockfish will lower the eval by more than 872 points?

[Dann Corbit]

If an engine scales better, it is most likely search that is better (lower branching factor).

The second most likely thing would be the SMP implementation.

The evaluation will not affect scaling much, except for improvement in the move ordering.

I think that better search does not mean lower branching factor

It is easy to get lower branching factor by dubious pruning.

I think that evaluation is important and I expect top engines not to scale well if you change their evaluation to simple piece square table evaluation.

Every single great advancement is chess engines has been due to a reduction in branching factor. While it is obviously a mistake to prune away good stuff let's take a quick look at the list:

1) Alpha-Beta : Enormous improvement over mini-max
2) Null move reduction: Enormous improvement over plain alpha-beta
3) PVS search: Modest improvement over null move reduction due to zero window searches
4) History Reductions: (As pioneered by Fruit) - huge improvent over plain PVS search
5) Smooth scaling reductions in null move pruning (As, for instance, Stockfish) - significant improvement over ordinary null move
6) Razoring (like Rybka and Strelka): Enormous improvement over plain pvs search
7) Late Move Reductions: (with Tord taking the lead in both effectiveness and publication) -- a huge improvement over not having LMR.

There are, of course, many others that I did not mention here.

It is not a coincidence that the top ten engines all have branching factors of about 2, and it is not a coincidence that most weak engines have a large branching factor.

Now, your point in well taken with individual cases. For instance, ExChess had the best branching factor of all engines at one point. But it was not the strongest engine by far. So poorly tuned reductions are not nearly so beneficial as properly tuned reductions.

But almost every big advancement comes from a reduction in branching factor and the next revolution will come from a reduction in branching factor.

There are, of course, some exceptions. The material imbalance table in Rybka was another revolution, and almost entirely due to evaluation improvement in that case (as a 'for instance'). We can thank Larry Kaufman for that, I think.

so, what makes you think Komodo has better BF than SF?

I did not say that. I do not think it is clear which is better, but both have very good branching factors.

what is the connection to LTC scaling?

Suppose that engine A evaluates twice as many nodes to advance one ply. BF=2

Suppose that engine B evaluates three times as many nodes to advance one ply. BF=3

To get to 30 ply how many more nodes will B examine than A in orders of magnitude?

[Guenther]

It is also true that better evaluation will reduce branching factor, principally by improvement in move ordering (which is very important to the fundamental alpha-beta step).

There are other things that tangentially improve branching factor like hash tables and IID.

It is also true that pure wood counting is not good enough. But examine the effectiveness of Olithink, which has an incredibly simply eval. It has more than just wood, but an engine can be made very strong almost exclusively through search. I guess that grafting Stockfish evaluation into a minimax engine you will get less than 2000 Elo.

I guess that grafting Olithink eval into Stockfish you will still get more than 3000 Elo.

Note that I did not test this, it is only a gedankenexperiment.

so, no search without eval.

I guess you are grossly wrong about both the 2000 and 3000 elo mark.

wanna try one of the 2?

Olithink eval into SF will play something like 1500 elo, wanna bet? :)

I guess it is time to change gedankenexperiment for realitaetsueberpruefung... :)

From CCRL 40/40;
216 OliThink 5.3.2 64-bit 2372 +19 −19 48.3% +12.5 25.6% 1011

With a super simple eval and a fairly simple search, it is already 2372.
Adding the incredible, sophisticated search of Stockfish will lower the eval by more than 872 points?

Discussing with him is not very fruitful, don't forget that.