Scaling from FGRL results with top 3 engines

[Dann Corbit]

If an engine scales better, it is most likely search that is better (lower branching factor).

The second most likely thing would be the SMP implementation.

The evaluation will not affect scaling much, except for improvement in the move ordering.

I think that better search does not mean lower branching factor

It is easy to get lower branching factor by dubious pruning.

I think that evaluation is important and I expect top engines not to scale well if you change their evaluation to simple piece square table evaluation.

Every single great advancement is chess engines has been due to a reduction in branching factor. While it is obviously a mistake to prune away good stuff let's take a quick look at the list:

1) Alpha-Beta : Enormous improvement over mini-max
2) Null move reduction: Enormous improvement over plain alpha-beta
3) PVS search: Modest improvement over null move reduction due to zero window searches
4) History Reductions: (As pioneered by Fruit) - huge improvent over plain PVS search
5) Smooth scaling reductions in null move pruning (As, for instance, Stockfish) - significant improvement over ordinary null move
6) Razoring (like Rybka and Strelka): Enormous improvement over plain pvs search
7) Late Move Reductions: (with Tord taking the lead in both effectiveness and publication) -- a huge improvement over not having LMR.

There are, of course, many others that I did not mention here.

It is not a coincidence that the top ten engines all have branching factors of about 2, and it is not a coincidence that most weak engines have a large branching factor.

Now, your point in well taken with individual cases. For instance, ExChess had the best branching factor of all engines at one point. But it was not the strongest engine by far. So poorly tuned reductions are not nearly so beneficial as properly tuned reductions.

But almost every big advancement comes from a reduction in branching factor and the next revolution will come from a reduction in branching factor.

There are, of course, some exceptions. The material imbalance table in Rybka was another revolution, and almost entirely due to evaluation improvement in that case (as a 'for instance'). We can thank Larry Kaufman for that, I think.

I agree about the history.
I do not think it means that always the future is going to be reduction of the branching factor.

The target is to play better and not to reduce the branching factor and I see no reason to assume that the next improvement is going to be more reductions and it also can be more extensions of the right lines.

Branching factor improvement is exponential improvement.
Other improvements will not be as astounding.
Until branching factor becomes one, it will always be possible to improve it.

I also agree that a perfect evaluation would lead to a branching factor of 1.
It is just that a perfect evaluation is probably many times more difficult to do and exponential improvements via search happen all the time.

In fact, I think that the key to beating SF is simple. Stop focusing on eval and focus on search. The SF team spends way too much time looking at eval and not enough time looking at search.

In this sense, you can call me a disciple of Christophe Theron, who said:
"Search is also knowledge."

if it were that simple, someone would already have done it.

from SF framework stats, search and eval patches are split about equal.

so what to do more with search, without improving eval on a par?

someone says, well, about the most important thing in chess programming
is move ordering. well, how do you achieve better move ordering without necessarily resorting to a more advanced move ordering function, which one way or another has to deal with a more refined eval?

Once move ordering passes 95% correct, the rest is fluff mathematically.

And someone will have already done it?
Yes, of course.
Every drop in branching factor (probably there are 50 by now) is a literal revolution in chess engine strength.

Do you never wonder why the expansion in strength of chess engines is exponential in time (possibly even super-exponential)?
Clearly, this is 99% due to branching factor.

Do you understand what the difference between:
36^40
and 1.8^40
IS?

Hint:
1.7868991024601705453143247728944e+62/16251752663 is a very big number.

The first number is the approximate value for a 40 ply search using mini-max
The second number is the same search using the strongest programs today.

The ratio is a truly enormous number.

[Uri Blass]

It is also true that better evaluation will reduce branching factor, principally by improvement in move ordering (which is very important to the fundamental alpha-beta step).

There are other things that tangentially improve branching factor like hash tables and IID.

It is also true that pure wood counting is not good enough. But examine the effectiveness of Olithink, which has an incredibly simply eval. It has more than just wood, but an engine can be made very strong almost exclusively through search. I guess that grafting Stockfish evaluation into a minimax engine you will get less than 2000 Elo.

I guess that grafting Olithink eval into Stockfish you will still get more than 3000 Elo.

Note that I did not test this, it is only a gedankenexperiment.

so, no search without eval.

I guess you are grossly wrong about both the 2000 and 3000 elo mark.

wanna try one of the 2?

Olithink eval into SF will play something like 1500 elo, wanna bet?

I guess it is time to change gedankenexperiment for realitaetsueberpruefung...

From CCRL 40/40;
216 OliThink 5.3.2 64-bit 2372 +19 −19 48.3% +12.5 25.6% 1011

With a super simple eval and a fairly simple search, it is already 2372.
Adding the incredible, sophisticated search of Stockfish will lower the eval by more than 872 points?

of course, it is all about tuning.

we are not speaking here of downgrading SF, leaving all its search and using just a dozen basic eval terms, in which case SF will still be somewhat strong, but of patching an entirely alien eval onto SF search.

as the eval and search will not be tuned to each other, you will mostly get completely random results.

Based on my experience with a different engine in the past(strelka) it is not the case.
I changed strelka's evaluation to a simple piece square table and was surprised to see that it is very strong at least in fast time control and beat engine like Joker in most games when joker has near 2300 CCRL rating.

Note that strelka's piece square table was simply the piece square table from the strelka's code that is not optimized and it was clear from watching the games that strelka could be better by increasing the value of the knight and bishop so it knows that knight and bishop are more than rook and a pawn.

In the original code strelka has imbalance table but I throwed all the code except piece square table.

I guess that strelka could get at least 2400 CCRL rating with piece square table evaluation at 40/4 time control and I have no reason to believe that it is going to be worse for stockfish inspite of the fact that stockfish is not tuned for olithink's evaluation(strelka is also not tuned for simplified strelka's evaluation that does not include pawn structure mobility and king safety).

[Uri Blass]

If an engine scales better, it is most likely search that is better (lower branching factor).

The second most likely thing would be the SMP implementation.

The evaluation will not affect scaling much, except for improvement in the move ordering.

I think that better search does not mean lower branching factor

It is easy to get lower branching factor by dubious pruning.

I think that evaluation is important and I expect top engines not to scale well if you change their evaluation to simple piece square table evaluation.

Every single great advancement is chess engines has been due to a reduction in branching factor. While it is obviously a mistake to prune away good stuff let's take a quick look at the list:

1) Alpha-Beta : Enormous improvement over mini-max
2) Null move reduction: Enormous improvement over plain alpha-beta
3) PVS search: Modest improvement over null move reduction due to zero window searches
4) History Reductions: (As pioneered by Fruit) - huge improvent over plain PVS search
5) Smooth scaling reductions in null move pruning (As, for instance, Stockfish) - significant improvement over ordinary null move
6) Razoring (like Rybka and Strelka): Enormous improvement over plain pvs search
7) Late Move Reductions: (with Tord taking the lead in both effectiveness and publication) -- a huge improvement over not having LMR.

There are, of course, many others that I did not mention here.

It is not a coincidence that the top ten engines all have branching factors of about 2, and it is not a coincidence that most weak engines have a large branching factor.

Now, your point in well taken with individual cases. For instance, ExChess had the best branching factor of all engines at one point. But it was not the strongest engine by far. So poorly tuned reductions are not nearly so beneficial as properly tuned reductions.

But almost every big advancement comes from a reduction in branching factor and the next revolution will come from a reduction in branching factor.

There are, of course, some exceptions. The material imbalance table in Rybka was another revolution, and almost entirely due to evaluation improvement in that case (as a 'for instance'). We can thank Larry Kaufman for that, I think.

I agree about the history.
I do not think it means that always the future is going to be reduction of the branching factor.

The target is to play better and not to reduce the branching factor and I see no reason to assume that the next improvement is going to be more reductions and it also can be more extensions of the right lines.

Branching factor improvement is exponential improvement.
Other improvements will not be as astounding.
Until branching factor becomes one, it will always be possible to improve it.

I also agree that a perfect evaluation would lead to a branching factor of 1.
It is just that a perfect evaluation is probably many times more difficult to do and exponential improvements via search happen all the time.

In fact, I think that the key to beating SF is simple. Stop focusing on eval and focus on search. The SF team spends way too much time looking at eval and not enough time looking at search.

In this sense, you can call me a disciple of Christophe Theron, who said:
"Search is also knowledge."

if it were that simple, someone would already have done it.

from SF framework stats, search and eval patches are split about equal.

so what to do more with search, without improving eval on a par?

someone says, well, about the most important thing in chess programming
is move ordering. well, how do you achieve better move ordering without necessarily resorting to a more advanced move ordering function, which one way or another has to deal with a more refined eval?

Once move ordering passes 95% correct, the rest is fluff mathematically.

And someone will have already done it?
Yes, of course.
Every drop in branching factor (probably there are 50 by now) is a literal revolution in chess engine strength.

Do you never wonder why the expansion in strength of chess engines is exponential in time (possibly even super-exponential)?
Clearly, this is 99% due to branching factor.

Do you understand what the difference between:
36^40
and 1.8^40
IS?

Hint:
1.7868991024601705453143247728944e+62/16251752663 is a very big number.

The first number is the approximate value for a 40 ply search using mini-max
The second number is the same search using the strongest programs today.

The ratio is a truly enormous number.

In case of sound pruning you are right but pruning is not sound pruning.
At fixed depth bigger branching factor is going to win and it is not clear that reducing the branching factor is the way to continue.

Of course we care about fixed time and not about fixed depth but it is not clear that reducing the effective branching factor is the way to get better only because it helped in the past.

[Lyudmil Tsvetkov]

It is also true that better evaluation will reduce branching factor, principally by improvement in move ordering (which is very important to the fundamental alpha-beta step).

There are other things that tangentially improve branching factor like hash tables and IID.

It is also true that pure wood counting is not good enough. But examine the effectiveness of Olithink, which has an incredibly simply eval. It has more than just wood, but an engine can be made very strong almost exclusively through search. I guess that grafting Stockfish evaluation into a minimax engine you will get less than 2000 Elo.

I guess that grafting Olithink eval into Stockfish you will still get more than 3000 Elo.

Note that I did not test this, it is only a gedankenexperiment.

so, no search without eval.

I guess you are grossly wrong about both the 2000 and 3000 elo mark.

wanna try one of the 2?

Olithink eval into SF will play something like 1500 elo, wanna bet?

I guess it is time to change gedankenexperiment for realitaetsueberpruefung...

From CCRL 40/40;
216 OliThink 5.3.2 64-bit 2372 +19 −19 48.3% +12.5 25.6% 1011

With a super simple eval and a fairly simple search, it is already 2372.
Adding the incredible, sophisticated search of Stockfish will lower the eval by more than 872 points?

of course, it is all about tuning.

we are not speaking here of downgrading SF, leaving all its search and using just a dozen basic eval terms, in which case SF will still be somewhat strong, but of patching an entirely alien eval onto SF search.

as the eval and search will not be tuned to each other, you will mostly get completely random results.

You are mostly right about that.
While good programming technique demands encapsulation, it is so ultra tempting to pierce that veil and get chummy with other parts of the program and show them your innards that virtually all programs do it.

I must mention Bas Hamstra's program, which was so beautifully crafted. But that is neither here nor there.

I guess that point I wanted to make is that branching factor (DONE PROPERLY) is the golden nail to better program success.

You point to eval. And eval has its place. But once (for instance) the fail high rate goes over 95% on the pv node, the rest is fluff, as far as BF goes. Now, there can be things to aim the engine better, I think everyone agrees on that. But if you are going to shock the world (and look at every world shocker) it is BF gains that drop the jaws and make the eyes bug out.

As I have said elsewhere, you are an interesting person and you know a lot about chess. But until you understand the complete implication of the branching factor, you cannot properly advice chess programmers.

The branching factor is the golden nail upon which all the kings will drape their mantles.

Mark my words,

Marking your words.
but before that, I will have to take a course on colloquial American.

neither me, nor you are chess programmers, as none of us has written/published a fully-fledged and functional chess engine from scratch.

I am not advising anyone, just sharing some thoughts.

with all your words and behaviour you want to deliver a single message:
'search is more important than evaluation what concerns performance of chess engines.'

and you drive your point home each and every time.

but this is simply not true.

just about everything in a chess engine revolves around eval, a specific estimate for each and every node:
- you call a function called eval() or something similar each and every node, unless you have a hash move
- to do move ordering, you are using hash moves, where the score is based on eval; killer moves are based on eval
- referring to main search functions, alpha is an evaluation estimate, beta too
- going to search routines, be it for futility pruning, razoring, null move reductions or something else, you are always using some kind of evaluation seed
- LMR/LMP, again, you need to order moves first, for which you use eval one way or another, and then reductions specifics again work only within a
particular evaluation framework

so that, to start a chess engine, after doing the move stack and generating the moves, the first thing you need is some kind of evaluation. search only comes in second.

of course, as a matter of fact, both are inseparable, but if I had to pick a more important factor, that would be evaluation.

and indeed, you can build a one-ply engine with some basic evaluation that will still pick some reasonable moves, while it is almost impossible to do the same with the most sophisticated search, provided the program does not know what the pieces are worth.

[Uri Blass]

It is also true that better evaluation will reduce branching factor, principally by improvement in move ordering (which is very important to the fundamental alpha-beta step).

There are other things that tangentially improve branching factor like hash tables and IID.

It is also true that pure wood counting is not good enough. But examine the effectiveness of Olithink, which has an incredibly simply eval. It has more than just wood, but an engine can be made very strong almost exclusively through search. I guess that grafting Stockfish evaluation into a minimax engine you will get less than 2000 Elo.

I guess that grafting Olithink eval into Stockfish you will still get more than 3000 Elo.

Note that I did not test this, it is only a gedankenexperiment.

so, no search without eval.

I guess you are grossly wrong about both the 2000 and 3000 elo mark.

wanna try one of the 2?

Olithink eval into SF will play something like 1500 elo, wanna bet?

I guess it is time to change gedankenexperiment for realitaetsueberpruefung...

From CCRL 40/40;
216 OliThink 5.3.2 64-bit 2372 +19 −19 48.3% +12.5 25.6% 1011

With a super simple eval and a fairly simple search, it is already 2372.
Adding the incredible, sophisticated search of Stockfish will lower the eval by more than 872 points?

of course, it is all about tuning.

we are not speaking here of downgrading SF, leaving all its search and using just a dozen basic eval terms, in which case SF will still be somewhat strong, but of patching an entirely alien eval onto SF search.

as the eval and search will not be tuned to each other, you will mostly get completely random results.

You are mostly right about that.
While good programming technique demands encapsulation, it is so ultra tempting to pierce that veil and get chummy with other parts of the program and show them your innards that virtually all programs do it.

I must mention Bas Hamstra's program, which was so beautifully crafted. But that is neither here nor there.

I guess that point I wanted to make is that branching factor (DONE PROPERLY) is the golden nail to better program success.

You point to eval. And eval has its place. But once (for instance) the fail high rate goes over 95% on the pv node, the rest is fluff, as far as BF goes. Now, there can be things to aim the engine better, I think everyone agrees on that. But if you are going to shock the world (and look at every world shocker) it is BF gains that drop the jaws and make the eyes bug out.

As I have said elsewhere, you are an interesting person and you know a lot about chess. But until you understand the complete implication of the branching factor, you cannot properly advice chess programmers.

The branching factor is the golden nail upon which all the kings will drape their mantles.

Mark my words,

Marking your words.
but before that, I will have to take a course on colloquial American.

neither me, nor you are chess programmers, as none of us has written/published a fully-fledged and functional chess engine from scratch.

I am not advising anyone, just sharing some thoughts.

with all your words and behaviour you want to deliver a single message:
'search is more important than evaluation what concerns performance of chess engines.'

and you drive your point home each and every time.

but this is simply not true.

just about everything in a chess engine revolves around eval, a specific estimate for each and every node:
- you call a function called eval() or something similar each and every node, unless you have a hash move
- to do move ordering, you are using hash moves, where the score is based on eval; killer moves are based on eval
- referring to main search functions, alpha is an evaluation estimate, beta too
- going to search routines, be it for futility pruning, razoring, null move reductions or something else, you are always using some kind of evaluation seed
- LMR/LMP, again, you need to order moves first, for which you use eval one way or another, and then reductions specifics again work only within a
particular evaluation framework

so that, to start a chess engine, after doing the move stack and generating the moves, the first thing you need is some kind of evaluation. search only comes in second.

of course, as a matter of fact, both are inseparable, but if I had to pick a more important factor, that would be evaluation.

and indeed, you can build a one-ply engine with some basic evaluation that will still pick some reasonable moves, while it is almost impossible to do the same with the most sophisticated search, provided the program does not know what the pieces are worth.

I agree that you need some evaluation to start but you can start with a very simple one.

I agree that search rules are basically based on some type of evaluation but it does not have to be the evaluation that give a single number to a position.

I would like engines to evaluate positions not in terms of advantage but in terms of confidence in the same way people know without search.

For example

1)Sure win for white

[D]2r5/Q7/4k3/1p6/2P5/1P1P4/P5PP/6K1 b - - 0 51

2)White does not lose
[D]4k3/3bppp1/8/8/8/8/3NPPPP/4K3 b - - 0 1

3)sure draw
[D]5rk1/5ppp/8/8/8/8/5PPP/5RK1 w - - 0 1

I think that these type of evaluations can be used simply for pruning.
For example as long as the computer evaluates that black is better then there is no reason to search forward lines that white does not lose and the computer can safely prune lines regardless of remaining depth.

[Kappatoo]

Do you understand what the difference between:
36^40
and 1.8^40
IS?

[...]

The second number is [about a 40 ply search] using the strongest programs today.

So modern engines calculate less than 2 moves per position in their search tree? That's amazing! Do you have an estimate what improvement the addition of specific pruning techniques yields, e.g. from minimax to alpha-beta, from there to null-move pruning, etc.?
And do you have a recommendation where I could learn more about this?

[Dann Corbit]

Do you understand what the difference between:
36^40
and 1.8^40
IS?

[...]

The second number is [about a 40 ply search] using the strongest programs today.

So modern engines calculate less than 2 moves per position in their search tree? That's amazing! Do you have an estimate what improvement the addition of specific pruning techniques yields, e.g. from minimax to alpha-beta, from there to null-move pruning, etc.?
And do you have a recommendation where I could learn more about this?

Pure minimax has a branching factor of 36 in the late opening and early middle game.

Pure alpha-beta has a branching factor of 6 under the same conditions, assuming that the moves are well ordered. It (6) is the square root of 36 and Knuth's analysis explains why this should be so.

The other reductions are variable because they are all implemented differently.

Consider null move reductions. Some people choose a number like R=3.
Other people have variable R, depending on game stage.
Some will have complicated zugzwang detection, and others simple (which prevents null move from being a good idea so it is turned off)
Other people will have much more clever implementations.

There are some engines that allow you to turn features off and on, and you can certainly do conditional compiles to find out what the answers are for a given engine.

[Dann Corbit]

Using this code:

Code: Select all

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>

#define NEED_STRTOK_R
#ifdef NEED_STRTOK_R
/*
 * Copyright &#40;c&#41; 1988 Regents of the University of California.
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met&#58;
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 * 3. Neither the name of the University nor the names of its contributors
 *    may be used to endorse or promote products derived from this software
 *    without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES &#40;INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION&#41;
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT &#40;INCLUDING NEGLIGENCE OR OTHERWISE&#41; ARISING IN ANY WAY
 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 */


/*
 * thread-safe version of strtok
 */
char           *strtok_r&#40;char *s, const char *delim, char **lasts&#41;
&#123;
    const char     *spanp;
    int             c,
                    sc;
    char           *tok;

    /* s may be NULL */
    assert&#40;delim != NULL&#41;;
    assert&#40;lasts != NULL&#41;;

    if &#40;s == NULL && &#40;s = *lasts&#41; == NULL&#41;
        return &#40;NULL&#41;;

    /* Skip &#40;span&#41; leading delimiters &#40;s += strspn&#40;s, delim&#41;, sort of&#41;. */
cont&#58;
    c = *s++;
    for &#40;spanp = delim; &#40;sc = *spanp++) != 0;) &#123;
        if &#40;c == sc&#41;
            goto cont;
    &#125;

    if &#40;c == 0&#41; &#123;               /* no non-delimiter characters */
        *lasts = NULL;
        return &#40;NULL&#41;;
    &#125;
    tok = s - 1;

    /* Scan token &#40;scan for delimiters&#58; s += strcspn&#40;s, delim&#41;, sort of&#41;.
     * Note that delim must have one NUL; we stop if we see that, too. */
    for (;;) &#123;
        c = *s++;
        spanp = delim;
        do &#123;
            if (&#40;sc = *spanp++) == c&#41; &#123;
                if &#40;c == 0&#41;
                    s = NULL;
                else
                    s&#91;-1&#93; = 0;
                *lasts = s;
                return &#40;tok&#41;;
            &#125;
        &#125;
        while &#40;sc != 0&#41;;
    &#125;
    /* NOTREACHED */
&#125;
#endif

char string0&#91;32767&#93;;
char string1&#91;32767&#93;;
char string2&#91;32767&#93;;

char *getsafe&#40;char *buffer, int count&#41;
&#123;
    char *result = buffer, *np;
    if (&#40;buffer == NULL&#41; || &#40;count < 1&#41;)
        result = NULL;
    else if &#40;count == 1&#41;
        *result = '\0';
    else if (&#40;result = fgets&#40;buffer, count, stdin&#41;) != NULL&#41;
        if &#40;np = strchr&#40;buffer, '\n'))
            *np = '\0';
    return result;
&#125;

static const char * p25 = "    25/";
static const char * tst = "PM, Time for this analysis&#58;";

int parsearg&#40;char *string&#41;
&#123;
    char *token=0;
    char *saveptr=0;
    int j;
    int seconds=0;
    token = strtok_r&#40;string," \t", &saveptr&#41;;
    if &#40;token != NULL&#41;
    &#123;
        token = strtok_r&#40;NULL," \t", &saveptr&#41;;
        if &#40;token != NULL&#41;
        &#123;
            char *sp = 0;
            char f0&#91;16&#93;= &#123;0&#125;;
            char f1&#91;16&#93;= &#123;0&#125;;
            char f2&#91;16&#93;= &#123;0&#125;;
            char *fields&#91;3&#93; = &#123;f0, f1, f2&#125;;
            char *p0 = strtok_r&#40;token, "&#58;", &sp&#41;;
            if &#40;p0&#41;
            &#123;
                strncpy&#40;fields&#91;0&#93;, p0, 15&#41;;
                char *p1 = strtok_r&#40;NULL, "&#58;", &sp&#41;;
                if &#40;p1&#41;
                &#123;
                    strncpy&#40;fields&#91;1&#93;, p1, 15&#41;;
                    char *p2 = strtok_r&#40;NULL, "&#58;", &sp&#41;;
                    if &#40;p2&#41;
                    &#123;
                        strncpy&#40;fields&#91;2&#93;, p2, 15&#41;;
                        seconds = atoi&#40;fields&#91;0&#93;) * 3600 + atoi&#40;fields&#91;1&#93;) * 60 + atoi&#40;fields&#91;2&#93;);
                    &#125;
                    else
                    &#123;
                        seconds = atoi&#40;fields&#91;0&#93;) * 60 + atoi&#40;fields&#91;1&#93;);
                    &#125;
                &#125;
            &#125;
        &#125;
    &#125;
    return seconds;
&#125;


int main&#40;void&#41;
&#123;
    size_t bfcnt = 0;
    double bfsum = 0;
    int bDoDiff = 0;
    int bIs25 = 0;
    puts&#40;"bf=");
    while &#40;getsafe&#40;string0, sizeof string0&#41;)
    &#123;
        if &#40;strstr&#40;string0, p25&#41; != NULL&#41; &#123;
            bIs25 = 1;
            bDoDiff = 1;
        &#125;
        else
        &#123;
            bIs25 = 0;
        &#125;

        if &#40;strstr&#40;string0, tst&#41; != NULL&#41; bDoDiff = 0;

        if &#40;bDoDiff && !bIs25&#41;
        &#123;
            strcpy&#40;string2, string0&#41;;
            int seconds1 = parsearg&#40;string2&#41;;
            strcpy&#40;string2, string1&#41;;
            int seconds2 = parsearg&#40;string2&#41;;
            double bf = seconds1 / &#40;double&#41; seconds2;
            bfsum += bf;
            bfcnt++;
            printf&#40;"%g,\n",bf&#41;;
        &#125;
        strcpy&#40;string1,string0&#41;;
    &#125;
    printf&#40;"\nAverage bf = %g\n", bfsum / bfcnt&#41;;
    return 0;
&#125;

on an Arena log file created by analyzing these EPD records to depth 36 with current Stockfish:

Code: Select all

r1b1qrk1/pp1n2bp/2pp2p1/4ppB1/2PPP1n1/2N2NP1/PP3PBP/R2Q1RK1 w - -
r1bq1rk1/pp3pb1/3p1n1p/2pPp1p1/2PnP3/2NB2B1/PP2NPPP/R2Q1RK1 w - c6
r4rk1/1p1b1pbp/p2p1np1/q1pPn3/P1P5/1PN1BP2/N2QB1PP/R4RK1 b - -
r4rk1/pp2qpbp/2pp1np1/4n3/2PNP1b1/1PN3P1/P1Q2PBP/R1BR2K1 w - -
r2q1rk1/1p1b1pb1/p2p1np1/2pPn3/P3P3/2N1BPPp/1P1NB2P/R2QK2R w KQ -
r4rk1/p2bq1b1/1p1p1n2/2NPpppp/2P1P3/P1N3BP/4BPP1/1R1Q1RK1 b - -
rnbq1rk1/1pp3b1/3p1n1p/p2Ppp2/2P5/2N2N1P/PP1BBPP1/R1Q1K2R b KQ -
r1b2rk1/1p2ppbp/p2p1np1/q1pPn3/2P1P3/2N1BPN1/PP1Q2PP/2R1KB1R b K -
r1bqnrk1/pp4bp/n2p2p1/3PpP2/6P1/2N1B3/PP1NBP1P/R2QK2R b KQ -
r1b2rk1/1p2ppbp/p1np1np1/q1pP4/2P1P3/2N1BP2/PP1QN1PP/2R1KB1R b K -
r1bqnrk1/pp4bp/n2p4/3PpP2/8/2N1B3/PP1NBP1P/R2QK2R b KQ -
r1bq1rk1/pppn2bp/3p2p1/1P1Pp3/4Pp2/3BBP2/PP2N1PP/R2Q1RK1 w - -
r1bqr1k1/pp1n1pbp/2pp1np1/8/2PPp3/BPN3P1/P1Q1PPBP/R3NRK1 b - -
r1bqnrk1/pp4bp/n2p4/3Ppp2/6P1/2N1B3/PP1NBP1P/R2QK2R w KQ -

I get this output:

Code: Select all

bf=
2,
1.95455,
1.05426,
1.44118,
1.08163,
1.24528,
1.81061,
1.31172,
1.20255,
1.36605,
1.10194,
1.17391,
1.40741,
1.39474,
1.30189,
1.11594,
1.24675,
1.46875,
1.80851,
1.70588,
1.33563,
1.27711,
3.30769,
1.81395,
1.08974,
1.14118,
1.2268,
1.35294,
2.20497,
1.24225,
1.05215,
1.62284,
1.19655,
1.875,
1.13333,
1.35294,
1.34783,
1.51613,
1.97872,
1.30108,
1.28099,
1.16774,
1.1326,
1.24878,
2.1,
1.95238,
1.58537,
1.15385,
1.24,
1.24731,
1.25862,
1.4589,
1.26761,
1.31852,
2.49719,
1.22222,
1.18182,
1.23077,
1.25,
1.85,
3.16216,
1.22222,
1.02797,
1.37415,
1.47525,
1.07718,
4.5,
1.01587,
1.39063,
1.05618,
1.11702,
1.07619,
1.16814,
1.0303,
1.125,
1.2549,
1.15104,
1.1,
1.09091,
1.16667,
1.17857,
1.30303,
1.69767,
1.27397,
1.13978,
1.10377,
1.16239,
2.41912,
1.125,
1.33333,
1.0625,
1.09804,
1.42857,
1.6,
1.42188,
1.78022,
1.17284,
1.14474,
1.15172,
1.4,
1.42857,
1.1,
3.09091,
1.20588,
1.07317,
1.72727,
1.51316,
1.2,
1.27536,
1.45455,
1.09677,
1.52941,
1.32692,
1.37681,
1.25263,
1.04202,
1.29032,
2.125,
1.15588,
1.25191,
1.15447,
1.33333,
1.05,
1.52381,
1.65625,
2.30189,
1.14754,
1.34286,
1.67553,
2.76508,
1.08955,
1.03899,
2.125,
2.05882,
1.28571,
1.11111,
1.22,
1.29508,
1.20253,
2.18947,
1.20673,
1.26693,
1.57862,
1.6,
1,
1.275,
1.80392,
1.63043,
1.13333,
1.06471,
1.72376,
1.22756,
1.51436,
1.73103,

Average bf = 1.43913

[Cardoso]

If an engine scales better, it is most likely search that is better (lower branching factor).

The second most likely thing would be the SMP implementation.

The evaluation will not affect scaling much, except for improvement in the move ordering.

sorry Dann, but that is all BS.

1) SMP is irrelevant, as Andreas' tests are conducted with single thread

2) BF has nothing to do with elo, are you aware of that? an engine with higher BF might have higher elo.

3) evaluation will not affect scaling much, hmm, I said 'king safety' and not 'evaluation', king safety might be related to both evaluation and search(and move ordering too, for that matter).

I have not based my observations on pure speculation, but on following extreme number of games at STC between the tops.

also, see Kai's statistics, seems to be pointing at exactly my hypothesis.

>>
2) BF has nothing to do with elo, are you aware of that? an engine with higher BF might have higher elo.
<<
You're a funny man.

you are even funnier.

2) BF has nothing to do with elo, are you aware of that? an engine with higher BF might have higher elo.

Ah ah, please understand booze and chess programming are incompatible
If BF has nothing to do with elo then it is easy to test this, we artificially increase BF and test the engine.

[Cardoso]

to do move ordering, you are using hash moves, where the score is based on eval; killer moves are based on eval

Lyudmil, the hashmove and it's associated score, comes not from the eval, but from a search that has an eval, same as the killer moves, counter moves, followupmoves you name it. if you simple assigned the eval result to the hashmove and to killers/countermoves/followupmoves etc that would hurt the engine badly.
Do you think Stockfish is so good because of eval? Just drop the SF eval and use a simplistic material only eval, and play some games against it, and you will finally understand the power of the search and the search also regulates Branching Factor.
To me SF success is 90% search and 10% eval!
Before making so assertive comments I think you should take a course on chess programming, steadily and gradually understand and implement the basics of an alphabeta searcher.

Also drop the "expert" attitude, as you are not one, as you are not even a student. Also that bit of overbearing pride is not heathy to you and those around you.