Trying to improve lazy smp

[bob]

If you make a change that supposedly gains +8 elo as you reported, you have to have the error bar much lower than +/- 8 to have any confidence in that result. IE a BUNCH of games is needed. If you are trying to prove a +40 Elo change, that takes fewer games. But you specifically gave something like +8 which needs 20-30K games to measure that accurately. If the error bar is +/- 40, you really don't have any idea whether your change is good or bad if the potential gain is only a few Elo...

Of course you have all the reason. But I'm trying to say that, as this is new and we don't know which is the upper bound, instead of spending efforts to be sure that a new point if sound, it's better to try higher points first to have an idea of the true upper bound and only then spend resources to refine the best points found.

If you are trying to find the value in cp of a rook, and you try 100 and then 150, instead of being sure of those 150, is sound to try first 200, even with a relative high margin of error.

How do you know if a change is good or bad? So how do you know in which direction to go for the next change not knowing that? That was my point. A few games makes this almost purely random. And while a "random walk" will eventually "get there" it won't get there very quickly or accurately.

[cdani]

How do you know if a change is good or bad? So how do you know in which direction to go for the next change not knowing that? That was my point. A few games makes this almost purely random. And while a "random walk" will eventually "get there" it won't get there very quickly or accurately.

I'm trying to slowly add more depth to threads and find if is at least a little better or similar.
If I have doubts if a new setup is good enough I will try either to add even more depth and try again or do more games.
I will repeat until I'm not able to add depth and win strength any more.
Then I will try seriously the bests setups to try to find the best one.

[cdani]

You can buy a used dual 6 core Xeon system on ebay pretty cheap. Here is one for $610:

http://www.ebay.com/itm/HP-Z800-Worksta ... 33a0db6e8d

For leasing, you can try Amazon as suggested. But also look at http://webnx.com/dedicated-servers/los- ... u-servers/. They have quad CPU systems too.

--Jon

Very interesting! Thanks!

[cdani]

Look at Daniel Shawul's code and Dr. Hyatt's code. They both have interesting Numa implementations.

Thanks! I will.

[bob]

How do you know if a change is good or bad? So how do you know in which direction to go for the next change not knowing that? That was my point. A few games makes this almost purely random. And while a "random walk" will eventually "get there" it won't get there very quickly or accurately.

I'm trying to slowly add more depth to threads and find if is at least a little better or similar.
If I have doubts if a new setup is good enough I will try either to add even more depth and try again or do more games.
I will repeat until I'm not able to add depth and win strength any more.
Then I will try seriously the bests setups to try to find the best one.

You are missing my point. How do you know if something "is a little better or similar". "little better" -> large number of games to have any certainty at all. If you play 50 games, the error margin is so large you could flip a coin and be just as certain, "heads it is better, tails it is worse".

It is an easy trap to fall into. Everyone has done it. But you have to realize, eventually, that you want to be pretty certain something is good before you keep it, or pretty certain something is bad before tossing it out.

[cdani]

You are missing my point. How do you know if something "is a little better or similar". "little better" -> large number of games to have any certainty at all. If you play 50 games, the error margin is so large you could flip a coin and be just as certain, "heads it is better, tails it is worse".

It is an easy trap to fall into. Everyone has done it. But you have to realize, eventually, that you want to be pretty certain something is good before you keep it, or pretty certain something is bad before tossing it out.

Sure.
For trying to find better setups, I tried from +-12 to +-8 elo. And when at some point I think I will nedd more, I will try do it. At 4 cores is a lot of hours for my resources.

[cdani]

Hi.
I have done a self test at 25+0.03 with 4 threads:

Code: Select all

Rank Name            Elo    +    - games score oppo. draw
1 Andscacs 0.74039    59    7    6  2512   69%   -59   45%
2 Andscacs 0.73174   -59    6    7  2512   31%    59   45%

The difference in % is equivalent to 140 elo, but bayeselo shows +-59. Someone knows what I'm interpreting bad? I used bayeselo for more than a year and I did not compared the % until now with standard elo.

I have done also some thousands games at 45+.03 and 60+.03 and the results are similar to the standard test I have done at 25+.03, so it seems that this idea scales well.

I have been also able to do a limited test at 20+.03 with 16 cores:

Code: Select all

 1 Andscacs 0.74042      114   19   19   704   69%   -23   27%
 2 Komodo 5.1r2 64-bit    86   40   40   140   46%   114   34%
 3 Senpai 1.0              6   41   42   140   34%   114   29%
 4 Gull 1.2 x64          -40   41   43   142   27%   114   31%
 5 Hannibal 1.3x64       -66   44   46   142   26%   114   20%
 6 Protector 1.7.0       -99   45   48   140   22%   114   21%

The result is very good, clearly over 3100 elo.

I will publish tomorrow the "official" version with what I have now, because I will be out for a week playing a tournament, and I prefer to have a version available for if someone wants to include it in a tournament.

[cdani]

I have been also able to do a limited test at 20+.03 with 16 cores

I forgot to mention the added depth for each thread:
{0, 1, 0, 2, 1, 3, 2, 4, 0, 1, 0, 1, 0, 1, 0, 1}

So is a basic initial approach.

I hope all this will be useful for someone, and also if someone uses it, that he shares their results if he wish

[Sven]

Hi.
I have done a self test at 25+0.03 with 4 threads:

Code: Select all

Rank Name            Elo    +    - games score oppo. draw
1 Andscacs 0.74039    59    7    6  2512   69%   -59   45%
2 Andscacs 0.73174   -59    6    7  2512   31%    59   45%

The difference in % is equivalent to 140 elo, but bayeselo shows +-59. Someone knows what I'm interpreting bad? I used bayeselo for more than a year and I did not compared the % until now with standard elo.

The difference between 59 and -59 is 118 which is quite a good match for 140.

[cdani]

Hi.
I have done a self test at 25+0.03 with 4 threads:

Code: Select all

Rank Name            Elo    +    - games score oppo. draw
1 Andscacs 0.74039    59    7    6  2512   69%   -59   45%
2 Andscacs 0.73174   -59    6    7  2512   31%    59   45%

The difference in % is equivalent to 140 elo, but bayeselo shows +-59. Someone knows what I'm interpreting bad? I used bayeselo for more than a year and I did not compared the % until now with standard elo.

The difference between 59 and -59 is 118 which is quite a good match for 140.

Yes, but with standar elo, 69% is 140 elo, not 118. Is simply different for bayeselo?