How do you know if a change is good or bad? So how do you know in which direction to go for the next change not knowing that? That was my point. A few games makes this almost purely random. And while a "random walk" will eventually "get there" it won't get there very quickly or accurately.cdani wrote:Of course you have all the reason. But I'm trying to say that, as this is new and we don't know which is the upper bound, instead of spending efforts to be sure that a new point if sound, it's better to try higher points first to have an idea of the true upper bound and only then spend resources to refine the best points found.bob wrote:If you make a change that supposedly gains +8 elo as you reported, you have to have the error bar much lower than +/- 8 to have any confidence in that result. IE a BUNCH of games is needed. If you are trying to prove a +40 Elo change, that takes fewer games. But you specifically gave something like +8 which needs 20-30K games to measure that accurately. If the error bar is +/- 40, you really don't have any idea whether your change is good or bad if the potential gain is only a few Elo...
If you are trying to find the value in cp of a rook, and you try 100 and then 150, instead of being sure of those 150, is sound to try first 200, even with a relative high margin of error.
Trying to improve lazy smp
Moderators: hgm, Rebel, chrisw
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Trying to improve lazy smp
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Trying to improve lazy smp
I'm trying to slowly add more depth to threads and find if is at least a little better or similar.bob wrote: How do you know if a change is good or bad? So how do you know in which direction to go for the next change not knowing that? That was my point. A few games makes this almost purely random. And while a "random walk" will eventually "get there" it won't get there very quickly or accurately.
If I have doubts if a new setup is good enough I will try either to add even more depth and try again or do more games.
I will repeat until I'm not able to add depth and win strength any more.
Then I will try seriously the bests setups to try to find the best one.
Daniel José - http://www.andscacs.com
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Trying to improve lazy smp
Very interesting! Thanks!jdart wrote:You can buy a used dual 6 core Xeon system on ebay pretty cheap. Here is one for $610:
http://www.ebay.com/itm/HP-Z800-Worksta ... 33a0db6e8d
For leasing, you can try Amazon as suggested. But also look at http://webnx.com/dedicated-servers/los- ... u-servers/. They have quad CPU systems too.
--Jon
Daniel José - http://www.andscacs.com
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Trying to improve lazy smp
Thanks! I will.Dann Corbit wrote: Look at Daniel Shawul's code and Dr. Hyatt's code. They both have interesting Numa implementations.
Daniel José - http://www.andscacs.com
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Trying to improve lazy smp
You are missing my point. How do you know if something "is a little better or similar". "little better" -> large number of games to have any certainty at all. If you play 50 games, the error margin is so large you could flip a coin and be just as certain, "heads it is better, tails it is worse".cdani wrote:I'm trying to slowly add more depth to threads and find if is at least a little better or similar.bob wrote: How do you know if a change is good or bad? So how do you know in which direction to go for the next change not knowing that? That was my point. A few games makes this almost purely random. And while a "random walk" will eventually "get there" it won't get there very quickly or accurately.
If I have doubts if a new setup is good enough I will try either to add even more depth and try again or do more games.
I will repeat until I'm not able to add depth and win strength any more.
Then I will try seriously the bests setups to try to find the best one.
It is an easy trap to fall into. Everyone has done it. But you have to realize, eventually, that you want to be pretty certain something is good before you keep it, or pretty certain something is bad before tossing it out.
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Trying to improve lazy smp
Sure.bob wrote: You are missing my point. How do you know if something "is a little better or similar". "little better" -> large number of games to have any certainty at all. If you play 50 games, the error margin is so large you could flip a coin and be just as certain, "heads it is better, tails it is worse".
It is an easy trap to fall into. Everyone has done it. But you have to realize, eventually, that you want to be pretty certain something is good before you keep it, or pretty certain something is bad before tossing it out.
For trying to find better setups, I tried from +-12 to +-8 elo. And when at some point I think I will nedd more, I will try do it. At 4 cores is a lot of hours for my resources.
Daniel José - http://www.andscacs.com
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Trying to improve lazy smp
Hi.
I have done a self test at 25+0.03 with 4 threads:
The difference in % is equivalent to 140 elo, but bayeselo shows +-59. Someone knows what I'm interpreting bad? I used bayeselo for more than a year and I did not compared the % until now with standard elo.
I have done also some thousands games at 45+.03 and 60+.03 and the results are similar to the standard test I have done at 25+.03, so it seems that this idea scales well.
I have been also able to do a limited test at 20+.03 with 16 cores:
The result is very good, clearly over 3100 elo.
I will publish tomorrow the "official" version with what I have now, because I will be out for a week playing a tournament, and I prefer to have a version available for if someone wants to include it in a tournament.
I have done a self test at 25+0.03 with 4 threads:
Code: Select all
Rank Name Elo + - games score oppo. draw
1 Andscacs 0.74039 59 7 6 2512 69% -59 45%
2 Andscacs 0.73174 -59 6 7 2512 31% 59 45%
I have done also some thousands games at 45+.03 and 60+.03 and the results are similar to the standard test I have done at 25+.03, so it seems that this idea scales well.
I have been also able to do a limited test at 20+.03 with 16 cores:
Code: Select all
1 Andscacs 0.74042 114 19 19 704 69% -23 27%
2 Komodo 5.1r2 64-bit 86 40 40 140 46% 114 34%
3 Senpai 1.0 6 41 42 140 34% 114 29%
4 Gull 1.2 x64 -40 41 43 142 27% 114 31%
5 Hannibal 1.3x64 -66 44 46 142 26% 114 20%
6 Protector 1.7.0 -99 45 48 140 22% 114 21%
I will publish tomorrow the "official" version with what I have now, because I will be out for a week playing a tournament, and I prefer to have a version available for if someone wants to include it in a tournament.
Daniel José - http://www.andscacs.com
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Trying to improve lazy smp
I forgot to mention the added depth for each thread:cdani wrote: I have been also able to do a limited test at 20+.03 with 16 cores
{0, 1, 0, 2, 1, 3, 2, 4, 0, 1, 0, 1, 0, 1, 0, 1}
So is a basic initial approach.
I hope all this will be useful for someone, and also if someone uses it, that he shares their results if he wish
Daniel José - http://www.andscacs.com
-
- Posts: 4052
- Joined: Thu May 15, 2008 9:57 pm
- Location: Berlin, Germany
- Full name: Sven Schüle
Re: Trying to improve lazy smp
The difference between 59 and -59 is 118 which is quite a good match for 140.cdani wrote:Hi.
I have done a self test at 25+0.03 with 4 threads:
The difference in % is equivalent to 140 elo, but bayeselo shows +-59. Someone knows what I'm interpreting bad? I used bayeselo for more than a year and I did not compared the % until now with standard elo.Code: Select all
Rank Name Elo + - games score oppo. draw 1 Andscacs 0.74039 59 7 6 2512 69% -59 45% 2 Andscacs 0.73174 -59 6 7 2512 31% 59 45%
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Trying to improve lazy smp
Yes, but with standar elo, 69% is 140 elo, not 118. Is simply different for bayeselo?Sven Schüle wrote:The difference between 59 and -59 is 118 which is quite a good match for 140.cdani wrote:Hi.
I have done a self test at 25+0.03 with 4 threads:
The difference in % is equivalent to 140 elo, but bayeselo shows +-59. Someone knows what I'm interpreting bad? I used bayeselo for more than a year and I did not compared the % until now with standard elo.Code: Select all
Rank Name Elo + - games score oppo. draw 1 Andscacs 0.74039 59 7 6 2512 69% -59 45% 2 Andscacs 0.73174 -59 6 7 2512 31% 59 45%
Daniel José - http://www.andscacs.com