Yeah...seems that SF will run over other engines like a damn TRUCK!
18 days from release date of SF4 and almost +30 ELO gain. -> http://95.47.140.100/tests/view/522bcb1 ... 2ee68dc04a
Have a nice day yo false magicians. Your days are counted.
18 days from SF4 release and about ~30+ ELO gain!
Moderators: hgm, Rebel, chrisw
-
- Posts: 28
- Joined: Fri Jul 26, 2013 6:24 am
-
- Posts: 1471
- Joined: Tue Mar 16, 2010 12:00 am
Re: 18 days from SF4 release and about ~30+ ELO gain!
Indeed, the pace of Stockfish improvement is amazing, the development framework constructed by Gary is awesome.
Clearly no individual or two-person team can keep up with this in the long run, so this could mean the end of commercial chess engines as we currently know them. Maybe in 2 years time only Stockfish and derivates will continue to be developed.
Robert
Clearly no individual or two-person team can keep up with this in the long run, so this could mean the end of commercial chess engines as we currently know them. Maybe in 2 years time only Stockfish and derivates will continue to be developed.
Robert
-
- Posts: 1968
- Joined: Wed Jul 13, 2011 9:04 pm
- Location: Madrid, Spain.
Re: 19 days from SF 4 release and about ~30 Elo gain!
Hello Alexandre:
So I think that it is safer to say that the current Elo gain of SF 130908 (8th September, 2013) is around +18 Elo to +20 Elo more less (plus/minus error bars). It is indeed an outstanding improvement path of more than 6 Elo/week... although we are talking about less than three weeks now, a very short period of time to extrapolate the Elo gain at the end of the year, for example.
Regards from Spain.
Ajedrecista.
Please remember that that test is a self test (one SF versus other SF), and self tests tend to exaggerate the Elo gap between versions. The regression test of SF 4 (one thread) versus SF 3 (one thread) showed an Elo gain of over 56 Elo plus/minus error bars, but IPON list showed a gain of around 38 Elo (BayesElo units) IIRC (similar gains were reported by other rating lists if I am not wrong). For three threads, the result was almost the same: over 55 Elo plus/minus error bars.Masta wrote:Yeah...seems that SF will run over other engines like a damn TRUCK!
18 days from release date of SF4 and almost +30 ELO gain. -> http://95.47.140.100/tests/view/522bcb1 ... 2ee68dc04a
Have a nice day yo false magicians. Your days are counted.
So I think that it is safer to say that the current Elo gain of SF 130908 (8th September, 2013) is around +18 Elo to +20 Elo more less (plus/minus error bars). It is indeed an outstanding improvement path of more than 6 Elo/week... although we are talking about less than three weeks now, a very short period of time to extrapolate the Elo gain at the end of the year, for example.
Regards from Spain.
Ajedrecista.
-
- Posts: 92
- Joined: Tue Jan 26, 2010 5:03 am
Re: 18 days from SF4 release and about ~30+ ELO gain!
Robert why don't you start your own frame work like stockfish did I am sure there is people who would sign on. then there will be two great engines competing all the time.
-
- Posts: 10282
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: 19 days from SF 4 release and about ~30 Elo gain!
There are rating lists that show more than 56 elo for stockfish4 relative to stockfish3Ajedrecista wrote:Hello Alexandre:
Please remember that that test is a self test (one SF versus other SF), and self tests tend to exaggerate the Elo gap between versions. The regression test of SF 4 (one thread) versus SF 3 (one thread) showed an Elo gain of over 56 Elo plus/minus error bars, but IPON list showed a gain of around 38 Elo (BayesElo units) IIRC (similar gains were reported by other rating lists if I am not wrong). For three threads, the result was almost the same: over 55 Elo plus/minus error bars.Masta wrote:Yeah...seems that SF will run over other engines like a damn TRUCK!
18 days from release date of SF4 and almost +30 ELO gain. -> http://95.47.140.100/tests/view/522bcb1 ... 2ee68dc04a
Have a nice day yo false magicians. Your days are counted.
So I think that it is safer to say that the current Elo gain of SF 130908 (8th September, 2013) is around +18 Elo to +20 Elo more less (plus/minus error bars). It is indeed an outstanding improvement path of more than 6 Elo/week... although we are talking about less than three weeks now, a very short period of time to extrapolate the Elo gain at the end of the year, for example.
Regards from Spain.
Ajedrecista.
CEGT 40/4 rating list shows 66 elo improvement so I think that the main reason is not that the 56 elo is too much but the fact that the IPON use slower time control.
http://www.husvankempen.de/nunn/40_4_Ra ... liste.html
18 Stockfish 4.0 x64 1CPU 3032 13 13 1700 66.6% 2911 41.2%
53 Stockfish 3.0 x64 1CPU 2966 12 12 2000 61.0% 2889 43.6%
Note also that the best stockfish in the CEGT 40/4 rating list is a different stockfish and 30 elo may not enough for stockfish developement version with 1 cpu to catch the first place there.
7 Stockfish 2.2.2 x64 4CPU 3081 13 13 1600 68.4% 2947 40.0%
-
- Posts: 3226
- Joined: Wed May 06, 2009 10:31 pm
- Location: Fuquay-Varina, North Carolina
Re: 19 days from SF 4 release and about ~30 Elo gain!
The regression results inside the Stockfish testing framework are not directly comparable to any rating list due to the large draw rate.Ajedrecista wrote:Hello Alexandre:
Please remember that that test is a self test (one SF versus other SF), and self tests tend to exaggerate the Elo gap between versions. The regression test of SF 4 (one thread) versus SF 3 (one thread) showed an Elo gain of over 56 Elo plus/minus error bars, but IPON list showed a gain of around 38 Elo (BayesElo units) IIRC (similar gains were reported by other rating lists if I am not wrong). For three threads, the result was almost the same: over 55 Elo plus/minus error bars.Masta wrote:Yeah...seems that SF will run over other engines like a damn TRUCK!
18 days from release date of SF4 and almost +30 ELO gain. -> http://95.47.140.100/tests/view/522bcb1 ... 2ee68dc04a
Have a nice day yo false magicians. Your days are counted.
So I think that it is safer to say that the current Elo gain of SF 130908 (8th September, 2013) is around +18 Elo to +20 Elo more less (plus/minus error bars). It is indeed an outstanding improvement path of more than 6 Elo/week... although we are talking about less than three weeks now, a very short period of time to extrapolate the Elo gain at the end of the year, for example.
Regards from Spain.
Ajedrecista.
-
- Posts: 3226
- Joined: Wed May 06, 2009 10:31 pm
- Location: Fuquay-Varina, North Carolina
Re: 18 days from SF4 release and about ~30+ ELO gain!
The main advantage of the Stockfish testing framework is actually the number of people contributing ideas/code to be tested. One or two people can only produce a limited number of changes to test in a week. The amount of cpu resources being used is great, but of limited value if there were only a handful of changes to test per week/month.gotogo wrote:Robert why don't you start your own frame work like stockfish did I am sure there is people who would sign on. then there will be two great engines competing all the time.
I do not think that taking on a lot of contributors is feasible for a commercial engine.
-
- Posts: 10282
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: 18 days from SF4 release and about ~30+ ELO gain!
I think that the main advantage is practically not the number of people but the number of machines that they can use when commercial programmers usually do not have 200 cores that work 24 hours per day for them.Adam Hair wrote:The main advantage of the Stockfish testing framework is actually the number of people contributing ideas/code to be tested. One or two people can only produce a limited number of changes to test in a week. The amount of cpu resources being used is great, but of limited value if there were only a handful of changes to test per week/month.gotogo wrote:Robert why don't you start your own frame work like stockfish did I am sure there is people who would sign on. then there will be two great engines competing all the time.
I do not think that taking on a lot of contributors is feasible for a commercial engine.
Inspite of it I am not sure that stockfish is going to be developed faster.
Stockfish has also the disadvantage that it is open source so other can learn from it to improve their programs.
-
- Posts: 2554
- Joined: Fri Nov 26, 2010 2:00 pm
- Location: Czech Republic
- Full name: Martin Sedlak
Re: 18 days from SF4 release and about ~30+ ELO gain!
I think Adam is right. Do you really think that commercials can't build such framework?!Uri Blass wrote: I think that the main advantage is practically not the number of people but the number of machines that they can use when commercial programmers usually do not have 200 cores that work 24 hours per day for them.
Inspite of it I am not sure that stockfish is going to be developed faster.
Stockfish has also the disadvantage that it is open source so other can learn from it to improve their programs.
As for the rest, do you understand what it means being one step ahead?
P.S. I certainly won't miss Houdini.
-
- Posts: 10282
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: 18 days from SF4 release and about ~30+ ELO gain!
I think that it is going to cost too much for the commercial to build the framework and use it.mar wrote:I think Adam is right. Do you really think that commercials can't build such framework?!Uri Blass wrote: I think that the main advantage is practically not the number of people but the number of machines that they can use when commercial programmers usually do not have 200 cores that work 24 hours per day for them.
Inspite of it I am not sure that stockfish is going to be developed faster.
Stockfish has also the disadvantage that it is open source so other can learn from it to improve their programs.
As for the rest, do you understand what it means being one step ahead?
P.S. I certainly won't miss Houdini.
It is clear that it is only a question of money and if the commercial are interested they can probably also buy most of the people who give ideas for stockfish and I think that it is going to cost them less money then the money that they need to use 200 cores for 24 hours per day.