TalkChess.com

Yeah...seems that SF will run over other engines like a damn TRUCK!

18 days from release date of SF4 and almost +30 ELO gain. -> http://95.47.140.100/tests/view/522bcb1 ... 2ee68dc04a

Have a nice day yo false magicians. Your days are counted.

Indeed, the pace of Stockfish improvement is amazing, the development framework constructed by Gary is awesome.
Clearly no individual or two-person team can keep up with this in the long run, so this could mean the end of commercial chess engines as we currently know them. Maybe in 2 years time only Stockfish and derivates will continue to be developed.

Robert

Hello Alexandre:

Masta wrote:Yeah...seems that SF will run over other engines like a damn TRUCK!

18 days from release date of SF4 and almost +30 ELO gain. -> http://95.47.140.100/tests/view/522bcb1 ... 2ee68dc04a

Have a nice day yo false magicians. Your days are counted.

Please remember that that test is a self test (one SF versus other SF), and self tests tend to exaggerate the Elo gap between versions. The regression test of SF 4 (one thread) versus SF 3 (one thread) showed an Elo gain of over 56 Elo plus/minus error bars, but IPON list showed a gain of around 38 Elo (BayesElo units) IIRC (similar gains were reported by other rating lists if I am not wrong). For three threads, the result was almost the same: over 55 Elo plus/minus error bars.

So I think that it is safer to say that the current Elo gain of SF 130908 (8th September, 2013) is around +18 Elo to +20 Elo more less (plus/minus error bars). It is indeed an outstanding improvement path of more than 6 Elo/week... although we are talking about less than three weeks now, a very short period of time to extrapolate the Elo gain at the end of the year, for example.

Regards from Spain.

Ajedrecista.

Robert why don't you start your own frame work like stockfish did I am sure there is people who would sign on. then there will be two great engines competing all the time.

Ajedrecista wrote:Hello Alexandre:

Masta wrote:Yeah...seems that SF will run over other engines like a damn TRUCK!

18 days from release date of SF4 and almost +30 ELO gain. -> http://95.47.140.100/tests/view/522bcb1 ... 2ee68dc04a

Have a nice day yo false magicians. Your days are counted.

Please remember that that test is a self test (one SF versus other SF), and self tests tend to exaggerate the Elo gap between versions. The regression test of SF 4 (one thread) versus SF 3 (one thread) showed an Elo gain of over 56 Elo plus/minus error bars, but IPON list showed a gain of around 38 Elo (BayesElo units) IIRC (similar gains were reported by other rating lists if I am not wrong). For three threads, the result was almost the same: over 55 Elo plus/minus error bars.

So I think that it is safer to say that the current Elo gain of SF 130908 (8th September, 2013) is around +18 Elo to +20 Elo more less (plus/minus error bars). It is indeed an outstanding improvement path of more than 6 Elo/week... although we are talking about less than three weeks now, a very short period of time to extrapolate the Elo gain at the end of the year, for example.

Regards from Spain.

Ajedrecista.

There are rating lists that show more than 56 elo for stockfish4 relative to stockfish3

CEGT 40/4 rating list shows 66 elo improvement so I think that the main reason is not that the 56 elo is too much but the fact that the IPON use slower time control.

http://www.husvankempen.de/nunn/40_4_Ra ... liste.html

18 Stockfish 4.0 x64 1CPU 3032 13 13 1700 66.6% 2911 41.2%

53 Stockfish 3.0 x64 1CPU 2966 12 12 2000 61.0% 2889 43.6%

Note also that the best stockfish in the CEGT 40/4 rating list is a different stockfish and 30 elo may not enough for stockfish developement version with 1 cpu to catch the first place there.

7 Stockfish 2.2.2 x64 4CPU 3081 13 13 1600 68.4% 2947 40.0%

Ajedrecista wrote:Hello Alexandre:

Masta wrote:Yeah...seems that SF will run over other engines like a damn TRUCK!

18 days from release date of SF4 and almost +30 ELO gain. -> http://95.47.140.100/tests/view/522bcb1 ... 2ee68dc04a

Have a nice day yo false magicians. Your days are counted.

Please remember that that test is a self test (one SF versus other SF), and self tests tend to exaggerate the Elo gap between versions. The regression test of SF 4 (one thread) versus SF 3 (one thread) showed an Elo gain of over 56 Elo plus/minus error bars, but IPON list showed a gain of around 38 Elo (BayesElo units) IIRC (similar gains were reported by other rating lists if I am not wrong). For three threads, the result was almost the same: over 55 Elo plus/minus error bars.

So I think that it is safer to say that the current Elo gain of SF 130908 (8th September, 2013) is around +18 Elo to +20 Elo more less (plus/minus error bars). It is indeed an outstanding improvement path of more than 6 Elo/week... although we are talking about less than three weeks now, a very short period of time to extrapolate the Elo gain at the end of the year, for example.

Regards from Spain.

Ajedrecista.

The regression results inside the Stockfish testing framework are not directly comparable to any rating list due to the large draw rate.

gotogo wrote:Robert why don't you start your own frame work like stockfish did I am sure there is people who would sign on. then there will be two great engines competing all the time.

The main advantage of the Stockfish testing framework is actually the number of people contributing ideas/code to be tested. One or two people can only produce a limited number of changes to test in a week. The amount of cpu resources being used is great, but of limited value if there were only a handful of changes to test per week/month.

I do not think that taking on a lot of contributors is feasible for a commercial engine.

Adam Hair wrote:

gotogo wrote:Robert why don't you start your own frame work like stockfish did I am sure there is people who would sign on. then there will be two great engines competing all the time.

The main advantage of the Stockfish testing framework is actually the number of people contributing ideas/code to be tested. One or two people can only produce a limited number of changes to test in a week. The amount of cpu resources being used is great, but of limited value if there were only a handful of changes to test per week/month.

I do not think that taking on a lot of contributors is feasible for a commercial engine.

I think that the main advantage is practically not the number of people but the number of machines that they can use when commercial programmers usually do not have 200 cores that work 24 hours per day for them.

Inspite of it I am not sure that stockfish is going to be developed faster.

Stockfish has also the disadvantage that it is open source so other can learn from it to improve their programs.

Uri Blass wrote: I think that the main advantage is practically not the number of people but the number of machines that they can use when commercial programmers usually do not have 200 cores that work 24 hours per day for them.

Inspite of it I am not sure that stockfish is going to be developed faster.

Stockfish has also the disadvantage that it is open source so other can learn from it to improve their programs.

I think Adam is right. Do you really think that commercials can't build such framework?!
As for the rest, do you understand what it means being one step ahead?
P.S. I certainly won't miss Houdini.

mar wrote:

Uri Blass wrote: I think that the main advantage is practically not the number of people but the number of machines that they can use when commercial programmers usually do not have 200 cores that work 24 hours per day for them.

Inspite of it I am not sure that stockfish is going to be developed faster.

Stockfish has also the disadvantage that it is open source so other can learn from it to improve their programs.

I think Adam is right. Do you really think that commercials can't build such framework?!
As for the rest, do you understand what it means being one step ahead?
P.S. I certainly won't miss Houdini.

I think that it is going to cost too much for the commercial to build the framework and use it.

It is clear that it is only a question of money and if the commercial are interested they can probably also buy most of the people who give ideas for stockfish and I think that it is going to cost them less money then the money that they need to use 200 cores for 24 hours per day.