18 days from SF4 release and about ~30+ ELO gain!

[Daniel Shawul]

P.S. I certainly won't miss Houdini.

I am tempted to take a cheap shot at Houdart for bad-mouthing my awesome bit-bases but I won't

[Houdini]

But there is another side effect of open development that could be more threatening for commercial engines, a threat that was not foreseen in advance and that even I didn't realize it would be a problem. This is the obsolescence of release process: just few days after Stockfish 4 is out, almost all power users have dismissed it in favor of last nightly build (I just come here now from Playchess where there is even not one SF 4 but are all nightly builds): this is something commercial engines have no defense against, simply they cannot do this. As long as the ELO gap is big it is ok, but when the open developed engine reaches the level of commercials, a new compile each day can really badly affect the commercial release because it greatly speeds up its obsolescent.

I have to say that this was not foreseen and I am sorry for this, it is not out target (daily binaries are even built outside of SF team) but it is something that, considered the open nature of the development, it is almost impossible to avoid.

Marco, there really is no reason to feel sorry, it's called progress. Commercial engines will have to adapt to the new situation or disappear.

You guys have done an awesome job - it's really outstanding to add 100+ Elo to a top engine in less than 6 months. Congratulations to all the contributors for this achievement!

Cheers,
Robert

[Stonewall]

Bad News for Houdini 3 .. Coz Stock fish latest development can now easily defeat it in my engine match .. I think that Robert must think something new to suppress this epidemic. Good Luck Commercial Houdini.

[Adam Hair]

I believe 'ideas' are fairly easy to come by, for ex:
bonus for rook behind passed pawn
penalty for undefended piece
etc.

The difficult part is the testing...
i.e. the truly beneficial ideas are found and verified thru extensive testing...
and the more processing power you have, the more ideas can be tested
(as result: more good ones can/may be identified)

My biggest assumptions are that it takes time to encode ideas and that a single developer will, at times, have no useful ideas or need some time to recharge. Stockfish suffers from neither of these problems.

I think the systems we are talking bout are beyond reach of most developers
(the cheapest modern 32 core IBM system I could find is $44,648.00)

A group of older systems is certainly better than nothing, but has the disadvantage of being very slow compared to today's moderen systems
i.e. the same # of tests would take much much longer to complete

I suspect that the average core in the Stockfish framework is not more than twice as fast than what I now have (Xeon L5420 2.5 GHz quad processors), and going half as fast is not much of a detriment considering that the queue of tests coming from one person is going to be shorter than that from a group of people.

[mwyoung]

Indeed, the pace of Stockfish improvement is amazing, the development framework constructed by Gary is awesome.
Clearly no individual or two-person team can keep up with this in the long run, so this could mean the end of commercial chess engines as we currently know them. Maybe in 2 years time only Stockfish and derivates will continue to be developed.

Robert

Improvement cannot be foreseen in advance. It could be that next month we got 0 ELO, it happens and you know it.

But there is another side effect of open development that could be more threatening for commercial engines, a threat that was not foreseen in advance and that even I didn't realize it would be a problem. This is the obsolescence of release process: just few days after Stockfish 4 is out, almost all power users have dismissed it in favor of last nightly build (I just come here now from Playchess where there is even not one SF 4 but are all nightly builds): this is something commercial engines have no defense against, simply they cannot do this. As long as the ELO gap is big it is ok, but when the open developed engine reaches the level of commercials, a new compile each day can really badly affect the commercial release because it greatly speeds up its obsolescent.

I have to say that this was not foreseen and I am sorry for this, it is not out target (daily binaries are even built outside of SF team) but it is something that, considered the open nature of the development, it is almost impossible to avoid.

This was my point also. And don't feel sorry, it is this type of innovation by design or accident that spurs more innovation. Now the commercial programs know there is a market for this type of release. If the commercial programs can do what you have done, they can charge for that service.

If the market wants more then yearly releases. I see nothing wrong with that, and I see how it could be profitable for the commercial engines to provide such a service.

[gladius]

I believe 'ideas' are fairly easy to come by, for ex:
bonus for rook behind passed pawn
penalty for undefended piece
etc.

The difficult part is the testing...
i.e. the truly beneficial ideas are found and verified thru extensive testing...
and the more processing power you have, the more ideas can be tested
(as result: more good ones can/may be identified)

My biggest assumptions are that it takes time to encode ideas and that a single developer will, at times, have no useful ideas or need some time to recharge. Stockfish suffers from neither of these problems.

I think the systems we are talking bout are beyond reach of most developers
(the cheapest modern 32 core IBM system I could find is $44,648.00)

A group of older systems is certainly better than nothing, but has the disadvantage of being very slow compared to today's moderen systems
i.e. the same # of tests would take much much longer to complete

I suspect that the average core in the Stockfish framework is not more than twice as fast than what I now have (Xeon L5420 2.5 GHz quad processors), and going half as fast is not much of a detriment considering that the queue of tests coming from one person is going to be shorter than that from a group of people.

I'd say it's a combination of the two:

1. Easy access to computing power. This can be duplicated for a cost, but it is in both time and money. Time spent managing servers, etc. is time not spent improving the engine. Also, the ongoing electricity costs should not be overlooked. Of course, fishtest is open source, and is a pretty good management system at this point .

2. Wide variety of ideas. This is quite true, that one developer can run out of ideas/inspiration. However, I don't think we are at that point yet. I have a ton of ideas I'd love to try out on SF, but I don't have the time! Especially things that the framework is not good at, like tuning multiple values.

[gladius]

Clearly no individual or two-person team can keep up with this in the long run, so this could mean the end of commercial chess engines as we currently know them. Maybe in 2 years time only Stockfish and derivates will continue to be developed.

Already, developing a chess engine is a labor of love. I have to imagine that is true even for commercial chess developers. If you are good enough to get your engine to the top of an incredibly competitive field, you could be making more money doing other things .

In the ELO race, who knows how things will go in the future. It's totally possible SF could hit a wall at any time (although I hope not!).

[shrapnel]

Marco, there really is no reason to feel sorry, it's called progress. Commercial engines will have to adapt to the new situation or disappear.

You guys have done an awesome job - it's really outstanding to add 100+ Elo to a top engine in less than 6 months. Congratulations to all the contributors for this achievement!

Cheers,
Robert

That's very sporting of you, Mr Houdart !
Good to see the sporting spirit is still alive in the Day and Age of Sore Losers one sees everywhere, in the chess and non-chess world !
Most people nowadays simply can't seem to swallow a defeat with grace !
Regards

[Red Hood]

Especially things that the framework is not good at, like tuning multiple values.

CLOP should be ready soon so you just might have a chance at that!

[gladius]

Especially things that the framework is not good at, like tuning multiple values.

CLOP should be ready soon so you just might have a chance at that!

Yes, hopefully I will have a chance to get CLOP going soon. It's a complicated system on the backend though. Probably more complicated than it should be, but oh well. Almost there!